PostgreSQL specific model fields ================================ All of these fields are available from the ``django.contrib.postgres.fields`` module. .. currentmodule:: django.contrib.postgres.fields ArrayField ---------- .. class:: ArrayField(base_field, size=None, **options) A field for storing lists of data. Most field types can be used, you simply pass another field instance as the :attr:`base_field `. You may also specify a :attr:`size `. ``ArrayField`` can be nested to store multi-dimensional arrays. If you give the field a :attr:`~django.db.models.Field.default`, ensure it's a callable such as ``list`` (for an empty default) or a callable that returns a list (such as a function). Incorrectly using ``default=[]`` creates a mutable default that is shared between all instances of ``ArrayField``. .. attribute:: base_field This is a required argument. Specifies the underlying data type and behavior for the array. It should be an instance of a subclass of :class:`~django.db.models.Field`. For example, it could be an :class:`~django.db.models.IntegerField` or a :class:`~django.db.models.CharField`. Most field types are permitted, with the exception of those handling relational data (:class:`~django.db.models.ForeignKey`, :class:`~django.db.models.OneToOneField` and :class:`~django.db.models.ManyToManyField`). It is possible to nest array fields - you can specify an instance of ``ArrayField`` as the ``base_field``. For example:: from django.db import models from django.contrib.postgres.fields import ArrayField class ChessBoard(models.Model): board = ArrayField( ArrayField( models.CharField(max_length=10, blank=True), size=8, ), size=8, ) Transformation of values between the database and the model, validation of data and configuration, and serialization are all delegated to the underlying base field. .. attribute:: size This is an optional argument. If passed, the array will have a maximum size as specified. This will be passed to the database, although PostgreSQL at present does not enforce the restriction. .. note:: When nesting ``ArrayField``, whether you use the `size` parameter or not, PostgreSQL requires that the arrays are rectangular:: from django.contrib.postgres.fields import ArrayField from django.db import models class Board(models.Model): pieces = ArrayField(ArrayField(models.IntegerField())) # Valid Board(pieces=[ [2, 3], [2, 1], ]) # Not valid Board(pieces=[ [2, 3], [2], ]) If irregular shapes are required, then the underlying field should be made nullable and the values padded with ``None``. Querying ArrayField ^^^^^^^^^^^^^^^^^^^ There are a number of custom lookups and transforms for :class:`ArrayField`. We will use the following example model:: from django.db import models from django.contrib.postgres.fields import ArrayField class Post(models.Model): name = models.CharField(max_length=200) tags = ArrayField(models.CharField(max_length=200), blank=True) def __str__(self): # __unicode__ on Python 2 return self.name .. fieldlookup:: arrayfield.contains contains ~~~~~~~~ The :lookup:`contains` lookup is overridden on :class:`ArrayField`. The returned objects will be those where the values passed are a subset of the data. It uses the SQL operator ``@>``. For example:: >>> Post.objects.create(name='First post', tags=['thoughts', 'django']) >>> Post.objects.create(name='Second post', tags=['thoughts']) >>> Post.objects.create(name='Third post', tags=['tutorial', 'django']) >>> Post.objects.filter(tags__contains=['thoughts']) [, ] >>> Post.objects.filter(tags__contains=['django']) [, ] >>> Post.objects.filter(tags__contains=['django', 'thoughts']) [] .. fieldlookup:: arrayfield.contained_by contained_by ~~~~~~~~~~~~ This is the inverse of the :lookup:`contains ` lookup - the objects returned will be those where the data is a subset of the values passed. It uses the SQL operator ``<@``. For example:: >>> Post.objects.create(name='First post', tags=['thoughts', 'django']) >>> Post.objects.create(name='Second post', tags=['thoughts']) >>> Post.objects.create(name='Third post', tags=['tutorial', 'django']) >>> Post.objects.filter(tags__contained_by=['thoughts', 'django']) [, ] >>> Post.objects.filter(tags__contained_by=['thoughts', 'django', 'tutorial']) [, , ] .. fieldlookup:: arrayfield.overlap overlap ~~~~~~~ Returns objects where the data shares any results with the values passed. Uses the SQL operator ``&&``. For example:: >>> Post.objects.create(name='First post', tags=['thoughts', 'django']) >>> Post.objects.create(name='Second post', tags=['thoughts']) >>> Post.objects.create(name='Third post', tags=['tutorial', 'django']) >>> Post.objects.filter(tags__overlap=['thoughts']) [, ] >>> Post.objects.filter(tags__overlap=['thoughts', 'tutorial']) [, , ] .. fieldlookup:: arrayfield.len len ~~~ Returns the length of the array. The lookups available afterwards are those available for :class:`~django.db.models.IntegerField`. For example:: >>> Post.objects.create(name='First post', tags=['thoughts', 'django']) >>> Post.objects.create(name='Second post', tags=['thoughts']) >>> Post.objects.filter(tags__len=1) [] .. fieldlookup:: arrayfield.index Index transforms ~~~~~~~~~~~~~~~~ This class of transforms allows you to index into the array in queries. Any non-negative integer can be used. There are no errors if it exceeds the :attr:`size ` of the array. The lookups available after the transform are those from the :attr:`base_field `. For example:: >>> Post.objects.create(name='First post', tags=['thoughts', 'django']) >>> Post.objects.create(name='Second post', tags=['thoughts']) >>> Post.objects.filter(tags__0='thoughts') [, ] >>> Post.objects.filter(tags__1__iexact='Django') [] >>> Post.objects.filter(tags__276='javascript') [] .. note:: PostgreSQL uses 1-based indexing for array fields when writing raw SQL. However these indexes and those used in :lookup:`slices ` use 0-based indexing to be consistent with Python. .. fieldlookup:: arrayfield.slice Slice transforms ~~~~~~~~~~~~~~~~ This class of transforms allow you to take a slice of the array. Any two non-negative integers can be used, separated by a single underscore. The lookups available after the transform do not change. For example:: >>> Post.objects.create(name='First post', tags=['thoughts', 'django']) >>> Post.objects.create(name='Second post', tags=['thoughts']) >>> Post.objects.create(name='Third post', tags=['django', 'python', 'thoughts']) >>> Post.objects.filter(tags__0_1=['thoughts']) [] >>> Post.objects.filter(tags__0_2__contains='thoughts') [, ] .. note:: PostgreSQL uses 1-based indexing for array fields when writing raw SQL. However these slices and those used in :lookup:`indexes ` use 0-based indexing to be consistent with Python. .. admonition:: Multidimensional arrays with indexes and slices PostgreSQL has some rather esoteric behavior when using indexes and slices on multidimensional arrays. It will always work to use indexes to reach down to the final underlying data, but most other slices behave strangely at the database level and cannot be supported in a logical, consistent fashion by Django. Indexing ArrayField ^^^^^^^^^^^^^^^^^^^ At present using :attr:`~django.db.models.Field.db_index` will create a ``btree`` index. This does not offer particularly significant help to querying. A more useful index is a ``GIN`` index, which you should create using a :class:`~django.db.migrations.operations.RunSQL` operation. HStoreField ----------- .. class:: HStoreField(**options) A field for storing mappings of strings to strings. The Python data type used is a ``dict``. To use this field, you'll need to: 1. Add ``'django.contrib.postgres'`` in your :setting:`INSTALLED_APPS`. 2. Setup the hstore extension in PostgreSQL before the first ``CreateModel`` or ``AddField`` operation by adding a migration with the :class:`~django.contrib.postgres.operations.HStoreExtension` operation. For example:: from django.contrib.postgres.operations import HStoreExtension class Migration(migrations.Migration): ... operations = [ HStoreExtension(), ... ] Creating the extension requires a database user with superuser privileges. If the Django database user doesn't have superuser privileges, you'll have to create the extension outside of Django migrations with a user that has the appropriate privileges. In that case, connect to your Django database and run the query ``CREATE EXTENSION IF NOT EXISTS hstore;`` You'll see an error like ``can't adapt type 'dict'`` if you skip the first step, or ``type "hstore" does not exist`` if you skip the second. .. note:: On occasions it may be useful to require or restrict the keys which are valid for a given field. This can be done using the :class:`~django.contrib.postgres.validators.KeysValidator`. Querying HStoreField ^^^^^^^^^^^^^^^^^^^^ In addition to the ability to query by key, there are a number of custom lookups available for ``HStoreField``. We will use the following example model:: from django.contrib.postgres.fields import HStoreField from django.db import models class Dog(models.Model): name = models.CharField(max_length=200) data = HStoreField() def __str__(self): # __unicode__ on Python 2 return self.name .. fieldlookup:: hstorefield.key Key lookups ~~~~~~~~~~~ To query based on a given key, you simply use that key as the lookup name:: >>> Dog.objects.create(name='Rufus', data={'breed': 'labrador'}) >>> Dog.objects.create(name='Meg', data={'breed': 'collie'}) >>> Dog.objects.filter(data__breed='collie') [] You can chain other lookups after key lookups:: >>> Dog.objects.filter(data__breed__contains='l') [, ] If the key you wish to query by clashes with the name of another lookup, you need to use the :lookup:`hstorefield.contains` lookup instead. .. warning:: Since any string could be a key in a hstore value, any lookup other than those listed below will be interpreted as a key lookup. No errors are raised. Be extra careful for typing mistakes, and always check your queries work as you intend. .. fieldlookup:: hstorefield.contains contains ~~~~~~~~ The :lookup:`contains` lookup is overridden on :class:`~django.contrib.postgres.fields.HStoreField`. The returned objects are those where the given ``dict`` of key-value pairs are all contained in the field. It uses the SQL operator ``@>``. For example:: >>> Dog.objects.create(name='Rufus', data={'breed': 'labrador', 'owner': 'Bob'}) >>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': 'Bob'}) >>> Dog.objects.create(name='Fred', data={}) >>> Dog.objects.filter(data__contains={'owner': 'Bob'}) [, ] >>> Dog.objects.filter(data__contains={'breed': 'collie'}) [] .. fieldlookup:: hstorefield.contained_by contained_by ~~~~~~~~~~~~ This is the inverse of the :lookup:`contains ` lookup - the objects returned will be those where the key-value pairs on the object are a subset of those in the value passed. It uses the SQL operator ``<@``. For example:: >>> Dog.objects.create(name='Rufus', data={'breed': 'labrador', 'owner': 'Bob'}) >>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': 'Bob'}) >>> Dog.objects.create(name='Fred', data={}) >>> Dog.objects.filter(data__contained_by={'breed': 'collie', 'owner': 'Bob'}) [, ] >>> Dog.objects.filter(data__contained_by={'breed': 'collie'}) [] .. fieldlookup:: hstorefield.has_key has_key ~~~~~~~ Returns objects where the given key is in the data. Uses the SQL operator ``?``. For example:: >>> Dog.objects.create(name='Rufus', data={'breed': 'labrador'}) >>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': 'Bob'}) >>> Dog.objects.filter(data__has_key='owner') [] .. fieldlookup:: hstorefield.has_any_keys has_any_keys ~~~~~~~~~~~~ .. versionadded:: 1.9 Returns objects where any of the given keys are in the data. Uses the SQL operator ``?|``. For example:: >>> Dog.objects.create(name='Rufus', data={'breed': 'labrador'}) >>> Dog.objects.create(name='Meg', data={'owner': 'Bob'}) >>> Dog.objects.create(name='Fred', data={}) >>> Dog.objects.filter(data__has_any_keys=['owner', 'breed']) [, ] .. fieldlookup:: hstorefield.has_keys has_keys ~~~~~~~~ Returns objects where all of the given keys are in the data. Uses the SQL operator ``?&``. For example:: >>> Dog.objects.create(name='Rufus', data={}) >>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': 'Bob'}) >>> Dog.objects.filter(data__has_keys=['breed', 'owner']) [] .. fieldlookup:: hstorefield.keys keys ~~~~ Returns objects where the array of keys is the given value. Note that the order is not guaranteed to be reliable, so this transform is mainly useful for using in conjunction with lookups on :class:`~django.contrib.postgres.fields.ArrayField`. Uses the SQL function ``akeys()``. For example:: >>> Dog.objects.create(name='Rufus', data={'toy': 'bone'}) >>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': 'Bob'}) >>> Dog.objects.filter(data__keys__overlap=['breed', 'toy']) [, ] .. fieldlookup:: hstorefield.values values ~~~~~~ Returns objects where the array of values is the given value. Note that the order is not guaranteed to be reliable, so this transform is mainly useful for using in conjunction with lookups on :class:`~django.contrib.postgres.fields.ArrayField`. Uses the SQL function ``avalues()``. For example:: >>> Dog.objects.create(name='Rufus', data={'breed': 'labrador'}) >>> Dog.objects.create(name='Meg', data={'breed': 'collie', 'owner': 'Bob'}) >>> Dog.objects.filter(data__values__contains=['collie']) [] JSONField --------- .. versionadded:: 1.9 .. class:: JSONField(**options) A field for storing JSON encoded data. In Python the data is represented in its Python native format: dictionaries, lists, strings, numbers, booleans and ``None``. If you give the field a :attr:`~django.db.models.Field.default`, ensure it's a callable such as ``dict`` (for an empty default) or a callable that returns a dict (such as a function). Incorrectly using ``default={}`` creates a mutable default that is shared between all instances of ``JSONField``. .. note:: PostgreSQL has two native JSON based data types: ``json`` and ``jsonb``. The main difference between them is how they are stored and how they can be queried. PostgreSQL's ``json`` field is stored as the original string representation of the JSON and must be decoded on the fly when queried based on keys. The ``jsonb`` field is stored based on the actual structure of the JSON which allows indexing. The trade-off is a small additional cost on writing to the ``jsonb`` field. ``JSONField`` uses ``jsonb``. **As a result, the usage of this field is only supported on PostgreSQL versions at least 9.4**. Querying JSONField ^^^^^^^^^^^^^^^^^^ We will use the following example model:: from django.contrib.postgres.fields import JSONField from django.db import models class Dog(models.Model): name = models.CharField(max_length=200) data = JSONField() def __str__(self): # __unicode__ on Python 2 return self.name .. fieldlookup:: jsonfield.key Key, index, and path lookups ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To query based on a given dictionary key, simply use that key as the lookup name:: >>> Dog.objects.create(name='Rufus', data={ ... 'breed': 'labrador', ... 'owner': { ... 'name': 'Bob', ... 'other_pets': [{ ... 'name': 'Fishy', ... }], ... }, ... }) >>> Dog.objects.create(name='Meg', data={'breed': 'collie'}) >>> Dog.objects.filter(data__breed='collie') [] Multiple keys can be chained together to form a path lookup:: >>> Dog.objects.filter(data__owner__name='Bob') [] If the key is an integer, it will be interpreted as an index lookup in an array:: >>> Dog.objects.filter(data__owner__other_pets__0__name='Fishy') [] If the key you wish to query by clashes with the name of another lookup, use the :lookup:`jsonfield.contains` lookup instead. If only one key or index is used, the SQL operator ``->`` is used. If multiple operators are used then the ``#>`` operator is used. .. warning:: Since any string could be a key in a JSON object, any lookup other than those listed below will be interpreted as a key lookup. No errors are raised. Be extra careful for typing mistakes, and always check your queries work as you intend. Containment and key operations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. fieldlookup:: jsonfield.contains .. fieldlookup:: jsonfield.contained_by .. fieldlookup:: jsonfield.has_key .. fieldlookup:: jsonfield.has_any_keys .. fieldlookup:: jsonfield.has_keys :class:`~django.contrib.postgres.fields.JSONField` shares lookups relating to containment and keys with :class:`~django.contrib.postgres.fields.HStoreField`. - :lookup:`contains ` (accepts any JSON rather than just a dictionary of strings) - :lookup:`contained_by ` (accepts any JSON rather than just a dictionary of strings) - :lookup:`has_key ` - :lookup:`has_any_keys ` - :lookup:`has_keys ` .. _range-fields: Range Fields ------------ There are five range field types, corresponding to the built-in range types in PostgreSQL. These fields are used to store a range of values; for example the start and end timestamps of an event, or the range of ages an activity is suitable for. All of the range fields translate to :ref:`psycopg2 Range objects ` in python, but also accept tuples as input if no bounds information is necessary. The default is lower bound included, upper bound excluded. IntegerRangeField ^^^^^^^^^^^^^^^^^ .. class:: IntegerRangeField(**options) Stores a range of integers. Based on an :class:`~django.db.models.IntegerField`. Represented by an ``int4range`` in the database and a :class:`~psycopg2:psycopg2.extras.NumericRange` in Python. BigIntegerRangeField ^^^^^^^^^^^^^^^^^^^^ .. class:: BigIntegerRangeField(**options) Stores a range of large integers. Based on a :class:`~django.db.models.BigIntegerField`. Represented by an ``int8range`` in the database and a :class:`~psycopg2:psycopg2.extras.NumericRange` in Python. FloatRangeField ^^^^^^^^^^^^^^^ .. class:: FloatRangeField(**options) Stores a range of floating point values. Based on a :class:`~django.db.models.FloatField`. Represented by a ``numrange`` in the database and a :class:`~psycopg2:psycopg2.extras.NumericRange` in Python. DateTimeRangeField ^^^^^^^^^^^^^^^^^^ .. class:: DateTimeRangeField(**options) Stores a range of timestamps. Based on a :class:`~django.db.models.DateTimeField`. Represented by a ``tztsrange`` in the database and a :class:`~psycopg2:psycopg2.extras.DateTimeTZRange` in Python. DateRangeField ^^^^^^^^^^^^^^ .. class:: DateRangeField(**options) Stores a range of dates. Based on a :class:`~django.db.models.DateField`. Represented by a ``daterange`` in the database and a :class:`~psycopg2:psycopg2.extras.DateRange` in Python. Querying Range Fields ^^^^^^^^^^^^^^^^^^^^^ There are a number of custom lookups and transforms for range fields. They are available on all the above fields, but we will use the following example model:: from django.contrib.postgres.fields import IntegerRangeField from django.db import models class Event(models.Model): name = models.CharField(max_length=200) ages = IntegerRangeField() start = models.DateTimeField() def __str__(self): # __unicode__ on Python 2 return self.name We will also use the following example objects:: >>> import datetime >>> from django.utils import timezone >>> now = timezone.now() >>> Event.objects.create(name='Soft play', ages=(0, 10), start=now) >>> Event.objects.create(name='Pub trip', ages=(21, None), start=now - datetime.timedelta(days=1)) and ``NumericRange``: >>> from psycopg2.extras import NumericRange Containment functions ~~~~~~~~~~~~~~~~~~~~~ As with other PostgreSQL fields, there are three standard containment operators: ``contains``, ``contained_by`` and ``overlap``, using the SQL operators ``@>``, ``<@``, and ``&&`` respectively. .. fieldlookup:: rangefield.contains contains '''''''' >>> Event.objects.filter(ages__contains=NumericRange(4, 5)) [] .. fieldlookup:: rangefield.contained_by contained_by '''''''''''' >>> Event.objects.filter(ages__contained_by=NumericRange(0, 15)) [] .. versionadded 1.9 The `contained_by` lookup is also available on the non-range field types: :class:`~django.db.models.fields.IntegerField`, :class:`~django.db.models.fields.BigIntegerField`, :class:`~django.db.models.fields.FloatField`, :class:`~django.db.models.fields.DateField`, and :class:`~django.db.models.fields.DateTimeField`. For example:: >>> from psycopg2.extras import DateTimeTZRange >>> Event.objects.filter(start__contained_by=DateTimeTZRange( ... timezone.now() - datetime.timedelta(hours=1), ... timezone.now() + datetime.timedelta(hours=1), ... ) [] .. fieldlookup:: rangefield.overlap overlap ''''''' >>> Event.objects.filter(ages__overlap=NumericRange(8, 12)) [] Comparison functions ~~~~~~~~~~~~~~~~~~~~ Range fields support the standard lookups: :lookup:`lt`, :lookup:`gt`, :lookup:`lte` and :lookup:`gte`. These are not particularly helpful - they compare the lower bounds first and then the upper bounds only if necessary. This is also the strategy used to order by a range field. It is better to use the specific range comparison operators. .. fieldlookup:: rangefield.fully_lt fully_lt '''''''' The returned ranges are strictly less than the passed range. In other words, all the points in the returned range are less than all those in the passed range. >>> Event.objects.filter(ages__fully_lt=NumericRange(11, 15)) [] .. fieldlookup:: rangefield.fully_gt fully_gt '''''''' The returned ranges are strictly greater than the passed range. In other words, the all the points in the returned range are greater than all those in the passed range. >>> Event.objects.filter(ages__fully_gt=NumericRange(11, 15)) [] .. fieldlookup:: rangefield.not_lt not_lt '''''' The returned ranges do not contain any points less than the passed range, that is the lower bound of the returned range is at least the lower bound of the passed range. >>> Event.objects.filter(ages__not_lt=NumericRange(0, 15)) [, ] .. fieldlookup:: rangefield.not_gt not_gt '''''' The returned ranges do not contain any points greater than the passed range, that is the upper bound of the returned range is at most the upper bound of the passed range. >>> Event.objects.filter(ages__not_gt=NumericRange(3, 10)) [] .. fieldlookup:: rangefield.adjacent_to adjacent_to ''''''''''' The returned ranges share a bound with the passed range. >>> Event.objects.filter(ages__adjacent_to=NumericRange(10, 21)) [, ] Querying using the bounds ~~~~~~~~~~~~~~~~~~~~~~~~~ There are three transforms available for use in queries. You can extract the lower or upper bound, or query based on emptiness. .. fieldlookup:: rangefield.startswith startswith '''''''''' Returned objects have the given lower bound. Can be chained to valid lookups for the base field. >>> Event.objects.filter(ages__startswith=21) [] .. fieldlookup:: rangefield.endswith endswith '''''''' Returned objects have the given upper bound. Can be chained to valid lookups for the base field. >>> Event.objects.filter(ages__endswith=10) [] .. fieldlookup:: rangefield.isempty isempty ''''''' Returned objects are empty ranges. Can be chained to valid lookups for a :class:`~django.db.models.BooleanField`. >>> Event.objects.filter(ages__isempty=True) [] Defining your own range types ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ PostgreSQL allows the definition of custom range types. Django's model and form field implementations use base classes below, and psycopg2 provides a :func:`~psycopg2:psycopg2.extras.register_range` to allow use of custom range types. .. class:: RangeField(**options) Base class for model range fields. .. attribute:: base_field The model field to use. .. attribute:: range_type The psycopg2 range type to use. .. attribute:: form_field The form field class to use. Should be a subclass of :class:`django.contrib.postgres.forms.BaseRangeField`. .. class:: django.contrib.postgres.forms.BaseRangeField Base class for form range fields. .. attribute:: base_field The form field to use. .. attribute:: range_type The psycopg2 range type to use.