This monster of a patch is the result of Alex Gaynor's 2009 Google Summer of Code project.
Congratulations to Alex for a job well done.
Big thanks also go to:
* Justin Bronn for keeping GIS in line with the changes,
* Karen Tracey and Jani Tiainen for their help testing Oracle support
* Brett Hoerner, Jon Loyens, and Craig Kimmerer for their feedback.
* Malcolm Treddinick for his guidance during the GSoC submission process.
* Simon Willison for driving the original design process
* Cal Henderson for complaining about ponies he wanted.
... and everyone else too numerous to mention that helped to bring this feature into fruition.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11952 bcc190cf-cafb-0310-a4f2-bffc1f526a37
See `docs/topics/db/raw.txt` for details.
Thanks to seanoc for getting the ball rolling, and to Russ for wrapping things up.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11921 bcc190cf-cafb-0310-a4f2-bffc1f526a37
We only clear ordering when doing so cannot change the result returned by
the get() method i.e. when the query does not already define limits.
Thanks to mattdennewitz@gmail.com for the report, and jdunck for the patch
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11916 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Thanks to Travis Cline for the patch, which gives about 35% speed increase
for the testsuite (with SQLite in-memory DB).
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11883 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Without this change, use of 'in' on a QuerySet resulted in ._result_cache
being fully populated, which sometimes is unnecessary work.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@11803 bcc190cf-cafb-0310-a4f2-bffc1f526a37
* Moved dimension field update logic out of `ImageFileDescriptor.__set__` and into its own method on `ImageField`.
* New `ImageField.update_dimension_fields` method is attached to model instance's `post_init` signal so that:
* Dimension fields are set when defined before the ImageField.
* Dimension fields are set when the field is assigned in the model constructor (fixes#11196), but only if the dimension fields don't already have values, so we avoid updating the dimensions every time an object is loaded from the database (fixes#11084).
* Clear dimension fields when the ImageField is set to None, which also causes dimension fields to be cleared when `ImageFieldFile.delete()` is used.
* Added many more tests for ImageField that test edge cases we weren't testing before, and moved the ImageField tests out of `file_storage` and into their own module within `model_fields`.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10858 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This bug actually exposed a related handful of inconsistancies in the underlying file handling and wraping, so a few related changes are in here as well:
* Dimensions are also now calculated the moment the image is assigned to the field instead of upon save.
* The base `File` object now when possible delegates its closed attribute down to the os-level file it wrapps.
* In-memory files' `close()` now is a no-op. Without this certain APIs that should be able to handle in-memory files were failing.
* Accessing `FieldFile.closed` used to open the file. That's silly, and it doesn't any more.
* Some over-eager error handling was squishing some errors that would normally be raised. One unit test was incorrectly depending on this behavior, so the test was removed.
Thanks to Armin Ronacher for much of this work.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10737 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This enables querysets with an extra clause to be used in an __in filter; as a side effect, it also means that as_sql() now returns the correct result for any query with an extra clause.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10648 bcc190cf-cafb-0310-a4f2-bffc1f526a37
We now pass numbers used in data queries as actualy numbers (integers) to the
database backends, rather than string forms. This is easier for some of the
less flexible backeds.
Based on a patch from Leo Soto and Ramiro Morales.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10530 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Loading related models when some fields were deferred was resulting in
incorrect offsets being used into the results row, causing the wrong data to be
assigned to attributes.
Refs #10710. This fixes the first of two bugs reported there.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10383 bcc190cf-cafb-0310-a4f2-bffc1f526a37
The original implementation had a few silly bugs in it that meant that data was
not being used only on the instance of the class that it was appropriate for
(one of the traps when using class-level things). No more!
Thanks to Justin Bronn and Alex Gaynor for the patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10382 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Using select_related(...) across a nullable relation to a multi-table
model inheritance situation no longer excludes results. Thanks to AdamG
for a test demonstrating part of the problem.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10136 bcc190cf-cafb-0310-a4f2-bffc1f526a37
A custom manager on a related object that filtered away objects would prevent
those objects being deleted via the relation. This is now fixed.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10104 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Calling the super() version of __reduce__ in Model.__reduce__ led to infinite
loops in Python prior to 2.5. We don't do that any longer.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10099 bcc190cf-cafb-0310-a4f2-bffc1f526a37
In extreme cases, some fields are expensive to load from the database
(e.g. GIS fields requiring conversion, or large text fields). This
commit adds defer() and only() methods to querysets that allow the
caller to specify which fields should not be loaded unless they are
accessed.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10090 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Large portions of this are needed for #5420, so I implemented it fully.
Thanks to Ryan Kelly for an initial patch to get this started.
Refs #5420.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10083 bcc190cf-cafb-0310-a4f2-bffc1f526a37
We can't use the default manager in Model.save_base(), since we need to
retrieve existing objects which might be filtered out by that manager. We now
always use a plain Manager instance at that point (or something that can
replace it, such as a GeoManager), making all existing rows in the
database visible to the saving code.
The logic for detecting a "suitable replacement" plain base is the same as for
related fields: if the use_for_related_fields is set on the manager subclass,
we can use it. The general requirement here is that we want a base class that
returns the appropriate QuerySet subclass, but does not restrict the rows
returned.
Fixed#8990, #9527.
Refs #2698 (which is not fixed by this change, but it's the first part of a
larger change to fix that bug.)
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10056 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Ensure to read the documentation before blindly enabling this: requires some
code audits first, but might well be worth it for busy sites.
Thanks to nicferrier, iamseb and Richard Davies for help with this patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10029 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Adding a get_db_prep_save() call to the UpdateQuery code path meant it
was being called twice if you updated an existing model attribute. This
change removes that double call and also makes TimeField.to_python() a
little more robust for the benefit of the Oracle backend (just in case).
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10013 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This allows a model to be defined which is not subject to database table
creation and removal. Useful for models that sit over existing tables or
database views.
Thanks to Alexander Myodov, Wolfgang Kriesing and Ryan Kelly for the bulk of
this patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10008 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This removes a long-standing FIXME in the update() handling and allows for
greater flexibility in the values passed in. In particular, it brings updates
into line with saves for django.contrib.gis fields, so fixed#10411.
Thanks to Justin Bronn and Russell Keith-Magee for help with this patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@10003 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Previously, you could explicitly assign None to a non-null ForeignKey
(or other) field when creating the model (Child(parent=None), etc). We
now throw an exception when you do that, which matches the behaviour
when you assign None to the attribute after creation.
Thanks to ales.zoulek@gmail.com and ondrej.kohout@gmail.com for some
analysis of this problem.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9983 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This fixes a broad class of bugs involving filters that look for missing
related models and fields. Most of them don't seem to have been reported
(the added tests cover the root cause). The exception is that this has
also fixed#9868.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9979 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This was always not working reliably (model initialization and serialization
were two of the problems). Now, it's an explicit error. Also, documented.
Fixed#10252.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9974 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Manually specifying both a OneToOneField(parent_link=True) and separate a
primary key field was causing invalid SQL to be generated. Thanks to Ramiro
Morales for some analysis on this one.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9971 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This includes a fairly large refactor of the update() query path (and
the initial portions of constructing the SQL for any query). The
previous code appears to have been only working more or less by accident
and was very fragile.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9967 bcc190cf-cafb-0310-a4f2-bffc1f526a37
If an update only affected an ancestor model (not the child), we were
returning 0 for the number of rows updated. This could have been
misleading if the value is used to detect an update occuring. So we now
return the rowcount from the first non-trivial query that is executed
(if any). Still a slight compromise, but better than what we had.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9966 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Some results were inadvertently being excluded if we were ordering across a
nullable relation which itself ordering by a non-nullable relation.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9916 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This involves a slight change to the interaction of annotate() and values() clauses that specify a list of columns. See the docs for details.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9888 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Many thanks to:
* Nicolas Lara, who worked on this feature during the 2008 Google Summer of Code.
* Alex Gaynor for his help debugging and fixing a number of issues.
* Malcolm Tredinnick for his invaluable review notes.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9792 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Previous behaviour was pretty stupid. Let's never speak of it again. New
behaviour both works and is documented.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9759 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Many thanks to:
* Nicolas Lara, who worked on this feature during the 2008 Google Summer of Code.
* Alex Gaynor for his help debugging and fixing a number of issues.
* Justin Bronn for his help integrating with contrib.gis.
* Karen Tracey for her help with cross-platform testing.
* Ian Kelly for his help testing and fixing Oracle support.
* Malcolm Tredinnick for his invaluable review notes.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9742 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This extends previous functionality that allowed passing Query objects as the
rvals to filters. You can now pass QuerySets, which requires less poking at
opaque attributes. See the documentation of the "__in" lookup type for the
details.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9701 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Extricated the code that works directly with SQL columns (standard
"where" stuff) from the the code that takes SQL fragments and combines
it with lookup types and values. The latter portion is now more
generally reusable. Any existing code that was poking at Query.having
will now break in very visible ways (no subtle miscalculations, which is
a good thing).
This patch, en passant, removes the existing "having" test, since the
new implementation requires more setting up than previously. The
aggregates support (currently in a separate codebase) has tests for this
functionality that work as a replacement for the removed test.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9700 bcc190cf-cafb-0310-a4f2-bffc1f526a37
The last use of it was removed in r9641 (it's internal API) and it's been
broken since r7477, as there's no longer a maximum of one OneToOneField per
model, so anything relying on it contained subtle bugs.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9643 bcc190cf-cafb-0310-a4f2-bffc1f526a37
that happens with MySQL when a "GROUP BY" clause is included. This is a
backend-specific operation, so any other databases requiring similar
encouragement can have a function added to their own backend code.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9637 bcc190cf-cafb-0310-a4f2-bffc1f526a37
lookup in the serializable_value() method. This means that abstract
parents that are multi-table children of other models(no, really!!) now
work again.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9618 bcc190cf-cafb-0310-a4f2-bffc1f526a37
"value" for a field that is a relation to another model.
This part adds the utility method on Model that should help in general.Also
cleans up the slightly ugly mess from r8957.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9601 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Descriptors now return themselves when accessed via the class, as per standard
Python descriptors like property().
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9550 bcc190cf-cafb-0310-a4f2-bffc1f526a37
backend.
This allows Querysets to be cached for Oracle and should provide a model for
adding pickling support to other (external) database backends that need a
custom Query class.
Thanks to Justin Bronn for some assistance with this patch.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9272 bcc190cf-cafb-0310-a4f2-bffc1f526a37
again later (order_by('foo')). Or, at least, it can now. Thanks to Ilya
Novoselov for diagnosing the problem here.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9206 bcc190cf-cafb-0310-a4f2-bffc1f526a37
more than 20 objects.
This means that accidentally executing HugeStoryArchive.objects.all() at the
interactive prompt (or in the debug template) won't try to load all 4,233,010
stories into memory and print them out. That would previously cause resource
starvation and other "interesting" crashes.
If you really, really want the previous behaviour (e.g. in a doctest that
prints more than 20 items), display "list(qs)" instead of just "qs".
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9202 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Any extra(select=...) columns can be ignored in the SQL for dates, since we are
only interested in extracting distinct date values. We were previously
including them by accident and it was generating incorrect SQL.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9091 bcc190cf-cafb-0310-a4f2-bffc1f526a37
"having" attributes, only the former was included in the resulting SQL, meaning
subclasses had to completely duplicate Query.as_sql() if they were using any
kind of grouping filtering on the results.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@9007 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This was triggered by r8794, but was, in fact, fairly fragile before then. The
current fix is the correct way we should be doing this.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8898 bcc190cf-cafb-0310-a4f2-bffc1f526a37
sometimes also sharing aliases, instead of creating their own. This was
generating incorrect SQL.
No representative test for this fix yet because I haven't had time to write one
that fits in nicely with the test suite. But it works for the monstrous example
in #8790 and a bunch of other complex examples I've created locally. Will write
a test later.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8853 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Also added documentation describing how manager inheritance works (and when
manager aren't inherited). Based on some patches from sebastian_noack and
emulbreh.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8851 bcc190cf-cafb-0310-a4f2-bffc1f526a37
efficient than possible SQL in some odd cases (found via code inspection, not
any particular failing example).
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8831 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Yes, this really is a commit that fixes an oversight in a commit that fixed an
oversight. One day I'll get it right.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8829 bcc190cf-cafb-0310-a4f2-bffc1f526a37
I can't reproduce the original error leading to #8115 and the patch. However, the only harm this change could cause is to raise more `CyclicDependency` exceptions than strictly necessary. That's better than infinite loops, at least, and it's easier to clean up in the future when we figure out the actual fix.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8807 bcc190cf-cafb-0310-a4f2-bffc1f526a37
fields (the latter was causing problems). This allows these values in URLs
(e.g. the admin filtering).
Not an ideal solution to the problem, but will do the job for the time being.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8691 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Backwards incompatible if you are using manually-specific primary key values
and relying on the previously documented behaviour that the new values would
always exist in the database (i.e. it would update the existing entry).
Fixed#8419.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8670 bcc190cf-cafb-0310-a4f2-bffc1f526a37
Also comes with approximately 67% less stupidity in the table joins for
filtering on generic relations.
Fixed#5937, hopefully for good, this time.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8644 bcc190cf-cafb-0310-a4f2-bffc1f526a37
* Removed `Manipulator`, `AutomaticManipulator`, and related classes.
* Removed oldforms specific bits from model fields:
* Removed `validator_list` and `core` arguments from constructors.
* Removed the methods:
* `get_manipulator_field_names`
* `get_manipulator_field_objs`
* `get_manipulator_fields`
* `get_manipulator_new_data`
* `prepare_field_objs_and_params`
* `get_follow`
* Renamed `flatten_data` method to `value_to_string` for better alignment with its use by the serialization framework, which was the only remaining code using `flatten_data`.
* Removed oldforms methods from `django.db.models.Options` class: `get_followed_related_objects`, `get_data_holders`, `get_follow`, and `has_field_type`.
* Removed oldforms-admin specific options from `django.db.models.fields.related` classes: `num_in_admin`, `min_num_in_admin`, `max_num_in_admin`, `num_extra_on_change`, and `edit_inline`.
* Serialization framework
* `Serializer.get_string_value` now calls the model fields' renamed `value_to_string` methods.
* Removed a special-casing of `models.DateTimeField` in `core.serializers.base.Serializer.get_string_value` that's handled by `django.db.models.fields.DateTimeField.value_to_string`.
* Removed `django.core.validators`:
* Moved `ValidationError` exception to `django.core.exceptions`.
* For the couple places that were using validators, brought over the necessary code to maintain the same functionality.
* Introduced a SlugField form field for validation and to compliment the SlugField model field (refs #8040).
* Removed an oldforms-style model creation hack (refs #2160).
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8616 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This isn't a complete solution to this class of problem, but it will do for
1.0, which only has generic relations as a multicolumn type. A more general
multicolumn solution will be available after that release.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8608 bcc190cf-cafb-0310-a4f2-bffc1f526a37
instances. We weren't skipping the correct output columns before processing
subsequent existing related instances. Thanks to mrmachine for the test case.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8598 bcc190cf-cafb-0310-a4f2-bffc1f526a37
used in "month" and "day" filters on date/datetime fields. Without this commit,
SQLite behaved inconsistently after [8494].
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8526 bcc190cf-cafb-0310-a4f2-bffc1f526a37
fields no longer creates duplicate copies of the join table(s). Basically, this
means filters on the join table (for ManyToManyField(through=...)) and complex
filters in the normal (non-through) case don't produce incorrect or duplicate
results.
git-svn-id: http://code.djangoproject.com/svn/django/trunk@8472 bcc190cf-cafb-0310-a4f2-bffc1f526a37
James Bennett
Luke Plant
Russell Keith-Magee
Joseph Kocherhans
Justin Bronn
Karen Tracey
Adrian Holovaty
Jannis Leidel
Ian Kelly
Jacob Kaplan-Moss
Gary Wilson Jr
Malcolm Tredinnick
Brian Rosner
Matt Boersma