Problem

Here is a branch with failing tests that I think should work (and that I think worked in earlier versions?): https://github.com/jazzband/django-model-utils/compare/master...jcushman:abstract-test-failure

This seems to be related to the stuff that @lucaswiman added in #317. @lucaswiman, I'm hoping based on that work you might have some clever idea for how to fix this. :)

Here's the situation: you have an abstract base class that defines a static attribute like is_active = True, and a concrete model inheriting from that class that defines a field like is_active = models.BooleanField(default=True). The model then throws an error on save():

from django.contrib.auth.models import AbstractUser

class MyUser(AbstractUser):
    tracker = FieldTracker()

MyUser().save()

Result:

    def __get__(self, instance, owner):
        if instance is None:
            return self
        was_deferred = self.field_name in instance.get_deferred_fields()
>       value = self.descriptor.__get__(instance, owner)
E       AttributeError: 'bool' object has no attribute '__get__'

model_utils/tracker.py:43: AttributeError

It would be great for this to work, because tracking changes on the Django user model is handy.

Debugging that error, I found the problem boils down to this:

class AbstractUser(models.Model):
    is_active = True

    class Meta:
        abstract = True

class MyUser(AbstractUser):
    is_active = models.BooleanField(default=True)
    tracker = FieldTracker()

MyUser().save()

The reason that fails is https://github.com/jazzband/django-model-utils/blob/master/model_utils/tracker.py#L218 :

descriptor = getattr(sender, field_name)
...
setattr(sender, field_name, wrapped_descriptor)

... which boils down to setting MyUser.is_active = DescriptorWrapper(MyUser.is_active). And that doesn't work because you expect MyUser.is_active to start with a value of DeferredAttribute('is_active'), but it actually returns True. For reasons I don't understand, when you override a static attribute on an abstract class, you get the base class's value back instead of the subclass's.

I tried tweaking tracker.py with variations on descriptor = getattr(sender, field_name) if field_name in sender.__dict__ else DeferredAttribute(field_name), but that broke foreign key fields and feels pretty janky anyway.

Any ideas on how to get this working?

Thanks!

Environment

• Django Model Utils version: master
• Django version: 2.1
• Python version: 3.6
• Other libraries used, if any:

Please note that I have made a choice in the feature addition that people might like to have full downcasting even if the associated select_related isn't available (because of django <1.6 bugs). If you don't like it, feel free to take it out. However, the second part is indeed a bug, if you have multiple grandchildren from a single child, the instance is not the first grandchild checked, but it is a sibling to the grandchild checked, then the child class will be taken instead of continuing on to the other grandchildren.

• Down-cast to final level even if we can't select the related data (in < 1.6)
• Fix bug where child classes are accepted if grand-child doesn't match even if there are more grand-child classes to check.

The second commit is a feature addition to allow casting of related fields (like foreign keys) in what I believe is a query efficient method. I have not yet written unit tests for it, but it does work in the testing I have done (in system testing).

I'm planning to implement tests and code for a model mixin that keeps track of the original values of the model. I often find this useful for performing tasks in save() methods or pre_save signals (like clearing cache on changes).

A very simple implementation. Another implementation that supports m2m fields

I don't think the implementation of this should create any extra DB queries... at least not by default. Feedback on implementation suggestion is welcome.

Does this sound like a good addition to django-model-utils?

There's an unfortunate side effect of referencing Django itself in setup.py or requirements.txt

It rather unexpectedly results in an upgrade to Django itself if I type: "pip install django-model-utils -U"

And really - your app doesn't 'require' django as much as it's an add-on for Django so not much is lost by removing it. Is anyone likely to install django-model-utils and expect it to install Django for them?

There's a discussion on the same issue here: https://github.com/charettes/django-admin-enhancer/issues/23

• Drops support for anything below Django 1.8 (current LTS release)
• ~~Cleans up some pep8 E303 warnings. I can remove this commit - just went ahead because my editor lit up like a christmas tree.~~
• ~~Resolves 183~~.
• Supersedes / closes #162.
• Supersedes / closes #204.

Problem

Six won't be used in Django 3.0 so it will cause ImportError

Solution

Drop six and add django 3.0 to the test matrix to ensure it is supported

Commandments

• [x] Write PEP8 compliant code.
• [x] Cover it with tests.
• [x] Update CHANGES.rst file to describe the changes, and quote according issue with GH-<issue_number>.
• [x] Pay attention to backward compatibility, or if it breaks it, explain why.
• [x] Update documentation (if relevant).

I was going to write a test for this but the tests currently fail up to 4 assertions when I run them without making a change. Anyway, I'm new at this whole pull thing so be gentle.

The issue is if i define a class with a related class:

class Opportunity(models.Model):
    objects=InheritanceManager()
    pass

class Contest(Opportunity):
    pass

class Terms(models.Model):
    opportunity = models.OneToOneField(Opportunity, related_name='contest_terms')

Calling:

contests = Opportunity.objects.select_related('contest_terms').filter(id=1)
t = contests[0].contest_terms
assert(len(connection.queries)==1)

works because it makes one DB hit and can access the contest_terms as it's been cached by select_related() changing the query in the first line to include select_subclasses:

contests = Opportunity.objects.select_related('contest_terms').select_subclasses('contest').filter(id=1)

fails because the related object cache is lost after subclassing so it makes the additional hit to the DB to get the terms.

The fix I proposed will copy the object caches from the base class when _get_sub_obj_recurse finds the appropriate subclass. This will keep the results of select_related() in the returned model instance.

This improvement will be useful for those who uses the field to store small or medium json-data sets. It is unlikely that the field will store complex data with a high level of nesting (for such problems it's better to use NoSQL), so the overhead of using deepcopy will be negligible, especially given the fact that the protective function (prevent_side_effects) will be called only during initialization. For models without json-fields only one extra loop will be made through the fields during initialization.

Hi, I had added ppc64le(Linux on Power) architecture support on travis-ci and looks like its been successfully added. I believe it is ready for the final review and merge. Travis-CI build job: https://travis-ci.com/github/kishorkunal-raj/django-model-utils/builds/188852916

Please have a look.

Thanks!! Kishor Kunal Raj

Fixes #278. cc @utapyngo @djstein

Problem

Prior to Django 1.10, deferred attributes were handled by constructing a new model class with custom descriptors for the deferred attributes. After 1.10, deferred fields are tracked by whether the attribute is present in the instance __dict__.

FieldTracker handled tracking on these fields by overwriting the descriptors on the model class. This means that in 1.10+, the instance model class is the model, so FieldTracker can introduce changes to the base class whenever deferred fields are used that persist on other queries. #278 This is reproduced in a test case.

Solution

I updated the finalize_class method to wrap field descriptors. This preserves the custom descriptor behavior, and means that deferring fields does not lead to permanent changes in base model classes. This is only done for django versions after 1.10: the previous behavior is preserved for 1.8 and 1.9.

Since the relevant code branches behavior on unsupported versions of Django, I also removed all branching behavior for versions of django listed as unsupported in the CHANGELOG.

Commandments

• [ ] Write PEP8 compliant code.
• [x] Cover it with tests. Reproduced in test cases, which fail prior to 54cc1507a79460497d40f6cba779f67a4b5f8041 and pass afterwards.
• [x] Update CHANGES.rst file to describe the changes, and quote according issue with GH-<issue_number>.
• [x] Pay attention to backward compatibility, or if it breaks it, explain why. The only backwards incompatible change is that the tracker._deferred_fields attribute is not set on django versions >=1.10. Since _deferred_fields is a private attribute, I wouldn't consider this a breaking change.
• [x] Update documentation (if relevant). _I don't think it is relevant.

Regarding PEP8, I ran flake8, and it generated a number of errors. I think my code didn't introduce new PEP8 violations, but I'm not sure. If you want, I can submit a separate PR fixing all the PEP8 violations, and rebase onto this branch.

This would be a first stab at the changes I was describing in #109, to allow for manually setting the modified attributes of a TimeStampedModel (or, really, the fields backing that model)

• Are there any other Django API methods that need demonstrating?
• Changelog/docs would need updating.

Problem

Model utils does not include type hints and py.typed. This prevents tools such as mypy to work properly with it.

Solution

Add model_utils/py.typed and a few typing hints.

Commandments

• [x] Write PEP8 compliant code.
• [_] Cover it with tests.
• [_] Update CHANGES.rst file to describe the changes, and quote according issue with GH-<issue_number>.
• [x] Pay attention to backward compatibility, or if it breaks it, explain why.
• [_] Update documentation (if relevant).

I can't get queryset for a particular subclass only. I want to get only products relating that subclass only

• Django Model Utils version: 4.3.1
• Django version: 4.1
• Python version: 3.9

class Product(models.Model)
...
objects  = InheritanceManager()


class Phone(Product)
...

class Laptop(Product)
...

How do I get querysets for the subclass Phone without any other subclasses instances I have gone through the documentation and didn't find any

I have built a custom filter that can do but it means I don't need the package and my concern is scalability of the custom implementation. I want to know if the package can help so I don't have to reinvent the wheel.

Problem

Python 3.11 was released on 2022-10-24 🚀

We should test it and declare support via the Trove classifier.

Solution

Add to tox.ini for local and CI testing, add to GitHub Actions for CI, and add the Trove classifier to declare support.

Also bump GitHub Actions and add colour to CI logs for readability.

Commandments

• [x] Write PEP8 compliant code.
• [x] Cover it with tests.
• [x] Update CHANGES.rst file to describe the changes, and quote according issue with GH-<issue_number>.
• [x] Pay attention to backward compatibility, or if it breaks it, explain why.
• [x] Update documentation (if relevant).

Problem

Hey, since Django ORM is starting to support async more and more, I'm wondering if you guys thought about adding async support to InheritanceManager (and InheritanceQuerySet respectively)?

Environment

• Django Model Utils version: 4.2.0
• Django version: 4.1
• Python version: 3.10
• Other libraries used, if any: not-relevant

Code examples

So far I was thinking of primitive solution to this problem and I've ended up with something like this, but I have not tested it yet (hopefully will get to testing it in few days). Do you think it should work, or is there some bigger problem about integrating async support to InheritanceManager?

class ExtendedInheritanceQuerySet(InheritanceQuerySet):
    """InheritanceQuerySet from django-model-utils extended for async methods."""

    async def aselect_subclasses(self, *subclasses):
        return await sync_to_async(self.select_subclasses)(*subclasses)

    async def aget_subclass(self, *args, **kwargs):
        return await sync_to_async(self.get_subclass)(*args, **kwargs)


class ExtendedInheritanceManager(InheritanceManager):
    """InheritanceManager from django-model-utils extended for async methods."""

    # todo also possible to create manager dynamically using `from_queryset` or `as_manager`

    _queryset_class = ExtendedInheritanceQuerySet

    async def aselect_subclasses(self, *subclasses):
        return await self.get_queryset().aselect_subclasses(*subclasses)

    async def aget_subclass(self, *args, **kwargs):
        return await self.get_queryset().aget_subclass(*args, **kwargs)

Problem

I wanted to do single query SQL selects of foreign key relations that are using Django model inheritance and found the following two improvements helpful.

Let me know what you think.

Code examples

Improvement to InheritanceQuerySet that is able to select inherited fields of foreign key relationship. Note that this works with multi-level inheritance by letting field_name be a tuple of list of names.

` class InheritanceQuerySet(ModelUtilsInheritanceQuerySet):

def select_field_subclasses(self, field_name):
    # Get field name iterable
    if isinstance(field_name, str):
        field_names = (field_name,)
    else:
        field_names = field_name

    # Lookup model
    model = self.model
    for field_name in field_names:
        field = model._meta.get_field(field_name)
        model = field.related_model

    # Lookup subclasses
    subclasses = self._get_subclasses_recurse(model)

    # Construct query
    subclasses = ['%s__%s' % ('__'.join(field_names), subclass) for subclass in subclasses]

    return self.select_related(*subclasses)

`

Using proper select queries with the above method, subclasses of foreign key relations can be retrieved with the following Model Mixin without touching the database.

` class InheritanceMixin:

def get_subclass(self):
    # Lookup subclasses
    helper = InheritanceQuerySetMixin() # hack to re-use private method of mixin
    subclasses = helper._get_subclasses_recurse(self.__class__)
    
    # Look for existing subclass
    for subclass in subclasses:
        try:
            return getattr(self, subclass)
        except getattr(self.__class__, subclass).RelatedObjectDoesNotExist:
            pass

    return self

`