Thanks Florian for isolating the shortest way to reproduce this issue:
./runtests.py \
django.contrib.auth.tests.test_context_processors.AuthContextProcessorTests.test_perms_attrs \
django.contrib.auth.tests.test_auth_backends.ChangedBackendSettingsTest.test_changed_backend_settings \
django.contrib.auth.tests.test_auth_backends.CustomUserModelBackendAuthenticateTest.test_authenticate \
django.contrib.auth.tests.test_basic.BasicTestCase.test_createsuperuser_management_command