p15693087
/
django
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Fixed #22667 -- Replaced leader/follower terminology with primary/replica

This commit is contained in:
Flavio Curella 2014-05-20 14:54:56 -05:00 committed by Tim Graham
parent ad994a3c5b
commit beec05686c
7 changed files with 85 additions and 82 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
docs/ref/settings.txt
View File

@ -649,10 +649,11 @@ Default: ``None``
The alias of the database that this database should mirror during
testing.
This setting exists to allow for testing of leader/follower
This setting exists to allow for testing of primary/replica
(referred to as master/slave by some databases)
configurations of multiple databases. See the documentation on
:ref:`testing leader/follower configurations
<topics-testing-leaderfollower>` for details.
:ref:`testing primary/replica configurations
<topics-testing-primaryreplica>` for details.
.. setting:: TEST_NAME

18
docs/topics/cache.txt
View File

@ -222,29 +222,29 @@ won't appear in the models cache, but the model details can be used
for routing purposes.
For example, the following router would direct all cache read
operations to ``cache_follower``, and all write operations to
``cache_leader``. The cache table will only be synchronized onto
``cache_leader``::
operations to ``cache_replica``, and all write operations to
``cache_primary``. The cache table will only be synchronized onto
``cache_primary``::
class CacheRouter(object):
"""A router to control all database cache operations"""
def db_for_read(self, model, **hints):
"All cache read operations go to the follower"
"All cache read operations go to the replica"
if model._meta.app_label in ('django_cache',):
return 'cache_follower'
return 'cache_replica'
return None
def db_for_write(self, model, **hints):
"All cache write operations go to leader"
"All cache write operations go to primary"
if model._meta.app_label in ('django_cache',):
return 'cache_leader'
return 'cache_primary'
return None
def allow_migrate(self, db, model):
"Only install the cache model on leader"
"Only install the cache model on primary"
if model._meta.app_label in ('django_cache',):
return db == 'cache_leader'
return db == 'cache_primary'
return None
If you don't specify routing directions for the database cache model,

61
docs/topics/db/multi-db.txt
View File

@ -197,17 +197,17 @@ Using routers
Database routers are installed using the :setting:`DATABASE_ROUTERS`
setting. This setting defines a list of class names, each specifying a
router that should be used by the leader router
router that should be used by the master router
(``django.db.router``).
The leader router is used by Django's database operations to allocate
The master router is used by Django's database operations to allocate
database usage. Whenever a query needs to know which database to use,
it calls the leader router, providing a model and a hint (if
it calls the master router, providing a model and a hint (if
available). Django then tries each router in turn until a database
suggestion can be found. If no suggestion can be found, it tries the
current ``_state.db`` of the hint instance. If a hint instance wasn't
provided, or the instance doesn't currently have database state, the
leader router will allocate the ``default`` database.
master router will allocate the ``default`` database.
An example
----------
@ -225,16 +225,17 @@ An example
introduce referential integrity problems that Django can't
currently handle.
The leader/follower configuration described is also flawed -- it
The primary/replica (referred to as master/slave by some databases)
configuration described is also flawed -- it
doesn't provide any solution for handling replication lag (i.e.,
query inconsistencies introduced because of the time taken for a
write to propagate to the followers). It also doesn't consider the
write to propagate to the replicas). It also doesn't consider the
interaction of transactions with the database utilization strategy.
So - what does this mean in practice? Let's consider another sample
configuration. This one will have several databases: one for the
``auth`` application, and all other apps using a leader/follower setup
with two read followers. Here are the settings specifying these
``auth`` application, and all other apps using a primary/replica setup
with two read replicas. Here are the settings specifying these
databases::
DATABASES = {
@ -244,20 +245,20 @@ databases::
'USER': 'mysql_user',
'PASSWORD': 'swordfish',
},
'leader': {
'NAME': 'leader',
'primary': {
'NAME': 'primary',
'ENGINE': 'django.db.backends.mysql',
'USER': 'mysql_user',
'PASSWORD': 'spam',
},
'follower1': {
'NAME': 'follower1',
'replica1': {
'NAME': 'replica1',
'ENGINE': 'django.db.backends.mysql',
'USER': 'mysql_user',
'PASSWORD': 'eggs',
},
'follower2': {
'NAME': 'follower2',
'replica2': {
'NAME': 'replica2',
'ENGINE': 'django.db.backends.mysql',
'USER': 'mysql_user',
'PASSWORD': 'bacon',
@ -309,30 +310,30 @@ send queries for the ``auth`` app to ``auth_db``::
return None
And we also want a router that sends all other apps to the
leader/follower configuration, and randomly chooses a follower to read
primary/replica configuration, and randomly chooses a replica to read
from::
import random
class LeaderFollowerRouter(object):
class PrimaryReplicaRouter(object):
def db_for_read(self, model, **hints):
"""
Reads go to a randomly-chosen follower.
Reads go to a randomly-chosen replica.
"""
return random.choice(['follower1', 'follower2'])
return random.choice(['replica1', 'replica2'])
def db_for_write(self, model, **hints):
"""
Writes always go to leader.
Writes always go to primary.
"""
return 'leader'
return 'primary'
def allow_relation(self, obj1, obj2, **hints):
"""
Relations between objects are allowed if both objects are
in the leader/follower pool.
in the primary/replica pool.
"""
db_list = ('leader', 'follower1', 'follower2')
db_list = ('primary', 'replica1', 'replica2')
if obj1._state.db in db_list and obj2._state.db in db_list:
return True
return None
@ -347,17 +348,17 @@ Finally, in the settings file, we add the following (substituting
``path.to.`` with the actual python path to the module(s) where the
routers are defined)::
DATABASE_ROUTERS = ['path.to.AuthRouter', 'path.to.LeaderFollowerRouter']
DATABASE_ROUTERS = ['path.to.AuthRouter', 'path.to.PrimaryReplicaRouter']
The order in which routers are processed is significant. Routers will
be queried in the order the are listed in the
:setting:`DATABASE_ROUTERS` setting . In this example, the
``AuthRouter`` is processed before the ``LeaderFollowerRouter``, and as a
``AuthRouter`` is processed before the ``PrimaryReplicaRouter``, and as a
result, decisions concerning the models in ``auth`` are processed
before any other decision is made. If the :setting:`DATABASE_ROUTERS`
setting listed the two routers in the other order,
``LeaderFollowerRouter.allow_migrate()`` would be processed first. The
catch-all nature of the LeaderFollowerRouter implementation would mean
``PrimaryReplicaRouter.allow_migrate()`` would be processed first. The
catch-all nature of the PrimaryReplicaRouter implementation would mean
that all models would be available on all databases.
With this setup installed, lets run some Django code::
@ -369,7 +370,7 @@ With this setup installed, lets run some Django code::
>>> # This save will also be directed to 'auth_db'
>>> fred.save()
>>> # These retrieval will be randomly allocated to a follower database
>>> # These retrieval will be randomly allocated to a replica database
>>> dna = Person.objects.get(name='Douglas Adams')
>>> # A new object has no database allocation when created
@ -379,10 +380,10 @@ With this setup installed, lets run some Django code::
>>> # the same database as the author object
>>> mh.author = dna
>>> # This save will force the 'mh' instance onto the leader database...
>>> # This save will force the 'mh' instance onto the primary database...
>>> mh.save()
>>> # ... but if we re-retrieve the object, it will come back on a follower
>>> # ... but if we re-retrieve the object, it will come back on a replica
>>> mh = Book.objects.get(title='Mostly Harmless')
@ -690,7 +691,7 @@ In addition, some objects are automatically created just after
database).
For common setups with multiple databases, it isn't useful to have these
objects in more than one database. Common setups include leader / follower and
objects in more than one database. Common setups include primary/replica and
connecting to external databases. Therefore, it's recommended:
- either to run :djadmin:`migrate` only for the default database;

39
docs/topics/testing/advanced.txt
View File

@ -64,16 +64,17 @@ The following is a simple unit test using the request factory::
Tests and multiple databases
============================
.. _topics-testing-leaderfollower:
.. _topics-testing-primaryreplica:
Testing leader/follower configurations
Testing primary/replica configurations
-----------------------------------
If you're testing a multiple database configuration with leader/follower
replication, this strategy of creating test databases poses a problem.
If you're testing a multiple database configuration with primary/replica
(referred to as master/slave by some databases) replication, this strategy of
creating test databases poses a problem.
When the test databases are created, there won't be any replication,
and as a result, data created on the leader won't be seen on the
follower.
and as a result, data created on the primary won't be seen on the
replica.
To compensate for this, Django allows you to define that a database is
a *test mirror*. Consider the following (simplified) example database
@ -83,34 +84,34 @@ configuration::
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'myproject',
'HOST': 'dbleader',
'HOST': 'dbprimary',
# ... plus some other settings
},
'follower': {
'replica': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'myproject',
'HOST': 'dbfollower',
'HOST': 'dbreplica',
'TEST_MIRROR': 'default'
# ... plus some other settings
}
}
In this setup, we have two database servers: ``dbleader``, described
by the database alias ``default``, and ``dbfollower`` described by the
alias ``follower``. As you might expect, ``dbfollower`` has been configured
by the database administrator as a read follower of ``dbleader``, so in
normal activity, any write to ``default`` will appear on ``follower``.
In this setup, we have two database servers: ``dbprimary``, described
by the database alias ``default``, and ``dbreplica`` described by the
alias ``replica``. As you might expect, ``dbreplica`` has been configured
by the database administrator as a read replica of ``dbprimary``, so in
normal activity, any write to ``default`` will appear on ``replica``.
If Django created two independent test databases, this would break any
tests that expected replication to occur. However, the ``follower``
tests that expected replication to occur. However, the ``replica``
database has been configured as a test mirror (using the
:setting:`TEST_MIRROR` setting), indicating that under testing,
``follower`` should be treated as a mirror of ``default``.
``replica`` should be treated as a mirror of ``default``.
When the test environment is configured, a test version of ``follower``
will *not* be created. Instead the connection to ``follower``
When the test environment is configured, a test version of ``replica``
will *not* be created. Instead the connection to ``replica``
will be redirected to point at ``default``. As a result, writes to
``default`` will appear on ``follower`` -- but because they are actually
``default`` will appear on ``replica`` -- but because they are actually
the same database, not because there is data replication between the
two databases.

2
tests/migrations/test_operations.py
View File

@ -980,7 +980,7 @@ class MultiDBOperationTests(MigrationTestBase):
multi_db = True
def setUp(self):
# Make the 'other' database appear to be a follower of the 'default'
# Make the 'other' database appear to be a replica of the 'default'
self.old_routers = router.routers
router.routers = [MigrateNothingRouter()]

2
tests/multiple_database/routers.py
View File

@ -4,7 +4,7 @@ from django.db import DEFAULT_DB_ALIAS
class TestRouter(object):
# A test router. The behavior is vaguely leader/follower, but the
# A test router. The behavior is vaguely primary/replica, but the
# databases aren't assumed to propagate changes.
def db_for_read(self, model, instance=None, **hints):
if instance:

38
tests/multiple_database/tests.py
View File

@ -854,7 +854,7 @@ class QueryTestCase(TestCase):
self.assertEqual(book.editor._state.db, 'other')
def test_subquery(self):
"""Make sure as_sql works with subqueries and leader/follower."""
"""Make sure as_sql works with subqueries and primary/replica."""
sub = Person.objects.using('other').filter(name='fff')
qs = Book.objects.filter(editor__in=sub)
@ -919,7 +919,7 @@ class RouterTestCase(TestCase):
multi_db = True
def setUp(self):
# Make the 'other' database appear to be a follower of the 'default'
# Make the 'other' database appear to be a replica of the 'default'
self.old_routers = router.routers
router.routers = [TestRouter()]
@ -1071,7 +1071,7 @@ class RouterTestCase(TestCase):
try:
dive.editor = marty
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments of original objects haven't changed...
self.assertEqual(marty._state.db, 'default')
@ -1089,7 +1089,7 @@ class RouterTestCase(TestCase):
except Book.DoesNotExist:
self.fail('Source database should have a copy of saved object')
# This isn't a real leader-follower database, so restore the original from other
# This isn't a real primary/replica database, so restore the original from other
dive = Book.objects.using('other').get(title='Dive into Python')
self.assertEqual(dive._state.db, 'other')
@ -1097,7 +1097,7 @@ class RouterTestCase(TestCase):
try:
marty.edited = [pro, dive]
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Assignment implies a save, so database assignments of original objects have changed...
self.assertEqual(marty._state.db, 'default')
@ -1111,7 +1111,7 @@ class RouterTestCase(TestCase):
except Book.DoesNotExist:
self.fail('Source database should have a copy of saved object')
# This isn't a real leader-follower database, so restore the original from other
# This isn't a real primary/replica database, so restore the original from other
dive = Book.objects.using('other').get(title='Dive into Python')
self.assertEqual(dive._state.db, 'other')
@ -1119,7 +1119,7 @@ class RouterTestCase(TestCase):
try:
marty.edited.add(dive)
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Add implies a save, so database assignments of original objects have changed...
self.assertEqual(marty._state.db, 'default')
@ -1133,7 +1133,7 @@ class RouterTestCase(TestCase):
except Book.DoesNotExist:
self.fail('Source database should have a copy of saved object')
# This isn't a real leader-follower database, so restore the original from other
# This isn't a real primary/replica database, so restore the original from other
dive = Book.objects.using('other').get(title='Dive into Python')
# If you assign a FK object when the base object hasn't
@ -1196,7 +1196,7 @@ class RouterTestCase(TestCase):
mark = Person.objects.using('default').create(pk=2, name="Mark Pilgrim")
# Now save back onto the usual database.
# This simulates leader/follower - the objects exist on both database,
# This simulates primary/replica - the objects exist on both database,
# but the _state.db is as it is for all other tests.
pro.save(using='default')
marty.save(using='default')
@ -1213,7 +1213,7 @@ class RouterTestCase(TestCase):
try:
marty.book_set = [pro, dive]
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments don't change
self.assertEqual(marty._state.db, 'default')
@ -1232,7 +1232,7 @@ class RouterTestCase(TestCase):
try:
marty.book_set.add(dive)
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments don't change
self.assertEqual(marty._state.db, 'default')
@ -1251,7 +1251,7 @@ class RouterTestCase(TestCase):
try:
dive.authors = [mark, marty]
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments don't change
self.assertEqual(marty._state.db, 'default')
@ -1273,7 +1273,7 @@ class RouterTestCase(TestCase):
try:
dive.authors.add(marty)
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments don't change
self.assertEqual(marty._state.db, 'default')
@ -1311,7 +1311,7 @@ class RouterTestCase(TestCase):
try:
bob.userprofile = alice_profile
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments of original objects haven't changed...
self.assertEqual(alice._state.db, 'default')
@ -1342,7 +1342,7 @@ class RouterTestCase(TestCase):
try:
review1.content_object = dive
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments of original objects haven't changed...
self.assertEqual(pro._state.db, 'default')
@ -1361,7 +1361,7 @@ class RouterTestCase(TestCase):
except Book.DoesNotExist:
self.fail('Source database should have a copy of saved object')
# This isn't a real leader-follower database, so restore the original from other
# This isn't a real primary/replica database, so restore the original from other
dive = Book.objects.using('other').get(title='Dive into Python')
self.assertEqual(dive._state.db, 'other')
@ -1369,7 +1369,7 @@ class RouterTestCase(TestCase):
try:
dive.reviews.add(review1)
except ValueError:
self.fail("Assignment across leader/follower databases with a common source should be ok")
self.fail("Assignment across primary/replica databases with a common source should be ok")
# Database assignments of original objects haven't changed...
self.assertEqual(pro._state.db, 'default')
@ -1444,7 +1444,7 @@ class RouterTestCase(TestCase):
self.assertEqual(pro.reviews.db_manager('default').all().db, 'default')
def test_subquery(self):
"""Make sure as_sql works with subqueries and leader/follower."""
"""Make sure as_sql works with subqueries and primary/replica."""
# Create a book and author on the other database
mark = Person.objects.using('other').create(name="Mark Pilgrim")
@ -1482,7 +1482,7 @@ class AuthTestCase(TestCase):
multi_db = True
def setUp(self):
# Make the 'other' database appear to be a follower of the 'default'
# Make the 'other' database appear to be a replica of the 'default'
self.old_routers = router.routers
router.routers = [AuthRouter()]