Moderately heavy rewrite of docs/translation.txt

git-svn-id: http://code.djangoproject.com/svn/django/trunk@1087 bcc190cf-cafb-0310-a4f2-bffc1f526a37

docs/translation.txt

@@ -1,79 +1,127 @@
-======================
+====================
-How to do translations
+Internationalization
-======================
+====================
 
-Django has support for internationalization of program strings and template
+Django has full support for internationalization of text in code and templates.
-content. Translations use the ``gettext`` library to produce strings in several
+Here's an overview of how translation works in Django.
-languages. Here's an overview of how translation works with Django.
 
-The goal of this document is to explain how to use translations in projects,
+.. admonition:: Behind the scenes
-how to add translations to Django patches and how to update and create
-translation files.
 
-Using translations in Python
+    Django's translation machinery uses the standard ``gettext`` module that
-============================
+    comes with Python.
 
-The translation machinery in Django uses the standard ``gettext`` module that
+Overview
-comes with Python. Django uses in its own functions and classes, but it uses
+========
-standard ``gettext`` machinery under the hood.
 
-To translate strings in your code, use one of the ``gettext`` helper functions.
+The goal of internationalization is to allow a single Web application to offer
-There are essentially two ways to use them:
+its content and functionality in multiple languages.
 
-    * Use the ``_()`` function, which is available globally. This function
+You, the Django developer, can accomplish this goal by adding a minimal amount
-      translates any string value.
+of hooks to your Python code and templates. These hooks are called
-    * Use ``django.utils.translation`` and import ``gettext`` or
+**translation strings**. They tell Django: "This text should be translated into
-      ``gettext_noop`` from there. ``gettext`` is identical to ``_()``.
+the end user's language, if a translation for this text is available in that
+language."
 
-Note one important thing about translations: The system can only translate
+Django takes care of using these hooks to translate Web apps, on the fly,
-strings it knows about. That means you have to mark strings for translation.
+according to users' language preferences.
-This is done either by calling ``_()``, ``gettext()`` or ``gettext_noop()`` on
-string constants. You can translate variable values or computed values, but the
-system needs to know those strings beforehand.
 
-The usual method is to build your strings using string interpolation and using
+Essentially, Django does two things:
-the ``gettext`` functions to do the actual translation. Example::
 
-    def hello_world(request, name, site):
+    * It lets developers and template authors specify which parts of their apps
-        page = _('Hello %(name)s, welcome to %(site)s!') % {
+      should be translatable.
-            'name': name,
+    * It uses these hooks to translate Web apps for particular users according
-            'site': site,
+      to their language preferences.
-        }
-        return HttpResponse(page)
 
-This short snippet shows one important thing: You shouldn't use positional
+How to internationalize your app: in three steps
-string interpolation (e.g., ``%s`` or ``%d``). Use the named string
+------------------------------------------------
-interpolation (e.g., ``%(name)s``), instead. Do this because other languages
-might require reordering of text.
 
-The other two helper functions are similar::
+    1. Embed translation strings in your Python code and templates.
+    2. Get translations for those strings, in whichever languages you want to
+       support.
+    2. Activate the locale middleware in your Django settings.
+
+How to specify translation strings
+==================================
+
+Translation strings specify "This text should be translated." These strings can
+appear in your Python code and templates. It's your responsibility to mark
+translatable strings; the system can only translate strings it knows about.
+
+In Python code
+--------------
+
+Standard translation
+~~~~~~~~~~~~~~~~~~~~
+
+Specify a translation string by using the function ``_()``. (Yes, the name of
+the function is the "underscore" character.) This function is available
+globally in any Python module; you don't have to import it.
+
+In this example, the text ``"Welcome to my site."`` is marked as a translation
+string::
+
+    def my_view(request):
+        output = _("Welcome to my site.")
+        return HttpResponse(output)
+
+The function ``django.utils.translation.gettext()`` is identical to ``_()``.
+This example is identical to the previous one::
 
     from django.utils.translation import gettext
-    def hello_world(request, name, site):
+    def my_view(request):
-        page = gettext('Hello %(name)s, welcome to %(site)s!') % {
+        output = gettext("Welcome to my site.")
-            'name': name,
+        return HttpResponse(output)
-            'site': site,
-        }
-        return HttpResponse(page)
 
-The difference here is that ``gettext`` is explicitly imported.
+Translation works on computed values. This example is identical to the previous
+two::
 
-Two important helper functions are available: ``gettext`` and ``gettext_noop``.
+    def my_view(request):
+        words = ['Welcome', 'to', 'my', 'site.']
+        output = _(' '.join(words))
+        return HttpResponse(output)
 
-    * ``gettext`` is just like ``_()`` -- it translates its argument.
+Translation works on variables. Again, here's an identical example::
-    * ``gettext_noop`` is different. It marks a string for inclusion into the
-      message file but doesn't do translation. Instead, the string is later
-      translated from a variable. Use this if you have constant strings that
-      should be stored in the source language because they are exchanged over
-      systems or users -- such as strings in a database -- but should be
-      translated at the last possible point in time, such as when the string is
-      presented to the user.
 
-One function, ``django.utils.translation.gettext_lazy()``, isn't available in
+    def my_view(request):
-the standard ``gettext`` module. Use it for lazily translated strings, such as
+        sentence = 'Welcome to my site.'
-messages in Django models that are stored internally and translated on access
+        output = _(sentence)
--- but not translated on storage, as that would only take the default language
+        return HttpResponse(output)
-into account.
+
+The strings you pass to ``_()`` or ``gettext()`` can take placeholders,
+specified with Python's standard named-string interpolation syntax. Example::
+
+    def my_view(request, n):
+        output = _('%(name)s is my name.') % {'name': n}
+        return HttpResponse(output)
+
+This technique lets language-specific translations reorder the placeholder
+text. For example, an English translation may be ``"Adrian is my name."``,
+while a Spanish translation may be ``"Me llamo Adrian."`` -- with the
+placeholder (the name) placed after the translated text instead of before it.
+
+For this reason, you should use named-string interpolation (e.g., ``%(name)s``)
+instead of positional interpolation (e.g., ``%s`` or ``%d``). If you used
+positional interpolation, translations wouldn't be able to reorder placeholder
+text.
+
+Marking strings as no-op
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Use the function ``django.utils.translation.gettext_noop()`` to mark a string
+as a translate string without translating it. The string is later translated
+from a variable.
+
+Use this if you have constant strings that should be stored in the source
+language because they are exchanged over systems or users -- such as strings in
+a database -- but should be translated at the last possible point in time, such
+as when the string is presented to the user.
+
+Lazy translation
+~~~~~~~~~~~~~~~~
+
+Use the function ``django.utils.translation.gettext_lazy()`` to translate
+strings lazily -- when the value is accessed rather than when the
+``gettext_lazy()`` function is called.
 
 For example, to translate a model's ``help_text``, do the following::
 
@@ -107,45 +155,57 @@ class, though::
             verbose_name = _('my thing')
             verbose_name_plural = _('mythings')
 
-A standard problem with translations is pluralization of strings. Use
+Pluralization
-``ngettext`` to solve this problem. Example::
+~~~~~~~~~~~~~
 
+Use the function ``django.utils.translation.ngettext()`` to specify pluralized
+messages. Example::
+
+    from django.utils.translation import ngettext
     def hello_world(request, count):
-        from django.utils.translation import ngettext
         page = ngettext('there is %(count)d object', 'there are %(count)d objects', count) % {
             'count': count,
         }
         return HttpResponse(page)
 
-Using translations in templates
+``ngettext`` takes three arguments: the singular translation string, the plural
-===============================
+translation string and the number of objects (which is passed to the
+translation languages as the ``count`` variable).
+
+In template code
+----------------
 
 Using translations in Django templates uses two template tags and a slightly
-different syntax than standard gettext. The ``{% trans %}`` template tag
+different syntax than in Python code. To give your template access to these
-translates a constant string or a variable content::
+tags, put ``{% load i18n %}`` toward the top of your template.
 
-    <title>{% trans 'This is the title.' %}</title>
+The ``{% trans %}`` template tag translates a constant string or a variable
+content::
 
-If you only want to mark some value for translation, but translate it
+    <title>{% trans "This is the title." %}</title>
-later from a variable, use the ``noop`` option::
 
-    <input name="field" value="{% trans "value" noop %}"/>
+If you only want to mark a value for translation, but translate it later from a
+variable, use the ``noop`` option::
 
-It is not possible to use variables in this constant string. If you
+    <title>{% trans "value" noop %}</title>
-have variables you need to put in your translations, you have to use the
-``{% blocktrans %}`` tag::
 
-    {% blocktrans %}This will have {{ value }} inside{% endblocktrans %}
+It's not possible to use template variables in ``{% trans %}`` -- only constant
+strings, in single or double quotes, are allowed. If your translations require
+variables (placeholders), use ``{% blocktrans %}``. Example::
 
-If your expressions are more complex (like you need to have filters applied),
+    {% blocktrans %}This will have {{ value }} inside.{% endblocktrans %}
-you need to bind them to local variables for the translation block::
 
-    {% blocktrans with value|filter as variable %}
+To translate a template expression -- say, using template filters -- you need
-    This will have {{ value }} inside
+to bind the expression to a local variable for use within the translation
+block::
+
+    {% blocktrans with value|filter as myvar %}
+    This will have {{ myvar }} inside.
     {% endblocktrans %}
 
-The last variant is the pluralization form: you need to specify both the singular
+To pluralize, specify both the singular and plural forms with the
-and plural sentence with intersparsed variables like this::
+``{% plural %}`` tag, which appears within ``{% blocktrans %}`` and
+``{% endblocktrans %}``. Example::
 
     {% blocktrans count list|counted as counter %}
     There is only one {{ name }} object.
@@ -153,8 +213,8 @@ and plural sentence with intersparsed variables like this::
     There are {{ counter }} {{ name }} objects.
     {% endblocktrans %}
 
-Internally all block translations and inline translations are translated into
+Internally, all block and inline translations use the appropriate
-the actual gettext/ngettext call.
+``gettext`` / ``ngettext`` call.
 
 Each ``DjangoContext`` has access to two translation-specific variables:
 
@@ -169,57 +229,141 @@ two tags::
     {% get_current_language as LANGUAGE_CODE %}
     {% get_available_languages as LANGUAGES %}
 
-All tags live in the ``i18n`` tag library, so you need to specify
+These tags also require a ``{% load i18n %}``.
-``{% load i18n %}`` in the head of your template to make use of them.
 
-There are some places where you will encounter constant strings in your template code.
+Translation hooks are also available within any template block tag that accepts
-One is filter arguments, the other are normal string constants for tags. If you need to
+constant strings. In those cases, just use ``_()`` syntax to specify a
-translate those, you can use the ``_("....")`` syntax::
+translation string. Example::
 
     {% some_special_tag _("Page not found") value|yesno:_("yes,no") %}
 
-In this case both the filter and the tag will see the already translated string, so they
+In this case, both the tag and the filter will see the already-translated
-don't need to be aware of translations. And both strings will be pulled out of the templates
+string, so they don't need to be aware of translations.
-for translation and stored in the .po files.
 
-The ``setlang`` redirect view
+How to create language files
------------------------------
+============================
 
-Django comes with a view, ``django.views.i18n.set_language`` that sets a user's
+Once you've tagged your strings for later translation, you need to write (or
-language preference and redirects back to the previous page. For example, put
+obtain) the language translations themselves. Here's how that works.
-this HTML code in your template::
 
-    <form action="/i18n/setlang/" method="POST">
+Message files
-    <input name="next" type="hidden" value="/next/page/" />
+-------------
-    <select name="language">
-    {% for lang in LANGUAGES %}
-    <option value="{{ lang.0 }}">{{ lang.1 }}</option>
-    {% endfor %}
-    </select>
-    <input type="submit" value="Go" />
-    </form>
 
-When a user submits the form, his chosen language will be saved in a cookie,
+The first step is to create a **message file** for a new language. A message
-and he'll be redirected either to the URL specified in the ``next`` field, or,
+file is a plain-text file, representing a single language, that contains all
-if ``next`` is empty, to the URL in the ``Referer`` header. If the ``Referer``
+available translation strings and how they should be represented in the given
-is blank -- say, if a user's browser suppresses that header -- then the user
+language. Message files have a ``.po`` file extension.
-will be redirected to ``/`` (the site root) as a fallback.
 
-Activate the ``setlang`` redirect view by adding the following line to your
+Django comes with a tool, ``bin/make-messages.py``, that automates the creation
-URLconf::
+and upkeep of these files.
 
-    (r'^i18n/', include('django.conf.urls.i18n'),
+To create or update a message file, run this command::
 
-Note that this example makes the view available at ``/i18n/setlang/``.
+    bin/make-messages.py -l de
 
-How language preference is discovered
+...where ``de`` is the language code for the message file you want to create.
-=====================================
+(The language code, in this case, is in locale format. So, for example, it's
+``pt_BR`` for Brazilian and ``de_AT`` for Austrian German.)
 
-Django has a very flexible model of deciding which language should be used --
+The script should be run from one of three places::
-installation-wide, for a particular user, or both.
+
+    * The root ``django`` directory (not a Subversion checkout, but the one
+      that is linked-to via ``$PYTHONPATH`` or is located somewhere on that
+      path).
+    * The root directory of your Django project.
+    * The root directory of your Django app.
+
+The script runs over the entire Django source tree and pulls out all strings
+marked for translation. It creates (or updates) a message file in the directory
+``conf/locale``. In the ``de`` example, the file will be
+``conf/locale/de/LC_MESSAGES/django.po``.
+
+.. admonition:: No gettext?
+
+    If you don't have the ``gettext`` utilities installed, ``make-messages.py``
+    will create empty files. If that's the case, either install the ``gettext``
+    utilities or just copy the English message file
+    (``conf/locale/en/LC_MESSAGES/django.po``) and use it as a starting point;
+    it's just an empty translation file.
+
+The format of ``.po`` files is straightforward. Each ``.po`` file contains a
+small bit of metadata, such as the translation maintainer's contact
+information, but the bulk of the file is a list of **messages** -- simple
+mappings between translation strings and the actual translated text for the
+particular language.
+
+For example, if your Django app contained a translation string for the text
+``"Welcome to my site.", like so::
+
+    _("Welcome to my site.")
+
+...then ``make-messages.py`` will have created a ``.po`` file containing the
+following snippet -- a message::
+
+    #: path/to/python/module.py:23
+    msgid "Welcome to my site."
+    msgstr ""
+
+A quick explanation:
+
+    * ``msgid`` is the translation string, which appears in the source. Don't
+      change it.
+    * ``msgstr`` is where you put the language-specific translation. It starts
+      out empty, so it's your responsibility to change it. Make sure you keep
+      the quotes around your translation.
+    * As a convenience, each message includes the filename and line number
+      from which the translation string was gleaned.
+
+Long messages are a special case. There, the first string directly after the
+``msgstr`` (or ``msgid``) is an empty string. Then the content itself will be
+written over the next few lines as one string per line. Those strings are
+directlyconcatenated. Don't forget trailing spaces within the strings;
+otherwise, they'll be tacked together without whitespace!
+
+.. admonition:: Mind your charset
+
+    When creating a ``.po`` file with your favorite text editor, first edit
+    the charset line (search for ``"CHARSET"``) and set it to the charset
+    you'll be using to edit the content. Generally, utf-8 should work for most
+    languages, but ``gettext`` can handle any charset you throw at it.
+
+To reexamine all source code and templates for new translation strings and
+update all message files for **all** languages, run ``make-messages.py -a``.
+
+Compiling message files
+-----------------------
+
+After you create your message file -- and each time you make changes to it --
+you'll need to compile it into a more efficient form, for use by ``gettext``.
+Do this with the ``bin/compile-messages.py`` utility.
+
+This tool runs over all available ``.po`` files and creates ``.mo`` files,
+which are binary files optimized for use by ``gettext``. In the same directory
+from which you ran ``make-messages.py``, run ``compile-messages.py`` like
+this::
+
+   bin/compile-messages.py
+
+That's it. Your translations are ready for use.
+
+.. admonition:: A note to translators
+
+    If you've created a translation in a language Django doesn't yet support,
+    please let us know! We'll add it to the global list of available languages
+    in the global Django settings (``settings.LANGUAGES``).
+
+How Django discovers language preference
+========================================
+
+Once you've prepared your translations -- or, if you just want to use the
+translations that come with Django -- you'll just need to activate translation
+for your app.
+
+Behind the scenes, Django has a very flexible model of deciding which language
+should be used -- installation-wide, for a particular user, or both.
 
 To set an installation-wide language preference, set ``LANGUAGE_CODE`` in your
-settings file. Django uses this language as the default translation -- the
+`settings file`_. Django uses this language as the default translation -- the
 final attempt if no other translator finds a translation.
 
 If all you want to do is run Django with your native language, and a language
@@ -228,8 +372,7 @@ file is available for your language, all you need to do is set
 
 If you want to let each individual user specify which language he or she
 prefers, use ``LocaleMiddleware``. ``LocaleMiddleware`` enables language
-selection based on data from the request. It lets each user have his or her own
+selection based on data from the request. It customizes content for each user.
-setting.
 
 To use ``LocaleMiddleware``, add ``'django.middleware.locale.LocaleMiddleware'``
 to your ``MIDDLEWARE_CLASSES`` setting. Because middleware order matters, you
@@ -247,11 +390,13 @@ For example, your ``MIDDLEWARE_CLASSES`` might look like this::
        'django.middleware.common.CommonMiddleware',
     )
 
+(For more on middleware, see the `middleware documentation`_.)
+
 ``LocaleMiddleware`` tries to determine the user's language preference by
 following this algorithm:
 
     * First, it looks for a ``django_language`` key in the the current user's
-      session.
+      `session`_.
     * Failing that, it looks for a cookie called ``django_language``.
     * Failing that, it looks at the ``Accept-Language`` HTTP header. This
       header is sent by your browser and tells the server which language(s) you
@@ -283,92 +428,52 @@ Note that, with static (middleware-less) translation, the language is in
 ``settings.LANGUAGE_CODE``, while with dynamic (middleware) translation, it's
 in ``request.LANGUAGE_CODE``.
 
+.. _settings file: http://www.djangoproject.com/documentation/settings/
+.. _middleware documentation: http://www.djangoproject.com/documentation/middleware/
+.. _session: http://www.djangoproject.com/documentation/sessions/
 .. _request object: http://www.djangoproject.com/documentation/request_response/#httprequest-objects
 
-Creating language files
+The ``set_language`` redirect view
-=======================
+==================================
 
-So, you've tagged all of your strings for later translation. But you need to
+As a convenience, Django comes with a view, ``django.views.i18n.set_language``,
-write the translations themselves.
+that sets a user's language preference and redirects back to the previous page.
 
-They need to be in a format grokable by ``gettext``. You need to update them.
+Activate this view by adding the following line to your URLconf::
-You may need to create new ones for new languages. This section shows you how
-to do it.
 
-Creating message files
+    (r'^i18n/', include('django.conf.urls.i18n'),
-----------------------
 
-The first step is to create a message file for a new language. Django comes
+(Note that this example makes the view available at ``/i18n/setlang/``.)
-with a tool, ``make-messages.py``, that automates this.
 
-To run it on the Django source tree, navigate to the ``django`` directory
+The view expects to be called via the ``GET`` method, with a ``language``
-itself -- not a Subversion check out, but the one linked to via ``$PYTHONPATH``
+parameter set in the query string. If session support is enabled, the view
-or located somewhere on that path.
+saves the language choice in the user's session. Otherwise, it saves the
+language choice in a ``django_language`` cookie.
 
-Then run this command::
+After setting the language choice, Django redirects the user, following this
+algorithm:
 
-    bin/make-messages.py -l de
+    * Django looks for a ``next`` parameter in the query string.
+    * If that doesn't exist, or is empty, Django tries the URL in the
+      ``Referer`` header.
+    * If that's empty -- say, if a user's browser suppresses that header --
+      then the user will be redirected to ``/`` (the site root) as a fallback.
 
-...where ``de`` is the language code for the message file you want to create.
+Here's example HTML template code::
 
-This script runs over the entire Django source tree and pulls out all strings
+    <form action="/i18n/setlang/" method="get">
-marked for translation, creating or updating the language's message file.
+    <input name="next" type="hidden" value="/next/page/" />
-
+    <select name="language">
-When it's done, it will have created (or updated) a message file under the
+    {% for lang in LANGUAGES %}
-directory ``conf/locale``. In this example, the file will be
+    <option value="{{ lang.0 }}">{{ lang.1 }}</option>
-``conf/locale/de/LC_MESSAGES/django.po``.
+    {% endfor %}
-
+    </select>
-If you don't have the ``gettext`` utilities installed, ``make-messages.py``
+    <input type="submit" value="Go" />
-will create empty files. If that's the case, either install the ``gettext``
+    </form>
-utilities or just copy the English message file
-(``conf/locale/en/LC_MESSAGES/django.po``) and use it as a starting point; it's
-just an empty translation file.
-
-Once you've created the ``.po`` file, edit the file with your favorite text
-editor. First, edit the charset line (search for ``"CHARSET"``) and set it to
-the charset you'll be using to edit the content. Then, proceed to write your
-translations.
-
-The language code for storage is in locale format -- so it's ``pt_BR`` for
-Brazilian and ``de_AT`` for Austrian German.
-
-Every message in the message file is in the same format:
-
-    * One line is the msgid. This is the actual string in the source. Don't
-      change it.
-    * The other line is msgstr. This is the translation. It starts out empty.
-      You change it.
-
-Long messages are a special case. There, the first string directly after the
-msgstr (or msgid) is an empty string. Then the content itself will be written
-over the next few lines as one string per line. Those strings are directly
-concatenated. Don't forget trailing spaces within the strings; otherwise,
-they'll be tacked together without whitespace!
-
-Compiling message files
------------------------
-
-After you create your message file, you'll need to transform it into a more
-efficient form to be read by ``gettext``. Do this with the
-``compile-messages.py`` utility. This tool runs over all available ``.po``
-files and creates ``.mo`` files. Run it like this::
-
-   bin/compile-messages.py
-
-That's it. You made your first translation. Now, if you configure your browser
-to request your language, Django apps will use your language preference.
-
-Another thing: Please submit the name of your newly-created language in that
-native language, so we can add it to the global list of available languages
-that is mirrored in ``settings.LANGUAGES`` (and the ``LANGUAGES`` template
-variable).
 
 Using translations in your own projects
 =======================================
 
-Of course, your own projects should make use of translations. Django makes this
-simple, because it looks for message files in several locations.
-
 Django looks for translations by following this algorithm:
 
     * First, it looks for a ``locale`` directory in the application directory
@@ -379,15 +484,15 @@ Django looks for translations by following this algorithm:
     * Finally, it checks the base translation in ``django/conf/locale``.
 
 This way, you can write applications that include their own translations, and
-you can override base translations in your project path if you want to do that.
+you can override base translations in your project path. Or, you can just build
-Or, you can just build a big project out of several apps and put all
+a big project out of several apps and put all translations into one big project
-translations into one big project message file. The choice is yours.
+message file. The choice is yours.
 
 All message file repositories are structured the same way. They are:
 
     * ``$APPPATH/locale/<language>/LC_MESSAGES/django.(po|mo)``
     * ``$PROJECTPATH/locale/<language>/LC_MESSAGES/django.(po|mo)``
-    * all paths listed in ``LOCALE_PATHS`` in your settings file are
+    * All paths listed in ``LOCALE_PATHS`` in your settings file are
       searched in that order for ``<language>/LC_MESSAGES/django.(po|mo)``
     * ``$PYTHONPATH/django/conf/locale/<language>/LC_MESSAGES/django.(po|mo)``
 
@@ -406,9 +511,9 @@ Finally, you should give some thought to the structure of your translation
 files. If your applications need to be delivered to other users and will
 be used in other projects, you might want to use app-specific translations.
 But using app-specific translations and project translations could produce
-weird problems with ``make-messages``: ``make-messages`` will traverse all directories
+weird problems with ``make-messages``: ``make-messages`` will traverse all
-below the current path and so might put message IDs into the project
+directories below the current path and so might put message IDs into the
-message file that are already in application message files.
+project message file that are already in application message files.
 
 The easiest way out is to store applications that are not part of the project
 (and so carry their own translations) outside the project tree. That way,
@@ -424,7 +529,7 @@ does translation:
 
     * The string domain is always ``django``. The string domain is used to
       differentiate between different programs that store their data in a
-      common messagefile library (usually ``/usr/share/locale/``). In Django's
+      common message-file library (usually ``/usr/share/locale/``). In Django's
       case, there are Django-specific locale libraries, so the domain itself
       isn't used. We could store app message files with different names and put
       them, say, in the project library, but we decided against this. With