Ubuntu Manpage: po4a - update both the PO files and translated documents in one shot

NAME

       po4a - update both the PO files and translated documents in one shot

SYNOPSIS

       po4a [options] config_file

DESCRIPTION

po4a (PO for anything) eases the maintenance of documentation translation using the classical gettext
tools. The main feature of po4a is that it decouples the translation of content from its document
structure. Please refer to the page po4a(7) for a gentle introduction to this project.

Upon execution, po4a parses all documentation files specified in its configuration file. It updates the
PO files (containing the translation) to reflect any change to the documentation, and produce a
translated documentation by injecting the content's translation (found in the PO files) into the
structure of the original master document.

At first, the PO files only contain the strings to translate from the original documentation. This file
format allows the translators to manually provide a translation for each paragraph extracted by po4a. If
the documentation is modified after translation, po4a marks the corresponding translations as "fuzzy" in
the PO file to request a manual review by the translators. The translators can also provide so-called
"addendum", that are extra content stating for example who did the translation and how to report bugs.

The workflow of po4a is asynchronous, as suited to open-source projects. The documentation writers author
the master documents at their own pace. The translators review and update the translations in the PO
files. The maintainers rerun po4a on need, to reflect any change to the original documentation to the PO
files, and to produce updated documentation translations, by injecting the latest translation into the
latest document structure.

By default, a given translated document is produced when at least 80% of its content is translated. The
untranslated text is kept in the original language. The produced documentation thus mixes languages if
the translation is not complete. You can change the 80% threshold with the --keep option described below.
Note however that discarding translations as soon as they are not 100% may be discouraging for the
translators whose work will almost never be shown to the users, while showing "translations" that are too
incomplete may be troubling for the end users.

Storing the translated documentation files in the version control system is probably a bad idea, since
they are automatically generated. The precious files are the PO files, that contain the hard work of your
fellow translators. Also, some people find it easier to interact with the translators through an online
platform such as weblate, but this is naturally fully optional.

Quick start tutorial
Let's assume you maintain a program named foo which has a man page man/foo.1 written in English (the
bridge language in most open-source projects, but po4a can be used from or to any language). Some times
ago, someone provided a German translation named man/foo.de.1 and disappeared. This is a problem because
you just got a bug report saying that your documentation contains a gravely misleading information that
must be fixed in all languages, but you don't speak German so you can only modify the original, not the
translation. Now, another contributor wants to contribute a translation to Japanese, a language that you
don't master either.

It is time to convert your documentation to po4a to solve your documentation maintenance nightmares. You
want to update the doc when needed, you want to ease the work of your fellow translators, and you want to
ensure that your users never see any outdated and thus misleading documentation.

The conversion includes two steps: setup the po4a infrastructure, and convert the previous German
translation to salvage the previous work. This latter part is done using po4a-gettextize, as follows. As
detailed in the documentation of po4a-gettextize(1), this process rarely fully automatic, but once it's
done, the de.po file containing the German translation can be integrated in your po4a workflow.

po4a-gettextize --format man --master foo.1 --localized foo.de.1 --po de.po

Let's now configure po4a. With the appropriate file layout, your configuration file could be as simple as
this:

[po_directory] man/po4a/

[type: man] man/foo.1 $lang:man/translated/foo.$lang.1

It specifies that all PO files (containing the work of the translators) are in the man/po4a/ directory,
and that you have only one master file, man/foo.1. If you had several master files, you would have
several lines similar to the second one. Each such line also specify where to write the corresponding
translation files. Here, the German translation of man/foo.1 is in man/translated/foo.de.1.

The last thing we need to complete the configuration of po4a is a POT file containing the template
material that should be used to start a new translation. Simply create an empty file with the .pot
extension in the specified po_directory (e.g. man/po4a/foo.pot), and po4a will fill it with the expected
content.

Here is a recap of the files in this setup:

├── man/
│ ├── foo.1 <- The original man page, in English
│ ├── po4a/
│ │ ├── de.po <- The German PO translation, from gettextization
│ │ └── foo.pot <- The POT template of future translations (empty at first)
│ └── translated/ <- Directory where the translations will be created
└── po4a.cfg <- The configuration file

Once setup, executing po4a will parse your documentation, update the POT template file, use it to update
the PO translation files, and use them to update the documentation translation files. All in one command:

po4a --verbose po4a.cfg

This is it. po4a is now fully configured. Once you've fixed your error in man/foo.1, the offending
paragraph in the German translation will be replaced by the fixed text in English. Mixing languages is
not optimal, but it's the only way to remove errors in translations that you don't even understand, and
ensure that the content presented to the users is never misleading. Updating the German translation is
also much easier in the corresponding PO file, so the language mix-up may not last long. Finally, when a
Japanese translator wants to contribute a new translation, she should rename the foo.pot into ja.po and
complete the translation. Once you have this file, just drop it in man/po4a/po/. A translated page will
appear as man/translated/foo.ja.1 (provided that enough content is translated) when you run po4a again.

OPTIONS

-k, --keep
Minimal threshold for translation percentage to keep (i.e. write) the resulting file (default: 80).
I.e. by default, files have to be translated at least at 80% to be written on disk.

-w, --width
Column at which we should wrap the resulting file if the format supports it (default: 76).

-h, --help
Show a short help message.

-M, --master-charset
Charset of the files containing the documents to translate. Note that all master documents must use
the same charset.

-L, --localized-charset
Charset of the files containing the localized documents. Note that all translated documents will use
the same charset.

-A, --addendum-charset
Charset of the addenda. Note that all the addenda should be in the same charset.

-V, --version
Display the version of the script and exit.

-v, --verbose
Increase the verbosity of the program.

-q, --quiet
Decrease the verbosity of the program.

-d, --debug
Output some debugging information.

-o, --option
Extra option(s) to pass to the format plugin. See the documentation of each plugin for more
information about the valid options and their meanings. For example, you could pass '-o tablecells'
to the AsciiDoc parser, while the text parser would accept '-o tabs=split'.

-f, --force
Always generate the POT and PO files, even if po4a considers it is not necessary.

The default behavior (when --force is not specified) is the following:

If the POT file already exists, it is regenerated if a master document or the configuration file
is more recent (unless --no-update is provided). The POT file is also written in a temporary
document and po4a verifies that the changes are really needed.

Also, a translation is regenerated only if its master document, the PO file, one of its addenda
or the configuration file is more recent. To avoid trying to regenerate translations which do
not pass the threshold test (see --keep), a file with the .po4a-stamp extension can be created
(see --stamp).

If a master document includes files, you should use the --force flag because the modification time of
these included files are not taken into account.

The PO files are always re-generated based on the POT with msgmerge -U.

--stamp
Tells po4a to create stamp files when a translation is not generated because it does not reach the
threshold. These stamp files are named according to the expected translated document, with the
.po4a-stamp extension.

Note: This only activates the creation of the .po4a-stamp files. The stamp files are always used if
they exist, and they are removed with --rm-translations or when the file is finally translated.

--no-translations
Do not generate the translated documents, only update the POT and PO files.

--no-update
Do not change the POT and PO files, only the translation may be updated.

--keep-translations
Keeps the existing translation files even if the translation doesn't meet the threshold specified by
--keep. This option does not create new translation files with few content, but it will save
existing translations which decay because of changes to the master files.

WARNING: This flag changes the po4a behavior in a rather drastic way: your translated files will not
get updated at all until the translation improves. Only use this flag if you prefer shipping an
outdated translated documentation rather than only shipping an accurate untranslated documentation.

--rm-translations
Remove the translated files (implies --no-translations).

--no-backups
This flag does nothing since 0.41, and may be removed in later releases.

--rm-backups
This flag does nothing since 0.41, and may be removed in later releases.

--translate-only translated-file
Translate only the specified file. It may be useful to speed up processing if a configuration file
contains a lot of files. Note that this option does not update PO and POT files. This option can be
used multiple times.

--variable var=value
Define a variable that will be expanded in the po4a configuration file. Every occurrence of $(var)
will be replaced by value. This option can be used multiple times.

--srcdir SRCDIR
Set the base directory for all input documents specified in the po4a configuration file.

If both destdir and srcdir are specified, input files are searched in the following directories, in
order: destdir, the current directory and srcdir. Output files are written to destdir if specified,
or to the current directory.

--destdir DESTDIR
Set the base directory for all the output documents specified in the po4a configuration file (see
--srcdir above).

Options modifying the POT header
--porefs type
Specify the reference format. Argument type can be one of never to not produce any reference, file to
only specify the file without the line number, counter to replace line number by an increasing
counter, and full to include complete references (default: full).

--wrap-po no|newlines|number (default: 76)
Specify how the po file should be wrapped. This gives the choice between either files that are nicely
wrapped but could lead to git conflicts, or files that are easier to handle automatically, but harder
to read for humans.

Historically, the gettext suite has reformatted the po files at the 77th column for cosmetics. This
option specifies the behavior of po4a. If set to a numerical value, po4a will wrap the po file after
this column and after newlines in the content. If set to newlines, po4a will only split the msgid and
msgstr after newlines in the content. If set to no, po4a will not wrap the po file at all. The
reference comments are always wrapped by the gettext tools that we use internally.

Note that this option has no impact on how the msgid and msgstr are wrapped, i.e. on how newlines
are added to the content of these strings.

--master-language
Language of the source files containing the documents to translate. Note that all master documents
must use the same language.

--msgid-bugs-address email@address
Set the report address for msgid bugs. By default, the created POT files have no Report-Msgid-Bugs-To
fields.

--copyright-holder string
Set the copyright holder in the POT header. The default value is "Free Software Foundation, Inc."

--package-name string
Set the package name for the POT header. The default is "PACKAGE".

--package-version string
Set the package version for the POT header. The default is "VERSION".

Options to modify the PO files
--msgmerge-opt options
Extra options for msgmerge(1).

Note: $lang will be extended to the current language.

--no-previous
This option removes --previous from the options passed to msgmerge. This is necessary to support
versions of gettext earlier than 0.16.

--previous
This option adds --previous to the options passed to msgmerge. It requires gettext 0.16 or later,
and is activated by default.

CONFIGURATION FILE

po4a expects a configuration file as argument. This file must contain the following elements:

• The path to the PO files and the list of languages existing in the project;

• Optionally, some global options and so-called configuration aliases that are used as templates to
configure individual master files;

• The list of each master file to translate, along with specific parameters.

All lines contain a command between square braces, followed by its parameters. Comments begin with the
char '#' and run until the end of the line. You can escape the end of line to spread a command over
several lines.

Some full examples are presented on this page, while other examples can be found in the "t/cfg" directory
of the source distribution.

Finding the PO and POT files
The simplest solution is to explicitly give the path to POT and PO files, as follows:

[po4a_paths] man/po/project.pot de:man/po/de.po fr:man/po/fr.po

This specifies the path to the POT file first, and then the paths to the German and French PO files.

The same information can be written as follows to reduce the risk of copy/paste errors:

[po4a_langs] fr de
[po4a_paths] man/po/project.pot $lang:man/po/$lang.po

The $lang component is automatically expanded using the provided languages list, reducing the risk of
copy/paste error when a new language is added.

You can further compact the same information by only providing the path to the directory containing your
translation project, as follows.

[po_directory] man/po/

The provided directory must contain a set of PO files, each named XX.po with "XX" the ISO 639-1 of the
language used in this file. The directory must also contain a single POT file, with the ".pot" file
extension. For the first run, this file can be empty but it must exist (po4a cannot guess the name to use
before the extension).

Note that you must choose only one between "po_directory" and "po4a_paths". The first one
("po_directory") is more compact, further reduces the risk of copy/paste error, but forces you to use the
expected project structure and file names. The second one ("po4a_paths"), is more explicit, probably more
readable, and advised when you setup your first project with po4a.

Centralized or split PO files?

By default, po4a produces one single PO file per target language, containing the whole content of your
translation project. As your project grows, the size of these files may become problematic. When using
weblate, it is possible to specify priorities for each translation segment (i.e., msgid) so that the
important ones get translated first. Still, some translation teams prefer to split the content in several
files.

To have one PO file per master file, you simply have to use the string $master in the name of your PO
files on the "[po4a_paths]" line, as follows.

[po4a_paths] doc/$master/$master.pot $lang:doc/$master/$lang.po

With this line, po4a will produce separate POT and PO files for each document to translate. For example,
if you have 3 documents and 5 languages, this will result in 3 POT files and 15 PO files. These files are
named as specified on the "po4a_paths" template, with $master substituted to the basename of each
document to translate. In case of name conflict, you can specify the POT file to use as follows, with the
"pot=" parameter.

This feature can also be used to group several translated files into the same POT file. The following
example only produces 2 POT files: l10n/po/foo.pot (containing the material from foo/gui.xml) and
l10n/po/bar.pot (containing the material from both bar/gui.xml and bar/cli.xml).

[po4a_langs] de fr ja
[po4a_paths] l10n/po/$master.pot $lang:l10n/po/$master.$lang.po
[type: xml] foo/gui.xml $lang:foo/gui.$lang.xml pot=foo
[type: xml] bar/gui.xml $lang:bar/gui.$lang.xml pot=bar
[type: xml] bar/cli.xml $lang:bar/cli.$lang.xml pot=bar

In split mode, po4a builds a temporary compendium during the PO update, to share the translations between
all the PO files. If two PO files have different translations for the same string, po4a will mark this
string as fuzzy and will submit both translations in all the PO files containing this string. When
unfuzzied by the translator, the translation is automatically used in every PO files.

Specifying the documents to translate
You must also list the documents that should be translated. For each master file, you must specify the
format parser to use, the location of the translated document to produce, and optionally some
configuration. File names should be quoted or escaped if they contain spaces. Here is an example:

[type: sgml] "doc/my stuff.sgml" "fr:doc/fr/mon truc.sgml" de:doc/de/mein\ kram.sgml
[type: man] script fr:doc/fr/script.1 de:doc/de/script.1
[type: docbook] doc/script.xml fr:doc/fr/script.xml \
de:doc/de/script.xml

But again, these complex lines are difficult to read and modify, e.g. when adding a new language. It is
much simpler to reorganize things using the $lang template as follows:

[type: sgml] doc/my_stuff.sgml $lang:doc/$lang/my_stuff.sgml
[type: man] script.1 $lang:po/$lang/script.1
[type: docbook] doc/script.xml $lang:doc/$lang/script.xml

Specifying options
There is two types of options: po4a options are default values to the po4a command line options while
format options are used to change the behavior of the format parsers. As a po4a options, you could for
example specify in your configuration file that the default value of the --keep command line parameter is
50% instead of 80%. Format options are documented on the specific page of each parsing module, e.g.
Locale::Po4a::Xml(3pm). You could for example pass nostrip to the XML parser to not strip the spaces
around the extracted strings.

You can pass these options for a specific master file, or even for a specific translation of that file,
using "opt:" and "opt_XX:" for the "XX" language. In the following example, the nostrip option is passed
to the XML parser (for all languages), while the threshold will be reduced to 0% for the French
translation (that is thus always kept).

[type:xml] toto.xml $lang:toto.$lang.xml opt:"-o nostrip" opt_fr:"--keep 0"

In any case, these configuration chunks must be located at the end of the line. The declaration of files
must come first, then the addendum if any (see below), and then only the options. The grouping of
configuration chunks is not very important, since elements are internally concatenated as strings. The
following examples are all equivalent:

[type:xml] toto.xml $lang:toto.$lang.xml opt:"--keep 20" opt:"-o nostrip" opt_fr:"--keep 0"
[type:xml] toto.xml $lang:toto.$lang.xml opt:"--keep 20 -o nostrip" opt_fr:"--keep 0"
[type:xml] toto.xml $lang:toto.$lang.xml opt:--keep opt:20 opt:-o opt:nostrip opt_fr:--keep opt_fr:0

Note that language specific options are not used when building the POT file. It is for example impossible
to pass nostrip to the parser only when building the French translation, because the same POT file is
used to update every languages. So the only options that can be language-specific are the ones that are
used when producing the translation, as the "--keep" option.

Configuration aliases

To pass the same options to several files, the best is to define a type alias as follows. In the next
example, "--keep 0" is passed to every Italian translation using this "test" type, that is an extension
of the "man" type.

[po4a_alias:test] man opt_it:"--keep 0"
[type: test] man/page.1 $lang:man/$lang/page.1

You can also extend an existing type reusing the same name for the alias as follows. This is not
interpreted as as an erroneous recursive definition.

[po4a_alias:man] man opt_it:"--keep 0"
[type: man] man/page.1 $lang:man/$lang/page.1

Global default options

You can also use "[options]" lines to define options that must be used for all files, regardless of their
type.

[options] --keep 20 --option nostrip

As with the command line options, you can abbreviate the parameters passed in the configuration file:

[options] -k 20 -o nostrip

Option priorities

The options of every sources are concatenated, ensuring that the default values can easily be overridden
by more specific options. The order is as follows:

• "[options]" lines provide default values that can be overridden by any other source.

• Type aliases are then used. Language specific settings override the ones applicable to all languages.

• Settings that are specific to a given master file override both the default ones and the ones coming
from the type alias. In this case also, language specific settings override the global ones.

• Finally, parameters provided on the po4a command line override any settings from the configuration
file.

Example

Here is an example showing how to quote the spaces and quotes:

[po_directory] man/po/

[options] --master-charset UTF-8

[po4a_alias:man] man opt:"-o \"mdoc=NAME,SEE ALSO\""
[type:man] t-05-config/test02_man.1 $lang:tmp/test02_man.$lang.1 \
opt:"-k 75" opt_it:"-L UTF-8" opt_fr:--verbose

Addendum: Adding extra content in the translation
If you want to add an extra section to the translation, for example to give credit to the translator,
then you need to define an addendum to the line defining your master file. Please refer to the page
po4a(7) for more details on the syntax of addendum files.

[type: pod] script fr:doc/fr/script.1 \
add_fr:doc/l10n/script.fr.add

You can also use language templates as follow:

[type: pod] script $lang:doc/$lang/script.1 \
add_$lang:doc/l10n/script.$lang.add

If an addendum fails to apply, the translation is discarded.

Modifiers for the addendum declaration

Addendum modifiers can simplify the configuration file in the case where not all languages provide an
addendum, or when the list of addenda changes from one language to the other. The modifier is a single
char located before the file name.

? Include addendum_path if this file does exist, otherwise do nothing.

@ addendum_path is not a regular addendum but a file containing a list of addenda, one by line. Each
addendum may be preceded by modifiers.

! addendum_path is discarded, it is not loaded and will not be loaded by any further addendum
specification.

The following includes an addendum in any language, but if only it exists. No error is reported if the
addendum does not exist.

[type: pod] script $lang:doc/$lang/script.1 add_$lang:?doc/l10n/script.$lang.add

The following includes a list of addendum for every language:

[type: pod] script $lang:doc/$lang/script.1 add_$lang:@doc/l10n/script.$lang.add

Filtering the translated strings
Sometimes, you want to hide some strings from the translation process. To that extend, you can give a
"pot_in" parameter to your master file to specify the name of the file to use instead of the real master
when building the POT file. Here is an example:

[type:docbook] book.xml \
pot_in:book-filtered.xml \
$lang:book.$lang.xml

With this setting, the strings to translate will be extracted from the book-filtered.xml (that must be
produced before calling po4a) while the translated files will be built from book.xml. As a result, any
string that is part of book.xml but not in book-filtered.xml will not be included in the PO files,
preventing the translators from providing a translation for them. So these strings will be left
unmodified when producing the translated documents. This naturally decreases the level of translation,
so you may need the "--keep" option to ensure that the document is produced anyway.

AUTHORS

        Denis Barbier <barbier@linuxfr.org>
        Nicolas François <nicolas.francois@centraliens.net>
        Martin Quinson (mquinson#debian.org)

COPYRIGHT AND LICENSE

       Copyright 2002-2023 by SPI, inc.

       This  program  is  free software; you may redistribute it and/or modify it under the terms of GPL v2.0 or
       later (see the COPYING file).

perl v5.38.2                                       2024-08-28                                         PO4A.1P(1)