Understand how to defuse localization in GNU Gettext
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	valerio.bozzolan
	Jun 30 2024, 19:21

Description

Imagine a wild amount of incoming environment variables like LANG, LANGUAGE, LC_MESSAGES and so on.

How to defuse localization?

🔴 `LANGUAGE=C`	Even if this has maximum priority, this is too risky since other specific things may call `LC_stuff` as far as I understand.
🔴 `LANG=C`	This is supposed to be the default for missing `LC_stuff` so, not correct. Moreover, `LC_ALL` can be available and will take precedence - https://www.gnu.org/software/gettext/manual/html_node/Locale-Environment-Variables.html
🔶 `LANG=C` + `LC_MESSAGES=C`	it should work since it was adopted by https://github.com/apache/subversion/blob/trunk/tools/client-side/bash_completion#L81 but `LC_ALL` may be available and could take precedence, or, `LC_NUMERIC` and other stuff will still be read
🔶 unset `LC_ALL` + `LC_MESSAGES=C`	OK for parsing messages, and in theory `LANG` has lower precedence, but not OK for parsing numbers and times and other stuff by `LC_*`
✅ `LC_ALL=C`	probably bombproof - in theory `LC_ALL` has highest precedence and the variable LANGUAGE is ignored if the locale is set to ‘C’. https://www.gnu.org/software/gettext/manual/html_node/The-LANGUAGE-variable.html#The-LANGUAGE-variable
✅ unset `LANGUAGE` + `LC_ALL=C`	probably bombproof - and in theory thanks to `LC_ALL=C` the `LANG` is not used to get defaults of `LC_stuff` including `LC_MESSAGES`
✅✅ `LC_ALL=C` + `LANG=C`	probably super bombproof, probably overkill, probably it's already OK the `LC_ALL=C`

IMPORTANT: If something above is uncorrect, please edit, add links, thanks 🌈

Related Objects
Search...

Status	Assigned	Task
Resolved	valerio.bozzolan	T15855 Fix "arc diff" in Subversion for non-English languages
Resolved	valerio.bozzolan	T15927 Fix Arcanist unit test ArcanistBundleTestCase->testNonlocalTrailingNewline() on non-English terminal
Resolved	• l2dy	T15872 Understand how to defuse localization in GNU Gettext

Event Timeline

valerio.bozzolan created this task.Jun 30 2024, 19:21

valerio.bozzolan mentioned this in D25691: Fix arc diff in Subversion for non-English languages.

valerio.bozzolan updated the task description. (Show Details)Jun 30 2024, 19:27

LC_ALL=C is not risky on GNU systems.

In Gettext, guess_category_value() ignores LANGUAGE if the locale is C or starts with C.. The locale is retrieved from Gnulib, which either calls uselocale(), setlocale(), or reads the environment variable with Gnulib's own implementation.

In Subversion, svn_cmdline_init() set the locale with setlocale(LC_ALL, ""), which means both uselocale() and setlocale() would read the locale set by setlocale(3). Gnulib's own implementation follows the GNU C Library's precedence rules for setlocale(3), which is:

first (regardless of category), the environment variable LC_ALL is inspected
next the environment variable with the same name as the category (see the table above), and
finally the environment variable LANG.
The first existing environment variable is used. If its value is not a valid locale specification, the locale is unchanged, and setlocale() returns NULL.

The details are implementation-dependent, but the Base Definitions volume of POSIX.1-2017 defines precedence as follows:

LC_ALL

This variable shall determine the values for all locale categories. The value of the LC_ALL environment variable has precedence over any of the other environment variables starting with LC_ (LC_COLLATE, LC_CTYPE, LC_MESSAGES, LC_MONETARY, LC_NUMERIC, LC_TIME) and the LANG environment variable.

I have not checked older standards or implementations, but for current POSIX-compliant and GNU systems, LC_ALL=C is certainly enough.

References:

• l2dy updated the task description. (Show Details)Jul 1 2024, 12:34

• l2dy updated the task description. (Show Details)Jul 1 2024, 12:39

OK thanks! you persuaded me

So, if we are sure that LC_ALL=C also ignores LANGUAGE, feel free to claim this and set as resolved to inherit endless glory and fix this truth on a runic stone

This [LANG=C] is supposed to be the default for missing LC_stuff

in theory LANG is never read since it's never used for defaults of LC_stuff including LC_MESSAGES

@valerio.bozzolan You may want to fix the contradiction between these two statements.

Anyways, LC_ALL=C is enough and LC_ALL=C + LANG=C should be bombproof.

valerio.bozzolan updated the task description. (Show Details)Jul 1 2024, 12:49

About this:

unset LC_ALL + LC_MESSAGES=C

I think it's not safe since other LC_STUFF can still be read 🤔

In T15872#18251, @valerio.bozzolan wrote:

About this:

unset LC_ALL + LC_MESSAGES=C

I think it's not safe since other LC_STUFF can still be read 🤔

Depends on what we are parsing, but yes that should be marked 🔶.

valerio.bozzolan updated the task description. (Show Details)Jul 1 2024, 12:56