id_translation.offline#

Offline (in-memory) translation classes.

Classes

`Format`(fmt)	Format specification for translations strings.
`FormatApplier`(translations, *[, transformer])	Application of `Format` specifications.
`MagicDict`(real_translations[, ...])	Dictionary type for translated IDs.
`TranslationMap`(source_translations, *[, ...])	Storage class for fetched translations.

class Format(fmt)[source]#

Bases: object

Format specification for translations strings.

Translation formats are similar to regular f-strings, with two important exceptions:

Positional placeholders ('{}') may not be used; correct form is '{placeholder-name}'.
Placeholders surrounded by '[]' denote an optional element. Optional elements are rendered…
- Only if all of its placeholders are defined.
- Without delimiting brackets.
- As literal text (with brackets) if there is no placeholder in the block.

Hint

Double the wanted bracket character to render as a literal, analogous to '{{' and '}}' in plain Python f-strings. See the example below for a demonstration.

Parameters:: fmt – A translation fstring.

Examples

Basic usage

Formats are created by passing a single str arguments, as described above.

>>> Format(Format.DEFAULT)
"Format('{id}:{name}')"
>>> Format(Format.DEFAULT).fstring().format(id=1, name="First")
'1:First'

Using Format.fstring() and str.format() is flexible but verbose. Formats can be applier either through Format.format()…

>>> fmt = Format(Format.DEFAULT_FAILED)
>>> fmt.format(id=1, name="First")

…or just Format.__call__().

>>> fmt(id=1, name="First")
'<Failed: id=1>'

Using either convenience method will use as many placeholders as possible.

>>> fmt = Format(Format.DEFAULT)
>>> fmt.placeholders
"('id', 'name')"
>>> fmt(id=1, name="First")
'1:First'
>>> fmt(id=1, name="First", unknown=20.19)
'1:First'

Unknown placeholders are simply ignored.

Optional placeholders

A format string using literal angle brackets and an optional element.

>>> from id_translation.offline import Format
>>> fmt = Format("{id}:[[{name}]][, nice={is_nice}]")

The Format class when used directly only returns required placeholders by default…

>>> fmt.fstring()
'{id}:[{name}]'
>>> fmt(id=0, name="Tarzan")
'0:[Tarzan]'

…but the placeholders attribute can be used to retrieve all placeholders, required and optional:

>>> fmt.placeholders
('id', 'name', 'is_nice')
>>> fmt(id=1, name="Morris", is_nice=True)
'1:[Morris], nice=True'

The Translator will automatically add optional placeholders, if they are present in the source.

Note

Python format specifications and conversions are preserved.

This is especially useful for long values such as UUIDs.

>>> from uuid import UUID
>>> uuid = UUID("550e8400-e29b-41d4-a716-446655440000")

Convert to string and truncate to eight characters.

>>> Format("{id!s:.8}:{name!r}").format(id=uuid, name="Sofia")
"550e8400:'Sofia'"

See the official Format Specification Mini-Language documentation for details.

DEFAULT = '{id}:{name}'#: Default translation format.

DEFAULT_FAILED = '<Failed: id={id!r}>'#: Default format for missing IDs.

format(**placeholders)[source]#

Apply the format.

Parameters:: **placeholders – Formats to use in the finals string.
Returns:: Formatting using placeholders.

fstring(placeholders=None, *, positional=False)[source]#

Create a format string for the given placeholders.

Parameters:

placeholders – Keys to keep. Passing None is equivalent to passing required_placeholders.
positional – If True, remove names to return a positional fstring.

Returns:

An fstring with optional elements removed unless included in placeholders.

Raises:

KeyError – If required placeholders are missing.

property optional_placeholders#: All optional placeholders in the order in which they appear.

classmethod parse(fmt)[source]#

Parse a format.

Parameters:: fmt – A str or Format instance.
Returns:: A Format instance.

partial(defaults)[source]#

Get a partially formatted fstring().

Parameters:: defaults – Keys which should be replaced with real values. Keys which are not part of defaults will be left as-is.
Returns:: A partially formatted fstring.

property placeholders#: All placeholders in the order in which they appear.

property required_placeholders#: All required placeholders in the order in which they appear.

class FormatApplier(translations, *, transformer=None)[source]#

Bases: Generic[NameType, SourceType, IdType]

Application of Format specifications.

This class converts raw translation data into ready-to-use dicts on the form {id: translation}, where the translation is always a plain string.

Parameters:

translations – A PlaceholderTranslations object returned by fetchers.
transformer – Initialized Transformer instance.

Raises:

ValueError – If default is given and any placeholder names are missing.

Examples

Basic usage.

>>> data = {1999: "Sofia", 1991: "Richard", 1904: "Fred"}
>>> translations = PlaceholderTranslations.from_dict("my-source", data)
>>> applier = FormatApplier(translations)

We used the simplified {id: name} to create the translation object above. Let’s create the formats to use:

>>> fmt = Format.parse("{id}:{name}")
>>> default_fmt = Format.parse("<Failed: id={id!r}>")

Using FormatApplier.__call__ delegates to apply().

>>> applier(fmt, default_fmt=default_fmt)
{1999: '1999:Sofia', 1991: '1991:Richard', 1904: '1904:Fred'}

The output may look like a regular dict, but is actually a MagicDict.

>>> magic_dict = applier(fmt, default_fmt=default_fmt)
>>> type(magic_dict)
<class 'id_translation.offline._magic_dict.MagicDict'>

Warning

The MagicDict is does not behave like a regular dict.

You can, for instance, use __getitem__ on unknown keys:

>>> magic_dict = applier(fmt, default_fmt=default_fmt)
>>> magic_dict[-1]
'<Failed: id=-1>'

See the MagicDict class documentation for more information.

apply(fmt, *, default_fmt, placeholders=None, default_fmt_placeholders=None, enable_uuid_heuristics=True)[source]#

Translate IDs.

Note

This method does not accept strings. Use Format.parse() to convert raw formats.

Parameters:

fmt – Translation Format to use.
placeholders – Tuple of placeholder names to include in the formatted output. If None, use the intersection of placeholders and fmt.placeholders.
default_fmt – Alternative format for default translation.
default_fmt_placeholders – Default placeholders, e.g. {'name': 'default name'}.
enable_uuid_heuristics – Improves matching when UUID-like IDs are in use.

Returns:

A dict {id: translation}.

Notes

This method is an alias of __call__.

property placeholders#: List of placeholder names; see PlaceholderTranslations.placeholders.

property records#: Records used by this instance; see PlaceholderTranslations.records.

property source#: Return translation source.

to_dict()[source]#

Get the underlying data used for translations as a dict.

Returns:: A dict {placeholder: [values...]}.

to_pandas()[source]#: Get the underlying data used for translations as a pandas.DataFrame.

property transformer#: Get the Transformer instance (or None) used by this FormatApplier.

class MagicDict(real_translations, default_value='<Failed: id={!r}>', enable_uuid_heuristics=True, transformer=None)[source]#

Bases: MutableMapping[IdType, str]

Dictionary type for translated IDs.

A dict-like mapping which returns “real” values if present in a backing dict. Values for unknown keys are generated using the default_value.

Parameters:

real_translations – A dict holding real translations.
default_value – A string with exactly one or zero positional placeholders.
enable_uuid_heuristics – Improves matching when UUID-like IDs are in use. Forcibly set to False if any of the real_translations are not UUID-like.
transformer – Initialized Transformer instance. The Transformer.update_translations()-method is called after UUID heuristics are applied.

Examples

Similarities with the built-in dict

>>> magic = MagicDict({1999: "Sofia", 1991: "Richard"})

Iteration, equality, and length are based on the real values.

>>> magic
{1999: 'Sofia', 1991: 'Richard'}
>>> len(magic)
2
>>> list(magic)
[1999, 1991]
>>> magic.real == magic
True
>>> magic == {1999: "Sofia"}  # Element missing
False

As you’d expect, casting to a regular dict removes all special handling.

Differences from the built-in dict

Methods __getitem__ and __contains__ never fail or return False. Using a default with get will generate a value rather than using the provided default.

>>> magic[1999]
'Sofia'
>>> magic[2019]
'<Failed: id=2019>'
>>> magic.get(2019, "foo")  
'<Failed: id=2019>'

ID translation heuristics

Special handling for uuid.UUID and UUID-like strings improve matching.

>>> string_uuid = "550e8400-e29b-41d4-a716-446655440000"
>>> magic = MagicDict(
...     {string_uuid: "Found!"},
...     enable_uuid_heuristics=True,
... )
>>> magic
{UUID('550e8400-e29b-41d4-a716-446655440000'): 'Found!'}

When enable_uuid_heuristics=True, the MagicDict will attempt to cast keys to uuid.UUID.

>>> from uuid import UUID
>>> magic[string_uuid], magic[UUID(string_uuid)]
('Found!', 'Found!')

Keys that cannot be converted are left as-is.

>>> magic["Hello"] = "World!"
>>> magic["unknown"], magic["Hello"]
("<Failed: id='unknown'>", 'World!')

To further customize ID matching behaviour, refer to the Transformer interface.

LOGGER = <Logger id_translation.offline.MagicDict (WARNING)>#

property default_value#: Return the default string value to return for unknown keys, if any.

get(key, /, _=None)[source]#

Same as __getitem__.

Values for missing keys are generated from default_value.

property real#: Returns the backing dict.

real_contains(key, /)[source]#: Check if an actual translation exists using real_get().

real_get(key)[source]#

Attempt to get an actual translation.

This method behaves like MagicDict.__getitem__, applying all appropriate heuristics except for falling back to the default_value. Returns None if the key cannot be mapped to a real values, like the regular dict.get method would.

To bypass the heuristics, use real and dict.get() instead. Note that the backing dict may still contain mappings added transformers, since the Transformer.update_translations() interface is called during initialization.

class TranslationMap(source_translations, *, fmt='{id}:{name}', default_fmt='<Failed: id={id!r}>', name_to_source=None, default_fmt_placeholders=None, enable_uuid_heuristics=True, transformers=None)[source]#

Bases: Generic[NameType, SourceType, IdType], HasSources[SourceType], Mapping[NameType | SourceType, MagicDict[IdType]]

Storage class for fetched translations.

Parameters:

source_translations – Fetched translations {source: PlaceholderTranslations}.
name_to_source – Mappings {name: source}, but may be overridden by the user.
fmt – A translation format. Must be given to use as a mapping.
default_fmt – Alternative format specification to use instead of fmt for fallback translation.
default_fmt_placeholders – Per-source default placeholder values.
enable_uuid_heuristics – Improves matching when UUID-like IDs are in use.
transformers – A dict {source: transformer} of initialized Transformer instances.

apply(name_or_source, fmt=None, *, default_fmt=None)[source]#

Create translations for a given name or source.

Parameters:

name_or_source – A name or source to translate.
fmt – Format to use. If None, fall back to init format.
default_fmt – Alternative format for default translation. Resolution: Arg -> init arg, fmt arg, init fmt arg

Returns:

Translations for name as a dict {id: translation}.

Raises:

ValueError – If fmt=None and initialized without fmt.
KeyError – If trying to translate name which is not known.

Notes

This method is called by __getitem__.

copy()[source]#: Make a copy of this TranslationMap.

property default_fmt#: Return the format specification to use instead of fmt for fallback translation.

property default_fmt_placeholders#: Return the default translations used for default_fmt_placeholders placeholders.

property enable_uuid_heuristics#: Return automatic UUID mitigation status.

property fmt#: Return the translation format.

classmethod from_pandas(frames, fmt='{id}:{name}', *, default_fmt='<Failed: id={id!r}>')[source]#

Create a new instance from a pandas.DataFrame dict.

Parameters:

frames – A dict {source: DataFrame}.
fmt – A translation format. Must be given to use as a mapping.
default_fmt – Alternative format specification to use instead of fmt for fallback translation.

Returns:

A new TranslationMap.

property len_per_source#: Number of IDs per source.

property name_to_source#: Return name-to-source mapping.

property names#: Return names that can be translated.

property placeholders#

Placeholders for all known Source names, such as id or name.

These are the (possibly unmapped) placeholders that may be used for translation.

Returns:: A dict {source: [placeholders..]}.

property reverse_mode#

Return reversed mode status flag.

If set, the mappings returned by apply() (and therefore also __getitem__) are reversed.

Returns:: Reversal status flag.

property sources#: A list of known Source names, such as cities or languages.

to_dicts()[source]#

Get the underlying data used for translations as dicts.

This is equivalent using to_pandas(), then calling DataFrame.to_dict(orient='list') on each frame.

Returns:: A dict {source: {placeholder: [values...]}}.

to_pandas()[source]#

Get the underlying data used for translations as pandas.DataFrame.

Returns:: A dict {source: DataFrame}.

to_translations(fmt=None)[source]#

Create translations for all sources.

Returned values are of type MagicDict. To convert to regular built-in dicts, run

translations = translation_map.to_translations()
as_regular_dicts = {
   source: dict(magic)
   for source, magic in translations.items()
}

on the returned dict-of-magic-dicts.

Parameters:: fmt – Format to use. If None, fall back to init format.
Returns:: A dict of translations {source: MagicDict}.

property transformers#: Get a dict {source: transformer} of Transformer instances used by this TranslationMap.

Modules

`parse_format_string`	Utility module for parsing raw `Format` input strings.
`types`	Types used for offline translation.