id_translation.dio.integration.pyarrow#

Integration for Arrow types.

Module Attributes

ArrowT

Supported arrow types.

Classes

ArrowIO(*[, missing_as_nan])

Optional IO implementation for pyarrow types.
class ArrowIO(*, missing_as_nan=False)[source]#

Bases: DataStructureIO[ArrowT, str, SourceType, IdType]

Optional IO implementation for pyarrow types.

Parameters:

missing_as_nan – If set, unknown IDs will be null. If False, placeholders such as '<Failed: id=-1>' will be used instead.

classmethod extract(translatable, names)[source]#

Extract IDs from translatable.

Parameters:

  • translatable – Data to extract IDs from.

  • names – List of names in translatable to extract IDs for.

Returns:

A dict {name: ids}.

classmethod handles_type(arg)[source]#

Return True if the implementation handles data for the type of arg.

insert(translatable, names, tmap, copy)[source]#

Insert translations into translatable.

Parameters:

  • translatable – Data to translate. Modified iff copy=False.

  • names – Names in translatable to translate.

  • tmap – Translations for IDs in translatable.

  • copy – If True, modify contents of the original translatable. Otherwise, returns a copy.

Returns:

A copy of translatable if copy=True, None otherwise.

Raises:

NotInplaceTranslatableError – If copy=False for a type which is not translatable in-place.

classmethod names(translatable)[source]#

Extract names from translatable.

Parameters:

translatable – Data to extract names from.

Returns:

A list of names to translate. Returns None if names cannot be extracted.

priority = -1900#

Determines order in which IOs are considered (higher = earlier).

Set priority < 0 to disable.

class ArrowT#

Supported arrow types.

alias of TypeVar(‘ArrowT’, ~pyarrow.lib.Table, ~pyarrow.lib.Array, ~pyarrow.lib.ChunkedArray)