[ð 0x4c8a] [20:30:00.001528] [DEBUG] [ð MultiFetcher.initialize_sources] Begin initialization of 2 children.
[ð 0x4c8a] [20:30:00.001644] [INFO] [â
MemoryFetcher.initialize_sources] Finished initialization of 'MemoryFetcher' in 3 Ξs: MemoryFetcher(sources=['category'], return_all=True)
[ð 0x4c8a] [20:30:00.071723] [DEBUG] [id_translation.fetching] Metadata for 'postgresql+pg8000://postgres:***@localhost:5002/sakila' created in 70 ms.
[ð 0x4c8a] [20:30:00.072792] [INFO] [â
SqlFetcher.initialize_sources] Finished initialization of 'SqlFetcher' in 71 ms: SqlFetcher('postgresql+pg8000://postgres:***@localhost:5002/sakila', sources=['address', 'film', 'language', 'customer', 'country', 'staff', 'actor', 'city'])
[ð 0x4c8a] [20:30:00.072865] [DEBUG] [â
MultiFetcher.initialize_sources] Finished initialization 2 children and 9 sources in 71 ms.
[ð 0xc0c9] [20:30:00.208977] [DEBUG] [id_translation.dio] Using rank-0 (priority=1999) implementation 'id_translation.dio.integration.pandas.PandasIO' for translatable of type='pandas.DataFrame'.
[ð 0xc0c9] [20:30:00.209183] [DEBUG] [ð Translator.translate] Begin translation of 'DataFrame'-type data. Names to translate: Derive based on type.
[ð 0xc0c9] [20:30:00.209249] [DEBUG] [id_translation.Translator] Name extraction complete. Found names=['customer_id', 'film_id', 'category_id', 'staff_id', 'rental_date', 'return_date'] for 'DataFrame'-type data.
[ð 0xc0c9] [20:30:00.209293] [DEBUG] [ð Translator.map] Begin name-to-source mapping of names=['customer_id', 'film_id', 'category_id', 'staff_id', 'rental_date', 'return_date'] in 'DataFrame' against sources=['category', 'address', 'film', 'language', 'customer', 'country', 'staff', 'actor', 'city'].
[ð 0xc0c9] [20:30:00.210815] [DEBUG] [id_translation.Translator.map] Computed 6x9 match scores in context=None in 153 Ξs:
candidates category address film language customer country staff actor city
values
customer_id 0.12 0.06 -0.00 -0.00 1.00 -0.01 -0.01 0.04 -0.01
film_id -0.00 0.00 1.00 -0.00 -0.00 -0.01 -0.01 -0.01 0.24
category_id 1.00 0.00 -0.00 0.12 0.12 0.14 -0.01 -0.01 0.04
staff_id -0.00 0.00 0.05 0.05 -0.00 -0.01 0.99 -0.01 -0.01
rental_date -inf -inf -inf -inf -inf -inf -inf -inf -inf
return_date -inf -inf -inf -inf -inf -inf -inf -inf -inf
[ð 0xc0c9] [20:30:00.211001] [INFO] [â
Translator.map] Finished mapping of 4/6 names in 'DataFrame' in 2 ms: {'customer_id': 'customer', 'film_id': 'film', 'category_id': 'category', 'staff_id': 'staff', 'rental_date': None, 'return_date': None}.
[ð 0xc0c9] [20:30:00.211264] [DEBUG] [ð MultiFetcher.fetch] Dispatch FETCH jobs for 4 sources using 2 different fetchers on 2 threads.
[ð 0xc0c9] [20:30:00.211508] [DEBUG] [id_translation.fetching.MultiFetcher] Begin FETCH job for 3 sources using rank-1 fetcher SqlFetcher('postgresql+pg8000://postgres:***@localhost:5002/sakila', sources=['address', 'film', 'language', 'customer', 'country', 'staff', 'actor', 'city']) at 0x74426fad36f0.
[ð 0xc0c9] [20:30:00.211625] [DEBUG] [ð SqlFetcher.fetch] Begin fetching placeholders=('id', 'name', 'last_name') for 22 IDs from 3 sources: {'staff': 2, 'customer': 10, 'film': 10}.
[ð 0xc0c9] [20:30:00.211684] [DEBUG] [ð SqlFetcher.map_placeholders] Begin mapping of wanted placeholders={'last_name', 'id', 'name'} to actual placeholders={'staff_id', 'first_name', 'address_id', 'username', 'picture', 'email', 'password', 'store_id', 'active', 'last_name', 'last_update'} for source='staff'.
[ð 0xc0c9] [20:30:00.213365] [DEBUG] [id_translation.fetching.map] Computed 3x11 match scores in context='staff' in 131 Ξs:
candidates staff_id first_name address_id username picture email password store_id active last_name last_update
values
last_name -inf -inf -inf -inf -inf -inf -inf -inf -inf inf -inf
id 0.99 -0.00 0.16 -0.00 0.06 0.12 0.24 0.61 0.04 -0.01 -0.01
name 0.00 0.99 0.02 0.20 0.06 0.18 0.03 -0.01 0.08 0.37 0.09
[ð 0xc0c9] [20:30:00.213885] [DEBUG] [â
SqlFetcher.map_placeholders] Finished placeholder mapping for source='staff': {'last_name': 'last_name', 'name': 'first_name', 'id': 'staff_id'}.
[ð 0xc0c9] [20:30:00.213958] [DEBUG] [ð SqlFetcher.fetch_translations] Begin fetching 2 IDs from source='staff'. Placeholders: ('staff_id', 'first_name', 'last_name').
[ð 0xc0c9] [20:30:00.214696] [DEBUG] [id_translation.fetching.MultiFetcher] Begin FETCH job for 1 sources using rank-0 fetcher MemoryFetcher(sources=['category'], return_all=True) at 0x74426f9a2650.
[ð 0xc0c9] [20:30:00.214788] [DEBUG] [ð MemoryFetcher.fetch] Begin fetching placeholders=('id', 'name', 'last_name') for 7 IDs from 1 sources: {'category': 7}.
[ð 0xc0c9] [20:30:00.214839] [DEBUG] [ð MemoryFetcher.map_placeholders] Begin mapping of wanted placeholders={'last_name', 'id', 'name'} to actual placeholders={'id', 'name'} for source='category'.
[ð 0xc0c9] [20:30:00.215789] [DEBUG] [id_translation.fetching.map] Computed 3x2 match scores in context='category' in 32 Ξs:
candidates id name
values
last_name 0.00 0.17
id inf -inf
name -inf inf
[ð 0xc0c9] [20:30:00.215976] [DEBUG] [â
MemoryFetcher.map_placeholders] Finished placeholder mapping for source='category': {'id': 'id', 'name': 'name', 'last_name': None}.
[ð 0xc0c9] [20:30:00.216046] [DEBUG] [ð MemoryFetcher.fetch_translations] Begin fetching 7 IDs from source='category'. Placeholders: ('id', 'name', 'last_name').
[ð 0xc0c9] [20:30:00.216087] [DEBUG] [â
MemoryFetcher.fetch_translations] Finished fetching 16 IDs from source='category' in 42 Ξs. Placeholders: ('id', 'name').
[ð 0xc0c9] [20:30:00.216136] [INFO] [â
MemoryFetcher.fetch] Finished fetching from 1 sources in 1 ms: ['category' x ('id', 'name') x 16/7 IDs].
[ð 0xc0c9] [20:30:00.217380] [DEBUG] [â
SqlFetcher.fetch_translations] Finished fetching 2 IDs from source='staff' in 3 ms. Placeholders: ('staff_id', 'first_name', 'last_name').
[ð 0xc0c9] [20:30:00.217463] [DEBUG] [ð SqlFetcher.map_placeholders] Begin mapping of wanted placeholders={'last_name', 'id', 'name'} to actual placeholders={'customer_id', 'first_name', 'address_id', 'activebool', 'email', 'store_id', 'active', 'last_name', 'create_date', 'last_update'} for source='customer'.
[ð 0xc0c9] [20:30:00.218809] [DEBUG] [id_translation.fetching.map] Computed 3x10 match scores in context='customer' in 120 Ξs:
candidates customer_id first_name address_id activebool email store_id active last_name create_date last_update
values
last_name -inf -inf -inf -inf -inf -inf -inf inf -inf -inf
id 0.99 -0.00 0.16 0.09 0.12 0.36 0.04 -0.01 0.07 0.16
name 0.00 0.99 -0.00 -0.00 0.12 -0.01 0.08 0.38 0.14 0.09
[ð 0xc0c9] [20:30:00.218938] [DEBUG] [â
SqlFetcher.map_placeholders] Finished placeholder mapping for source='customer': {'last_name': 'last_name', 'name': 'first_name', 'id': 'customer_id'}.
[ð 0xc0c9] [20:30:00.218997] [DEBUG] [ð SqlFetcher.fetch_translations] Begin fetching 10 IDs from source='customer'. Placeholders: ('customer_id', 'first_name', 'last_name').
[ð 0xc0c9] [20:30:00.221355] [DEBUG] [â
SqlFetcher.fetch_translations] Finished fetching 10 IDs from source='customer' in 2 ms. Placeholders: ('customer_id', 'first_name', 'last_name').
[ð 0xc0c9] [20:30:00.221443] [DEBUG] [ð SqlFetcher.map_placeholders] Begin mapping of wanted placeholders={'last_name', 'id', 'name'} to actual placeholders={'title', 'special_features', 'description', 'language_id', 'rating', 'replacement_cost', 'original_language_id', 'rental_rate', 'release_year', 'rental_duration', 'fulltext', 'film_id', 'length', 'last_update'} for source='film'.
[ð 0xc0c9] [20:30:00.223089] [DEBUG] [id_translation.fetching.map] Computed 3x14 match scores in context='film' in 146 Ξs:
candidates title special_features description language_id rating replacement_cost original_language_id rental_rate release_year rental_duration fulltext film_id length last_update
values
last_name 0.04 -0.00 -0.00 -0.00 -0.01 0.06 -0.01 0.10 0.07 -0.01 -0.01 -0.01 0.03 0.07
id -0.00 0.01 0.02 0.10 0.10 -0.01 0.05 -0.01 -0.01 0.02 0.23 0.98 -0.01 0.00
name inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf -inf
[ð 0xc0c9] [20:30:00.223260] [DEBUG] [â
SqlFetcher.map_placeholders] Finished placeholder mapping for source='film': {'name': 'title', 'id': 'film_id', 'last_name': None}.
[ð 0xc0c9] [20:30:00.223307] [DEBUG] [ð SqlFetcher.fetch_translations] Begin fetching 10 IDs from source='film'. Placeholders: ('film_id', 'title').
[ð 0xc0c9] [20:30:00.225296] [DEBUG] [â
SqlFetcher.fetch_translations] Finished fetching 10 IDs from source='film' in 2 ms. Placeholders: ('film_id', 'title').
[ð 0xc0c9] [20:30:00.225383] [INFO] [â
SqlFetcher.fetch] Finished fetching from 3 sources in 14 ms: ['staff' x ('id', 'name', 'last_name') x 2/2 IDs], ['customer' x ('id', 'name', 'last_name') x 10/10 IDs], ['film' x ('id', 'name') x 10/10 IDs].
[ð 0xc0c9] [20:30:00.225606] [DEBUG] [â
MultiFetcher.fetch] Finished FETCH jobs for 4 sources using 2 different fetchers in 14 ms.
[ð 0xc0c9] [20:30:00.227411] [DEBUG] [id_translation.Translator] Performance counters [ms]: {'map': 1.758, 'extract': 0.187, 'fetch': 14.618, 'verify': 0.315, 'insert': 1.379, 'overhead': 0.475}
[ð 0xc0c9] [20:30:00.227481] [INFO] [â
Translator.translate] Finished translation of 29 unique IDs (4 names) in 'DataFrame' in 19 ms.