2025-02-24 07:15:08,405 - data_prep_class.py - INFO - 94 - Get data from Big Query with SELECT firma, datum, fg_sg, fg_bez, ladenvkp, katalogvkp, oeffnungszeit, fg_typ
FROM `test-ww-ki-witt.co_stat_daily_prognosis_bronzelayer.sales`
2025-02-24 07:15:13,332 - data_prep_class.py - INFO - 102 - firma datum fg_sg fg_bez ladenvkp katalogvkp oeffnungszeit fg_typ date
0 Sieh an! GmbH 2024-11-28 21 Neuburg a. d. Donau 135.80 166.00 Montag bis Freitag von 09.00 bis 18.00 Uhr | k... Basisfiliale 2024-11-28
1 Josef Witt GmbH 2024-11-28 21 Neuburg a. d. Donau 1680.85 2036.40 Montag bis Freitag von 09.00 bis 18.00 Uhr | k... Basisfiliale 2024-11-28
2 Sieh an! GmbH 2024-11-28 193 Dingolfing 56.00 56.00 Montag bis Freitag von 9.00 bis 18.00 Uhr | Sa... Basisfiliale 2024-11-28
3 Heinrich Heine GmbH 2024-11-28 193 Dingolfing 14.98 29.99 Montag bis Freitag von 9.00 bis 18.00 Uhr | Sa... Basisfiliale 2024-11-28
4 Josef Witt GmbH 2024-11-28 193 Dingolfing 1307.04 1383.89 Montag bis Freitag von 9.00 bis 18.00 Uhr | Sa... Basisfiliale 2024-11-28
2025-02-24 07:15:13,346 - dataprep_main.py - DEBUG - 44 - shape: (257665, 9) shops: 122 start: 2021-01-04 end: 2025-02-05 sumladenvkp: 246324285
2025-02-24 07:15:13,409 - data_prep_class.py - DEBUG - 132 - output of shops and codes found in sales data to fg_sg_bez.xlsx
2025-02-24 07:15:13,641 - dataprep_main.py - DEBUG - 55 - shape: (126213, 3) shops: 122 start: 2021-01-04 end: 2025-02-05 sumladenvkp: 246324285
2025-02-24 07:15:13,641 - data_prep_class.py - DEBUG - 175 - max date in dataframe: 2025-02-05 00:00:00
2025-02-24 07:15:13,642 - data_prep_class.py - DEBUG - 188 - date_range: 2025-02-06 00:00:00 2025-04-30 00:00:00
2025-02-24 07:15:13,735 - data_prep_class.py - DEBUG - 201 - size of new dataset: (10248, 3) vs 10248
2025-02-24 07:15:13,746 - data_prep_class.py - DEBUG - 205 - final size: (136461, 3)
2025-02-24 07:15:13,750 - dataprep_main.py - DEBUG - 63 - shape: (136461, 3) shops: 122 start: 2021-01-04 end: 2025-04-30 sumladenvkp: 246324285
2025-02-24 07:15:14,019 - dataprep_main.py - DEBUG - 74 - shape: (132299, 24) shops: 113 start: 2021-01-04 end: 2025-04-30 sumladenvkp: 242259077
2025-02-24 07:15:14,019 - data_prep_class.py - DEBUG - 250 - check Friedrichshafen
2025-02-24 07:15:14,035 - data_prep_class.py - DEBUG - 254 - date fg_sg fg_bez ladenvkp
114476 2024-01-02 183 Friedrichshafen 643.92
2025-02-24 07:15:50,323 - dataprep_main.py - DEBUG - 81 - shape: (132299, 24) shops: 113 start: 2021-01-04 end: 2025-04-30 sumladenvkp: 242259077
2025-02-24 07:15:50,340 - dataprep_main.py - DEBUG - 85 - shape: (132299, 24) shops: 113 start: 2021-01-04 end: 2025-04-30 sumladenvkp: 242259077
2025-02-24 07:15:57,311 - dataprep_main.py - DEBUG - 99 - shape: (109987, 38) shops: 113 start: 2022-01-03 end: 2025-04-30 sumladenvkp: 204495209
2025-02-24 07:15:57,311 - data_prep_class.py - DEBUG - 211 - before removing holidays: (109987, 38)
2025-02-24 07:15:57,325 - data_prep_class.py - DEBUG - 213 - aftere removing holidays: (109987, 38)
2025-02-24 07:15:57,329 - dataprep_main.py - DEBUG - 104 - shape: (109987, 38) shops: 113 start: 2022-01-03 end: 2025-04-30 sumladenvkp: 204495209
2025-02-24 07:15:57,750 - data_prep_class.py - DEBUG - 436 - missing values last year: lastyear_date 36754
lastyear_ladenvkp 36754
lastyear_fg_sg 36754
dtype: int64
2025-02-24 07:15:59,160 - dataprep_main.py - DEBUG - 143 - shape: (109987, 259) shops: 113 start: 2022-01-03 end: 2025-04-30 sumladenvkp: 204495209
2025-02-24 07:16:08,966 - data_prep_class.py - DEBUG - 393 - merge on agp - missing: lastyear_date 31609
lastyear_ladenvkp 31609
lastyear_fg_sg 31609
imputed_lastyear_ladenvkp 28833
imputed2_lastyear_ladenvkp 27534
av_lastyear_ladenvkp 27534
agp 2713
dtype: int64
2025-02-24 07:16:08,970 - dataprep_main.py - DEBUG - 151 - shape: (104842, 260) shops: 113 start: 2022-03-01 end: 2025-04-30 sumladenvkp: 196830741
2025-02-24 07:16:08,972 - data_prep_class.py - DEBUG - 400 - fill missing agp: 0
2025-02-24 07:16:08,975 - dataprep_main.py - DEBUG - 156 - shape: (104842, 260) shops: 113 start: 2022-03-01 end: 2025-04-30 sumladenvkp: 196830741