2025-04-02 12:15:37,974 - data_prep_class.py - INFO - 95 - Get data from Big Query with SELECT firma, datum, fg_sg, fg_bez, ladenvkp, katalogvkp, oeffnungszeit, fg_typ
FROM `test-ww-ki-witt.co_stat_daily_prognosis_bronzelayer.sales`
2025-04-02 12:15:43,180 - data_prep_class.py - INFO - 104 - firma datum fg_sg fg_bez ladenvkp katalogvkp oeffnungszeit fg_typ date
0 Sieh an! GmbH 2024-09-13 21 Neuburg a. d. Donau 39.49 67.00 Montag bis Freitag von 09.00 bis 18.00 Uhr | k... Basisfiliale 2024-09-13
1 Josef Witt GmbH 2024-09-13 21 Neuburg a. d. Donau 2101.97 2159.53 Montag bis Freitag von 09.00 bis 18.00 Uhr | k... Basisfiliale 2024-09-13
2 Heinrich Heine GmbH 2024-09-13 193 Dingolfing 134.96 149.97 Montag bis Freitag von 9.00 bis 18.00 Uhr | Sa... Basisfiliale 2024-09-13
3 Sieh an! GmbH 2024-09-13 193 Dingolfing 194.90 245.00 Montag bis Freitag von 9.00 bis 18.00 Uhr | Sa... Basisfiliale 2024-09-13
4 Josef Witt GmbH 2024-09-13 193 Dingolfing 1488.82 1668.76 Montag bis Freitag von 9.00 bis 18.00 Uhr | Sa... Basisfiliale 2024-09-13
2025-04-02 12:15:43,195 - dataprep_main.py - DEBUG - 39 - shape: (269726, 9) shops: 122 start: 2021-01-04 end: 2025-04-01 sumladenvkp: 254785264
2025-04-02 12:15:43,261 - data_prep_class.py - DEBUG - 134 - output of shops and codes found in sales data to fg_sg_bez.xlsx
2025-04-02 12:15:43,678 - dataprep_main.py - DEBUG - 50 - shape: (131423, 3) shops: 122 start: 2021-01-04 end: 2025-04-01 sumladenvkp: 254785264
2025-04-02 12:15:43,678 - data_prep_class.py - DEBUG - 177 - max date in dataframe: 2025-04-01 00:00:00
2025-04-02 12:15:43,679 - data_prep_class.py - DEBUG - 190 - date_range: 2025-04-02 00:00:00 2025-06-30 00:00:00
2025-04-02 12:15:43,783 - data_prep_class.py - DEBUG - 203 - size of new dataset: (10980, 3) vs 10980
2025-04-02 12:15:43,795 - data_prep_class.py - DEBUG - 207 - final size: (142403, 3)
2025-04-02 12:15:43,799 - dataprep_main.py - DEBUG - 58 - shape: (142403, 3) shops: 122 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 254785264
2025-04-02 12:15:44,086 - dataprep_main.py - DEBUG - 69 - shape: (138187, 24) shops: 113 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 250720056
2025-04-02 12:15:44,086 - data_prep_class.py - DEBUG - 252 - check Friedrichshafen
2025-04-02 12:15:44,101 - data_prep_class.py - DEBUG - 256 - date fg_sg fg_bez ladenvkp
119169 2024-01-02 183 Friedrichshafen 643.92
2025-04-02 12:16:20,786 - dataprep_main.py - DEBUG - 76 - shape: (138187, 24) shops: 113 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 250720056
2025-04-02 12:16:20,809 - dataprep_main.py - DEBUG - 80 - shape: (138187, 24) shops: 113 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 250720056
2025-04-02 12:16:28,129 - dataprep_main.py - DEBUG - 94 - shape: (115313, 38) shops: 113 start: 2022-01-03 end: 2025-06-30 sumladenvkp: 212920849
2025-04-02 12:16:28,130 - data_prep_class.py - DEBUG - 213 - before removing holidays: (115313, 38)
2025-04-02 12:16:28,147 - data_prep_class.py - DEBUG - 215 - aftere removing holidays: (115313, 38)
2025-04-02 12:16:28,151 - dataprep_main.py - DEBUG - 99 - shape: (115313, 38) shops: 113 start: 2022-01-03 end: 2025-06-30 sumladenvkp: 212920849
2025-04-02 12:16:28,588 - data_prep_class.py - DEBUG - 468 - missing values last year: lastyear_date 37070
lastyear_ladenvkp 37070
lastyear_fg_sg 37070
dtype: int64
2025-04-02 12:16:30,125 - dataprep_main.py - DEBUG - 138 - shape: (115313, 259) shops: 113 start: 2022-01-03 end: 2025-06-30 sumladenvkp: 212920849
2025-04-02 12:16:39,942 - data_prep_class.py - DEBUG - 425 - merge on agp - missing: lastyear_date 31925
lastyear_ladenvkp 31925
lastyear_fg_sg 31925
imputed_lastyear_ladenvkp 28931
imputed2_lastyear_ladenvkp 27534
av_lastyear_ladenvkp 27534
agp 8040
dtype: int64
2025-04-02 12:16:39,946 - dataprep_main.py - DEBUG - 146 - shape: (110168, 260) shops: 113 start: 2022-03-01 end: 2025-06-30 sumladenvkp: 205256381
2025-04-02 12:16:39,948 - data_prep_class.py - DEBUG - 432 - fill missing agp: 0
2025-04-02 12:16:39,951 - dataprep_main.py - DEBUG - 151 - shape: (110168, 260) shops: 113 start: 2022-03-01 end: 2025-06-30 sumladenvkp: 205256381