2025-04-07 08:08:27,886 - data_prep_class.py - INFO - 98 - Get data from Big Query with SELECT firma, datum, fg_sg, fg_bez, ladenvkp, katalogvkp, oeffnungszeit, fg_typ
FROM `test-ww-ki-witt.co_stat_daily_prognosis_bronzelayer.sales`
2025-04-07 08:08:33,569 - data_prep_class.py - INFO - 107 - firma datum fg_sg fg_bez ladenvkp katalogvkp oeffnungszeit fg_typ date
0 Sieh an! 2023-03-06 20 Furth im Wald 65.99 183.99 Montag bis Freitag von 09.00 bis 18.00 Uhr | S... Preislandgeschäft 2023-03-06
1 Witt 2023-03-06 20 Furth im Wald 1166.12 3224.83 Montag bis Freitag von 09.00 bis 18.00 Uhr | S... Preislandgeschäft 2023-03-06
2 Sieh an! 2022-10-05 20 Furth im Wald 137.32 223.98 Montag bis Freitag von 09.00 bis 18.00 Uhr | S... Preislandgeschäft 2022-10-05
3 Witt 2022-10-05 20 Furth im Wald 1119.74 2497.91 Montag bis Freitag von 09.00 bis 18.00 Uhr | S... Preislandgeschäft 2022-10-05
4 Sieh an! 2022-10-05 51 Frankenberg 10.00 20.00 Montag bis Freitag von 09.00 bis 18.00 Uhr | S... Preislandgeschäft 2022-10-05
2025-04-07 08:08:33,582 - dataprep_main.py - DEBUG - 39 - shape: (270522, 9) shops: 122 start: 2021-01-04 end: 2025-04-04 sumladenvkp: 255342707
2025-04-07 08:08:33,644 - data_prep_class.py - DEBUG - 137 - output of shops and codes found in sales data to fg_sg_bez.xlsx
2025-04-07 08:08:33,950 - dataprep_main.py - DEBUG - 50 - shape: (131754, 3) shops: 122 start: 2021-01-04 end: 2025-04-04 sumladenvkp: 255342707
2025-04-07 08:08:34,060 - data_prep_class.py - DEBUG - 180 - max date in dataframe: 2025-04-04 00:00:00
2025-04-07 08:08:34,061 - data_prep_class.py - DEBUG - 193 - date_range: 2025-04-05 00:00:00 2025-06-30 00:00:00
2025-04-07 08:08:34,153 - data_prep_class.py - DEBUG - 206 - size of new dataset: (10614, 3) vs 10614
2025-04-07 08:08:34,163 - data_prep_class.py - DEBUG - 210 - final size: (142368, 3)
2025-04-07 08:08:34,166 - dataprep_main.py - DEBUG - 58 - shape: (142368, 3) shops: 122 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 255342707
2025-04-07 08:08:34,424 - dataprep_main.py - DEBUG - 69 - shape: (138179, 24) shops: 113 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 251277499
2025-04-07 08:08:34,424 - data_prep_class.py - DEBUG - 255 - check Friedrichshafen
2025-04-07 08:08:34,437 - data_prep_class.py - DEBUG - 259 - date fg_sg fg_bez ladenvkp
119467 2024-01-02 183 Friedrichshafen 643.92
2025-04-07 08:09:09,442 - dataprep_main.py - DEBUG - 76 - shape: (138179, 24) shops: 113 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 251277499
2025-04-07 08:09:09,459 - dataprep_main.py - DEBUG - 80 - shape: (138179, 24) shops: 113 start: 2021-01-04 end: 2025-06-30 sumladenvkp: 251277499
2025-04-07 08:09:16,687 - dataprep_main.py - DEBUG - 94 - shape: (115311, 38) shops: 113 start: 2022-01-03 end: 2025-06-30 sumladenvkp: 213478292
2025-04-07 08:09:16,688 - data_prep_class.py - DEBUG - 216 - before removing holidays: (115311, 38)
2025-04-07 08:09:16,701 - data_prep_class.py - DEBUG - 218 - aftere removing holidays: (115311, 38)
2025-04-07 08:09:16,705 - dataprep_main.py - DEBUG - 99 - shape: (115311, 38) shops: 113 start: 2022-01-03 end: 2025-06-30 sumladenvkp: 213478292
2025-04-07 08:09:17,096 - data_prep_class.py - DEBUG - 476 - missing values last year: lastyear_date 37070
lastyear_ladenvkp 37070
lastyear_fg_sg 37070
dtype: int64
2025-04-07 08:09:18,604 - dataprep_main.py - DEBUG - 138 - shape: (115311, 259) shops: 113 start: 2022-01-03 end: 2025-06-30 sumladenvkp: 213478292
2025-04-07 08:09:27,985 - data_prep_class.py - DEBUG - 433 - merge on agp - missing: lastyear_date 31925
lastyear_ladenvkp 31925
lastyear_fg_sg 31925
imputed_lastyear_ladenvkp 28931
imputed2_lastyear_ladenvkp 27534
av_lastyear_ladenvkp 27534
agp 8038
dtype: int64
2025-04-07 08:09:27,989 - dataprep_main.py - DEBUG - 146 - shape: (110166, 260) shops: 113 start: 2022-03-01 end: 2025-06-30 sumladenvkp: 205813825
2025-04-07 08:09:27,991 - data_prep_class.py - DEBUG - 440 - fill missing agp: 0
2025-04-07 08:09:27,994 - dataprep_main.py - DEBUG - 151 - shape: (110166, 260) shops: 113 start: 2022-03-01 end: 2025-06-30 sumladenvkp: 205813825