[06:46:31] 06Data-Engineering, 06Product-Analytics: dbt: Weekly scheduling dependency checks only check the first week of the date - https://phabricator.wikimedia.org/T436744 (10KCVelaga_WMF) 03NEW [07:36:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Collaboration-Services, 06Reader Growth Team: Structured Data packages size on GitLab - https://phabricator.wikimedia.org/T435199#12278925 (10ABran-WMF) [07:47:14] 06Data-Engineering, 06Data-Engineering-Radar, 06Traffic: Normalize URI host in Turnilo's webrequest_sampled_live - https://phabricator.wikimedia.org/T434766#12278956 (10hashar) Awesome, thank you @Fabfur ! [07:49:42] (03CR) 10Joal: [C:03+1] "Code looks god to me, but I think we need to manually delete the data projects like '%test%'from the existing table for the effect to be m" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1333219 (https://phabricator.wikimedia.org/T436658) (owner: 10Milimetric) [08:17:28] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12279078 (10brouberol) Alright, I'll deploy a new image and we'll deal with the... [08:23:20] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12279087 (10brouberol) a:03brouberol [08:23:26] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12279089 (10brouberol) 05Open→03In progress [08:32:07] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12279146 (10brouberol) Now that the patch has been deployed, I will stop all ong... [08:39:14] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12279179 (10brouberol) I've now stopped all dump jobs. Before re-running them wi... [09:01:49] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12279250 (10brouberol) {F101365107} Errors have stopped. I'm going to regress th... [09:02:19] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12279251 (10brouberol) p:05Unbreak!→03High [10:07:45] (03CR) 10Milimetric: [V:03+2 C:03+2] "yep, I did that with an insert overwrite. And overwrote that `/wmf/data/wmf/mediawiki/database/grouped_wikis.csv` file too." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1333219 (https://phabricator.wikimedia.org/T436658) (owner: 10Milimetric) [11:57:31] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12280173 (10brouberol) No new errors have crept up in the past 2 hours. If this... [12:02:15] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Move webrequest sequence statistics to a dashboard - https://phabricator.wikimedia.org/T436700#12280179 (10Milimetric) [13:03:39] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12280536 (10JMonton-WMF) We had a meeting yesterday to talk about this topi... [13:57:27] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12280896 (10brouberol) No errors in the last 5 hours. Closing. [13:57:33] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12280898 (10brouberol) 05In progress→03Resolved [14:04:04] 06Data-Engineering, 06Data-Engineering-Icebox, 10Tool-Pageviews: Pageviews Analysis 2026 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#12280946 (10MusikAnimal) [14:24:55] 06Data-Engineering, 10Test Kitchen, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 13Patch-For-Review: Airflow instance for Experiment Platform - https://phabricator.wikimedia.org/T416709#12281066 (10atsuko) [14:27:23] 06Data-Engineering, 10Test Kitchen, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 13Patch-For-Review: Airflow instance for Experiment Platform - https://phabricator.wikimedia.org/T416709#12281078 (10atsuko) Instance https://airflow-experiment-platform.wikimedia.org/ is up. @mpopov should I close the task,... [14:34:33] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Relative Trending - Milestone 4 - Storage - https://phabricator.wikimedia.org/T431555#12281133 (10JMonton-WMF) We have changed the design of the storage, as described in https://phabricator.wikimedia.org/T432352#... [15:15:04] 06Data-Engineering, 10Test Kitchen, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 13Patch-For-Review: Airflow instance for Experiment Platform - https://phabricator.wikimedia.org/T416709#12281403 (10atsuko) 05In progress→03Resolved [15:15:45] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12281406 (10Ottomata) > Change our Flink application to produce a new outpu... [15:23:52] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12281548 (10JMonton-WMF) Yes, the regular output doesn't change, we create... [16:32:27] 06Data-Engineering, 06Product-Analytics: dbt: Weekly scheduling dependency checks only check the first week of the date - https://phabricator.wikimedia.org/T436744#12281838 (10amastilovic) [16:33:48] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting Kerberos access for kamila - https://phabricator.wikimedia.org/T436703#12281847 (10Ahoelzl) Approved. [16:34:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Documentation: Complete Documentation of Article Feature Counts - https://phabricator.wikimedia.org/T436717#12281851 (10Ahoelzl) [16:34:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Documentation: Create Article Feature Counts Wikitech page - https://phabricator.wikimedia.org/T436653#12281853 (10Ahoelzl) [16:34:37] 06Data-Engineering, 06Product-Analytics: dbt: Weekly scheduling dependency checks only check the first week of the date - https://phabricator.wikimedia.org/T436744#12281854 (10GGoncalves-WMF) a:03amastilovic [16:36:23] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Commons-Impact-Metrics, 10Commons-Impact-Metrics-Requests: Update Commons Impact Metrics allow-list August 2026 - https://phabricator.wikimedia.org/T436416#12281860 (10Ahoelzl) [16:37:27] 06Data-Engineering, 06Traffic: Add backend information to webrequests in data lake - https://phabricator.wikimedia.org/T436223#12281866 (10Ahoelzl) a:03GGoncalves-WMF [16:37:47] 06Data-Engineering, 06Traffic: Add backend information to webrequests in data lake - https://phabricator.wikimedia.org/T436223#12281869 (10Ahoelzl) @GGoncalves-WMF can you triage please? [16:39:02] 06Data-Engineering, 06Data-Platform-SRE: Enable Superset feature flag for Alerts - https://phabricator.wikimedia.org/T430682#12281881 (10Ahoelzl) a:05Ahoelzl→03None [16:39:11] 06Data-Engineering, 07Essential-Work: Revive data engineering alert metrics dashboard - https://phabricator.wikimedia.org/T399518#12281883 (10Ahoelzl) a:05Ahoelzl→03None [16:40:06] 06Data-Engineering, 06Product-Analytics: dbt: Weekly scheduling dependency checks only check the first date of the week - https://phabricator.wikimedia.org/T436744#12281890 (10KCVelaga_WMF) [16:46:40] 06Data-Engineering, 06Data-Platform-SRE, 10WMDE Analytics: [Analytics] [Tech Debt] Switch bi-yearly data cleanup email DAG to email all cluster users - https://phabricator.wikimedia.org/T436324#12281932 (10Ahoelzl) [16:47:38] 06Data-Engineering, 06Data-Platform-SRE, 10WMDE Analytics: [Analytics] [Tech Debt] Switch bi-yearly data cleanup email DAG to email all cluster users - https://phabricator.wikimedia.org/T436324#12281944 (10Ottomata) #data-platform-sre is this something yall would consider taking on? I'm not sure if every ai... [16:48:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 07Wikimedia-production-error: "User::loadFromSession called before the end of Setup.php" when EventBus is serializing user groups - https://phabricator.wikimedia.org/T436345#12281949 (10Ahoelzl) p:05Triage→03High a:03Ottomata [16:49:24] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Wikimedia-production-error: dumps are generating 'PHP Warning: Undefined array key x4' errors - https://phabricator.wikimedia.org/T436663#12281954 (10dancy) @brouberol Thanks! [16:50:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Replace deployment-schema-3 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436463#12281978 (10Ahoelzl) a:03tchin [16:51:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Replace deployment-eventgate-4 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436474#12281989 (10Ahoelzl) a:03tchin [16:51:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Replace deployment-schema-3 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436463#12281993 (10Ottomata) →14Duplicate dup:03T429497 [16:51:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06cloud-services-team, 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12281995 (10Ottomata) [16:52:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12282004 (10taavi) [16:52:22] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12282005 (10Ottomata) [16:54:23] 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12282038 (10Ahoelzl) @GGoncalves-WMF is this related to dbt project structuring? [16:54:25] 06Data-Engineering, 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Replace deployment-eventstreams-2 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436475#12282035 (10Ottomata) →14Duplicate dup:03T429497 [16:54:27] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12282039 (10Ottomata) [16:55:06] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Replace deployment-eventgate-4 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436474#12282050 (10Ottomata) →14Duplicate dup:03T429497 [16:55:10] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12282052 (10Ottomata) [16:56:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12282067 (10Ottomata) @Southparkfan thanks for the tasks! For ease of project... [16:56:40] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12282075 (10ppelberg) [16:58:19] 06Data-Engineering: Investigate using --partial-parse for further isolating dbt models in different scheduling groups - https://phabricator.wikimedia.org/T436834 (10GGoncalves-WMF) 03NEW [16:59:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Linked Artifact Cache - Automated MediaWiki JobQueue precompute - https://phabricator.wikimedia.org/T436695#12282122 (10Ahoelzl) a:03Ottomata [17:00:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics: dbt: Weekly scheduling dependency checks only check the first date of the week - https://phabricator.wikimedia.org/T436744#12282127 (10Ahoelzl) [17:13:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 07Wikimedia-production-error: "User::loadFromSession called before the end of Setup.php" when EventBus is serializing user groups - https://phabricator.wikimedia.org/T436345#12282289 (10Dreamy_Jazz) (Looking further this may be fixe... [17:19:07] (03PS1) 10Milimetric: Refresh sitematrix parsing [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1333961 (https://phabricator.wikimedia.org/T411938) [17:19:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12282313 (10ppelberg) [17:22:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Engineering-Wikistats, 13Patch-For-Review: The stat site show INVALID on Cantonese Wikipedia. - https://phabricator.wikimedia.org/T411938#12282340 (10Milimetric) [17:23:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Engineering-Wikistats, 13Patch-For-Review: The stat site show INVALID on Cantonese Wikipedia. - https://phabricator.wikimedia.org/T411938#12282350 (10Milimetric) These parent tasks that I added are very worth checking out, @Ahoelzl. I won't h... [17:23:43] (03CR) 10Milimetric: [C:03+2] "it builds, ship it!" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1333961 (https://phabricator.wikimedia.org/T411938) (owner: 10Milimetric) [17:24:46] (03Merged) 10jenkins-bot: Refresh sitematrix parsing [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1333961 (https://phabricator.wikimedia.org/T411938) (owner: 10Milimetric) [17:26:26] (03PS1) 10Milimetric: Release 2.13.0 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1333964 [17:28:48] (03CR) 10Milimetric: [C:03+2] Release 2.13.0 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1333964 (owner: 10Milimetric) [17:29:56] (03Merged) 10jenkins-bot: Release 2.13.0 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1333964 (owner: 10Milimetric) [18:49:11] 06Data-Engineering, 06Data-Engineering-Icebox, 10Tool-Pageviews: Pageviews Analysis 2026 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#12282719 (10MusikAnimal) 05In progress→03Resolved [19:29:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 07Wikimedia-production-error: "User::loadFromSession called before the end of Setup.php" when EventBus is serializing user groups - https://phabricator.wikimedia.org/T436345#12282845 (10Ottomata) Okay thank you! FWIW, I hope that {... [19:30:15] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): IR2.1.2d: Feature count data product [Attribution API] - https://phabricator.wikimedia.org/T432604#12282861 (10AKhatun_WMF) [21:06:53] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12283145 (10Southparkfan) >>! In T429497#12282039, @Ottomata wrote: > @Southp... [23:10:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Fix dataset.yaml entries for wmf_raw.mediawiki.* tables - https://phabricator.wikimedia.org/T436879 (10amastilovic) 03NEW