[00:40:37] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Load Google Search Console data into the Data Lake - https://phabricator.wikimedia.org/T420996#12259987 (10nshahquinn-wmf) @Ahoelzl did you see my proposal T436193? Let's discuss there 😊 [01:11:21] 06Data-Engineering: Load Google Search Console data from BigQuery as soon as it's available and reload when data corrections occur - https://phabricator.wikimedia.org/T436193#12260044 (10nshahquinn-wmf) >>! In T420996#12259661, @Ahoelzl wrote: > @nshahquinn-wmf Would an immediate daily update paired with a 2-4 d... [01:14:52] 06Data-Engineering: Load Google Search Console data from BigQuery as soon as it's available and reload when data corrections occur - https://phabricator.wikimedia.org/T436193#12260046 (10nshahquinn-wmf) > So we'll need to pick how long to retain the data in BigQuery based on how long we want to be able to receiv... [01:17:43] 06Data-Engineering: Load Google Search Console data from BigQuery as soon as it's available and reload when data corrections occur - https://phabricator.wikimedia.org/T436193#12260064 (10nshahquinn-wmf) [05:57:09] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06Wikimedia Enterprise: Provide auth-less access to Enterprise APIs from WMF Analytics cluster - https://phabricator.wikimedia.org/T403298#12260329 (10awight) I think there is a lack of clarity around ownership of this work. In [[ https://wikimedia... [09:42:18] 06Data-Engineering, 06Traffic: Add backend information to webrequests in data lake - https://phabricator.wikimedia.org/T436223 (10Clement_Goubert) 03NEW [10:42:13] 06Data-Engineering, 10Test Kitchen, 06Data-Platform-SRE (2026-08-07 - 2026-08-28), 13Patch-For-Review: Airflow instance for Experiment Platform - https://phabricator.wikimedia.org/T416709#12261013 (10atsuko) [] Create the HDFS folders [] Configure the new instance in airflow-dags repo [] configure ldap `ai... [11:35:36] 06Data-Engineering, 10AbuseFilter, 07Schema-change: Create afl_reverted column in abuse_filter_log table - https://phabricator.wikimedia.org/T436239 (10Dragoniez) 03NEW [12:26:42] (03PS2) 10Mforns: Switch pageview_actor consumers to use webrequest_v2 instead. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) [13:24:47] (03CR) 10Joal: "The name of the field doesn't match the one defined in the linked PR: `is_thumb_generated` vs `thumb_generated`." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1328608 (https://phabricator.wikimedia.org/T435634) (owner: 10Ladsgroup) [13:55:41] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 13Patch-For-Review: migrate dbt_jobs pipelines to use MWH incremental - https://phabricator.wikimedia.org/T429770#12261640 (10APizzata-WMF) [13:56:03] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 13Patch-For-Review: migrate dbt_jobs pipelines to use MWH incremental - https://phabricator.wikimedia.org/T429770#12261642 (10APizzata-WMF) [14:06:23] (03PS3) 10Mforns: Switch pageview_actor consumers to use webrequest_v2 instead. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) [14:19:52] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10MediaWiki-extensions-PageAssessments, 06Content-Platform-Team (Work In Progress), 07Essential-Work, and 3 others: PHP Warning: MediaWiki\ResourceLoader\Context::encodeJson failed to encode "wgPag... - https://phabricator.wikimedia.org/T432770#12261759 [15:56:04] (03CR) 10Ladsgroup: "I already did that in I743845a27 and that extract the field as is_thumb_generated. Should that be merged first before this?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1328608 (https://phabricator.wikimedia.org/T435634) (owner: 10Ladsgroup) [15:57:02] 06Data-Engineering, 10WMDE Analytics: Airflow testing instances made with airflow-devenv aren't able to send emails - https://phabricator.wikimedia.org/T436270 (10AndrewTavis_WMDE) 03NEW [16:40:31] (03CR) 10Joal: [C:03+1] "Great! I had missed the other patch :) I prefer to have the other merged first, validate the data looking at kafka, and then merge this on" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1328608 (https://phabricator.wikimedia.org/T435634) (owner: 10Ladsgroup) [18:09:22] (03CR) 10Ladsgroup: "Thanks! I go ping Luca about it." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1328608 (https://phabricator.wikimedia.org/T435634) (owner: 10Ladsgroup) [19:03:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [19:43:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [19:58:01] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform: Create new mediawiki.tags_change event stream - https://phabricator.wikimedia.org/T434143#12263101 (10tchin) I added a `LogRecord` immutable value class under T434143. Unfortunately, it seems like `RecentChanges` also does not have a... [21:12:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [22:02:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [22:10:27] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Ingest developer metrics related data sources - https://phabricator.wikimedia.org/T431155#12263749 (10Ahoelzl) →14Duplicate dup:03T433683 [22:10:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Make developer metrics related data sets available in the data lake - https://phabricator.wikimedia.org/T433683#12263751 (10Ahoelzl) [22:12:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Make developer metrics related data sets available in the data lake - https://phabricator.wikimedia.org/T433683#12263758 (10Ahoelzl) [22:15:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Make developer metrics related data sets available in the data lake - https://phabricator.wikimedia.org/T433683#12263763 (10Ahoelzl) [23:52:51] 06Data-Engineering, 06Data-Engineering-Radar, 06Commons, 06Data-Persistence, and 5 others: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) - https://phabricator.wikimedia.org/T28741#12263855 (10Bawolff) Just mentioning here for visibility - There was a bug in fi...