[06:34:17] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=eqiad%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [06:39:17] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=eqiad%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [09:46:39] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop cl_to and cl_collation from categorylinks in wmf production - https://phabricator.wikimedia.org/T402925#11184054 (10Ladsgroup) [09:50:06] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop cl_to and cl_collation from categorylinks in wmf production - https://phabricator.wikimedia.org/T402925#11184070 (10Ladsgroup) [10:57:45] 06Data-Engineering, 06Data-Engineering-Radar, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Add default value for afl_ip and remove default value for afl_ip_hex in abuse_filter_log table - https://phabricator.wikimedia.org/T401906#11184418 (10Ladsgroup) [11:10:26] (03PS1) 10Gerrit maintenance bot: Add ms.wikiquote to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1188748 (https://phabricator.wikimedia.org/T404700) [13:19:33] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for later deploy. Fixed in production." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1188401 (https://phabricator.wikimedia.org/T401666) (owner: 10Joal) [13:51:40] 06Data-Engineering, 06Traffic: Reduce noise from duplicate sequence-gap alerts on HaProxy-webrequests - https://phabricator.wikimedia.org/T401383#11185133 (10Fabfur) We deployed a change in HAProxy logging (see T403176) to avoid sending non-utf8 encoded headers to DLQ, this *could* also affect this issue as we... [14:18:35] (03PS3) 10Ottomata: Sqoop CentralAuth's globaluser and localuser tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1184925 (https://phabricator.wikimedia.org/T389666) [14:18:40] (03CR) 10Ottomata: Sqoop CentralAuth's globaluser and localuser tables (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1184925 (https://phabricator.wikimedia.org/T389666) (owner: 10Ottomata) [14:44:29] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11185422 (10Eevans) [14:46:38] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Data-Persistence, 10Data-Persistence-Design-Review: Data Persistence Design Review: Global Editor Metrics (YiR, Impact Module, App Activity Tab) - https://phabricator.wikimedia.org/T401260#11185428 (10Eevans) [15:26:03] (03PS4) 10Ottomata: Sqoop CentralAuth's globaluser and localuser tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1184925 (https://phabricator.wikimedia.org/T389666) [15:49:10] 06Data-Engineering, 06Infrastructure-Foundations, 13Patch-For-Review: proposal: allow analytics-admins to also trigger puppet runs - https://phabricator.wikimedia.org/T404630#11185750 (10CDanis) [16:01:53] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Fix CommonsCategoryGraphBuilder to reflect latest changes to categorylinks table - https://phabricator.wikimedia.org/T404735 (10amastilovic) 03NEW [16:10:49] (03CR) 10Ottomata: "I tested this! Output data can be found in HDFS at" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1184925 (https://phabricator.wikimedia.org/T389666) (owner: 10Ottomata) [16:15:32] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Experimentation Lab, 13Patch-For-Review: NEW/CHANGE FEATURE REQUEST: make available the centralauth.globaluser table in Data Lake - https://phabricator.wikimedia.org/T389666#11185908 (10Ottomata) I was able to test my sqoop patch and get globaluser... [16:42:34] (03PS5) 10Ottomata: Sqoop CentralAuth's globaluser and localuser tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1184925 (https://phabricator.wikimedia.org/T389666) [16:43:27] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Experimentation Lab, 13Patch-For-Review: NEW/CHANGE FEATURE REQUEST: make available the centralauth.globaluser table in Data Lake - https://phabricator.wikimedia.org/T389666#11185993 (10Ottomata) Just tested with Hive tables too. Looks good. [18:42:18] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Spark jobs do not honor default conf - https://phabricator.wikimedia.org/T404754#11186542 (10xcollazo) [20:35:36] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10Event-Platform: mediawiki.content_history: flink applications experiencing frequent restarts due to JobManager OOMs - https://phabricator.wikimedia.org/T397330#11187169 (10tchin) I'm going to try to upgrade `mw-content-history-reconcile-enrich-next`... [20:39:06] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Growth-Team, 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog: Global Editor Metrics for YiR, Apps Activity Tab, and Growth Impact Module - https://phabricator.wikimedia.org/T403660#11187203 (10Ottomata) > Could we instead: > count to... [21:00:35] 06Data-Engineering, 10CampaignEvents, 06DBA, 10Connection-Team (Connection-Current-Sprint), 07Schema-change-in-production: Create DB column campaign_events.event_track_contributions in production (x1) - https://phabricator.wikimedia.org/T404789 (10Daimona) 03NEW [21:17:47] 06Data-Engineering, 06SRE, 06Traffic-Icebox, 10MobileFrontend (Tracking): QA features on the new mobile URLs - https://phabricator.wikimedia.org/T403638#11187380 (10vaughnwalters) >>! In T403638#11178012, @vaughnwalters wrote: > I made a note of this when testing the campaign events extension T404244#11177... [21:20:45] 06Data-Engineering, 10CampaignEvents, 06DBA, 10Connection-Team (Connection-Current-Sprint), 07Schema-change-in-production: Create DB column campaign_events.event_track_contributions in production (x1) - https://phabricator.wikimedia.org/T404789#11187400 (10Daimona) [21:20:59] 06Data-Engineering, 10CampaignEvents, 06DBA, 10Connection-Team (Connection-Current-Sprint), 07Schema-change-in-production: Create DB column campaign_events.event_track_contributions in production (x1) - https://phabricator.wikimedia.org/T404789#11187401 (10Daimona) [21:21:03] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence (work done), and 4 others: Update DB schema to store whether contribution tracking is enabled for a given event - https://phabricator.wikimedia.org/T402816#11187402 (10Daimona) [21:21:51] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence (work done), and 4 others: Update DB schema to store whether contribution tracking is enabled for a given event - https://phabricator.wikimedia.org/T402816#11187410 (10Daimona) [21:21:54] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence (work done), and 4 others: Update DB schema to store whether contribution tracking is enabled for a given event - https://phabricator.wikimedia.org/T402816#11187413 (10Daimona) [23:15:28] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Growth-Team, 10MediaWiki-Page-derived-data, 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog: Global Editor Metrics for YiR, Apps Activity Tab, and Growth Impact Module - https://phabricator.wikimedia.org/T403660#11187732 (10Ottomata) [23:23:50] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10Event-Platform: mediawiki.content_history: flink applications experiencing frequent restarts due to JobManager OOMs - https://phabricator.wikimedia.org/T397330#11187737 (10Ottomata) > Updating to 1.20 would mean running it on experimental support for... [23:25:21] 06Data-Engineering, 06Discovery-Search, 07Epic, 07Essential-Work: [EPIC] Upgrade flink jobs to java 17 - https://phabricator.wikimedia.org/T404340#11187739 (10Ottomata) IIUC, recent 1.20.2 [[ https://docker-registry.wikimedia.org/flink/tags/ | flink image versions ]] are only Java 17? Could we consider a... [23:26:08] 06Data-Engineering, 06Discovery-Search, 07Epic, 07Essential-Work: [EPIC] Upgrade flink jobs to java 17 - https://phabricator.wikimedia.org/T404340#11187743 (10Ottomata) Oh, or do we need T401694 first?