[08:27:43] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): change metric name for prometheus slo - https://phabricator.wikimedia.org/T411973 (10APizzata-WMF) 03NEW [09:20:48] 14Analytics, 06Data-Engineering, 10Data-Engineering-Wikistats: دعانویس تضمینی 09051560602 - https://phabricator.wikimedia.org/T411976 (10doajateltelesm) 03NEW [09:22:32] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): refine_to_hive dag optimizations - https://phabricator.wikimedia.org/T392668#11439389 (10Antoine_Quhen) [09:23:20] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Move more of refine_hive_hourly dag logic into RefineConfiguration - https://phabricator.wikimedia.org/T375064#11439396 (10Antoine_Quhen) [09:23:20] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): refine_to_hive dag optimizations - https://phabricator.wikimedia.org/T392668#11439397 (10Antoine_Quhen) [09:26:54] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Reduce `refine_to_hive_hourly` airflow task number - https://phabricator.wikimedia.org/T380856#11439423 (10Antoine_Quhen) 05Open→03Resolved We have already merged 2 features to improve on that: * all the preparation tasks are gone * all evolve+... [09:53:01] 06Data-Engineering: Airflow main instance optimization - https://phabricator.wikimedia.org/T411988 (10Antoine_Quhen) 03NEW [09:55:15] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): refine_to_hive dag optimizations - https://phabricator.wikimedia.org/T392668#11439649 (10Antoine_Quhen) [09:58:54] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Optimize canary event generation resources consumption on Airflow - https://phabricator.wikimedia.org/T411989 (10Antoine_Quhen) 03NEW [10:00:42] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Analyze and optimize Airflow Postgres backend performance - https://phabricator.wikimedia.org/T411990 (10Antoine_Quhen) 03NEW [10:02:19] 06Data-Engineering: Reduce main Airflow DB size and consider splitting heavy workloads into separate instances - https://phabricator.wikimedia.org/T411992 (10Antoine_Quhen) 03NEW [10:02:34] 06Data-Engineering: Airflow main instance optimization - https://phabricator.wikimedia.org/T411988#11439721 (10Antoine_Quhen) [10:02:35] 06Data-Engineering: Reduce main Airflow DB size and consider splitting heavy workloads into separate instances - https://phabricator.wikimedia.org/T411992#11439722 (10Antoine_Quhen) [10:02:37] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Analyze and optimize Airflow Postgres backend performance - https://phabricator.wikimedia.org/T411990#11439723 (10Antoine_Quhen) [10:02:38] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Optimize canary event generation resources consumption on Airflow - https://phabricator.wikimedia.org/T411989#11439724 (10Antoine_Quhen) [10:02:41] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): refine_to_hive dag optimizations - https://phabricator.wikimedia.org/T392668#11439725 (10Antoine_Quhen) [10:02:42] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Move more of refine_hive_hourly dag logic into RefineConfiguration - https://phabricator.wikimedia.org/T375064#11439726 (10Antoine_Quhen) [10:13:04] 14Analytics, 06Data-Engineering, 10Data-Engineering-Wikistats: دعانویس تضمینی 09051560602 - https://phabricator.wikimedia.org/T411976#11439765 (10Bugreporter) 05Open→03Invalid [10:28:19] 06Data-Engineering: Migrate cleanup jobs for snapshot datasets from systemd timers to Airflow - https://phabricator.wikimedia.org/T411999 (10Antoine_Quhen) 03NEW [10:50:53] 06Data-Engineering: Airflow main instance optimization - https://phabricator.wikimedia.org/T411988#11439918 (10BTullis) [13:18:35] 06Data-Engineering: Reduce main Airflow DB size and consider splitting heavy workloads into separate instances - https://phabricator.wikimedia.org/T411992#11440274 (10BTullis) We're also investigating the possibility of the Airflow metadata DB being implicated in: {T412003} For reference, we also have an Airflo... [13:35:44] !log purged airflow-main database records older than 6 months for T412003 [13:35:47] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:35:47] T412003: Airflow-main scheduler loop sometimes slows down markedly - https://phabricator.wikimedia.org/T412003 [15:57:53] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Troubleshoot duplicates issue in mw_content_merge_events_to_mw_content_history_daily - https://phabricator.wikimedia.org/T410431#11440818 (10APizzata-WMF) Current situation after the monthly reconciliation (query executed on 2025-12-08): ` spark.sql(""... [16:24:04] 06Data-Engineering, 10Dumps-Generation, 10MediaWiki-Core-Snapshots, 07Wikimedia-production-error: PHP Notice: fwrite(): write of X bytes failed with errno=32 Broken pipe (via TextPassDumper) - https://phabricator.wikimedia.org/T377136#11440951 (10Umherirrender) [16:31:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): GobblinLastSuccessfulRunTooLongAgo alerts - https://phabricator.wikimedia.org/T406526#11440996 (10Antoine_Quhen) 05Open→03Resolved [16:37:12] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Upgrade Airflow HdfsEmailOperator to take both a String or a List(String) email addresses. - https://phabricator.wikimedia.org/T412035 (10Snwachukwu) 03NEW [16:38:55] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Upgrade Airflow HdfsEmailOperator to take both a String or a List(String) email addresses. - https://phabricator.wikimedia.org/T412035#11441041 (10Snwachukwu) [16:39:33] 06Data-Engineering: Productize Data for Monthly Active Moderator Actions - https://phabricator.wikimedia.org/T410940#11441043 (10GGoncalves-WMF) @fkaelin and I just chatted a little more about this, quoting here: > Here are the datasets, > - [[ https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPl... [16:40:00] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Troubleshoot duplicates issue in mw_content_merge_events_to_mw_content_history_daily - https://phabricator.wikimedia.org/T410431#11441045 (10APizzata-WMF) a:05xcollazo→03APizzata-WMF [16:54:38] !log Test Kitchen mw-user experiment (poll 76991) - adds: none; removes: none; fields: we-3-3-4-reading-list-test1 - xLab/MPIC/TK tips at https://w.wiki/FwuD [16:54:40] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:54:58] !log Test Kitchen mw-user experiment (poll 76992) - adds: none; removes: none; fields: we-3-3-4-reading-list-test1-en - xLab/MPIC/TK tips at https://w.wiki/FwuD [16:55:00] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:50:09] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 13Patch-For-Review: Review and productionize the WME differential privacy data set - https://phabricator.wikimedia.org/T409601#11441659 (10Snwachukwu) @Htriedman I created an [[ https://gitlab.wikimedia.org/htriedman/wme-pageviews/-/merge_requests/1... [21:17:19] 06Data-Engineering, 10Dumps-Generation: Update dump mirror rsync allowlist to reflect new IP address for Scatter - https://phabricator.wikimedia.org/T409006#11442083 (10jeremyb) originally added in {T354679} see also current move {T306550}