[01:53:33] (03PS1) 10Zabe: Add anp.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/901729 (https://phabricator.wikimedia.org/T332379) [01:54:05] (03PS2) 10Zabe: Add anp.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/901729 (https://phabricator.wikimedia.org/T332379) [01:54:16] (03PS3) 10Zabe: Add anp.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/901729 (https://phabricator.wikimedia.org/T332379) [03:19:40] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: systemd-timedated.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [03:21:42] RECOVERY - Check systemd state on an-worker1132 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [03:42:34] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [03:46:16] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [03:50:08] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [03:57:40] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [04:14:54] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [04:28:50] PROBLEM - Check systemd state on an-launcher1002 is CRITICAL: CRITICAL - degraded: The following units failed: monitor_refine_event_sanitized_analytics_immediate.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [04:41:30] RECOVERY - Check systemd state on an-worker1132 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [04:51:52] (03CR) 10Krinkle: [C: 04-1] "I think we need painttiming to stay around for a little while since these cannot be atomically deployed at the same instant." [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/901169 (https://phabricator.wikimedia.org/T328256) (owner: 10Barakat Ajadi) [06:16:37] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team: Prepare and check storage layer for anpwiki - https://phabricator.wikimedia.org/T332458 (10Marostegui) [06:16:52] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team: Prepare and check storage layer for vewikimedia - https://phabricator.wikimedia.org/T330704 (10Marostegui) [06:17:03] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team: Prepare and check storage layer for gurwiki - https://phabricator.wikimedia.org/T327841 (10Marostegui) [06:17:12] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team: Prepare and check storage layer for gucwiki - https://phabricator.wikimedia.org/T326235 (10Marostegui) [06:17:20] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team: Prepare and check storage layer for azwikimedia - https://phabricator.wikimedia.org/T330442 (10Marostegui) [06:34:34] PROBLEM - Check systemd state on an-worker1110 is CRITICAL: CRITICAL - degraded: The following units failed: systemd-timedated.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [06:38:24] RECOVERY - Check systemd state on an-worker1110 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [06:49:20] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [06:53:14] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [06:58:52] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [07:04:36] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [07:08:32] PROBLEM - Check systemd state on an-worker1132 is CRITICAL: CRITICAL - degraded: The following units failed: export_smart_data_dump.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [07:09:50] (03CR) 10Gergő Tisza: Add fields to analytics/mediawiki/mentor_dashboard/visit (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/901539 (https://phabricator.wikimedia.org/T325117) (owner: 10Urbanecm) [07:40:52] RECOVERY - Check systemd state on an-worker1132 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [08:51:39] (03CR) 10Urbanecm: Add fields to analytics/mediawiki/mentor_dashboard/visit (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/901539 (https://phabricator.wikimedia.org/T325117) (owner: 10Urbanecm) [08:52:19] 10Data-Engineering-Planning, 10Patch-For-Review, 10Shared-Data-Infrastructure (Shared-Data-Infra Sprint 10): Upgrade Hadoop test cluster to Bullseye - https://phabricator.wikimedia.org/T329363 (10ops-monitoring-bot) Cookbook cookbooks.sre.ganeti.reimage was started by stevemunene@cumin1001 for host an-test-c... [09:30:42] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for next deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/901729 (https://phabricator.wikimedia.org/T332379) (owner: 10Zabe) [10:00:05] 10Data-Engineering-Planning, 10Shared-Data-Infrastructure: Upgrade the spark YARN shuffler service on Hadoop workers from version 2 to 3 - https://phabricator.wikimedia.org/T332765 (10BTullis) [10:07:40] 10Data-Engineering-Planning, 10Patch-For-Review, 10Shared-Data-Infrastructure (Shared-Data-Infra Sprint 10): Upgrade Hadoop test cluster to Bullseye - https://phabricator.wikimedia.org/T329363 (10ops-monitoring-bot) Cookbook cookbooks.sre.ganeti.reimage started by stevemunene@cumin1001 for host an-test-clien... [10:31:40] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10MW-1.40-notes (1.40.0-wmf.24; 2023-02-20), 10MW-1.41-notes (1.41.0-wmf.2; 2023-03-27), and 2 others: Remove StreamConfig::INTERNAL_SETTINGS logic from EventStreamConfig and do it in EventLogging client ... - https://phabricator.wikimedia.org/T286344 [10:37:59] (03PS3) 10Urbanecm: Add fields to analytics/mediawiki/mentor_dashboard/visit [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/901539 (https://phabricator.wikimedia.org/T325117) [12:03:45] 10Data-Engineering-Planning, 10Patch-For-Review, 10Shared-Data-Infrastructure (Shared-Data-Infra Sprint 10): Upgrade Hadoop test cluster to Bullseye - https://phabricator.wikimedia.org/T329363 (10Stevemunene) [12:14:03] (03PS1) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [12:59:58] (03PS1) 10Barakat Ajadi: PaintTiming: Add paint timing metrics to Navigation timing schema [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/902079 (https://phabricator.wikimedia.org/T328256) [13:02:37] 10Data-Engineering, 10Cloud-Services, 10WMDE-References-FocusArea, 10WMDE-TechWish-Sprint-2023-03-14: Shut down our previous Cloud VPS project and create a new one - https://phabricator.wikimedia.org/T332040 (10awight) I kind-of arbitrarily increased our vCPU request, after getting hints from initial perfo... [13:15:45] (03CR) 10Barakat Ajadi: "closing this, new changes can be found in https://gerrit.wikimedia.org/r/c/schemas/event/secondary/+/902079" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/901169 (https://phabricator.wikimedia.org/T328256) (owner: 10Barakat Ajadi) [13:16:36] (03Abandoned) 10Barakat Ajadi: PaintTiming: Move painttiming to navtiming schema [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/901169 (https://phabricator.wikimedia.org/T328256) (owner: 10Barakat Ajadi) [13:47:11] (03CR) 10Joal: "Comments on comments and som changes. I also would like the files to be in the hql/referer folder with names: compute_archive_daily.hql an" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [13:58:24] 10Data-Engineering, 10Event-Platform Value Stream (Sprint 10): [SPIKE] tune memory and latency of mediawiki-event-enrichment on k8s - https://phabricator.wikimedia.org/T332166 (10gmodena) I experimented with tunables recommended on Flink's Slack, and was able to run the application for over 6 hours. Mem... [14:02:23] 10Data-Engineering-Planning, 10Patch-For-Review, 10Shared-Data-Infrastructure (Shared-Data-Infra Sprint 10): Upgrade Hadoop test cluster to Bullseye - https://phabricator.wikimedia.org/T329363 (10ops-monitoring-bot) Cookbook cookbooks.sre.ganeti.reimage was started by stevemunene@cumin1001 for host an-test-c... [14:13:10] 10Data-Engineering-Planning, 10MW-1.40-notes (1.40.0-wmf.5; 2022-10-10), 10Metrics-Platform-Planning (Metrics Platform Kanban): Add mediawiki.db_name to Metrics Platform Event schema - https://phabricator.wikimedia.org/T304689 (10phuedx) 05Open→03Resolved a:03phuedx Being **bold**. [14:39:04] (03CR) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job (035 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [14:41:58] (03PS2) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [14:42:14] (03CR) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job (033 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [14:43:09] 10Quarry: Add a possibility to delete a draft - https://phabricator.wikimedia.org/T135908 (10Nikki) >>! In T135908#6827926, @Krinkle wrote: > As I understand it, past versions of draft queries (not published queries) are not kept. So the dozen drafts I have at have (by my cho... [14:44:03] (03PS3) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [14:49:44] hello teamm! [14:52:59] Hello mforns [15:00:49] hi! [15:01:18] Welcome back! [15:02:55] thanks steve_munene! :] [15:03:36] 10Quarry: Add a possibility to delete a draft - https://phabricator.wikimedia.org/T135908 (10rook) >>! In T135908#8718121, @Nikki wrote: >>>! In T135908#6827926, @Krinkle wrote: >> As I understand it, past versions of draft queries (not published queries) are not kept. So the dozen drafts I have at Hey mforns! [15:34:58] heya aqu! [15:56:23] 10Data-Engineering, 10Equity-Landscape: 2022 - Affiliates input metrics - https://phabricator.wikimedia.org/T330295 (10ntsako) [15:56:43] 10Data-Engineering, 10Equity-Landscape: 2022 - Social Progress index - https://phabricator.wikimedia.org/T330897 (10ntsako) [16:16:09] (03PS1) 10Htriedman: coalesce join_titles.hql to 1 partition [analytics/refinery] - 10https://gerrit.wikimedia.org/r/901637 [16:31:15] (03PS4) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [16:59:30] (03PS3) 10Snwachukwu: Copy add_partition hql script from Oozie to Hql folder. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/900389 (https://phabricator.wikimedia.org/T330200) [16:59:59] (03CR) 10Snwachukwu: Copy add_partition hql script from Oozie to Hql folder. (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/900389 (https://phabricator.wikimedia.org/T330200) (owner: 10Snwachukwu) [17:01:51] (03CR) 10Mforns: [V: 03+2 C: 03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/894104 (owner: 10Addshore) [17:18:31] (03CR) 10Joal: "You missed the file rename and folder change from my previous comment 😊 Almost there!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [17:19:30] (03CR) 10Joal: [C: 03+1] "Adding a parameter to coalesce is a great idea :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/901637 (owner: 10Htriedman) [17:20:14] (03CR) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [17:22:00] 10Data-Engineering-Planning, 10Product-Analytics: Add log_search to monthly sqoop list - https://phabricator.wikimedia.org/T332621 (10lbowmaker) [17:22:43] 10Data-Engineering-Planning, 10Data Pipelines, 10Product-Analytics: Add log_search to monthly sqoop list - https://phabricator.wikimedia.org/T332621 (10lbowmaker) [17:28:08] 10Data-Engineering-Planning, 10Data Pipelines: Review Superset permissions and assign roles as appropriate - https://phabricator.wikimedia.org/T328457 (10WMDE-leszek) I kindly ask granting `sql_lab` permissions to myself and the following WMDE colleagues of mine: @Aline_Bruenger_WMDE @Siko_WMDE. We'd like to q... [17:36:49] 10Data-Engineering-Planning, 10Patch-For-Review, 10Shared-Data-Infrastructure (Shared-Data-Infra Sprint 10): Upgrade Hadoop test cluster to Bullseye - https://phabricator.wikimedia.org/T329363 (10Stevemunene) 'an-test-client1002.eqiad.wmnet' was successfully created with same spec as an-test-client1001.eqiad... [17:41:09] 10Data-Engineering-Planning, 10Shared-Data-Infrastructure (Shared-Data-Infra Sprint 10): Investigate DB connection issues faced from airflow on an-launcher1002 - https://phabricator.wikimedia.org/T331265 (10lbowmaker) [17:42:40] 10Data-Engineering-Planning, 10Data Pipelines: Delete empty tables unique_devices_*_wide_* - https://phabricator.wikimedia.org/T329978 (10lbowmaker) a:03JAllemandou [17:43:04] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 11): Delete empty tables unique_devices_*_wide_* - https://phabricator.wikimedia.org/T329978 (10lbowmaker) [17:44:49] 10Data-Engineering-Planning, 10Data Pipelines: Deprecate old mobile datasets - https://phabricator.wikimedia.org/T329310 (10JAllemandou) a:03JAllemandou [17:47:40] (03PS5) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [17:49:38] (03CR) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job (034 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [18:36:36] (03CR) 10Joal: "One nit in one file, and some problems with files:" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [18:46:15] (03CR) 10Joal: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [18:47:37] (03PS6) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [18:48:19] (03CR) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [19:00:58] (03PS7) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [19:07:25] (03CR) 10Joal: [C: 03+1] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [19:20:55] (03PS8) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 [19:23:12] (03CR) 10Jennifer Ebe: T305842-Migrate-The-Referrer-Job-Daily-hql Create archive and referrer hql for referrer daily airflow job (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/902068 (owner: 10Jennifer Ebe) [21:01:24] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 11): Deprecate old mobile datasets - https://phabricator.wikimedia.org/T329310 (10lbowmaker)