[00:32:29] <wikibugs>	 10Analytics-Canonical-Data, 06Movement-Insights, 07Epic: Add language and incubating wikis data to canonical data - https://phabricator.wikimedia.org/T388201#10818945 (10nshahquinn-wmf)
[06:56:01] <wmf-insecte>	 Starting build #31 for job analytics-refinery-update-jars
[06:56:37] <wikibugs>	 (03PS1) 10Maven-release-user: Add refinery-source jars for v0.2.64 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1145745
[06:56:38] <wmf-insecte>	 Project analytics-refinery-update-jars build #31: 09SUCCESS in 37 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars/31/
[07:28:48] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23): Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10820364 (10brouberol) I'm happy to have it be deployed prod, as we've enabled it in staging for a month an a h...
[07:54:34] <wikibugs>	 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-Engineering-Wikistats, 10Data Pipelines, and 2 others: Move translations from ks-arab to ks on Translatewiki.net - https://phabricator.wikimedia.org/T394269 (10srishakatux) 03NEW
[07:54:55] <wikibugs>	 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-Engineering-Wikistats, 10Data Pipelines, and 2 others: Move translations from ks-arab to ks on Translatewiki.net - https://phabricator.wikimedia.org/T394269#10820478 (10srishakatux) p:05Triage→03Medium
[08:04:47] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PageMovedEvent should carry article redirect data - https://phabricator.wikimedia.org/T394049#10820516 (10gmodena) >>! In T394049#10818491, @daniel wrote: > I'd suggest adding the following...
[08:11:50] <wikibugs>	 (03Abandoned) 10Aqu: Add refinery-source jars for v0.2.64 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1145745 (owner: 10Maven-release-user)
[08:13:11] <wikibugs>	 (03CR) 10Gehel: Add refinery-job-lite jars for non-Spark jobs (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1143022 (owner: 10Aqu)
[08:42:45] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PageMovedEvent should carry article redirect data - https://phabricator.wikimedia.org/T394049#10820617 (10daniel) > Is a ?PageRecord a better type than ?WikiPage? WikiPage is the type carri...
[09:47:19] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PageMovedEvent should carry article redirect data - https://phabricator.wikimedia.org/T394049#10820914 (10gmodena) @daniel Thanks for clarifying. +1 for returning an immutable object. I'll...
[10:16:40] <wikibugs>	 (03Abandoned) 10Aqu: Add refinery-job-lite jars for non-Spark jobs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1143022 (owner: 10Aqu)
[11:40:12] <moritzm>	 FYI, I've failed over the kadmin server to krb1002.eqiad.wmnet (the server where to run kadmin commands, e.g. to generate new keytabs)
[11:41:02] <moritzm>	 is there any kerberized data platform server waiting to be racked for a test of generate_keytabs.py with the new bookworm stack?
[12:11:53] <wikibugs>	 (03CR) 10Joal: [V:03+2 C:03+2] "Merging for later deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139419 (https://phabricator.wikimedia.org/T391006) (owner: 10Joal)
[12:15:31] <wikibugs>	 (03CR) 10Joal: [V:03+2 C:03+2] "Merging for later deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1145164 (https://phabricator.wikimedia.org/T394011) (owner: 10Joal)
[12:23:42] <joal>	 !log Deploying refinery with scap
[12:23:44] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[12:39:36] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PagedMovedEvent should carry reason data - https://phabricator.wikimedia.org/T394046#10821575 (10gmodena) @daniel   Here I'm thinking of adding ` getReason(): string|null `  Does the naming...
[12:40:19] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PagedMovedEvent should carry reason data - https://phabricator.wikimedia.org/T394046#10821579 (10gmodena) >>! In T394046#10818020, @Ahoelzl wrote: > @gmodena I assumed this should be addres...
[12:40:41] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 07Epic, 10Event-Platform, and 2 others: EventBus: replace PageMoveCompleteHook with PageMovedEvent - https://phabricator.wikimedia.org/T393890#10821581 (10gmodena)
[13:35:44] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PagedMovedEvent should carry reason data - https://phabricator.wikimedia.org/T394046#10821841 (10daniel) >>! In T394046#10821575, @gmodena wrote: > @daniel  >  > Here I'm thinking of adding...
[13:44:23] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012#10821876 (10xcollazo) set `"spark.sql.iceberg.locality.enabled":"true"` on `wmf_content.mediawiki_content_current_v1` ingest.
[13:50:05] <joal>	 !log Delete deprecated-webrequest DAGs in Airflow
[13:50:06] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[13:55:17] <wikibugs>	 (03CR) 10Lucas Werkmeister (WMDE): "I’m not sure when I’ll get a +2 from someone here so if you want to +2 it, be my guest ^^ (and then I can cherry-pick it to the `productio" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145144 (owner: 10Lucas Werkmeister (WMDE))
[14:09:13] <wikibugs>	 06Data-Engineering, 06MW-Interfaces-Team, 10observability: [Needs grooming] Turnilo: include authentication status in request data cube - https://phabricator.wikimedia.org/T332864#10822105 (10lmata) Hi, we're moving to radar for now. Please let us know if there's anything we can do to assist.
[14:09:45] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Investigate first set of DQ alerts - https://phabricator.wikimedia.org/T394310 (10xcollazo) 03NEW
[14:09:55] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Investigate first set of DQ alerts - https://phabricator.wikimedia.org/T394310#10822120 (10xcollazo)
[14:09:56] <wikibugs>	 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 07Epic: Dumps 2.0 Phase III: Production level dumps - https://phabricator.wikimedia.org/T366752#10822121 (10xcollazo)
[14:15:08] <moritzm>	 is there any kerberized data platform server waiting to be racked for a test of generate_keytabs.py with the new bookworm stack?
[14:32:50] <wikibugs>	 (03CR) 10Filippo Giunchedi: [C:03+1] "I'd like to +2 and I don't permissions, this is good to go from my POV though" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145144 (owner: 10Lucas Werkmeister (WMDE))
[14:41:07] <wikibugs>	 (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] "Ah, okay. Then I’ll just self-+2 this." [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145144 (owner: 10Lucas Werkmeister (WMDE))
[14:41:20] <wikibugs>	 (03PS1) 10Lucas Werkmeister (WMDE): Add -w 1 flag to WikimediaGraphite nc calls [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145992
[14:41:31] <wikibugs>	 (03Merged) 10jenkins-bot: Add -w 1 flag to WikimediaGraphite nc calls [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145144 (owner: 10Lucas Werkmeister (WMDE))
[15:05:01] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23), 13Patch-For-Review: Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10822501 (10xcollazo) a:05xcollazo→03brouberol
[15:07:48] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23), 13Patch-For-Review: Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10822525 (10xcollazo)
[15:13:54] <wikibugs>	 (03CR) 10Lucas Werkmeister (WMDE): [C:03+2] Add -w 1 flag to WikimediaGraphite nc calls [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145992 (owner: 10Lucas Werkmeister (WMDE))
[15:14:22] <wikibugs>	 (03Merged) 10jenkins-bot: Add -w 1 flag to WikimediaGraphite nc calls [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1145992 (owner: 10Lucas Werkmeister (WMDE))
[15:18:45] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7006 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:19:35] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7012 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:19:43] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7011 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:19:53] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7016 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:20:11] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7009 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:20:35] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7001 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:20:41] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7003 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:20:51] <sukhe>	 fabfur is working on this and should clear up
[15:21:01] <fabfur>	 yep tnx
[15:21:47] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7013 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:22:41] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7014 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:22:55] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7005 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:26:01] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7002 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:27:45] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7015 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:28:22] <fabfur>	 acked these varnishkafka alerts
[15:38:47] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7007 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:40:47] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7004 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:42:45] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7010 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:43:41] <icinga-wm>	 PROBLEM - Webrequests Varnishkafka log producer on cp7008 is CRITICAL: PROCS CRITICAL: 0 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/webrequest.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[15:48:59] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Cassandra, 13Patch-For-Review: Audit and update AQS Cassandra roles & grants - https://phabricator.wikimedia.org/T313877#10822875 (10Eevans) >>! In T313877#10817966, @Ahoelzl wrote: > @Eevans should we expedite this?  That would be lovely; It'd be great t...
[15:54:16] <joal>	 Wow, looks like you're getting rid of some varnishkafka fabfur :)
[15:55:44] <fabfur>	 :)
[15:57:58] <elukey>	 so long varnishkafka!
[15:58:51] <joal>	 o/
[16:01:17] <fabfur>	 atm only in magru, and we're trying to fix alerts
[16:15:17] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23), 13Patch-For-Review: Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10823031 (10brouberol) The change has been deployed to the hive metastore, and the servic...
[16:21:15] <joal>	 ack fabfur - all good :)
[16:24:48] <wikibugs>	 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-Engineering-Wikistats, 10Data Pipelines, and 2 others: Move translations from ks-arab to ks on Translatewiki.net - https://phabricator.wikimedia.org/T394269#10823075 (10srishakatux)
[17:18:03] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th): Facilitate automatic artifact cache warming for airflow-dags artifacts - https://phabricator.wikimedia.org/T392244#10823369 (10JAllemandou) Copy pasting here a discussion from this gitlab code-review: https://gitlab.wikimedia.org/repos/data-engineering/airflow-...
[18:39:51] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th): Modify scap config files so that we pull artifacts from main rather than deprecated analytics config - https://phabricator.wikimedia.org/T394343 (10xcollazo) 03NEW
[19:54:16] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23): Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10823986 (10xcollazo) a:05brouberol→03xcollazo >>! In T386854#10823031, @brouberol wrote: > The change has...
[19:54:26] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23): Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10823991 (10xcollazo)
[19:57:19] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Data-Platform-SRE (2025.05.02 - 2025.05.23): Enable lock transaction management on the Hive Metastore - https://phabricator.wikimedia.org/T386854#10824002 (10xcollazo) Production config looks good: ` xcollazo@stat1011:/etc/hive/conf$ cat hive-site.xml | gre...
[20:09:59] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 10Event-Platform: PagedMovedEvent should carry reason data - https://phabricator.wikimedia.org/T394046#10824034 (10gmodena) a:03gmodena
[20:46:45] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Investigate first set of DQ alerts - https://phabricator.wikimedia.org/T394310#10824224 (10xcollazo) Below I explain the behavior just for `enwiki` for brevity, but it applies to all wikis.  Here are the alerts for `enwiki`: ` spark.s...
[20:47:55] <wikibugs>	 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Investigate first set of DQ alerts - https://phabricator.wikimedia.org/T394310#10824226 (10xcollazo)
[21:19:10] <wikibugs>	 06Data-Engineering, 06Research, 10Event-Platform: Implement stream of HTML content on mw.page_change event - https://phabricator.wikimedia.org/T360794#10824323 (10leila) FYI that I declined Research to spend time on T394065 b/c once this task is done, that issue will be addressed as well (Albeit it will be t...