[09:36:27] (03CR) 10Gmodena: [C: 03+1] Switch from git-fat to git-lfs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1007690 (https://phabricator.wikimedia.org/T328472) (owner: 10Ahmon Dancy) [13:29:33] (03CR) 10Nik Gkountas: [C: 03+2] cx event: add event sources for return to the dashboard [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/991111 (https://phabricator.wikimedia.org/T355200) (owner: 10KCVelaga) [13:31:21] 06Data-Engineering, 10Data Pipelines, 06SRE, 06Traffic-Icebox: Mobile redirects drop provenance parameters - https://phabricator.wikimedia.org/T252227#9600748 (10dr0ptp4kt) Originally, the thought was to be able to simply count relative volume of these types of inbound taps/clicks. Although we want fidelit... [13:34:29] (03Merged) 10jenkins-bot: cx event: add event sources for return to the dashboard [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/991111 (https://phabricator.wikimedia.org/T355200) (owner: 10KCVelaga) [13:42:47] 06Data-Engineering: [NEEDS GROOMING] we should provide DQ integration with Python - https://phabricator.wikimedia.org/T353940#9600823 (10gmodena) a:03gmodena [14:12:19] 06Data-Engineering, 10Data-Platform-SRE (2024.03.04 - 2024.03.24): Alerts Review: determine if we can use Prometheus to alert based on historical datasets - https://phabricator.wikimedia.org/T357537#9601077 (10Ottomata) Oh cool! @bking I read the linked notes but I'm missing how its gonna work? How can you a... [14:16:55] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic: Check statsv and eventlogging VarnishKafka instances - https://phabricator.wikimedia.org/T359178 (10Fabfur) [14:18:41] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic: Create a new temporary topic on Kafka jumbo cluster to test Benthos - https://phabricator.wikimedia.org/T359172#9601109 (10brouberol) ` brouberol@kafka-jumbo1010:~$ kafka topics --create --topic webrequest_text_test --replication-fact... [14:18:50] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic, 13Patch-For-Review: Install new Benthos instance on cp hosts - https://phabricator.wikimedia.org/T358109#9601115 (10brouberol) [14:18:59] (03CR) 10Joal: [C: 03+1] "LGTM! Thanks for doing this Ahmon" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1007690 (https://phabricator.wikimedia.org/T328472) (owner: 10Ahmon Dancy) [14:19:12] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic: Create a new temporary topic on Kafka jumbo cluster to test Benthos - https://phabricator.wikimedia.org/T359172#9601113 (10brouberol) 05Open→03Resolved All done! ` brouberol@kafka-jumbo1010:~$ kafka topics --describe --topic webr... [14:25:38] (03CR) 10Ottomata: Extract RefineSingleApp code from Refine (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1003745 (https://phabricator.wikimedia.org/T356363) (owner: 10Joal) [15:01:11] (03PS18) 10Joal: Extract RefineSingleApp code from Refine [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1003745 (https://phabricator.wikimedia.org/T356363) [15:07:53] (03CR) 10CI reject: [V: 04-1] Extract RefineSingleApp code from Refine [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1003745 (https://phabricator.wikimedia.org/T356363) (owner: 10Joal) [15:10:36] (03CR) 10Joal: "recheck" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1003745 (https://phabricator.wikimedia.org/T356363) (owner: 10Joal) [15:34:00] (03CR) 10Joal: "recheck" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1003745 (https://phabricator.wikimedia.org/T356363) (owner: 10Joal) [16:16:30] (03CR) 10Ahmon Dancy: "I'd like to coordinate with one of you to select a time to test out merge and deployment of this change. Anyone up for it? I'm in US/Paci" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1007690 (https://phabricator.wikimedia.org/T328472) (owner: 10Ahmon Dancy) [16:33:12] moritzm: I'm seeing ferm_srange related puppet errors on an-launcher1002: https://puppetboard.wikimedia.org/report/an-launcher1002.eqiad.wmnet/402be710e014faec691d6d2aa8c16fd4d4f079b9 is that on your radar? [16:34:18] I wonder if we somehow missed something during the firewall migration for this host. I can dig if need be [16:35:49] 10Data-Engineering (Sprint 9), 06Data-Platform, 06Movement-Insights: Add movement insights group/users to MWH denormalize job alerts - https://phabricator.wikimedia.org/T357472#9602070 (10Rmaung) Please also send to rmaung@wikimedia.org and aigumenshcheva@wikimedia.org, if we are not too late! Thank you!! [16:39:10] I think the issue stems from https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/hieradata/role/common/analytics_cluster/launcher.yaml#88 [16:39:17] brouberol: I'll make a patch [16:39:26] appreciated, thank you! [16:39:38] the airflow instance on an-launcher was missed when converting [16:41:06] https://gerrit.wikimedia.org/r/c/operations/puppet/+/1008896 [16:41:31] this only caused puppet to fail, the pre-existing ferm rules should still be loaded [16:42:55] approved, thank you! [16:47:19] merged and forced a puppet run on an-launcher, confirmed fixed [16:47:52] moritzm: Many thanks. [17:27:17] btullis: Heya - I've noticed you're in ops-week and there have been some alert emails lately - Would you take some time to look at them? [17:56:54] (03PS1) 10DLynch: Desktopwebuiactionstracking: add missing `show` to action enum [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1008928 (https://phabricator.wikimedia.org/T359182) [18:35:38] 10Data-Engineering (Sprint 9), 06Data Products, 06Movement-Insights, 10Movement-Metrics, 13Patch-For-Review: Skip Wikidata when loading XML dumps to the Data Lake - https://phabricator.wikimedia.org/T357859#9602789 (10nshahquinn-wmf) [18:47:13] joal: yes, will do. Sorry, I was distracted by Archiva today. [18:57:08] Oh, I think mforns has stepped in and re-run the failed jobs. Thanks so much. [20:14:35] 14Analytics-Radar, 06Data-Engineering-Icebox, 10Data-Services, 06cloud-services-team: Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#9603083 (10Superyetkin) 05Declined→03Open This needs to stay open. What do we need to do in or... [20:25:12] (03CR) 10Ottomata: Extract RefineSingleApp code from Refine (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1003745 (https://phabricator.wikimedia.org/T356363) (owner: 10Joal) [20:27:15] (03CR) 10Ottomata: Extract RefineSingleApp code from Refine (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1003745 (https://phabricator.wikimedia.org/T356363) (owner: 10Joal) [20:32:49] 14Analytics-Radar, 06Data-Engineering-Icebox, 10Data-Services, 06cloud-services-team: Implement technical details and process for "datasets_p" on wikireplica hosts - https://phabricator.wikimedia.org/T173511#9603122 (10bd808) >>! In T173511#9603083, @Superyetkin wrote: > This needs to stay open. I find yo... [22:28:06] joal: I have attempted to fix the `mediawiki_history_check_denormalize.mediawiki_page_history_check_error` job, but I have been unable to. I have to reach out to the rest of the DE team for assistance.