[01:31:48] PROBLEM - Check unit status of monitor_refine_event on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [02:16:52] RECOVERY - Check unit status of monitor_refine_event on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [07:14:06] 10Analytics-Wikistats, 10Data-Engineering, 10Browser-Support-Opera: Opera 15+ seems not to be recognized correctly - https://phabricator.wikimedia.org/T61816 (10JAllemandou) 05Open→03Declined Discarding - but is way to old. [07:34:57] (03PS1) 10Urbanecm: homepagemodule: Add mentorship-optout action [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/786268 [07:35:11] (03PS2) 10Urbanecm: homepagemodule: Add mentorship-optout action [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/786268 (https://phabricator.wikimedia.org/T287915) [07:36:30] (03CR) 10Urbanecm: [C: 04-2] "pending confirmation the feature won't be available outside of homepage" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/786268 (https://phabricator.wikimedia.org/T287915) (owner: 10Urbanecm) [08:17:23] hey, pontoon-1.analytics.eqiad1.wikimedia.cloud and an-db-1.analytics.eqiad1.wikimedia.cloud have been failing puppet for a while and are not accepting connection from cloud-cumin - could someone look at fixing those? thanks! [08:25:40] taavi: Will do, thanks for the heads-up. [08:27:29] (03PS3) 10Urbanecm: homepagemodule: Add mentorship-optout action [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/786268 (https://phabricator.wikimedia.org/T287915) [08:29:29] 10Data-Engineering, 10Cassandra, 10Platform Team Workboards (Platform Engineering Reliability): Decommission old AQS cluster nodes - https://phabricator.wikimedia.org/T302277 (10BTullis) Removing this from kanban whilst we work on the dependent tickets - namely migrating cassandra loading from oozie to airflow. [08:52:00] 10Data-Engineering, 10Data-Engineering-Kanban: Create Analytics Network Diagram & Documentation - https://phabricator.wikimedia.org/T298577 (10BTullis) I believe that we can resolve this ticket. * The analytics network diagram (as of Feb2022) is [[https://app.diagrams.net/#G1G_ql8YQ0JPQVyebAk4YT7Z5ZE9RCWs8e|h... [09:03:03] (03CR) 10Urbanecm: [C: 03+1] "should be good to go now" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/786268 (https://phabricator.wikimedia.org/T287915) (owner: 10Urbanecm) [09:15:45] (03PS4) 10Urbanecm: homepagemodule: Add mentorship-optout/mentorship-optin actions [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/786268 (https://phabricator.wikimedia.org/T287915) [09:57:30] 10Data-Engineering, 10Data-Engineering-Kanban: Update ua-parser library for traffic data - https://phabricator.wikimedia.org/T306829 (10Antoine_Quhen) a:03Antoine_Quhen Currently, UAP-java https://github.com/ua-parser/uap-java last version is almost a year old. Some minor fixes have been committed to uap-cor... [10:45:02] (03PS1) 10Phuedx: Remove the GettingStarted* allowlist entries [analytics/refinery] - 10https://gerrit.wikimedia.org/r/786291 (https://phabricator.wikimedia.org/T306879) [10:49:14] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering, 10User-TheresNoTime, and 2 others: Wikistats Bug - easy to understand language for pageviews - https://phabricator.wikimedia.org/T263973 (10TheresNoTime) a:05TheresNoTime→03None [10:58:36] 10Data-Engineering, 10Observability-Logging: Indexing errors / malformed logs for aqs on cassandra timeout - https://phabricator.wikimedia.org/T262920 (10BTullis) Can we decline or resolve this ticket? It's been open for 18 months, it's a known issue, with an ultimate resolution of adopting ECS common logging... [12:02:19] !log Rerun cassandra-daily-wf-local_group_default_T_mediarequest_per_file-2022-4-23 [12:02:21] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:09:32] 10Analytics-Clusters, 10Data-Engineering: Create analytics-test-eqiad zookeeper cluster - https://phabricator.wikimedia.org/T289056 (10BTullis) 05Open→03Declined This is no longer necessary, now that the analytics vlan firewall has gone. We can now use the `test-eqiad` zookeeper cluster from the analytics... [12:23:09] 10Data-Engineering: LVS in Analytics VLANs - https://phabricator.wikimedia.org/T288750 (10BTullis) Now that the analytics vlan outbound firewall restrictions have been removed in {T298087} - what is the impact on this ticket? Is it still the case that the current LVS servers can't L2 forward packets to the anal... [12:23:58] 10Data-Engineering, 10Data-Engineering-Kanban, 10Cassandra, 10Platform Team Workboards (Platform Engineering Reliability): Stop ingesting data to the old AQS cluster - https://phabricator.wikimedia.org/T302276 (10BTullis) [12:24:19] joal: Thanks for that rerun. [12:25:47] yo're welcome btullis :) [12:26:36] Heads up - I'm going to be rebooting the machine hosting archiva in 5 minutes, unless anyone has any objections. [12:27:35] all good for me btullis [12:27:53] 👍 Thanks [12:29:17] btullis: thanks for fixing puppet on the analytics project hosts! could you also rebase the puppet git repo there so that some recent changes to vm ldap authentication get applied? [12:31:11] taavi: Yes, will do. I fixed pontoon-1 but I didn't think that an-db-1 was fixed yet. I'd like to check with ottomata before rebasing, because I think that he was the last person to use these two hosts. [12:31:54] oh, oops, an-db-1 is still broken (only looked at pontoon-1) :/ [12:33:04] Yeah, compilation error so I thought I'd check. We might just be able to delete it, but I'll keep you posted. I'll try to do that rebase on pontoon-1 today as well. [12:33:43] thank you! [12:34:36] 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Pageview definition relies on X-Analytics to determine special pages - https://phabricator.wikimedia.org/T304362 (10mforns) Thanks @Krinkle, that is very informative, will help when implementing the fix. [12:57:35] 10Data-Engineering, 10Data-Engineering-Kanban, 10Cassandra: Update HiveToCassandra job to read cassandra password from file - https://phabricator.wikimedia.org/T306895 (10JAllemandou) [12:59:30] 10Data-Engineering, 10Data-Catalog: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896 (10BTullis) [13:04:08] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Pageviews integration testing - https://phabricator.wikimedia.org/T299735 (10BPirkle) [13:04:28] 10Data-Engineering-Radar, 10API Platform, 10Platform Engineering Roadmap: Audit/review pageviews test cases - https://phabricator.wikimedia.org/T305502 (10BPirkle) [13:11:44] 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Pageview definition relies on X-Analytics to determine special pages - https://phabricator.wikimedia.org/T304362 (10awight) >>! In T304362#7878093, @mforns wrote: > I executed a quick query to proxy the proportion of pageviews affected by thi... [13:19:06] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top-by-country endpoint of the pageviews API - https://phabricator.wikimedia.org/T299733 (10DAbad) Update 4/26/2022 - Contract is done - still need to tests (Wendy may need help on getting caught up to tests) M... [13:19:55] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top-per-country endpoint of the pageviews API - https://phabricator.wikimedia.org/T299734 (10DAbad) Update 4/26/2022 - Contract is done - still need to tests (Wendy may need help on getting caught up to tests)... [13:21:38] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top endpoint of the pageviews API - https://phabricator.wikimedia.org/T299732 (10DAbad) Update 4/26/2022 - Contract is done - tests done and some merged tests still need to be merged and reviewed. [13:22:31] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top endpoint of the pageviews API - https://phabricator.wikimedia.org/T299732 (10DAbad) 05Open→03In progress p:05Triage→03High [13:22:33] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10DAbad) [13:24:26] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10DAbad) Update 4/26/2022 halfway through tests and contract is fully implemented for all 5 endpoints [13:24:44] (03PS6) 10Vivian Rook: Use one_or_none to handle non-existent queries [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/682030 (https://phabricator.wikimedia.org/T280915) (owner: 10BrandonXLF) [13:27:30] 10Data-Engineering-Radar, 10API Platform, 10Platform Engineering Roadmap: Audit/review pageviews test cases - https://phabricator.wikimedia.org/T305502 (10DAbad) Have list of tests here: https://phabricator.wikimedia.org/T299735 [13:29:31] 10Data-Engineering, 10Observability-Logging: Indexing errors / malformed logs for aqs on cassandra timeout - https://phabricator.wikimedia.org/T262920 (10fgiunchedi) 05Open→03Declined Agreed @BTullis ! I don't think we've seen a reoccurence (though I might be mistaken) so I'm boldly closing. Please reopen... [13:29:35] 10Data-Engineering-Radar, 10API Platform, 10Platform Engineering Roadmap: Retroactively fix uses of fmt.Sprintf where no formatting occurs - https://phabricator.wikimedia.org/T305510 (10DAbad) have a merge request for this, just need to get it pushed [13:30:04] 10Data-Engineering-Radar, 10API Platform, 10Platform Engineering Roadmap: Retroactively fix uses of fmt.Sprintf where no formatting occurs - https://phabricator.wikimedia.org/T305510 (10DAbad) @bpirkle for peer QA/review [13:32:57] 10Analytics, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0 documentation - https://phabricator.wikimedia.org/T288664 (10DAbad) a:03apaskulin We should probably get this going now that we have the first set of endpoints starting to look better. Team to reach out to @apaskulin to discuss next steps. [13:51:32] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Update DNS for the DataHub MVP services - https://phabricator.wikimedia.org/T301460 (10BTullis) 05Open→03Resolved [13:51:36] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [13:51:38] 10Data-Engineering, 10Data-Catalog, 10SRE, 10serviceops, and 2 others: New Service Request: DataHub - https://phabricator.wikimedia.org/T303049 (10BTullis) [13:56:33] 10Data-Engineering, 10Data-Catalog, 10SRE, 10serviceops, and 2 others: New Service Request: DataHub - https://phabricator.wikimedia.org/T303049 (10BTullis) Should we call this done, or should we leave it open pending an outcome on {T305358}? Many thanks again for all your support with this request @JMeybohm. [13:57:44] 10Data-Engineering, 10Data-Catalog, 10SRE, 10serviceops, and 2 others: New Service Request: DataHub - https://phabricator.wikimedia.org/T303049 (10JMeybohm) Please keep this open as it is absolutely in a hacky state currently (DNS + service::catalog wise) [13:57:52] 10Data-Engineering, 10Data-Catalog: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896 (10BTullis) [13:57:56] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [14:02:25] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [14:02:51] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog, 10Patch-For-Review: Set up karapace instance for datahub - https://phabricator.wikimedia.org/T301562 (10BTullis) 05Open→03Resolved [14:03:28] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog, 10Patch-For-Review: Set up karapace instance for datahub - https://phabricator.wikimedia.org/T301562 (10BTullis) [14:03:32] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog, 10Patch-For-Review: Create debian package of karapace - https://phabricator.wikimedia.org/T301565 (10BTullis) 05Open→03Resolved p:05Triage→03Medium [14:08:28] 10Data-Engineering, 10Data-Catalog: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896 (10Ottomata) Interesting, how would this contrast with lineage from Airflow that schedules Spark jobs? [14:09:31] (03PS15) 10Aqu: Make caching mechanisms thread ready [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) [14:11:51] 10Data-Engineering, 10Data-Catalog, 10Epic: Integrate Superset with DataHub - https://phabricator.wikimedia.org/T306903 (10BTullis) [14:11:54] 10Analytics, 10API Platform (Product Roadmap), 10Code-Health-Objective, 10Epic, and 3 others: AQS 2.0 - https://phabricator.wikimedia.org/T263489 (10DAbad) [14:11:58] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10DAbad) 05Open→03In progress [14:12:19] 10Data-Engineering, 10Data-Catalog: Integrate Superset with DataHub - https://phabricator.wikimedia.org/T306903 (10BTullis) [14:12:44] 10Analytics, 10API Platform, 10Generated Data Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement per-article endpoint of the pageviews API - https://phabricator.wikimedia.org/T289265 (10DAbad) [14:14:09] 10Analytics, 10API Platform, 10Code-Health-Objective, 10Epic, and 3 others: Implement aggregate endpoint of the pageviews API - https://phabricator.wikimedia.org/T299731 (10DAbad) [14:14:46] 10Analytics, 10API Platform, 10Code-Health-Objective, 10Epic, and 3 others: Implement aggregate endpoint of the pageviews API - https://phabricator.wikimedia.org/T299731 (10DAbad) 05Open→03In progress [14:14:50] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10DAbad) [14:15:40] 10Analytics, 10API Platform, 10Code-Health-Objective, 10Epic, and 3 others: Implement aggregate endpoint of the pageviews API - https://phabricator.wikimedia.org/T299731 (10DAbad) p:05Triage→03High [14:16:56] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10DAbad) [14:16:58] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top-by-country endpoint of the pageviews API - https://phabricator.wikimedia.org/T299733 (10DAbad) 05Open→03In progress p:05Triage→03High [14:17:01] 10Data-Engineering, 10Data-Catalog: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896 (10BTullis) >>! In T306896#7881056, @Ottomata wrote: > Interesting, how would this contrast with lineage from Airflow that schedules Spark jobs? Good question! I honestly don't know at the moment. They... [14:17:15] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top-by-country endpoint of the pageviews API - https://phabricator.wikimedia.org/T299733 (10DAbad) [14:18:23] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top-per-country endpoint of the pageviews API - https://phabricator.wikimedia.org/T299734 (10DAbad) 05Open→03In progress p:05Triage→03High [14:18:25] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10DAbad) [14:18:43] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Implement top-per-country endpoint of the pageviews API - https://phabricator.wikimedia.org/T299734 (10DAbad) [14:19:57] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10DAbad) [14:19:59] 10Data-Engineering-Radar, 10API Platform, 10Platform Engineering Roadmap: Retroactively fix uses of fmt.Sprintf where no formatting occurs - https://phabricator.wikimedia.org/T305510 (10DAbad) 05Open→03In progress [14:20:35] 10Data-Engineering-Radar, 10API Platform, 10Platform Engineering Roadmap: Audit/review pageviews test cases - https://phabricator.wikimedia.org/T305502 (10DAbad) 05Open→03Resolved [14:20:37] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: Pageviews integration testing - https://phabricator.wikimedia.org/T299735 (10DAbad) [14:21:39] 10Data-Engineering-Radar, 10API Platform, 10Platform Engineering Roadmap: Retroactively fix logging to use a RequestScopedLogger where applicable - https://phabricator.wikimedia.org/T305504 (10DAbad) [14:23:15] 10Analytics, 10API Platform (Product Roadmap), 10Code-Health-Objective, 10Epic, and 3 others: AQS 2.0 - https://phabricator.wikimedia.org/T263489 (10DAbad) 05Open→03In progress p:05Medium→03High a:03DAbad [14:24:01] (03CR) 10Aqu: [C: 03+2] "Comments added." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [14:24:05] 10Analytics, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0 documentation - https://phabricator.wikimedia.org/T288664 (10DAbad) p:05Triage→03Medium [14:27:51] (03CR) 10Aqu: [V: 03+2 C: 03+2] Make caching mechanisms thread ready [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [14:34:01] (03Merged) 10jenkins-bot: Make caching mechanisms thread ready [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [14:51:44] (03CR) 10Aqu: [C: 03+2] Migrate wikidata/item_page_link/weekly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/761876 (https://phabricator.wikimedia.org/T300023) (owner: 10Aqu) [15:03:01] (03Merged) 10jenkins-bot: Migrate wikidata/item_page_link/weekly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/761876 (https://phabricator.wikimedia.org/T300023) (owner: 10Aqu) [15:12:48] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Connect MVP to Hive metastore [Mile Stone 4] - https://phabricator.wikimedia.org/T299897 (10BTullis) [15:12:50] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [15:40:55] (03CR) 10David Caro: [C: 03+1] "LGTM, nits can be ignored" [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/682030 (https://phabricator.wikimedia.org/T280915) (owner: 10BrandonXLF) [15:56:51] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for shnwikivoyage - https://phabricator.wikimedia.org/T302798 (10BTullis) When running the `maintain-views` script for shnwikivoyage, we received the following warning. ` 2022-04-26 15:53:24,038 WARN... [15:58:13] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for shnwikivoyage - https://phabricator.wikimedia.org/T302798 (10Marostegui) @BTullis Probably a question for #cloud-services-team I would say. [16:28:30] 10Data-Engineering-Kanban, 10Data-Engineering-Radar, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Upgrade clouddb* hosts to Bullseye - https://phabricator.wikimedia.org/T299480 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=644878e4-4323-41a9-afa2-a71944aff698... [16:35:44] 10Data-Engineering-Kanban, 10Data-Engineering-Radar, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Upgrade clouddb* hosts to Bullseye - https://phabricator.wikimedia.org/T299480 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by razzi@cumin1001 for host clou... [16:54:10] (03CR) 10Snwachukwu: "Hi Joseph," [analytics/refinery] - 10https://gerrit.wikimedia.org/r/785153 (https://phabricator.wikimedia.org/T300028) (owner: 10Snwachukwu) [17:09:25] 10Data-Engineering-Kanban, 10Data-Engineering-Radar, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Upgrade clouddb* hosts to Bullseye - https://phabricator.wikimedia.org/T299480 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by razzi@cumin1001 for host clouddb1... [17:18:53] 10Data-Engineering, 10Data-Engineering-Kanban, 10DC-Ops, 10Infrastructure-Foundations, 10Patch-For-Review: clouddb1021 missing network firmware bnx2x/bnx2x-e2-7.13.21.0.fw in Debian 11 Bullseye - https://phabricator.wikimedia.org/T306148 (10razzi) I did the reimage again just now and it worked fine selec... [17:19:07] 10Data-Engineering-Kanban, 10Data-Engineering-Radar, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Upgrade clouddb* hosts to Bullseye - https://phabricator.wikimedia.org/T299480 (10razzi) [17:22:00] 10Data-Engineering-Kanban, 10Data-Engineering-Radar, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Upgrade clouddb* hosts to Bullseye - https://phabricator.wikimedia.org/T299480 (10razzi) 05Open→03Resolved [17:24:33] 10Data-Engineering, 10Data-Engineering-Kanban, 10DC-Ops, 10Infrastructure-Foundations, 10Patch-For-Review: clouddb1021 missing network firmware bnx2x/bnx2x-e2-7.13.21.0.fw in Debian 11 Bullseye - https://phabricator.wikimedia.org/T306148 (10MoritzMuehlenhoff) Ack, I feel confident that the patch does the... [18:26:42] (03PS1) 10Aqu: Update changelog.md to v0.1.27 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/786362 [18:27:48] (03CR) 10Aqu: [V: 03+2 C: 03+2] Update changelog.md to v0.1.27 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/786362 (owner: 10Aqu) [18:30:04] Starting build #104 for job analytics-refinery-maven-release-docker [18:45:15] Project analytics-refinery-maven-release-docker build #104: 09SUCCESS in 15 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release-docker/104/ [18:47:28] Starting build #63 for job analytics-refinery-update-jars-docker [18:48:05] (03PS1) 10Maven-release-user: Add refinery-source jars for v0.1.27 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/786370 [18:48:09] Project analytics-refinery-update-jars-docker build #63: 09SUCCESS in 41 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars-docker/63/ [18:51:41] (03CR) 10Aqu: [C: 03+2] Add refinery-source jars for v0.1.27 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/786370 (owner: 10Maven-release-user) [18:51:47] (03CR) 10Aqu: [V: 03+2 C: 03+2] Add refinery-source jars for v0.1.27 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/786370 (owner: 10Maven-release-user) [19:02:10] !log About to deploy analytics/refinery: Weekly deployment train + Artifacts to 0.1.27 [19:02:12] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:16:06] 10Data-Engineering, 10DBA, 10Data-Services, 10cloud-services-team (Kanban): Prepare and check storage layer for shnwikivoyage - https://phabricator.wikimedia.org/T302798 (10bd808) >>! In T302798#7881457, @BTullis wrote: > When running the `maintain-views` script for shnwikivoyage, we received the following... [20:19:21] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering-Radar, 10Product-Analytics, and 3 others: Wikistats pageview data missing counts for Mobile App pageviews on Commons, going back to 2020-11 - https://phabricator.wikimedia.org/T299439 (10LGoto) [20:58:40] 10Data-Engineering: Crash of artifact-cache in scap deploy context - https://phabricator.wikimedia.org/T305868 (10Antoine_Quhen) Same error today, but I may have found a pattern: # scap deploy some code with a newly declared artifact https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/commit/e1... [22:06:38] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog POC - https://phabricator.wikimedia.org/T293647 (10odimitrijevic)