[00:04:25] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Add an option to define an alert email at the level of team YAML config dbt scheduler file - https://phabricator.wikimedia.org/T434001#12249654 (10amastilovic) This issue is now implemented. [00:31:44] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Load Google Search Console data into the Data Lake - https://phabricator.wikimedia.org/T420996#12249700 (10nshahquinn-wmf) >>! In T420996#12206331, @nshahquinn-wmf wrote: > Given that, I think the best tradeoff is to load the data from BigQuery as so... [06:24:42] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12250022 (10JMonton-WMF) We are preparing this data for the Attribution API... [09:22:21] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10MediaWiki-extensions-Wikibase-Client, 10Wikidata, 10Wikidata Integration in Wikimedia projects: Data Lake - sqoop of wikidata_entity_usage needs to adapt to move to x1 MariaDB cluster - https://phabricator.wikimedia.org/T431477#12250501 (10Jakob_W... [09:44:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12250597 (10Ladsgroup) Thanks. I assume there aren't that many trending art... [11:36:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12251161 (10JMonton-WMF) That's correct, we don't have yet exact numbers, b... [12:22:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12251302 (10pmiazga) FYI - what I'm currently working on is https://gerrit.... [13:26:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Java-Scala-Standardization, 07Essential-Work, 13Patch-For-Review: Publish all wmf-sparksqlclidriver-*.jar versions to GitLab Package Registry - https://phabricator.wikimedia.org/T435126#12251589 (10xcollazo) [13:32:03] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12251621 (10Ladsgroup) To be clear, I mean it should be per-wiki (not per -... [13:32:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Essential-Work: With spill_enabled = true, Presto fails queries with CROSS JOIN - https://phabricator.wikimedia.org/T435862#12251622 (10xcollazo) Ok, seems like Presto is full of surprising behavior. Even with `spill_enabled` globally off: ` presto... [13:43:01] (03CR) 10Xcollazo: [C:03+2] Update user_agents_info with Cloudflare Radar [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1325932 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [13:43:35] (03CR) 10Xcollazo: [V:03+2 C:03+2] Update user_agents_info with Cloudflare Radar [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1325932 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [13:56:47] (03PS1) 10Xcollazo: Update changelog.md to release v0.3.25 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1329288 [13:58:31] (03CR) 10Xcollazo: [V:03+2 C:03+2] Update changelog.md to release v0.3.25 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1329288 (owner: 10Xcollazo) [14:00:13] Starting build #72 for job analytics-refinery-maven-release [14:23:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Ops Week: Increase wait time of webrequest_actor_metrics_hourly sensor task before it fails - https://phabricator.wikimedia.org/T430247#12251997 (10Snwachukwu) Sure @Ahoelzl [14:31:04] Project analytics-refinery-maven-release build #72: 09SUCCESS in 30 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/72/ [14:35:38] Starting build #41 for job analytics-refinery-update-jars [14:36:58] 06Data-Engineering, 06Data-Engineering-Radar, 06MediaWiki-API-Platform-Team (MediaWiki-Interfaces-Sprints), 07OKR-Work: REST API vNext: Create dedicated REST shared-schema repository - https://phabricator.wikimedia.org/T432799#12252160 (10HCoplin-WMF) [14:37:49] (03PS1) 10Maven-release-user: Add refinery-source jars for v0.3.25 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1329298 [14:37:49] Project analytics-refinery-update-jars build #41: 09SUCCESS in 2 min 10 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars/41/ [14:39:21] (03CR) 10Xcollazo: [C:03+2] Add refinery-source jars for v0.3.25 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1329298 (owner: 10Maven-release-user) [14:39:24] (03CR) 10Xcollazo: [V:03+2 C:03+2] Add refinery-source jars for v0.3.25 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1329298 (owner: 10Maven-release-user) [14:43:06] !log Deployed refinery-source using jenkins [14:43:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:09:14] !log Deployed refinery using scap, then deployed onto hdfs [15:09:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:33:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Collaboration-Services, 10Phabricator, 13Patch-For-Review: data lake: phabricator events - https://phabricator.wikimedia.org/T434047#12252565 (10LSobanski) [15:57:03] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: [Mediawiki History] make array fields empty as opposed to null when relevant but no values present - https://phabricator.wikimedia.org/T433511#12252697 (10Isaac) [17:37:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Load Google Search Console data into the Data Lake - https://phabricator.wikimedia.org/T420996#12253261 (10nshahquinn-wmf) @Ahoezl said he's just waiting on a definite statement of what we want the loading schedule to be. It just so happened that Go... [18:27:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [18:37:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [18:49:24] 06Data-Engineering, 06Data-Engineering-Radar, 06Java-Scala-Standardization, 06Discovery-Search (2026.08.03 - 2026.08.31), and 2 others: Publish search instance artifact JARs to GitLab Package Registry - https://phabricator.wikimedia.org/T435141#12253653 (10EBernhardson) This is merged and deployed, we will... [20:15:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Ops Week: Increase wait time of webrequest_actor_metrics_hourly sensor task before it fails - https://phabricator.wikimedia.org/T430247#12254075 (10Snwachukwu) The root cause of the issue can be found in the slack [[ https://wikimed... [20:36:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): wmf_contributors.edit_per_editor_per_page_daily claims `Martin Urbanec` is a system user - https://phabricator.wikimedia.org/T434217#12254164 (10Snwachukwu) `user_is_system` value is copied [[ https://gerrit.wikimedia.org/r/plugins/gitiles/analytics/refi... [20:40:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12254181 (10pmiazga) ah, sorry. My bad, missed the part that `7k-15k` items... [21:26:44] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): wmf_contributors.edit_per_editor_per_page_daily claims `Martin Urbanec` is a system user - https://phabricator.wikimedia.org/T434217#12254289 (10Urbanecm_WMF) >>! In T434217#12254164, @Snwachukwu wrote: > The [[ https://gerrit.wikimedia.org/r/plugins/git...