[01:27:22] PROBLEM - Check systemd state on an-launcher1002 is CRITICAL: CRITICAL - degraded: The following units failed: monitor_refine_event.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [01:30:20] PROBLEM - Check unit status of monitor_refine_event on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [05:46:12] (VarnishkafkaNoMessages) firing: varnishkafka on cp2035 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2035%3A9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [05:51:12] (VarnishkafkaNoMessages) resolved: varnishkafka on cp2035 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2035%3A9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [09:32:13] 10Data-Engineering: Add the requestctl element of the x-analytics map to turnlio's webrequest_sampled_128 - https://phabricator.wikimedia.org/T314578 (10JAllemandou) This is indeed not complicated - Andrew's description of what to do is comprehensive :) [09:44:56] (03PS3) 10Joal: Adapt sqoop to templatelinks schema changes [analytics/refinery] - 10https://gerrit.wikimedia.org/r/821312 (https://phabricator.wikimedia.org/T314666) (owner: 10Milimetric) [10:03:29] (03CR) 10Joal: "@Amir: Thank you for the original patch! I have added the linktarget table, but it seems it's not yet replicated to labs :S Do you have an" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/821312 (https://phabricator.wikimedia.org/T314666) (owner: 10Milimetric) [10:04:34] (03CR) 10Ladsgroup: Adapt sqoop to templatelinks schema changes (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/821312 (https://phabricator.wikimedia.org/T314666) (owner: 10Milimetric) [11:15:04] (03CR) 10Ladsgroup: Adapt sqoop to templatelinks schema changes (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/821312 (https://phabricator.wikimedia.org/T314666) (owner: 10Milimetric) [12:25:32] (03CR) 10Kosta Harlan: Instrument blocked account registration (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/821711 (https://phabricator.wikimedia.org/T306018) (owner: 10Sergio Gimeno) [12:42:14] (03PS4) 10Joal: Adapt sqoop to templatelinks schema changes [analytics/refinery] - 10https://gerrit.wikimedia.org/r/821312 (https://phabricator.wikimedia.org/T314666) (owner: 10Milimetric) [12:42:40] (03CR) 10Kosta Harlan: Instrument blocked account registration (039 comments) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/821711 (https://phabricator.wikimedia.org/T306018) (owner: 10Sergio Gimeno) [12:45:07] (03CR) 10Joal: Adapt sqoop to templatelinks schema changes (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/821312 (https://phabricator.wikimedia.org/T314666) (owner: 10Milimetric) [13:05:31] Hey ottomata - May I ask for some of your time today before you go in holidays? I'd like to talk about the spark3 installation progress, and about streams, please [13:41:58] 10Analytics-Radar, 10Machine-Learning-Team, 10SRE: Using docker in WMF production network outside of kubernetes - https://phabricator.wikimedia.org/T275551 (10akosiaris) >>! In T275551#8176053, @fkaelin wrote: > Reviving this discussion, though I renamed the phab to "Running docker containers in a non-produc... [14:00:13] 10Data-Engineering, 10Event-Platform Value Stream, 10Lift-Wing, 10Epic, and 2 others: Send score to eventgate when requested - https://phabricator.wikimedia.org/T301878 (10Ottomata) [14:13:04] 10Data-Engineering: Update Search Engine list - https://phabricator.wikimedia.org/T315329 (10mforns) Code and changes make sense to me @Isaac! I'll confirm with the team, but feel free to create a patch for that if you feel like it! [14:37:11] (03PS1) 10Isaac Johnson: Update search engine detection [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/826581 (https://phabricator.wikimedia.org/T315329) [14:45:24] 10Data-Engineering, 10Patch-For-Review: Update Search Engine list - https://phabricator.wikimedia.org/T315329 (10Isaac) Thanks -- patch uploaded! Big caveat that while I tested a Python version to make sure no adverse effects etc., I don't have a good way of testing this Java version. [14:56:22] (03CR) 10Ottomata: "Hi, that looks about right to me! I see you" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826373 (https://phabricator.wikimedia.org/T316047) (owner: 10Bearloga) [14:56:26] (03CR) 10Ottomata: [C: 03+1] Retroactively add http.client_ip to Android schemas [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826373 (https://phabricator.wikimedia.org/T316047) (owner: 10Bearloga) [14:57:11] (03CR) 10Ottomata: [C: 03+1] "I think it is safe to merge this and nothing will break." [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826373 (https://phabricator.wikimedia.org/T316047) (owner: 10Bearloga) [15:24:19] joal: yes please! [15:25:03] joal how about in 1h? [15:25:15] let me check ottomata :) [15:25:29] Works for me ottomata ! [15:50:57] (03PS1) 10Snwachukwu: [WIP]Repurpose Refinery-Tool module to contain codes reused across other modules. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/826599 (https://phabricator.wikimedia.org/T316120) [15:53:29] (03CR) 10CI reject: [V: 04-1] [WIP]Repurpose Refinery-Tool module to contain codes reused across other modules. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/826599 (https://phabricator.wikimedia.org/T316120) (owner: 10Snwachukwu) [16:00:19] (03CR) 10Ottomata: [C: 03+1] "looks fine to me once Jenkins is happy :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/826599 (https://phabricator.wikimedia.org/T316120) (owner: 10Snwachukwu) [16:05:29] milimetric: Heya - Could you please review https://gerrit.wikimedia.org/r/c/analytics/refinery/+/821312 that you originated and I updated? [16:12:31] (03CR) 10Milimetric: [C: 03+2] "looks good, I'm not 100% sure if just altering the templatelinks hive table will work. If not, then we can just drop the old columns and " [analytics/refinery] - 10https://gerrit.wikimedia.org/r/821312 (https://phabricator.wikimedia.org/T314666) (owner: 10Milimetric) [16:30:43] Thanks a lot for the quick turnaround milimetric :) [16:30:56] ottomata: batcave? [16:32:12] 10Data-Engineering, 10Cassandra: Decommission old AQS cluster nodes - https://phabricator.wikimedia.org/T302277 (10hnowlan) [16:32:39] 10Data-Engineering, 10Cassandra: Final cleanup tasks related to the AQS cluster migration - https://phabricator.wikimedia.org/T302278 (10hnowlan) [16:32:47] 10Data-Engineering, 10Cassandra: Stop ingesting data to the old AQS cluster - https://phabricator.wikimedia.org/T302276 (10hnowlan) [16:34:07] (just doing some board cleanup, sorry for the noise) [17:38:58] (03CR) 10Bearloga: [C: 03+2] Retroactively add http.client_ip to Android schemas [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826373 (https://phabricator.wikimedia.org/T316047) (owner: 10Bearloga) [17:39:39] (03Merged) 10jenkins-bot: Retroactively add http.client_ip to Android schemas [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826373 (https://phabricator.wikimedia.org/T316047) (owner: 10Bearloga) [18:00:43] (03PS1) 10MNeisler: content_translation_event: update schema documentation with new event source values [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826632 (https://phabricator.wikimedia.org/T287403) [18:12:25] (03CR) 10NOkafor: "All resolved." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/812095 (https://phabricator.wikimedia.org/T311507) (owner: 10NOkafor) [18:13:42] (03PS13) 10NOkafor: Add cassandra loading queries for aiflow in the hql folder Bug: T311507 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/812095 (https://phabricator.wikimedia.org/T311507) [19:15:00] (03CR) 10MNeisler: [C: 03+2] content_translation_event: update schema documentation with new event source values [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826632 (https://phabricator.wikimedia.org/T287403) (owner: 10MNeisler) [19:15:34] (03Merged) 10jenkins-bot: content_translation_event: update schema documentation with new event source values [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/826632 (https://phabricator.wikimedia.org/T287403) (owner: 10MNeisler) [20:07:23] 10Data-Engineering, 10Event-Platform Value Stream: Consider supporting incompatible schema changes in Event Platform streams - https://phabricator.wikimedia.org/T316288 (10Ottomata) [20:07:32] 10Data-Engineering, 10Event-Platform Value Stream: Consider supporting incompatible schema changes in Event Platform streams - https://phabricator.wikimedia.org/T316288 (10Ottomata) p:05Triage→03Low