[05:07:21] (03PS4) 10Snwachukwu: Add Custom Authentication Configuration Class for Cassandra. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) [05:13:24] (03CR) 10Snwachukwu: Add Custom Authentication Configuration Class for Cassandra. (033 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) (owner: 10Snwachukwu) [08:13:38] (03CR) 10Joal: "Two requests for changes, the overall looks great 😊 Thanks Sandra!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) (owner: 10Snwachukwu) [09:23:36] (03PS1) 10Phuedx: analytics/mediawiki/client/metrics_event: Handle custom_data[].data_type of "null" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/855493 (https://phabricator.wikimedia.org/T322758) [09:42:47] 10Analytics, 10Data-Engineering, 10Event-Platform Value Stream, 10SRE: ~1 request/minute to intake-logging.wikimedia.org times out at the traffic/service interface - https://phabricator.wikimedia.org/T264021 (10Vgutierrez) Are we sure that this is a service side issue? this sounds a lot like a FetchError t... [09:45:40] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10elukey) Assumptions/notes for the first prototype: * Benthos reads webrequest_{text,upload... [10:26:55] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10JAllemandou) happy to talk about sampling when you wish @elukey :) I also think you should... [10:34:07] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10elukey) >>! In T314981#8385855, @JAllemandou wrote: > happy to talk about sampling when you... [10:49:59] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10JAllemandou) Something I forgot: It'd be great to have the new stream setup as an "event" (... [10:53:25] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10JAllemandou) >> I also think you should be pointed to T319344, as it requests a new feature... [11:52:59] 10Data-Engineering, 10Equity-Landscape: Grants input metric - https://phabricator.wikimedia.org/T309276 (10KCVelaga_WMF) @ntsako All the grants and weighted columns align, but there are issues with the "change" columns Comparisons at https://docs.google.com/spreadsheets/d/1smlxmLZN3igND0vW1Zhsr5BRnXgWxx_zbr... [11:57:51] 10Analytics-Radar, 10SRE, 10Traffic, 10Patch-For-Review: Consider adding X-Analytics subfield for 'has a session cookie' - https://phabricator.wikimedia.org/T319324 (10Vgutierrez) 05Open→03Resolved a:03Vgutierrez CR merged and https://wikitech.wikimedia.org/wiki/X-Analytics#Keys updated, thanks for c... [12:05:19] 10Data-Engineering-Planning, 10Shared-Data-Infrastructure, 10Event-Platform Value Stream (Sprint 04): [SPIKE] Deploy event driven stateless Flink service to DSE cluster - https://phabricator.wikimedia.org/T320812 (10gmodena) Now that a namespace has been created, I'm picking this back up and wanted to sync.... [13:59:26] 10Data-Engineering-Planning, 10Shared-Data-Infrastructure, 10Event-Platform Value Stream (Sprint 04): [SPIKE] Deploy event driven stateless Flink service to DSE cluster - https://phabricator.wikimedia.org/T320812 (10BTullis) Hi @gmodena - I believe that since the `flink-rdf-streaming-updater` service [[https... [15:16:39] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10Ottomata) > overridden by the DE batch jobs QQ: do the existent DE batch jobs already produ... [15:59:58] (03PS5) 10Snwachukwu: Add Custom Authentication Configuration Class for Cassandra. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) [16:25:00] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10elukey) >>! In T314981#8386970, @Ottomata wrote: >> overridden by the DE batch jobs > QQ: d... [16:27:55] 10Data-Engineering-Planning, 10Data Pipelines, 10Foundational Technology Requests, 10Traffic, and 2 others: Add a webrequest sampled topic and ingest into druid/turnilo - https://phabricator.wikimedia.org/T314981 (10Ottomata) > The idea is to proceed for a first iteration (namely a new kafka topic + druid... [16:36:01] (03PS3) 10Aqu: Create dataset from HDFS fsimage.xml [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/852315 (https://phabricator.wikimedia.org/T321168) [16:40:38] (03CR) 10CI reject: [V: 04-1] Create dataset from HDFS fsimage.xml [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/852315 (https://phabricator.wikimedia.org/T321168) (owner: 10Aqu) [16:46:41] (03PS4) 10Aqu: Create dataset from HDFS fsimage.xml [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/852315 (https://phabricator.wikimedia.org/T321168) [16:54:54] (03CR) 10Snwachukwu: Add Custom Authentication Configuration Class for Cassandra. (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) (owner: 10Snwachukwu) [17:00:55] (03CR) 10Joal: "2 minor changes in comments please, then we can merge!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) (owner: 10Snwachukwu) [17:23:17]  [17:34:01] PROBLEM - Check systemd state on aqs1016 is CRITICAL: CRITICAL - degraded: The following units failed: cassandra-a.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [17:35:49] RECOVERY - Check systemd state on aqs1016 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [18:07:57] (03PS6) 10Snwachukwu: Add Custom Authentication Configuration Class for Cassandra. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) [18:10:34] (03CR) 10Joal: [C: 03+1] "Thanks Sandra - +1, I let you +2 when you have tested this latest version" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/851077 (https://phabricator.wikimedia.org/T306895) (owner: 10Snwachukwu) [18:19:23] (03PS1) 10Mazevedo: Add http client_ip to iOS schemas [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/855675 [19:00:38] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 04), 10Patch-For-Review: Allow Cormac Parle and Marco Fossati to deploy analytics-platform-eng Airflow instance - https://phabricator.wikimedia.org/T321925 (10Ottomata) Okay, did some things, @cparle or @mfossati can you try deploying now? [19:59:12] (03CR) 10Tsevener: "Looking good to me, just one little thing I noticed." [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/855675 (owner: 10Mazevedo) [20:06:01] (03PS2) 10Mazevedo: Add http client_ip to iOS schemas [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/855675 [20:13:16] (03CR) 10Tsevener: [C: 03+1] Add http client_ip to iOS schemas (032 comments) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/855675 (owner: 10Mazevedo) [21:48:08] 10Data-Engineering, 10Product-Analytics, 10Wmfdata-Python: Remove Spark session timeout functionality from Wmfdata-Python - https://phabricator.wikimedia.org/T298179 (10nshahquinn-wmf) >>! In T298179#8378525, @Antoine_Quhen wrote: > If the timeout is removed, it could be possible to detect and alert when `no...