[01:25:15] (EventgateLoggingExternalLatency) firing: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLoggingExternalLatency [01:30:15] (EventgateLoggingExternalLatency) resolved: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLoggingExternalLatency [06:06:30] 10Data-Engineering, 10Airflow: Add a new member to the airlfow-dags repository - https://phabricator.wikimedia.org/T305719 (10bmansurov) 05Open→03Resolved @mforns, thanks! I'm able to push. [10:28:18] 10Analytics: jmads requesting Kerberos password - https://phabricator.wikimedia.org/T250560 (10Volans) @jmads your kerberos account should still be valid, as far as I can tell. Please verify it and feel free to close this task if all is working as expected. [10:30:47] (03PS1) 10Sbisson: Fix typo in wikistories contribution schema [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/784652 [11:18:43] (03CR) 10Joal: [C: 03+1] "Nits - comments about comments - ready to be merged as is." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/772027 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [11:26:28] (03CR) 10Joal: Implement job to extract officewiki webrequests (033 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/779952 (https://phabricator.wikimedia.org/T306136) (owner: 10Milimetric) [12:21:04] (03CR) 10Eamedina: [C: 03+2] "Thanks" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/784652 (owner: 10Sbisson) [12:21:35] (03Merged) 10jenkins-bot: Fix typo in wikistories contribution schema [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/784652 (owner: 10Sbisson) [12:33:30] (03CR) 10Gergő Tisza: Image Suggestions feature schema (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/779052 (owner: 10Luke Bowmaker) [12:52:18] (03CR) 10Joal: [WIP] Create a Hive to Graphite job (034 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/775376 (https://phabricator.wikimedia.org/T304623) (owner: 10Snwachukwu) [13:03:06] ottomata: meeting! [13:04:38] (03CR) 10Ottomata: Image Suggestions feature schema (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/779052 (owner: 10Luke Bowmaker) [13:28:29] (03CR) 10Gergő Tisza: Image Suggestions feature schema (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/779052 (owner: 10Luke Bowmaker) [13:32:52] (03PS1) 10Sbisson: Wikistories consumption: make story_open_time optional [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/784675 [13:41:08] (03CR) 10Ottomata: Image Suggestions feature schema (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/779052 (owner: 10Luke Bowmaker) [13:46:32] (03CR) 10Huei Tan: [C: 03+1] Wikistories consumption: make story_open_time optional [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/784675 (owner: 10Sbisson) [14:08:50] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure Datahub Authentication and Authorization for the MVP - https://phabricator.wikimedia.org/T306448 (10BTullis) p:05Triage→03High a:03BTullis [14:15:11] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure Datahub Authentication and Authorization for the MVP - https://phabricator.wikimedia.org/T306448 (10BTullis) I've manually added the users to their correct `wmf` and `nda` groups. {F35060060,width=80%} I've also added a `datahubadmin` g... [14:18:06] (03CR) 10Eamedina: [C: 03+2] Wikistories consumption: make story_open_time optional [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/784675 (owner: 10Sbisson) [14:19:04] (03Merged) 10jenkins-bot: Wikistories consumption: make story_open_time optional [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/784675 (owner: 10Sbisson) [14:39:02] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure Datahub Authentication and Authorization for the MVP - https://phabricator.wikimedia.org/T306448 (10BTullis) I think this is the bare minimum that we need for the MVP - https://datahub.wikimedia.org/policies {F35060121,width=75%} * All... [14:41:34] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure Datahub Authentication and Authorization for the MVP - https://phabricator.wikimedia.org/T306448 (10BTullis) Marking as done, although I will gather feedback on whether logged in users see what they expect. If we wish to restrict what... [14:47:02] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure Datahub Authentication and Authorization for the MVP - https://phabricator.wikimedia.org/T306448 (10BTullis) [14:47:04] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [14:47:10] (03CR) 10Joal: "wrong patch version reviewed - my bad" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/775376 (https://phabricator.wikimedia.org/T304623) (owner: 10Snwachukwu) [15:22:02] 10Data-Engineering, 10Data-Catalog: Connect Kafka to the MVP [Mile Stone 5] - https://phabricator.wikimedia.org/T299899 (10BTullis) I have ingested kafka-jumbo with the following recipe. ` source: type: "kafka" config: connection: bootstrap: "kafka-jumbo1001.eqiad.wmnet:9092" schema_registr... [15:22:50] 10Data-Engineering, 10Data-Catalog: Connect Kafka to the MVP [Mile Stone 5] - https://phabricator.wikimedia.org/T299899 (10BTullis) a:03BTullis [15:47:09] folks I am skipping the sre sync since my team meeting got anticipated by an hour, sorry [15:47:33] No worries elukey. Catch up another time. :-) [15:48:34] 10Analytics-Clusters, 10Data-Engineering, 10Data-Engineering-Kanban, 10Cassandra, and 2 others: Stop ingesting data to the old AQS cluster - https://phabricator.wikimedia.org/T302276 (10BTullis) p:05Triage→03Medium [16:05:00] 10Data-Engineering, 10Generated Data Platform, 10SRE, 10SRE-Access-Requests, 10Patch-For-Review: Request to grant cparle and mfossati login to an-airflow1003.eqiad.wmne - https://phabricator.wikimedia.org/T306057 (10Volans) I've +1ed the patch, @Ottomata feel free to merge whenever works for you. [16:18:55] 10Data-Engineering: Request to delete tumult_temp__ tables/schemas from analytics cluster - https://phabricator.wikimedia.org/T306551 (10Htriedman) [16:22:33] 10Data-Engineering, 10Generated Data Platform, 10SRE, 10SRE-Access-Requests, 10Patch-For-Review: Request to grant cparle and mfossati login to an-airflow1003.eqiad.wmne - https://phabricator.wikimedia.org/T306057 (10Ottomata) Thanks got caught in meetings, will do soon! [16:39:48] btullis: https://meet.google.com/kti-iybt-ekv [16:45:36] 10Data-Engineering, 10Generated Data Platform, 10SRE, 10SRE-Access-Requests, 10Patch-For-Review: Request to grant cparle and mfossati login to an-airflow1003.eqiad.wmne - https://phabricator.wikimedia.org/T306057 (10Ottomata) 05Open→03Resolved a:03Ottomata [16:57:16] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) rack/setup/install an-worker11[42-48].eqiad.wmnet - https://phabricator.wikimedia.org/T293922 (10Cmjohnson) vlans updated [16:59:40] 10Analytics-Clusters, 10Data-Engineering, 10Data-Engineering-Kanban, 10Cassandra, and 2 others: Stop ingesting data to the old AQS cluster - https://phabricator.wikimedia.org/T302276 (10BTullis) Spoke to @JAllemandou about this. There are two fundamental options: # Disable cassandra2 loading from the oozi... [17:07:03] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) rack/setup/install an-worker11[42-48].eqiad.wmnet - https://phabricator.wikimedia.org/T293922 (10Cmjohnson) [17:35:49] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: Pageviews integration testing - https://phabricator.wikimedia.org/T299735 (10BPirkle) [17:38:24] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [17:38:26] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog, 10Epic, 10Patch-For-Review: Update DataHub to latest version for MVP - https://phabricator.wikimedia.org/T306019 (10BTullis) 05Open→03Resolved [17:38:51] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Catalog: Configure Datahub Authentication and Authorization for the MVP - https://phabricator.wikimedia.org/T306448 (10BTullis) 05Open→03Resolved [17:38:53] 10Data-Engineering, 10Data-Catalog, 10Epic: Data Catalog MVP - https://phabricator.wikimedia.org/T299910 (10BTullis) [17:39:01] 10Data-Engineering, 10Platform Engineering Roadmap: Audit/review pageviews test cases - https://phabricator.wikimedia.org/T305502 (10BPirkle) > This is great. Should we turn this into a checklist of completion criteria in T299735: Pageviews integration testing? Done. >> @Eevans , is the [[ https://gerrit.wik... [18:54:15] (EventgateLoggingExternalLatency) firing: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLoggingExternalLatency [18:59:15] (EventgateLoggingExternalLatency) resolved: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateLoggingExternalLatency