[06:40:19] 14Analytics, 06Data-Engineering-Icebox, 10MediaWiki-Page-history: Add historical page protection status to MediaWiki history - https://phabricator.wikimedia.org/T246723#10001349 (10Aklapper) [07:44:39] (03CR) 10DCausse: "I think "clear" and "set" are fairly different, clear does only need to know the tag group so I believe that clear should just be an array" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) (owner: 10Peter Fischer) [08:15:08] 06Data-Engineering, 10Observability-Metrics, 10Sustainability (Incident Followup): Site Issue: Delayed data in the `webrequest_sampled_live` kafka topic - https://phabricator.wikimedia.org/T369737#10001495 (10fgiunchedi) [08:15:38] 06Data-Engineering, 10Observability-Metrics, 10Sustainability (Incident Followup): Site Issue: Delayed data in the `webrequest_sampled_live` kafka topic - https://phabricator.wikimedia.org/T369737#10001496 (10fgiunchedi) [08:17:06] !log deploy istio (adding securityContext) to dse-k8s-eqiad cluster - T362978 [08:17:09] 06Data-Engineering, 10Observability-Metrics, 10Sustainability (Incident Followup): Site Issue: Delayed data in the `webrequest_sampled_live` kafka topic - https://phabricator.wikimedia.org/T369737#10001497 (10fgiunchedi) 05Open→03Resolved a:03fgiunchedi I'm calling this done / good enough for now,... [08:17:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:17:10] T362978: Update all helm modules and charts to be compatible with the restricted PSS - https://phabricator.wikimedia.org/T362978 [09:30:40] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization, 06Release-Engineering-Team: Validate CI integration so that Ci can release Maven artifacts on user's demand - https://phabricator.wikimedia.org/T367403#10001637 (10Gehel) @hashar... [11:03:13] 06Data-Engineering, 06Data Products, 06DBA, 10MediaWiki-extensions-Newsletter, 07Schema-change-in-production: Apply Newsletter schema change (make indices non-unique) - https://phabricator.wikimedia.org/T370602#10001913 (10Ladsgroup) a:03Ladsgroup [11:05:58] 06Data-Engineering, 06Data Products, 06DBA, 10MediaWiki-extensions-Newsletter, 07Schema-change-in-production: Apply Newsletter schema change (make indices non-unique) - https://phabricator.wikimedia.org/T370602#10001917 (10Ladsgroup) 05Open→03Resolved Ran it with replication on testwiki, mediawik... [11:52:44] (03PS6) 10Peter Fischer: Introducing cirrussearch/weighted_tags [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) [11:56:22] (03CR) 10Peter Fischer: "After looking into the `MultiListHandler` code I learned about the option of encoding sub-list-deletions. That was not obvious from the do" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) (owner: 10Peter Fischer) [11:56:54] (03CR) 10Peter Fischer: "Done" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) (owner: 10Peter Fischer) [13:11:26] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#10002205 (10Ottomata... [13:27:14] !log restarting eventlogging-processor on eventlog1003 - something is wrong with the consumer...https://phabricator.wikimedia.org/T353817#10002205 [13:27:17] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:32:32] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#10002273 (10Ottomata... [13:41:21] 06Data-Engineering, 06Discovery-Search, 06Java-Scala-Standardization, 06Release-Engineering-Team: Java projects hosted on Gerrit should publish artifacts to Gitlab - https://phabricator.wikimedia.org/T370400#10002341 (10Gehel) >>! In T370400#9995143, @Ottomata wrote: > @gehel, would it be easier if project... [13:45:16] (03CR) 10DCausse: "lgtm! left a couple nits, adding Andrew to make we don't miss anything important" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) (owner: 10Peter Fischer) [13:45:54] 06Data-Engineering, 06Discovery-Search, 06Java-Scala-Standardization, 06Release-Engineering-Team: Java projects hosted on Gerrit should publish artifacts to Gitlab - https://phabricator.wikimedia.org/T370400#10002386 (10Gehel) p:05Triage→03Medium [13:55:03] FIRING: VarnishKafkaDeliveryErrors: varnishkafka has cache_text errors on cp2037:9132 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?panelId=20&fullscreen&orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2037 - https://alerts.wikimedia.org/?q=alertname%3DVarnishKafkaDeliveryErrors [13:59:50] RESOLVED: VarnishKafkaDeliveryErrors: varnishkafka has cache_text errors on cp2037:9132 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?panelId=20&fullscreen&orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2037 - https://alerts.wikimedia.org/?q=alertname%3DVarnishKafkaDeliveryErrors [14:14:40] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10CirrusSearch, 03Discovery-Search (Current work), 10MW-1.43-notes (1.43.0-wmf.15; 2024-07-23), 13Patch-For-Review: [Search Update Pipeline] Source streams for private wikis - https://phabricator.wikimedia.org/T346046#10002484 (10Ottomata) [14:38:58] 10Data-Engineering (Q1 2024 July 1st - September 30th): Develop Airflow ExternalTaskSensor to orchestrate DAG dependencies - https://phabricator.wikimedia.org/T369900#10002656 (10amastilovic) [14:39:16] 10Data-Engineering (Q1 2024 July 1st - September 30th): Develop Airflow ExternalTaskSensor to orchestrate DAG dependencies - https://phabricator.wikimedia.org/T369900#10002657 (10amastilovic) [14:57:24] 06Data-Engineering, 06Discovery-Search, 06Java-Scala-Standardization, 06Release-Engineering-Team: Java projects hosted on Gerrit should publish artifacts to Gitlab - https://phabricator.wikimedia.org/T370400#10002753 (10Ottomata) 👍 [15:09:27] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Drop gb_by from globalblocks table - https://phabricator.wikimedia.org/T370394#10002856 (10VirginiaPoundstone) CC @Ottomata @Milimetric please have a look at this schema change. Please let me know what the risks and impact are of... [15:10:27] (03CR) 10Ottomata: Introducing cirrussearch/weighted_tags (032 comments) [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) (owner: 10Peter Fischer) [16:36:21] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic: New software: haproxykafka - https://phabricator.wikimedia.org/T370668#10003563 (10Fabfur) [16:41:28] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic: New software: haproxykafka - https://phabricator.wikimedia.org/T370668#10003577 (10Vgutierrez) https://github.com/negasus/haproxy-spoe-go could be handy if we go down that road and we don't want to get dirty writing C code :) [16:42:30] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization, 06Release-Engineering-Team: Validate CI integration so that Ci can release Maven artifacts on user's demand - https://phabricator.wikimedia.org/T367403#10003586 (10amastilovic) A... [17:13:59] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: [Refine Refactoring] Changes to EventStreamConfig needed for scheduling Refine via airflow - https://phabricator.wikimedia.org/T367134#10003814 (10tchin) Diffing the output of deeply merging stream defaults, all of the changes are eith... [17:46:00] 06Data-Engineering, 03Discovery-Search (Current work), 10MW-1.43-notes (1.43.0-wmf.14; 2024-07-16), 07Wikimedia-production-error: '.event.pageViewId' should be string, '.event.subTest' should be string, '.event.searchSessionId' should be string - https://phabricator.wikimedia.org/T286814#10003984 (10EBernha... [18:56:17] 10Data-Engineering (Q1 2024 July 1st - September 30th), 13Patch-For-Review: [Refine Refactoring] Changes to EventStreamConfig needed for scheduling Refine via airflow - https://phabricator.wikimedia.org/T367134#10004344 (10Ottomata) Okay great, I think we can merge the ESC deep merge patch then. Just +2ed it. [19:09:12] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#10004395 (10Ottomata... [19:23:18] 06Data-Engineering: Handle Late-Arrived Events from Gobblin into Airflow triggered Refine - https://phabricator.wikimedia.org/T370665#10004432 (10Ottomata) [19:23:35] 06Data-Engineering: Handle Late-Arrived Events from Gobblin into Airflow triggered Refine - https://phabricator.wikimedia.org/T370665#10004433 (10Ottomata) Edited to add ^ and add context. [20:19:45] 06Data-Engineering, 10Wmfdata-Python: Specify Conda-Pack as a dependency - https://phabricator.wikimedia.org/T370718 (10nshahquinn-wmf) 03NEW p:05Triage→03Low [21:11:55] 06Data-Engineering: Event Utilities partially downloads schemas - https://phabricator.wikimedia.org/T309717#10005018 (10Ottomata) This happened again today, the exception was: ` at [Source: (StringReader); line: 220, column: 14] at com.fasterxml.jackson.dataformat.yaml.snakeyaml.error.MarkedYAMLExcepti... [21:12:32] 06Data-Engineering: Event Utilities partially downloads schemas - https://phabricator.wikimedia.org/T309717#10005020 (10Ottomata) As a next step, we could try adding debug logging in BasicHttpClient and/or BasicHttpResponse. [21:59:33] 07Analytics-Data-Problem, 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform, 06Movement-Insights: NEW BUG REPORT Mediawiki_history contains duplicate rows for some revisions - https://phabricator.wikimedia.org/T369851#10005159 (10Milimetric) Checking: * wmf.mediawiki_history: duplica... [22:19:05] (03PS7) 10Peter Fischer: Introducing cirrussearch/weighted_tags [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) [22:20:11] (03CR) 10Peter Fischer: "Thank you for your comments! I renamed/moved the schema, inlined the definitions and added descriptions." [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/1055226 (https://phabricator.wikimedia.org/T366253) (owner: 10Peter Fischer)