[01:53:34] 10Data-Engineering (Q3 2024 January 1st - March 31th): Analyze Dumps Usage Through Apache Logs - https://phabricator.wikimedia.org/T383175#10478129 (10VirginiaPoundstone) @jebe-wmf any progress updates or questions? [02:10:36] 06Data-Engineering, 10Data-Engineering-Wikistats: Enable arabic language - https://phabricator.wikimedia.org/T384265 (10Mr.Ibrahem) 03NEW [02:16:36] (03PS3) 10Mr.Ibrahem: enable arabic language [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1112849 (https://phabricator.wikimedia.org/T384265) [03:41:24] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Experimentation Lab, 10Metrics Platform: Make jsonschema-tools merge values of enums when merging allOf - https://phabricator.wikimedia.org/T345317#10478178 (10VirginiaPoundstone) [03:41:40] 06Data-Engineering, 06Data-Engineering-Icebox, 06Experimentation Lab, 06Growth-Team, and 4 others: Create a test for end-to-end event logging data verification of happy path with Special:Homepage and suggested edits - https://phabricator.wikimedia.org/T301463#10478182 (10VirginiaPoundstone) [03:42:25] 06Data-Engineering, 06Experimentation Lab, 10Metrics Platform: Access to aggregate User Agent statistics - https://phabricator.wikimedia.org/T298912#10478192 (10VirginiaPoundstone) [03:42:37] 14Analytics-Radar, 06Data-Engineering, 06Data-Engineering-Icebox, 06Experimentation Lab, and 2 others: Draft of full process for instrumentation using new client libraries - https://phabricator.wikimedia.org/T275694#10478195 (10VirginiaPoundstone) [03:43:09] 14Analytics, 06Data-Engineering, 06Experimentation Lab, 10Metrics Platform, 10Event-Platform: Client-side error logging should use Elastic Common Schema (ECS) fields when possible - https://phabricator.wikimedia.org/T267602#10478201 (10VirginiaPoundstone) [03:48:16] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Experimentation Lab: Make jsonschema-tools merge values of enums when merging allOf - https://phabricator.wikimedia.org/T345317#10478234 (10VirginiaPoundstone) [03:48:36] 06Data-Engineering, 06Data-Engineering-Icebox, 06Experimentation Lab, 06Growth-Team, and 3 others: Create a test for end-to-end event logging data verification of happy path with Special:Homepage and suggested edits - https://phabricator.wikimedia.org/T301463#10478238 (10VirginiaPoundstone) [03:49:32] 06Data-Engineering, 06Data-Engineering-Radar, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging: Remove deprecated $logger parameter from EventLogging::submit() - https://phabricator.wikimedia.org/T376410#10478253 (10VirginiaPoundstone) [03:49:50] 06Data-Engineering, 06Data-Engineering-Radar, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, and 4 others: Decide on how data platform wants to monitor bundle sizes - https://phabricator.wikimedia.org/T378772#10478249 (10VirginiaPoundstone) [03:50:05] 06Data-Engineering, 06Experimentation Lab: Access to aggregate User Agent statistics - https://phabricator.wikimedia.org/T298912#10478260 (10VirginiaPoundstone) [03:50:21] 14Analytics-Radar, 06Data-Engineering, 06Data-Engineering-Icebox, 06Experimentation Lab, 06Product-Analytics: Draft of full process for instrumentation using new client libraries - https://phabricator.wikimedia.org/T275694#10478262 (10VirginiaPoundstone) [03:50:53] 06Data-Engineering, 06Data-Engineering-Radar, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, 07Technical-Debt: Migrate EventLogging to use DefaultEventSubmitter - https://phabricator.wikimedia.org/T375749#10478270 (10VirginiaPoundstone) [03:51:23] 14Analytics, 06Data-Engineering, 06Experimentation Lab, 10Event-Platform: Client-side error logging should use Elastic Common Schema (ECS) fields when possible - https://phabricator.wikimedia.org/T267602#10478279 (10VirginiaPoundstone) [07:05:55] 06Data-Engineering, 06DBA, 06Growth-Team, 13Patch-For-Review, 07Schema-change-in-production: Make growthexperiments_link_recommendations.gelr_data nullable in the per-wiki x1 databases - https://phabricator.wikimedia.org/T384177#10478495 (10Marostegui) This is now running in the master with some sleep in... [10:50:50] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Movement-Insights, 13Patch-For-Review: Test airflow cascading-reruns for projectview-hourly dependent jobs - https://phabricator.wikimedia.org/T383804#10478876 (10Antoine_Quhen) 05Open→03Resolved a:03Antoine_Quhen Feedback on Cascading Cleari... [10:59:58] !log disabled gobblin timers on the analytics-airflow instance [11:00:00] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:00:22] !log paused all un-paused DAGs on airflow-analaytics, prior to the migration. [11:00:24] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:46:54] 06Data-Engineering, 06DBA, 06Growth-Team, 13Patch-For-Review, 07Schema-change-in-production: Make growthexperiments_link_recommendations.gelr_data nullable in the per-wiki x1 databases - https://phabricator.wikimedia.org/T384177#10479089 (10Marostegui) 05Open→03Resolved This is all done [12:02:50] 06Data-Engineering, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops: Add support for JIT in PHP8.1 images - https://phabricator.wikimedia.org/T384294 (10jijiki) 03NEW [12:08:29] 06Data-Engineering, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops: Add support for JIT in PHP8.1 images - https://phabricator.wikimedia.org/T384294#10479188 (10jijiki) [12:16:21] FYI, the root partition on stat1008 is full [12:16:34] (just noticed since a package update failed on it) [12:47:07] 06Data-Engineering, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops: Add support for JIT in PHP8.1 images - https://phabricator.wikimedia.org/T384294#10479258 (10jijiki) [13:16:35] 06Data-Engineering, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, 07Spike: [SPIKE] Investigate possible event loss on navigation in Google Chrome - https://phabricator.wikimedia.org/T384307 (10phuedx) 03NEW [13:20:07] 06Data-Engineering, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, 07Spike: [SPIKE] Investigate possible event loss on navigation in Google Chrome - https://phabricator.wikimedia.org/T384307#10479433 (10phuedx) @Krinkle: I'm going to continue to investigate but I would appreciate your input if... [13:24:04] !log stopped airflow services on an-launcher1002 for T380619 [13:24:07] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:24:07] T380619: Migrate the airflow-analytics scheduler to Kubernetes - https://phabricator.wikimedia.org/T380619 [13:33:29] 06Data-Engineering, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, 07Spike: [SPIKE] Investigate possible event loss on navigation in Google Chrome - https://phabricator.wikimedia.org/T384307#10479482 (10Krinkle) @phuedx Two things to check: 1. Can you reproduce this locally? Preferably local... [13:33:38] 06Data-Engineering, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, 10MediaWiki-Platform-Team (Radar), 07Spike: [SPIKE] Investigate possible event loss on navigation in Google Chrome - https://phabricator.wikimedia.org/T384307#10479484 (10Krinkle) [14:06:45] Thanks moritzm - We'll get to it asap. [14:07:27] it actually seems to have resolved itself [14:08:05] I ran ncdu on the / partition to figure out if there's any culprits and given all the mounts it took over half an hour :-) [14:09:15] and when that was done there was a lot in /tmp, but the root device had recovered to 12G free space [14:43:20] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence, and 5 others: Add "event_is_test_event" field to "campaign_events" table - https://phabricator.wikimedia.org/T381759#10479885 (10MHorsey-WMF) @Daimona yes, I was planning to retain this ticket for that process. [14:52:23] 06Data-Engineering, 06Data-Platform-SRE: Update canary_events DAG to use the internal domain to obtain its eventstream config - https://phabricator.wikimedia.org/T384329 (10BTullis) 03NEW [14:52:56] 06Data-Engineering, 06Data-Platform-SRE: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10479941 (10BTullis) [15:07:12] 06Data-Engineering, 06Data-Platform-SRE: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10480045 (10BTullis) @aqu has mentioned that we could change the `stream_config_url` to be an HDFS URL, rather than an... [15:28:06] 06Data-Engineering, 06Product-Analytics, 10Event-Platform, 13Patch-For-Review: Enable Event Platform instruments to opt out of collecting User-Agent data - https://phabricator.wikimedia.org/T382173#10480130 (10Ottomata) [15:31:38] 06Data-Engineering, 06Product-Analytics, 10Event-Platform, 13Patch-For-Review: Enable Event Platform instruments to opt out of collecting User-Agent data - https://phabricator.wikimedia.org/T382173#10480171 (10Ottomata) [15:32:51] 06Data-Engineering, 10Commons-Impact-Metrics: "Pageview counts for a given category" not offering "deep" metrics - https://phabricator.wikimedia.org/T382733#10480190 (10mforns) @GFontenelle_WMF The deep metrics are only available for the primary categories (the ones in the allow-list). This restriction is unfo... [15:34:40] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0 (Kanban Board): Calculate rough HDFS storage requirements for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T383816#10480202 (10xcollazo) [15:35:01] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0 (Kanban Board): Calculate rough HDFS storage requirements for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T383816#10480205 (10xcollazo) 05Open→03Resolved [15:35:25] 06Data-Engineering, 06Data-Platform-SRE: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10480210 (10Ottomata) > change the stream_config_url to be an HDFS URL, rather than an HTTPS URL, as a workaround. Th... [15:37:47] 06Data-Engineering, 06Data-Platform-SRE: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10480223 (10BTullis) Yes, this should be a short-term workaround only. The better solution would be to use the envoy s... [15:49:19] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence, and 5 others: Add "event_is_test_event" field to "campaign_events" table - https://phabricator.wikimedia.org/T381759#10480294 (10MHorsey-WMF) [15:51:35] 06Data-Engineering, 10Add-Link, 10GrowthExperiments-NewcomerTasks, 10Growth-Team (Current Sprint), and 2 others: Make growthexperiments_link_recommendations.gelr_data nullable in GrowthExperiments - https://phabricator.wikimedia.org/T383864#10480314 (10Urbanecm_WMF) 05Open→03Resolved [16:13:11] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Experimentation Lab, 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Dashboard and alerting of data quality metrics for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T357684#10480395 (10xcollazo) [16:17:01] 06Data-Engineering, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops, 13Patch-For-Review: Add support for JIT in PHP8.1 images - https://phabricator.wikimedia.org/T384294#10480405 (10Joe) @jijiki I've looked into JIT in php 8.1 a bit, and it seems like it had several serious bugs (see for exam... [16:28:13] 06Data-Engineering, 10Dumps 2.0 (Kanban Board), 10Event-Platform: [Event Platform] Instrument EventBus with prometheus MW Statslib - https://phabricator.wikimedia.org/T363587#10480434 (10Ottomata) 05Open→03Resolved [16:32:27] 06Data-Engineering, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, 10MediaWiki-Platform-Team (Radar), 07Spike: [SPIKE] Investigate possible event loss on navigation in Google Chrome - https://phabricator.wikimedia.org/T384307#10480450 (10phuedx) >>! In T384307#10479482, @Krinkle wrote: > @ph... [16:39:11] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 06Release-Engineering-Team, 06Web-Team: Allow JavaScript errors to fail CI builds - https://phabricator.wikimedia.org/T318902#10480485 (10Ottomata) No opinion on who best suited to do task. Q: is using `mediawiki.client.error` to detect this the bes... [16:39:57] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06SRE, 06Traffic, 13Patch-For-Review: Refine add_is_wmf_domain TransformFunction fails if no source field exists - https://phabricator.wikimedia.org/T383914#10480486 (10Ahoelzl) [16:41:45] 06Data-Engineering, 07Sustainability: Move some anlaytics jobs to day time in Virginia - https://phabricator.wikimedia.org/T384166#10480495 (10Ottomata) Related: {T371321}. I'd expect that energy consumption would go down if we stopped searching for needles in the webrequest haystack. [16:53:06] 10Data-Engineering (Q3 2024 January 1st - March 31th): Analyze Dumps Usage Through Apache Logs - https://phabricator.wikimedia.org/T383175#10480572 (10JEbe-WMF) Hi @VirginiaPoundstone yes, thank you for asking. I just finished a preliminary analysis of the logs by months - months because I kept running into netw... [16:57:41] 06Data-Engineering, 10GrowthExperiments, 10Growth-Team (Current Sprint), 10MW-1.44-notes (1.44.0-wmf.13; 2025-01-21), 07Schema-change: The `growthexperiments_mentor_mentee.gemm_mentee_is_active` DB column usees the "boolean" type, which is not supported... - https://phabricator.wikimedia.org/T383778#10480603 [17:06:46] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 06Release-Engineering-Team, 06Web-Team: Allow JavaScript errors to fail CI builds - https://phabricator.wikimedia.org/T318902#10480672 (10Jdlrobson) FWIW Right now, if code was committed that caused JS errors this would likely be picked up in log tri... [17:06:52] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 06Release-Engineering-Team, 06Web-Team: Allow JavaScript errors to fail CI builds - https://phabricator.wikimedia.org/T318902#10480675 (10Jdlrobson) p:05Triage→03Medium [17:26:57] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Experimentation Lab (Experiment Platform Sprint 1), 10MediaWiki-Platform-Team (Radar), 07Spike: [SPIKE] Investigate possible event loss on navigation in Google Chrome - https://phabricator.wikimedia.org/T384307#10480808 (10phuedx) [17:56:52] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10Data-Platform-SRE (2025.01.11 - 2025.01.31): Data Platform access streamlining for WMDE staff - https://phabricator.wikimedia.org/T381824#10480892 (10jcrespo) Is there anything else to do here (are there any concerns left?), other than fixing documenta... [18:00:08] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10Data-Platform-SRE (2025.01.11 - 2025.01.31): Data Platform access streamlining for WMDE staff - https://phabricator.wikimedia.org/T381824#10480907 (10Ottomata) Olja approved, so no concerns left. Just needs to be implemented by fixing docs, etc. Than... [18:00:33] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10Data-Platform-SRE (2025.01.11 - 2025.01.31): Data Platform access streamlining for WMDE staff - https://phabricator.wikimedia.org/T381824#10480909 (10jcrespo) a:03jcrespo [18:00:36] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10Data-Platform-SRE (2025.01.11 - 2025.01.31): Data Platform access streamlining for WMDE staff - https://phabricator.wikimedia.org/T381824#10480910 (10jcrespo) p:05Triage→03Medium [18:22:41] 10Data-Engineering (Q3 2024 January 1st - March 31th): [Developer Experience] Implement CI hql Linting - https://phabricator.wikimedia.org/T360967#10481133 (10Ahoelzl) Moving to backlog / next up for now. [18:27:12] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0 (Kanban Board): Create and test a new produced_by config for the datalake table - https://phabricator.wikimedia.org/T381432#10481159 (10Ahoelzl) [18:29:53] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence, and 5 others: Add "event_is_test_event" field to "campaign_events" table - https://phabricator.wikimedia.org/T381759#10481160 (10Daimona) [18:30:33] 06Data-Engineering: Migrate refinery HQL files to CI/CD supported GitLab repository - https://phabricator.wikimedia.org/T362832#10481163 (10Ahoelzl) [18:30:44] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence, and 6 others: Add "event_is_test_event" field to "campaign_events" table - https://phabricator.wikimedia.org/T381759#10481165 (10Daimona) [18:30:56] 10Data-Engineering (Q3 2024 January 1st - March 31th): Migrate and re-deploy eventgate using new service-utils - https://phabricator.wikimedia.org/T361768#10481167 (10Ahoelzl) [18:31:29] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence, and 7 others: Add "event_is_test_event" field to "campaign_events" table - https://phabricator.wikimedia.org/T381759#10481170 (10Daimona) [18:32:08] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Epic, 13Patch-For-Review: [Maintenance] Safeguard VarnishKafka to HAProxy analytics transition - https://phabricator.wikimedia.org/T354694#10481175 (10Ahoelzl) a:05gmodena→03Antoine_Quhen [18:32:49] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0 (Kanban Board): Create and test a new produced_by config for the datalake table - https://phabricator.wikimedia.org/T381432#10481181 (10xcollazo) 05Open→03In progress [18:35:46] 06Data-Engineering: [SPIKE] Learn and document how to use Flink-CDC from MediaWiki MariaDB locally - https://phabricator.wikimedia.org/T373144#10481198 (10Ahoelzl) [18:45:59] 06Data-Engineering: [Data Quality] Improve Superset visualizations - https://phabricator.wikimedia.org/T372678#10481243 (10Ahoelzl) [18:46:15] 10Data-Engineering (Q3 2024 January 1st - March 31th): [Data Quality] Improve Superset visualizations - https://phabricator.wikimedia.org/T372678#10481246 (10Ahoelzl) [18:51:30] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Traffic, 10Experimentation Lab Radar: Cookie % has been rejected because it is foreign and does not have the "Partitioned" attribute - https://phabricator.wikimedia.org/T375256#10481296 (10Ahoelzl) [18:51:38] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Traffic, 07Essential-Work, 10Experimentation Lab Radar: Cookie % has been rejected because it is foreign and does not have the "Partitioned" attribute - https://phabricator.wikimedia.org/T375256#10481297 (10Ottomata) > @mforns to confirm whether this... [18:52:11] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Traffic, 07Essential-Work, 10Experimentation Lab Radar: Cookie % has been rejected because it is foreign and does not have the "Partitioned" attribute - https://phabricator.wikimedia.org/T375256#10481301 (10Ahoelzl) a:03mforns [18:54:21] 06Data-Engineering, 07Epic: [Epic] Migrate Data Engineering maintained NodeJS repositories to GitLab - https://phabricator.wikimedia.org/T366614#10481320 (10tchin) [18:54:28] 06Data-Engineering, 06Data-Platform-SRE, 07Essential-Work: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10481321 (10Ottomata) p:05Triage→03High a:03Antoine_Quhen [18:56:44] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 07Essential-Work, 10Experimentation Lab (Experiment Platform Sprint 1), and 2 others: [SPIKE] Investigate possible event loss on navigation in Google Chrome - https://phabricator.wikimedia.org/T384307#10481327 (10Ottomata) [18:57:40] 06Data-Engineering, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops, 13Patch-For-Review: Add support for JIT in PHP8.1 images - https://phabricator.wikimedia.org/T384294#10481330 (10Ottomata) [19:01:07] 06Data-Engineering, 10Data-Engineering-Wikistats, 07Essential-Work: Migrate from Webpack to Vite - https://phabricator.wikimedia.org/T384045#10481350 (10Ottomata) [19:03:04] 06Data-Engineering: Migrate refinery HQL files to CI/CD supported GitLab repository - https://phabricator.wikimedia.org/T362832#10481355 (10Ahoelzl) p:05Triage→03High [19:12:23] 10Data-Engineering (Q3 2024 January 1st - March 31th): Analyze Dumps Usage Through Apache Logs - https://phabricator.wikimedia.org/T383175#10481404 (10Ahoelzl) Preparation work - Download all files comprising 6 months of data - Making files available for data analysis Analysis goals - Counts of detailed access... [19:31:08] (03CR) 10Ottomata: "Thank you! Some nits." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1112392 (https://phabricator.wikimedia.org/T383914) (owner: 10Aqu) [19:56:45] 06Data-Engineering, 10DPE Temporary Accounts (Sprint 1), 07Epic: [Epic] Modify MediaWiki History pipeline for Temp Accounts - https://phabricator.wikimedia.org/T377304#10481605 (10Ahoelzl) [19:56:51] 06Data-Engineering, 10DPE Temporary Accounts (Sprint 1), 07Epic: [Epic] Modify DPE pipelines to account for Temp Accounts - https://phabricator.wikimedia.org/T377293#10481606 (10Ahoelzl) [20:34:16] (03CR) 10GergesShamon: "recheck" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1112849 (https://phabricator.wikimedia.org/T384265) (owner: 10Mr.Ibrahem) [20:36:31] 14Analytics-Kanban, 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Event-Platform, 05Goal: Modern Event Platform - https://phabricator.wikimedia.org/T185233#10481690 (10Ottomata) [20:36:44] 14Analytics-Kanban, 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10Event-Platform, 05Goal: Modern Event Platform - https://phabricator.wikimedia.org/T185233#10481691 (10Ottomata) [20:38:44] <_Gerges> Hi [20:43:53] 06Data-Engineering, 06Product-Analytics, 10Event-Platform: Migrate legacy metawiki schemas to Event Platform - https://phabricator.wikimedia.org/T259163#10481697 (10Ottomata) Deleted sudo -u analytics kerberos-run-command analytics hdfs dfs -rm -f -r /wmf/data/event/autoblockipblock /wmf/data/event/cookieb... [20:54:02] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Optimize XML Dump code to be able to handle wikis from simplewiki to enwiki - https://phabricator.wikimedia.org/T381016#10481739 (10xcollazo) [21:07:00] 06Data-Engineering: Airflow job to do monthly XML dumps - https://phabricator.wikimedia.org/T384381 (10xcollazo) 03NEW [21:13:30] 06Data-Engineering, 10Data-Engineering-Wikistats, 13Patch-For-Review: Enable arabic language - https://phabricator.wikimedia.org/T384265#10481808 (10Mr.Ibrahem) a:05Mr.Ibrahem→03None @Milimetric to review [21:14:08] 06Data-Engineering, 10Data-Engineering-Wikistats, 13Patch-For-Review: Enable arabic language - https://phabricator.wikimedia.org/T384265#10481811 (10Mr.Ibrahem) a:03Mr.Ibrahem [21:15:16] 06Data-Engineering, 10Dumps 2.0: Productionization of code to dump in XML - https://phabricator.wikimedia.org/T384382 (10xcollazo) 03NEW [21:16:22] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Optimize XML Dump code to be able to handle wikis from simplewiki to enwiki - https://phabricator.wikimedia.org/T381016#10481828 (10xcollazo) [21:16:24] 06Data-Engineering, 10Dumps 2.0: Productionization of code to dump in XML - https://phabricator.wikimedia.org/T384382#10481829 (10xcollazo) [21:16:39] 06Data-Engineering, 10Dumps 2.0: Productionization of code to dump in XML - https://phabricator.wikimedia.org/T384382#10481831 (10xcollazo) [21:20:03] 06Data-Engineering, 10Dumps 2.0: Modify XML dumping code to be able to do 'partial' dumps - https://phabricator.wikimedia.org/T384383 (10xcollazo) 03NEW [21:23:17] 06Data-Engineering, 10Dumps 2.0: Refactor code to use new table and column names - https://phabricator.wikimedia.org/T384385 (10xcollazo) 03NEW [21:28:13] 14Data-Engineering (Q2 2024 October 1st - December 31th), 10MediaWiki-Platform-Team (Radar): Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#10481911 (10Ottomata) [21:56:12] 06Data-Engineering, 10Dumps 2.0: Modify XML dumping code to be able to do 'partial' dumps - https://phabricator.wikimedia.org/T384383#10482026 (10xcollazo) [21:58:21] 06Data-Engineering, 10Dumps 2.0: Put together a DPE Deep Dive session on learnings from optimizing this code - https://phabricator.wikimedia.org/T384392 (10xcollazo) 03NEW [22:07:52] 06Data-Engineering: HDFS capacity needs for XML Dumps temporary storage - https://phabricator.wikimedia.org/T384397 (10xcollazo) 03NEW [22:08:22] 06Data-Engineering: HDFS capacity needs for XML Dumps temporary storage - https://phabricator.wikimedia.org/T384397#10482131 (10xcollazo) [22:08:38] 06Data-Engineering, 10Dumps 2.0: HDFS capacity needs for XML Dumps temporary storage - https://phabricator.wikimedia.org/T384397#10482133 (10xcollazo) [22:08:58] 06Data-Engineering, 10Dumps 2.0 (Kanban Board): HDFS capacity needs for XML Dumps temporary storage - https://phabricator.wikimedia.org/T384397#10482137 (10xcollazo)