[04:53:27] <jinxer-wm>	 (HiveServerHeapUsage) firing: Hive Server JVM Heap usage is above 80% on an-test-coord1001:10100 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hive/Alerts#Hive_Server_Heap_Usage - https://grafana.wikimedia.org/d/000000379/hive?panelId=7&fullscreen&orgId=1&var-instance=an-test-coord1001:10100 - https://alerts.wikimedia.org
[05:18:27] <jinxer-wm>	 (HiveServerHeapUsage) resolved: Hive Server JVM Heap usage is above 80% on an-test-coord1001:10100 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hive/Alerts#Hive_Server_Heap_Usage - https://grafana.wikimedia.org/d/000000379/hive?panelId=7&fullscreen&orgId=1&var-instance=an-test-coord1001:10100 - https://alerts.wikimedia.org
[07:00:27] <jinxer-wm>	 (HiveServerHeapUsage) firing: Hive Server JVM Heap usage is above 80% on an-test-coord1001:10100 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hive/Alerts#Hive_Server_Heap_Usage - https://grafana.wikimedia.org/d/000000379/hive?panelId=7&fullscreen&orgId=1&var-instance=an-test-coord1001:10100 - https://alerts.wikimedia.org
[07:10:27] <jinxer-wm>	 (HiveServerHeapUsage) resolved: Hive Server JVM Heap usage is above 80% on an-test-coord1001:10100 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hive/Alerts#Hive_Server_Heap_Usage - https://grafana.wikimedia.org/d/000000379/hive?panelId=7&fullscreen&orgId=1&var-instance=an-test-coord1001:10100 - https://alerts.wikimedia.org
[07:42:35] <elukey>	 https://feast.dev/blog/a-state-of-feast/ - very interesting :)
[07:59:41] <elukey>	 joal: bonjour, during the next days I'd need to talk about cassandra for the online feature store use case :)
[09:04:27] <jinxer-wm>	 (HiveServerHeapUsage) firing: Hive Server JVM Heap usage is above 80% on an-test-coord1001:10100 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hive/Alerts#Hive_Server_Heap_Usage - https://grafana.wikimedia.org/d/000000379/hive?panelId=7&fullscreen&orgId=1&var-instance=an-test-coord1001:10100 - https://alerts.wikimedia.org
[09:09:27] <jinxer-wm>	 (HiveServerHeapUsage) resolved: Hive Server JVM Heap usage is above 80% on an-test-coord1001:10100 - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hive/Alerts#Hive_Server_Heap_Usage - https://grafana.wikimedia.org/d/000000379/hive?panelId=7&fullscreen&orgId=1&var-instance=an-test-coord1001:10100 - https://alerts.wikimedia.org
[10:09:51] <milimetric>	 hi joal, I'm around now
[10:19:24] <elukey>	 wow good morning milimetric :D
[10:20:21] <milimetric>	 morning elukey :) I'm on kid duty today so my morning TV watching goes to work instead :P
[10:33:49] <elukey>	 :D
[12:11:23] <joal>	 Hi milimetric - sorry I missed your ping earlier
[12:15:43] <joal>	 I'm assuming you're with kids - let me know when you have some time :)
[12:24:41] <milimetric>	 yep, kids until nanny gets here
[14:00:27] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban: Some varnishkafka instances dropped traffic for a long time due to the wrong version of the package installed - https://phabricator.wikimedia.org/T300164 (10elukey) >>! In T300164#7730882, @elukey wrote: > The varnishkafka package version will be handled in T30230...
[15:22:43] <wikibugs>	 10Data-Engineering, 10Airflow: [Airflow] Spike investigate of better ways to organize/access Airflow logs - https://phabricator.wikimedia.org/T302500 (10mforns)
[15:30:29] <milimetric>	 joal: I'm free!
[15:56:53] <wikibugs>	 (03CR) 10Sergio Gimeno: [C: 03+2] Add an image: add confirm_reject_suggestion action [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/765351 (https://phabricator.wikimedia.org/T302429) (owner: 10MewOphaswongse)
[15:57:32] <wikibugs>	 (03Merged) 10jenkins-bot: Add an image: add confirm_reject_suggestion action [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/765351 (https://phabricator.wikimedia.org/T302429) (owner: 10MewOphaswongse)
[16:50:09] <wikibugs>	 (03CR) 10Tchanders: Basic ipinfo instrument setup (034 comments) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte)
[17:12:24] <wikibugs>	 10Data-Engineering-Kanban, 10Airflow: [Airflow] Spike investigate of better ways to organize/access Airflow logs - https://phabricator.wikimedia.org/T302500 (10EChetty)
[17:23:04] <wikibugs>	 10Data-Engineering-Kanban: Modify HiveToDruid Job - https://phabricator.wikimedia.org/T302514 (10EChetty)
[17:24:56] <wikibugs>	 10Data-Engineering-Kanban, 10Data-Engineering-Radar: The network_internal druid load job fails if data is not present - https://phabricator.wikimedia.org/T302263 (10EChetty)
[17:28:47] <wikibugs>	 10Data-Engineering-Kanban, 10Product-Analytics: Improvements to mediawiki_geoeditors_monthly dimensions - https://phabricator.wikimedia.org/T302079 (10EChetty)
[17:29:12] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Product-Analytics: Improvements to mediawiki_geoeditors_monthly dimensions - https://phabricator.wikimedia.org/T302079 (10EChetty)
[17:32:43] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Log_param is redacted in wiki replica when only comment and/or user should be - https://phabricator.wikimedia.org/T301943 (10EChetty)
[17:33:07] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Patch-For-Review, 10cloud-services-team (Kanban): Log_param is redacted in wiki replica when only comment and/or user should be - https://phabricator.wikimedia.org/T301943 (10EChetty) a:03razzi
[17:36:14] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Platform Engineering, and 2 others: Log_param is redacted in wiki replica when only comment and/or user should be - https://phabricator.wikimedia.org/T301943 (10EChetty)
[17:38:27] <wikibugs>	 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform, 10TimedMediaHandler, 10Wikimedia-Video: Record and report metrics for audio and video playback - https://phabricator.wikimedia.org/T108522 (10EChetty)
[17:40:00] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban: Upgrade Turnilo - https://phabricator.wikimedia.org/T301990 (10EChetty)
[17:41:35] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Product-Analytics: conda-create-stacked breaks wmfdata.presto - https://phabricator.wikimedia.org/T301734 (10EChetty)
[17:42:27] <wikibugs>	 10Data-Engineering: krb1001's auth.log grows a lot causing disk space issues for the root partition - https://phabricator.wikimedia.org/T302518 (10elukey)
[17:45:06] <wikibugs>	 10Data-Engineering, 10Growth-Team, 10GrowthExperiments, 10MediaWiki-extensions-EventLogging, and 2 others: Create a test for end-to-end event logging data verification of happy path with Special:Homepage and suggested edits - https://phabricator.wikimedia.org/T301463 (10EChetty)
[17:46:28] <wikibugs>	 10Data-Engineering: krb1001's auth.log grows a lot causing disk space issues for the root partition - https://phabricator.wikimedia.org/T302518 (10EChetty) a:03razzi
[17:48:39] <wikibugs>	 10Analytics, 10Data-Engineering, 10Pageviews-API: Track page views by page ID rather than title (handles moved pages) - https://phabricator.wikimedia.org/T159046 (10EChetty) 05Open→03Declined
[17:50:44] <wikibugs>	 10Analytics-Wikistats, 10Data-Engineering: Provide link to csv file for every table in every report - https://phabricator.wikimedia.org/T62811 (10EChetty) 05Open→03Resolved
[17:51:49] <wikibugs>	 10Analytics-Radar, 10Analytics-Wikistats, 10Data-Engineering: Restore WikiStats features disabled for mere performance reasons - https://phabricator.wikimedia.org/T44318 (10EChetty) 05Open→03Declined
[17:53:48] <wikibugs>	 10Data-Engineering: [Anomaly detection] Create a heatmap view in Superset - https://phabricator.wikimedia.org/T301572 (10EChetty) a:03EChetty
[17:58:21] <wikibugs>	 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform, 10Platform Engineering, and 2 others: eventlogging_VisualEditorTemplateDialogUse: '.event.template_names[0]' should be string - https://phabricator.wikimedia.org/T299779 (10EChetty)
[17:58:46] <wikibugs>	 (03PS1) 10Milimetric: Add wikis to sqoop list [analytics/refinery] - 10https://gerrit.wikimedia.org/r/765578 (https://phabricator.wikimedia.org/T299548)
[17:59:39] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Airflow: [Anomaly detection] Allow for custom email alert content - https://phabricator.wikimedia.org/T301571 (10EChetty)
[18:03:59] <wikibugs>	 10Data-Engineering, 10Research-Backlog, 10Stewards-and-global-tools: Collect information about users affected by blocks - https://phabricator.wikimedia.org/T297051 (10EChetty) p:05Triage→03Low
[18:04:57] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Airflow: [Airflow] Research, discuss and decide on DAG/task dependencies VS. success/failure files (Oozie style) - https://phabricator.wikimedia.org/T301568 (10EChetty)
[18:07:47] <wikibugs>	 (03CR) 10Razzi: [C: 03+1] Add wikis to sqoop list [analytics/refinery] - 10https://gerrit.wikimedia.org/r/765578 (https://phabricator.wikimedia.org/T299548) (owner: 10Milimetric)
[18:16:43] <wikibugs>	 10Analytics, 10Data-Engineering, 10Event-Platform, 10Metrics-Platform: jsonschema-tools tests should fail if schema $id does not match title or path - https://phabricator.wikimedia.org/T300404 (10EChetty)
[18:19:33] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10SRE, 10observability, 10serviceops: Upgrade Kafka to 2.x - https://phabricator.wikimedia.org/T300102 (10EChetty)
[18:22:48] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Anti-Harassment, 10Product-Analytics: Distinguish between types of block events in the Mediawiki user history table - https://phabricator.wikimedia.org/T213583 (10EChetty)
[18:24:39] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Anti-Harassment, 10Product-Analytics: Mediawiki history has no data on IP blocks - https://phabricator.wikimedia.org/T211627 (10EChetty)
[18:26:39] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban: Some varnishkafka instances dropped traffic for a long time due to the wrong version of the package installed - https://phabricator.wikimedia.org/T300164 (10JAllemandou) >>! In T300164#7735054, @elukey wrote: >>>! In T300164#7730882, @elukey wrote: >> The varnishk...
[18:28:18] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10Documentation, 10cloud-services-team (Kanban): Document on wikitech the general process of getting a table/column exposed to Wiki Replica users - https://phabricator.wikimedia.org/T209992 (10EChetty)
[18:28:55] <wikibugs>	 10Data-Engineering, 10Data-Engineering-Kanban: Consider resizing an-test-coord1001 partitions - https://phabricator.wikimedia.org/T299930 (10EChetty) 05Open→03Resolved
[18:59:12] <wikibugs>	 (03PS6) 10Joal: [WIP] Add flink job reporting webrequest patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/763610
[19:17:59] <wikibugs>	 (03PS41) 10AGueyte: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415)
[19:18:46] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte)
[19:29:20] <wikibugs>	 (03PS42) 10AGueyte: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415)
[19:34:13] <wikibugs>	 (03CR) 10AGueyte: Basic ipinfo instrument setup (034 comments) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte)
[21:26:29] <wikibugs>	 (03CR) 10Sharvaniharan: "After an eye-opening discussion with @Mikhail Popov, will be making a new patch for this change which will hold the new app-related variab" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/761452 (owner: 10Sharvaniharan)
[21:26:40] <wikibugs>	 (03Abandoned) 10Sharvaniharan: Add a required variable to app analytics fragment [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/761452 (owner: 10Sharvaniharan)
[22:11:34] <wikibugs>	 (03PS1) 10Milimetric: Fix as many security vulnerabilities as possible [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/765647
[22:30:39] <wikibugs>	 10Analytics, 10Code-Health-Objective, 10Epic, 10Platform Engineering Roadmap, and 2 others: [DISCUSS]: Problem details for HTTP APIs (rfc7807) - https://phabricator.wikimedia.org/T302536 (10Eevans)
[22:33:23] <wikibugs>	 10Data-Engineering, 10Product-Analytics: kerberos::systemd_timer should have a smarter default for syslog_identifier - https://phabricator.wikimedia.org/T302533 (10mpopov)
[23:03:45] <wikibugs>	 10Data-Engineering, 10Patch-For-Review, 10Product-Analytics (Kanban): Test log file and error notification - https://phabricator.wikimedia.org/T295733 (10Mayakp.wiki) 05Open→03Resolved
[23:03:59] <wikibugs>	 10Data-Engineering, 10Patch-For-Review, 10Product-Analytics (Kanban): Test log file and error notification - https://phabricator.wikimedia.org/T295733 (10Mayakp.wiki) Thanks @mpopov for helping with filing the task T302533.   I will go ahead and close this task as we've officially resolved this issue. Once a...