[06:14:12] <_joe_> there are good reasons to block HTTP methods like TRACE and PATCH by default [06:14:40] <_joe_> there's a lot of httpd servers out there that do stupid things if you allow TRACE [06:14:47] <_joe_> including apache HTTPD [10:26:13] 10serviceops, 10Maps, 10Product-Infrastructure-Team-Backlog: Tegola cronjob cannot connect to kafka-main cluster - https://phabricator.wikimedia.org/T293134 (10Jgiannelos) 05Open→03Resolved a:03Jgiannelos [10:30:27] 10serviceops, 10Wikidata, 10Wikidata-Query-Service, 10wdwb-tech: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes - https://phabricator.wikimedia.org/T293063 (10jijiki) [10:31:16] 10serviceops, 10Prod-Kubernetes, 10Wikidata, 10Wikidata-Query-Service, 10wdwb-tech: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes - https://phabricator.wikimedia.org/T293063 (10jijiki) [11:52:46] 10serviceops: Upgrade mc* and mc-gp* hosts to Debian Bullseye - https://phabricator.wikimedia.org/T293216 (10jijiki) [12:57:38] 10serviceops, 10MW-on-K8s, 10SRE: Evaluate istio as an ingress for production usage - https://phabricator.wikimedia.org/T287007 (10JMeybohm) [18:32:39] 10serviceops, 10Shellbox, 10User-brennen, 10Wikimedia-production-error: Shellbox\ShellboxError: Shellbox server returned status code 503 - https://phabricator.wikimedia.org/T292663 (10dancy) @Legoktm A question that came up during train log triage today: Is there a logstash dashboard for Shellbox? [18:37:40] deploying gitlab backup-restore timer by Arnold on gitlab machines [18:42:35] talking on operations [19:31:58] 10serviceops, 10MediaWiki-General, 10Performance-Team (Radar), 10SecTeam-Processed, 10Security: Create a tmp directory just for MediaWiki - https://phabricator.wikimedia.org/T179901 (10Mstyles) [19:45:13] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup/install kubernetes10[18-21] - https://phabricator.wikimedia.org/T290202 (10Jclark-ctr) [19:48:12] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup/install kubernetes10[18-21] - https://phabricator.wikimedia.org/T290202 (10Jclark-ctr) kubernetes1021 is up and bios configured [20:01:27] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup/install kubernetes10[18-21] - https://phabricator.wikimedia.org/T290202 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by robh@cumin1001 for host kubernetes1021.eqiad.wmnet with OS stretch [20:04:06] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup/install kubernetes10[18-21] - https://phabricator.wikimedia.org/T290202 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by robh@cumin1001 for host kubernetes1021.eqiad.wmnet with OS stretch executed with errors: -... [20:04:45] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup/install kubernetes10[18-21] - https://phabricator.wikimedia.org/T290202 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by robh@cumin1001 for host kubernetes1021.eqiad.wmnet with OS stretch [20:27:48] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup/install kubernetes10[18-21] - https://phabricator.wikimedia.org/T290202 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by robh@cumin1001 for host kubernetes1021.eqiad.wmnet with OS stretch completed: - kubernetes1... [20:30:18] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup/install kubernetes10[18-21] - https://phabricator.wikimedia.org/T290202 (10RobH) 05Open→03Resolved a:05Cmjohnson→03RobH [21:17:53] 10serviceops, 10Toolhub, 10Patch-For-Review: Get mcrouter & prometheus-mcrouter-exporter tags for helmfile.d from upstream config - https://phabricator.wikimedia.org/T291530 (10bd808) 05Open→03In progress p:05Triage→03Medium a:03Joe The #toolhub specific bits of this are done thanks @joe. I'm assig...