[07:50:56] 10serviceops: httpbb needs full path of test file - https://phabricator.wikimedia.org/T285294 (10Joe) [07:51:04] 10serviceops: httpbb needs full path of test file - https://phabricator.wikimedia.org/T285294 (10Joe) p:05Triage→03Lowest [08:02:31] 10serviceops, 10MW-on-K8s, 10SRE: Make all httpbb tests pass on the mwdebug deployment. - https://phabricator.wikimedia.org/T285298 (10Joe) [08:02:46] 10serviceops, 10MW-on-K8s, 10SRE: Make all httpbb tests pass on the mwdebug deployment. - https://phabricator.wikimedia.org/T285298 (10Joe) p:05Triage→03High a:03Joe [08:14:49] 10serviceops, 10CX-cxserver, 10Wikidata, 10wdwb-tech, 10Language-Team (Language-2021-April-June): cxserver: https://cxserver.wikimedia.org/v2/suggest/source/Paneer/ca?sourcelanguages=en occasionally fails with HTTP 503 - https://phabricator.wikimedia.org/T285219 (10Nikerabbit) It's likely that the error... [08:51:02] 10serviceops, 10MW-on-K8s, 10SRE: Make all httpbb tests pass on the mwdebug deployment. - https://phabricator.wikimedia.org/T285298 (10Joe) On second thoughts: we're mostly using http where we could just use: ` request_headers: X-Forwarded-Proto: http ` instead. I'll do that where appropriate. [08:57:23] 10serviceops, 10CX-cxserver, 10Wikidata, 10wdwb-tech, 10Language-Team (Language-2021-April-June): cxserver: https://cxserver.wikimedia.org/v2/suggest/source/Paneer/ca?sourcelanguages=en occasionally fails with HTTP 503 - https://phabricator.wikimedia.org/T285219 (10Addshore) I'm not sure of anything that... [09:01:49] 10serviceops, 10CX-cxserver, 10Wikidata, 10wdwb-tech, 10Language-Team (Language-2021-April-June): cxserver: https://cxserver.wikimedia.org/v2/suggest/source/Paneer/ca?sourcelanguages=en occasionally fails with HTTP 503 - https://phabricator.wikimedia.org/T285219 (10Nikerabbit) See * https://github.com/wi... [09:31:05] 10serviceops, 10Wikidata, 10Wikidata-Query-Service, 10wdwb-tech, and 2 others: Deploy Flink (rdf-streaming-updater) to kubernetes (k8s) - https://phabricator.wikimedia.org/T264006 (10jijiki) [09:33:28] 10serviceops, 10Wikidata, 10Wikidata-Query-Service, 10wdwb-tech, and 2 others: Deploy Flink (rdf-streaming-updater) to kubernetes (k8s) - https://phabricator.wikimedia.org/T264006 (10jijiki) @Zbyszko, @dcausse, we will try to clear any leftovers this week so you can do a first deployment to staging. Sinc... [09:34:19] 10serviceops, 10Wikidata, 10Wikidata-Query-Service, 10wdwb-tech, and 2 others: Deploy Flink (rdf-streaming-updater) to kubernetes (k8s) - https://phabricator.wikimedia.org/T264006 (10Zbyszko) @jijiki Great, thank you! [10:10:10] 10serviceops, 10MW-on-K8s, 10SRE, 10Patch-For-Review: Make all httpbb tests pass on the mwdebug deployment. - https://phabricator.wikimedia.org/T285298 (10Joe) [10:30:56] 10serviceops, 10SRE, 10Patch-For-Review, 10Performance-Team (Radar), 10User-jijiki: Enable TLS on memcached for cross-dc replication - https://phabricator.wikimedia.org/T271967 (10jijiki) [10:31:24] 10serviceops, 10SRE, 10Patch-For-Review, 10Performance-Team (Radar), 10User-jijiki: Enable TLS on memcached for cross-dc replication - https://phabricator.wikimedia.org/T271967 (10jijiki) Tested using the codfw TLS pool on eqiad canaries https://gerrit.wikimedia.org/r/c/operations/puppet/+/699908 [12:46:49] 10serviceops, 10MW-on-K8s, 10SRE: Install wiki-specific php extensions in the mediawiki production image - https://phabricator.wikimedia.org/T285309 (10Joe) [12:47:22] 10serviceops, 10MW-on-K8s, 10SRE, 10Patch-For-Review: Make all httpbb tests pass on the mwdebug deployment. - https://phabricator.wikimedia.org/T285298 (10Joe) [13:03:00] 10serviceops, 10MW-on-K8s, 10SRE: Install wiki-specific php extensions in the mediawiki production image - https://phabricator.wikimedia.org/T285309 (10Joe) p:05Triage→03High [15:43:30] 10serviceops, 10MW-on-K8s, 10Release-Engineering-Team, 10SRE: Check out www-portals repo in the mediawiki-webserver and in the mediawiki-multiversion images - https://phabricator.wikimedia.org/T285325 (10Joe) [15:43:59] 10serviceops, 10MW-on-K8s, 10SRE, 10Patch-For-Review: Make all httpbb tests pass on the mwdebug deployment. - https://phabricator.wikimedia.org/T285298 (10Joe) [16:57:22] hello folks, anythings against me merging https://gerrit.wikimedia.org/r/c/operations/docker-images/production-images/+/700396 ? [16:57:37] those are the new istio docker images, basically a new version [16:57:39] nothing big [17:22:20] 10serviceops, 10SRE, 10Datacenter-Switchover: Siteinfo timeout during switch datacenter - https://phabricator.wikimedia.org/T266618 (10Legoktm) > it's connecting to port 80 with the x-forwarded-proto header, and that should probably be updated. This is easy, I'll upload a patch. > the warmup script didn't... [20:28:32] 10serviceops, 10Performance-Team, 10SRE, 10MW-1.36-notes, and 3 others: Enable "/*/mw-with-onhost-tier/" route for MediaWiki where safe - https://phabricator.wikimedia.org/T264604 (10Krinkle) Based on very rudimentary testing it seems that there is quite a direct correlation between the number of (concurre... [21:17:59] _joe_: wondering if you have a chance soon to look over the latest set of changes for this helm chart! https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/688358