[07:38:33] 06serviceops, 06MW-Interfaces-Team, 10RESTBase Sunsetting, 13Patch-For-Review: Switchover plan from RESTbase to REST Gateway for rest_v1/page/html and rest_v1/page/title endpoints - https://phabricator.wikimedia.org/T374683#10236661 (10akosiaris) > I suggest we let the change ride next week's train and act... [08:52:35] 06serviceops, 06Data-Persistence, 13Patch-For-Review: Sessionstore's discovery TLS cert will expire before end of May 2024 - https://phabricator.wikimedia.org/T363996#10236821 (10elukey) @hnowlan @Eevans we could do the following: 1) Test staging to verify that everything is good. 2) Depool eqiad from disco... [09:21:42] 06serviceops, 06Data-Persistence, 13Patch-For-Review: Sessionstore's discovery TLS cert will expire before end of May 2024 - https://phabricator.wikimedia.org/T363996#10236882 (10hnowlan) >>! In T363996#10236821, @elukey wrote: > @hnowlan @Eevans we could do the following: > > 1) Test staging to verify that... [10:02:48] jayme: what kind of traffic levels are appropriate for Ingress? [10:08:26] cdanis: I'd say everything that is not mediawiki scale :) [10:08:54] hmm [10:09:06] the thing is that we have said not to trust ingress with huge amounts of traffic (for now) - without explicitely defining huge amounts [10:09:10] so hundreds but not thousands of rps? [10:09:49] something like that, yes. Although I'm not sure that still holds. I can go check what amount of traffic is already flowing through ingress [10:09:52] Ingress would probably be fine for this, then -- AIUI there's the usual MW-level caching in parsercache of output from the service [10:53:11] 06serviceops, 10[DEPRECATED] wdwb-tech, 10Citoid, 06Content-Transform-Team, and 9 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#10237211 (10Mvolz) >>! In T349118#10214863, @Ottomata wrote: >> What service are you using this for? > > https://wikitec... [11:12:07] cdanis: jayme: unless something huge shows up and blocks my plans, I think we will slowly move into using ingress for MW this FY. It's the only place where we can centrally make decisions for routing wikis to specific MW versions [12:46:24] <_joe_> akosiaris: uhm, OTOH using ingress is a big change and it's less efficient. [12:46:41] <_joe_> We've purposefully avoided using istio ingress for larger payloads [12:47:39] <_joe_> the other place where such decisions could be made is our service mesh, ofc, plus the traffic layer [12:57:15] 06serviceops: kafka-main100[6789] and kafka-main1010 implementation tracking - https://phabricator.wikimedia.org/T363214#10237614 (10akosiaris) Mistakenly removed @dcausse, re-adding. [13:01:27] 06serviceops: kafka-main100[6789] and kafka-main1010 implementation tracking - https://phabricator.wikimedia.org/T363214#10237627 (10jijiki) p:05Triage→03Medium [13:55:50] 06serviceops, 10[DEPRECATED] wdwb-tech, 10Citoid, 06Content-Transform-Team, and 9 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#10237935 (10akosiaris) >>! In T349118#10237211, @Mvolz wrote: > Well, I'm not sure we need them but we have the default... [14:31:00] 06serviceops, 06Content-Transform-Team, 10Electron-PDFs, 07Essential-Work: Download to PDF: HTTP 500 error on some wikis for some users - https://phabricator.wikimedia.org/T376438#10238103 (10MSantos) [14:40:55] 06serviceops, 06Data-Persistence, 13Patch-For-Review: Sessionstore's discovery TLS cert will expire before end of May 2024 - https://phabricator.wikimedia.org/T363996#10238167 (10Eevans) >>! In T363996#10236821, @elukey wrote: > @hnowlan @Eevans we could do the following: > > 1) Test staging to verify that... [14:43:23] urandom, swfrench-wmf: let's aim for Monday for sessionstore? I'll get the CRs etc lined up tomorrow [14:44:24] hnowlan: SGTM [14:52:23] hnowlan: +1 [14:56:17] 06serviceops: Cannot Run Golang or Rust Binaries with Provided AppArmor Profile - https://phabricator.wikimedia.org/T377468 (10cmassaro) 03NEW [14:58:29] hnowlan: <3 [15:05:53] 06serviceops, 06Content-Transform-Team, 10Electron-PDFs, 07Essential-Work: Download to PDF: HTTP 500 error on some wikis for some users - https://phabricator.wikimedia.org/T376438#10238271 (10Jgiannelos) I suspect that the issue is that we don't close or somehow we end up in a sitation with stale browser i... [15:09:24] 06serviceops, 06collaboration-services, 06Infrastructure-Foundations, 10GitLab (Integrations), and 2 others: Container image reports in debmonitor are broken - https://phabricator.wikimedia.org/T348876#10238279 (10elukey) Getting back to the original pain points to review their current status: > debmonito... [15:37:48] 06serviceops, 06Data-Persistence, 13Patch-For-Review: Sessionstore's discovery TLS cert will expire before end of May 2024 - https://phabricator.wikimedia.org/T363996#10238411 (10elukey) It would be ideal to run this during a MediaWiki infrastructure window (https://wikitech.wikimedia.org/wiki/Deployments),... [15:45:08] 06serviceops, 10[DEPRECATED] wdwb-tech, 10Citoid, 06Content-Transform-Team, and 9 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#10238469 (10Ottomata) > First thing would be to set num_workers to 0 in the Chart. This shortcircuits service-runner work... [18:44:21] 06serviceops, 10MW-on-K8s: Functional replacement for importImages.php on Kubernetes - https://phabricator.wikimedia.org/T377497 (10RLazarus) 03NEW [18:51:39] 06serviceops, 10MW-on-K8s: Functional replacement for importImages.php on Kubernetes - https://phabricator.wikimedia.org/T377497#10239328 (10Pppery) > sometimes dozens of gigabytes The current maximum size of a file that MediaWiki/Swift can handle is 5GB. But see T191802 which proposes to increase it further. [19:12:37] 06serviceops, 10MW-on-K8s: Functional replacement for importImages.php on Kubernetes - https://phabricator.wikimedia.org/T377497#10239432 (10RLazarus) [19:13:16] 06serviceops, 10MW-on-K8s: Functional replacement for importImages.php on Kubernetes - https://phabricator.wikimedia.org/T377497#10239429 (10RLazarus) Thanks -- that was in reference to: >>! In T341553#10217525, @Urbanecm_WMF wrote: > The mediafiles can be very large – I've certainly uploaded files that had d... [19:14:39] 06serviceops, 10MW-on-K8s: Functional replacement for importImages.php on Kubernetes - https://phabricator.wikimedia.org/T377497#10239435 (10Urbanecm_WMF) >>! In T377497#10239328, @Pppery wrote: >> sometimes dozens of gigabytes > > The current maximum size of a file that MediaWiki/Swift can handle is 5GB. But...