[04:47:04] (03PS1) 10Hslater: Add new Extension:PDFCreator [integration/config] - 10https://gerrit.wikimedia.org/r/1135151 [08:38:51] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391434 (10Sirchanmp) 03NEW [08:54:42] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391436 (10Sirchanmp) 03NEW [08:57:00] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391434#10724831 (10Superpes15) [08:57:41] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391436#10724836 (10Superpes15) 05Open→03Invalid Duplicate of T391434 [09:04:22] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 3 others: move query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10724863 (10Jelto) Querybuilder for `query`, `query-main` and `query-scholarly` were switched to wikikube succe... [09:07:05] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 3 others: move query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10724875 (10Jelto) [09:07:07] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391436#10724877 (10taavi) →14Duplicate dup:03T391434 [09:07:09] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for [Sirchanmp] - https://phabricator.wikimedia.org/T391434#10724879 (10taavi) [09:41:34] 06Gerrit-Privilege-Requests, 10WDLexAudio: Request for plus 2 rights - https://phabricator.wikimedia.org/T391414#10724979 (10Peachey88) [09:43:16] 10GitLab (Upstream pit of despair 🕳️): 404 link in header notice about "push and pull" terminology - https://phabricator.wikimedia.org/T377070#10724994 (10Aklapper) [09:43:37] 06Gerrit-Privilege-Requests, 10WDLexAudio: Request for plus 2 rights - https://phabricator.wikimedia.org/T391414#10724997 (10taavi) The relevant [[ https://gerrit.wikimedia.org/r/admin/groups/5a653dd39f435e24b659cb1ceedc873b34abffa0 | Gerrit group ]] is self-owned so @Eugene233 should be able to do this withou... [09:43:43] 10GitLab (Upstream pit of despair 🕳️): 404 link in header notice about "push and pull" terminology - https://phabricator.wikimedia.org/T377070#10724998 (10Aklapper) [09:53:16] 10GitLab, 06Release-Engineering-Team: Add WMF staff instructions to GitLab account approval banner - https://phabricator.wikimedia.org/T391446 (10Aklapper) 03NEW [10:04:32] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 06serviceops: Migrate mwdebug* hosts to PHP8.1 - https://phabricator.wikimedia.org/T391452 (10jijiki) 03NEW [10:36:56] 06Release-Engineering-Team, 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Refactor scap's kubernetes DeploymentsConfig to support selection of image kinds - https://phabricator.wikimedia.org/T389499#10725388 (10BTullis) >>! In T389499#10671841, @Scott_French wrote: > Thanks, @BTullis - I was aware of Joe'... [11:38:12] 10Gerrit, 10CAS-SSO, 06Infrastructure-Foundations: gerrit.wikimedia.org should use IDP for login - https://phabricator.wikimedia.org/T147864#10725535 (10taavi) [12:04:32] 10Beta-Cluster-Infrastructure, 10Wikifunctions, 10Abstract Wikipedia team (25Q3 (Jan–Mar)): "Exec error in changeprop" for wikifunctions.beta.wmflabs.org - https://phabricator.wikimedia.org/T389274#10725614 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF [12:22:43] 10Phabricator, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469 (10bvibber) 03NEW [12:23:22] 10Beta-Cluster-Infrastructure, 10NavigationTiming, 06SRE Observability: navtiming: Loss of Kafka connection fills multiple log files with identical stack traces - https://phabricator.wikimedia.org/T391273#10725743 (10Peter) I think the root issue is in T346402 [12:28:01] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Migrate mwdebug* hosts to PHP8.1 - https://phabricator.wikimedia.org/T391452#10725756 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jiji@cumin1002 for host mwdebug1002.eqiad.wmnet wi... [12:42:47] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725848 (10Aklapper) What does "Deprecate" imply/mean? Phabricator has no such concept. (This seems to be about a project tag and not the software config, thus updating tags.) [12:43:47] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725851 (10bvibber) [12:44:41] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725856 (10bvibber) >>! In T391469#10725848, @Aklapper wrote: > What does "Deprecate" imply/mean? Phabricator has no such concept. > > (This seems to be about a project tag and not the... [12:46:23] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725861 (10bvibber) Note: I've added a specific work task for going in and cleaning up until large files can be self-uploaded reliably: T391473 This is on my maintenance shortlist now u... [12:52:24] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725885 (10Aklapper) >>! In T391469#10725856, @bvibber wrote: > making it hard for people to use it anew by accident. We could adjust the preamble on https://phabricator.wikimedia.org... [12:53:59] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725888 (10bvibber) Well, "You probably shouldn't use it" is what deprecated means. :) I defer to whatever wording you recommend. [12:54:55] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725893 (10bvibber) (note the main preamble is at the top of the page and says: "Warning, deprecated: Server-side upload queues are unmaintained and requests are not guaranteed to be h... [12:56:38] 06Project-Admins, 10Server-side-upload-request: Deprecate server-side upload tag - https://phabricator.wikimedia.org/T391469#10725905 (10Aklapper) Maybe. I rarely ever see the top of any wiki pages as my browser address bar autocompletion jumps directly to page anchors. [13:01:20] 10Phabricator, 10Server-side-upload-request: Communicate in Phabricator that server-side upload tag is deprecated (project description; form 106) - https://phabricator.wikimedia.org/T391469#10725916 (10Aklapper) p:05Triage→03Medium [13:01:27] 10Phabricator, 10Server-side-upload-request: Communicate in Phabricator that server-side upload tag is deprecated (project description; form 106) - https://phabricator.wikimedia.org/T391469#10725919 (10Aklapper) 05Open→03Resolved I updated https://phabricator.wikimedia.org/transactions/editengine/manip... [13:04:57] 10Phabricator, 10Server-side-upload-request: Communicate in Phabricator that server-side upload tag is deprecated (project description; form 106) - https://phabricator.wikimedia.org/T391469#10725950 (10bvibber) >>! In T391469#10725916, @Aklapper wrote: > I updated https://phabricator.wikimedia.org/transact... [13:21:25] 06Project-Admins: Propose to add "stack trace requested" column to #wikimedia-production-error - https://phabricator.wikimedia.org/T391206#10726023 (10Aklapper) There is a semi-regular #wikimedia-production-error grooming on Thursdays but not sure how often they/we end up digging up stacktraces of "older" ti... [13:24:08] 10Release-Engineering-Team (Priority Backlog 📥), 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Migrate mwdebug* hosts to PHP8.1 - https://phabricator.wikimedia.org/T391452#10726028 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jiji@cumin1002 for host mwdebug1002.eqiad.wmnet with O... [14:14:52] 06Release-Engineering-Team, 10Scap (SpiderPig 🕸️), 10logspam-watch, 10observability, 10Observability-Logging: Add a log view to SpiderPig - https://phabricator.wikimedia.org/T391005#10726271 (10lmata) [14:25:08] 06Release-Engineering-Team, 10Scap (SpiderPig 🕸️), 10logspam-watch, 10observability, 10Observability-Logging: Add a log view to SpiderPig - https://phabricator.wikimedia.org/T391005#10726337 (10colewhite) A couple points for awareness as you plan the project: - Someday(tm), we will enable the security... [14:26:22] 10Continuous-Integration-Infrastructure, 10Testing Support, 10ci-test-error (WMF-deployed Build Failure), 10MW-1.44-notes (1.44.0-wmf.23; 2025-04-01), and 2 others: Selenium timeouts can cause the job to remain stuck until the build times out - https://phabricator.wikimedia.org/T389536#10726340 (10zeljkofil... [15:07:01] 10GitLab (Project Migration), 10BDC-Implementation: Move Analytics from Gerrit to Gitlab - https://phabricator.wikimedia.org/T391404#10726499 (10Aklapper) [15:19:25] 10Beta-Cluster-Infrastructure, 06Data-Engineering, 06Data-Engineering-Radar: beta/deployment-prep Kafka clusters do not support TLS client connections - https://phabricator.wikimedia.org/T346402#10726551 (10Ottomata) [16:47:54] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team (25Q4 (Apr–Jun)): Beta Cluster orchestrator / evaluator broken, blocking WikiLambda CI (and use of Beta Cluster) - https://phabricator.wikimedia.org/T374242#10726888 (10DSantamaria) @Jdforrester-WMF is this #essential-work? [16:57:10] (03update) 10dancy: kubernetes.py: Fix monitoring of deployment to fresh cluster [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/743 [16:59:09] (03merge) 10dancy: kubernetes.py: Fix monitoring of deployment to fresh cluster [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/743 [17:00:28] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap (SpiderPig 🕸️), 07OKR-Work, 13Patch-For-Review: Scap SpiderPig: user group/admin set up - https://phabricator.wikimedia.org/T383947#10726985 (10thcipriani) 05Open→03Resolved a:03dancy This is done. Status: - Anyone in the `releng`, `ops`, o... [17:21:33] 10Beta-Cluster-Infrastructure, 10Abstract Wikipedia team (25Q4 (Apr–Jun)), 07Essential-Work: Beta Cluster orchestrator / evaluator broken, blocking WikiLambda CI (and use of Beta Cluster) - https://phabricator.wikimedia.org/T374242#10727078 (10Jdforrester-WMF) 05In progress→03Resolved Oops, forgot to... [17:29:46] 06Project-Admins: Alfredo Zumaya Robles (biografía) - https://phabricator.wikimedia.org/T391504 (10alzuro6364) 03NEW [18:25:19] 10Beta-Cluster-Infrastructure, 10WikiLambda: Betacluster exception thrown on login `Domain 'wikifunctionswiki' is not recognized.` - https://phabricator.wikimedia.org/T391511 (10vaughnwalters) 03NEW [18:26:50] 10Beta-Cluster-Infrastructure, 10WikiLambda: Betacluster exception thrown on login `Domain 'wikifunctionswiki' is not recognized.` - https://phabricator.wikimedia.org/T391511#10727323 (10vaughnwalters) [18:28:58] 10Beta-Cluster-Infrastructure, 10WikiLambda: Betacluster exception thrown on login `Domain 'wikifunctionswiki' is not recognized.` - https://phabricator.wikimedia.org/T391511#10727340 (10vaughnwalters) [18:29:52] 10Beta-Cluster-Infrastructure, 10WikiLambda: Betacluster exception thrown on login: Domain 'wikifunctionswiki' is not recognized. - https://phabricator.wikimedia.org/T391511#10727341 (10vaughnwalters) [18:42:59] (03open) 10dancy: timers: Append stage name to some per-stage timers [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/750 [18:43:01] (03update) 10dancy: timers: Append stage name to some per-stage timers [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/750 [18:47:23] (03merge) 10dancy: timers: Append stage name to some per-stage timers [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/750 [18:54:05] 06Release-Engineering-Team, 10MW-on-K8s, 06serviceops, 13Patch-For-Review: Refactor scap's kubernetes DeploymentsConfig to support selection of image kinds - https://phabricator.wikimedia.org/T389499#10727380 (10Scott_French) Thanks for the additional details on the mechanics, @BTullis! >>! In T389499#107... [19:13:12] 10WikimediaDebug, 10Arc-Lamp, 10observability, 07Regression: https://performance.wikimedia.org/php-profiling/ leads to 404 for all listed sources - https://phabricator.wikimedia.org/T391516 (10Krinkle) 03NEW [19:14:32] 10WikimediaDebug, 10Arc-Lamp, 10observability, 07Regression: Daily flame graph for "fn-EditAction" missing as since 28 March 2025 - https://phabricator.wikimedia.org/T391517 (10Krinkle) 03NEW [19:14:41] 10WikimediaDebug, 10Arc-Lamp, 10observability, 07Regression: Daily flame graph for "fn-EditAction" missing since 28 March 2025 - https://phabricator.wikimedia.org/T391517#10727473 (10Krinkle) [19:14:45] (03open) 10swfrench: config: update mediawiki_runtime_image to use PHP 8.1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/751 (https://phabricator.wikimedia.org/T390225) [19:21:50] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for "UnrivaledIr" - https://phabricator.wikimedia.org/T391519 (10UnrivaledIr) 03NEW [19:23:33] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for "UnrivaledIr" - https://phabricator.wikimedia.org/T391519#10727509 (10Arian_Ar) The user is a member of the Iranian Wikimedians User Group and is contributing to several technical projects run by the User Group. @... [19:28:00] (03merge) 10dancy: config: update mediawiki_runtime_image to use PHP 8.1 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/751 (https://phabricator.wikimedia.org/T390225) (owner: 10swfrench) [19:28:40] (03open) 10dancy: Release 4.153.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/752 [19:30:42] (03merge) 10dancy: Release 4.153.0 [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/752 [20:42:01] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [labs/tools/mostvisitedarticle] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135492 [20:42:02] (03CR) 10QChris: [V:03+2 C:03+2] Allow “Gerrit Managers” to import history [labs/tools/mostvisitedarticle] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135492 (owner: 10QChris) [20:42:51] (03PS1) 10QChris: Import done. Revoke import grants [labs/tools/mostvisitedarticle] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135493 [20:42:51] (03CR) 10QChris: [V:03+2 C:03+2] Import done. Revoke import grants [labs/tools/mostvisitedarticle] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135493 (owner: 10QChris) [20:49:19] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [labs/tools/WdTmCollab] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135494 [20:49:19] (03CR) 10QChris: [V:03+2 C:03+2] Allow “Gerrit Managers” to import history [labs/tools/WdTmCollab] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135494 (owner: 10QChris) [20:49:52] (03PS1) 10QChris: Import done. Revoke import grants [labs/tools/WdTmCollab] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135495 [20:49:52] (03CR) 10QChris: [V:03+2 C:03+2] Import done. Revoke import grants [labs/tools/WdTmCollab] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/1135495 (owner: 10QChris) [21:09:11] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for "UnrivaledIr" - https://phabricator.wikimedia.org/T391519#10727797 (10Ladsgroup) +1 on my side [21:13:11] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for "UnrivaledIr" - https://phabricator.wikimedia.org/T391519#10727833 (10thcipriani) 05Open→03Resolved a:03thcipriani Done! For future reference, if you add folks you know and trust to #trusted-contributor... [21:26:08] (03open) 10dancy: sync_wikiversions -> sync-wikiversions in log messages [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/753 (https://phabricator.wikimedia.org/T340745) [21:26:10] (03update) 10dancy: sync_wikiversions -> sync-wikiversions in log messages [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/753 (https://phabricator.wikimedia.org/T340745) [21:42:20] 10Gerrit, 07Upstream: Allow for group admins to create project repositories inside a parent project - https://phabricator.wikimedia.org/T38937#10727933 (10Dzahn) >>! In T38937#9566625, @QChris wrote: > But given that we're sunsetting Gerrit (and unless we're challenging that decision), I'm not sure it's worth... [22:01:22] 10Gerrit, 07Upstream: Allow for group admins to create project repositories inside a parent project - https://phabricator.wikimedia.org/T38937#10727992 (10bd808) If upstreaming this functionality is tricky I could see it being possible via an external tool that has rights in all the needed places (gerrit, gith... [22:06:50] 10Beta-Cluster-Infrastructure, 06Abstract Wikipedia team, 10WikiLambda, 07Wikimedia-production-error: Betacluster exception thrown on login: Domain 'wikifunctionswiki' is not recognized. - https://phabricator.wikimedia.org/T391511#10728000 (10A_smart_kitten) [22:08:49] 10Scap, 06serviceops, 13Patch-For-Review: Migrate scap's maintenance script invocations to PHP 8.1 - https://phabricator.wikimedia.org/T390225#10728008 (10Scott_French) p:05Triage→03High The `mediawiki_runtime_image` default change is now live with scap 4.153.0, so all that remains here is to clean up th... [22:54:54] 10Beta-Cluster-Infrastructure, 06Abstract Wikipedia team, 10WikiLambda, 07Wikimedia-production-error: Betacluster exception thrown on login: Domain 'wikifunctionswiki' is not recognized. - https://phabricator.wikimedia.org/T391511#10728106 (10Jdforrester-WMF) Oh, bleh, we probably need to hack CentralAuth'... [23:07:56] 10Beta-Cluster-Infrastructure, 06Abstract Wikipedia team, 10WikiLambda, 07Wikimedia-production-error: Betacluster exception thrown on login: Domain 'wikifunctionswiki' is not recognized. - https://phabricator.wikimedia.org/T391511#10728181 (10Jdforrester-WMF) As I thought, I had dropped it from `wmgCentral... [23:11:09] 10Beta-Cluster-Infrastructure, 06Abstract Wikipedia team, 10WikiLambda, 07Wikimedia-production-error: Betacluster exception thrown on login: Domain 'wikifunctionswiki' is not recognized. - https://phabricator.wikimedia.org/T391511#10728195 (10matmarex) https://en.wikipedia.beta.wmflabs.org/wiki/Special:Cen... [23:19:17] 10Beta-Cluster-Infrastructure, 06Abstract Wikipedia team, 10WikiLambda, 07Wikimedia-production-error: Betacluster exception thrown on login: Domain 'wikifunctionswiki' is not recognized. - https://phabricator.wikimedia.org/T391511#10728200 (10Jdforrester-WMF) >>! In T391511#10728195, @matmarex wrote: > htt... [23:31:41] 10Beta-Cluster-Infrastructure, 06Abstract Wikipedia team, 10WikiLambda, 13Patch-For-Review, 07Wikimedia-production-error: Betacluster exception thrown on login: Domain 'wikifunctionswiki' is not recognized. - https://phabricator.wikimedia.org/T391511#10728234 (10Jdforrester-WMF) For a maintenance script,... [23:43:51] 10Beta-Cluster-Infrastructure, 10NavigationTiming, 06SRE Observability: navtiming: Loss of Kafka connection fills multiple log files with identical stack traces - https://phabricator.wikimedia.org/T391273#10728260 (10bd808) >>! In T391273#10725743, @Peter wrote: > I think the root issue is in T346402 There... [23:50:55] 10Beta-Cluster-Infrastructure, 06Data-Engineering, 06Data-Engineering-Radar: beta/deployment-prep Kafka clusters do not support TLS client connections - https://phabricator.wikimedia.org/T346402#10728268 (10bd808) I think the TLS listener problems were fixed by @elukey in {T383096}. `lang=shell-session bd80... [23:57:15] 10Beta-Cluster-Infrastructure, 06Data-Engineering, 06Data-Engineering-Radar: beta/deployment-prep Kafka clusters do not support TLS client connections - https://phabricator.wikimedia.org/T346402#10728275 (10bd808) 05Open→03Resolved a:03elukey I'm giving @elukey credit for this no longer being an ac...