[00:17:23] 06Project-Admins: Create project tag for WikiConnect-ParseWiki - https://phabricator.wikimedia.org/T398147#10956958 (10Aklapper) Hi, do you plan to disable https://github.com/wiki-connect/ParseWiki/issues ? [00:22:05] 06Project-Admins: Create project tag for WikiConnect-ParseWiki - https://phabricator.wikimedia.org/T398147#10956960 (10Gerges) Sure, I will move from github to gerrit.wikimedia.org after creating project. [04:30:21] 10Phabricator maintenance bot, 07Security: @Maintenance_bot can't see/doesn't act on security tasks - https://phabricator.wikimedia.org/T391674#10957070 (10Bugreporter) We can add the bot to #acl*security, but the not still can not see NDA protected tasks until being added to #WMF-NDA. [09:14:10] 10Gerrit: Slow connection to gerrit via ssh - https://phabricator.wikimedia.org/T398030#10957622 (10hashar) 05Openβ†’03Resolved a:03hashar I don't think we throttle bandwidth, but we do have systems limiting concurrent connections which would end up blocking for some minutes. I don't think it would affec... [09:26:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:26:32] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176 (10wmcs-alerts) 03NEW [09:41:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [09:41:32] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10957721 (10wmcs-alerts) [10:25:29] 10Deployments, 10Release-Engineering-Team (Radar), 06serviceops, 13Patch-For-Review, 07Wikimedia-production-error: httpb sometimes fails upon deployment with a HTTP 503 - https://phabricator.wikimedia.org/T380958#10957904 (10akosiaris) The 2 patches have been merged and will ride out today's deployments.... [10:59:42] 10Phabricator (Search): Cannot find some Tasks using simple search - https://phabricator.wikimedia.org/T305065#10958058 (10Aklapper) Decreasing the scope of that query to https://phabricator.wikimedia.org/search/query/aYltXfrCu2Iv/#R, I get four tasks, missing `T304193`. The fulltext search code executed first... [11:00:11] 10Phabricator (Search): Global fulltext search results miss some tasks - https://phabricator.wikimedia.org/T305065#10958059 (10Aklapper) [11:08:18] (03CR) 10Hashar: [C:03+2] Zuul: [mediawiki/extensions/PDFCreator] Add VueJsPlus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1164221 (owner: 10Fomafix) [11:08:41] (03CR) 10Hashar: [C:03+2] Zuul: [extensions/Scribunto] Add CodeMirror as phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1164653 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [11:10:10] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/PDFCreator] Add VueJsPlus dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1164221 (owner: 10Fomafix) [11:10:44] (03Merged) 10jenkins-bot: Zuul: [extensions/Scribunto] Add CodeMirror as phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/1164653 (https://phabricator.wikimedia.org/T373711) (owner: 10MusikAnimal) [11:41:24] 10Phabricator (Search): "Tags" field in Maniphest Advanced Search does not offer MediaWiki-API in proposals when using "any" prefix - https://phabricator.wikimedia.org/T218103#10958198 (10Aklapper) 05Openβ†’03Invalid Same as T215085 [11:46:04] 10Phabricator (Search): Incomplete and inconsistent (Maniphest vs global) search results in Phabricator - https://phabricator.wikimedia.org/T344113#10958218 (10Aklapper) Testcases not applicable anymore but I assume it's the same game as T305065. [11:46:13] 10Phabricator (Search): Incomplete and inconsistent (Maniphest vs global) search results in Phabricator - https://phabricator.wikimedia.org/T344113#10958221 (10Aklapper) β†’14Duplicate dup:03T305065 [11:46:15] 10Phabricator (Search): Global fulltext search results miss some tasks - https://phabricator.wikimedia.org/T305065#10958223 (10Aklapper) [11:48:42] 10Phabricator (Search): Searching by Document Status "Open" includes resolved tasks - https://phabricator.wikimedia.org/T397392#10958233 (10Aklapper) @A_smart_kitten: Oh well, that would be {T288905}. [11:48:47] 10Phabricator (Search), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Have a more scope-restricted default global (primary) search - https://phabricator.wikimedia.org/T397558#10958236 (10Aklapper) @A_smart_kitten: That would be {T288905}. [11:51:18] 10Phabricator (Search): Searching by Document Status "Open" includes resolved tasks - https://phabricator.wikimedia.org/T397392#10958242 (10Aklapper) This is a similar underlying problem as {T365128}: `ManiphestTaskFulltextEngine` checks for `PhabricatorSearchRelationship::RELATIONSHIP_OPEN` or `_CLOSED`. But op... [11:53:02] 10Phabricator (Search): Searching by Document Status "Open" includes resolved tasks - https://phabricator.wikimedia.org/T397392#10958250 (10Aklapper) Notes to myself: Getting affected tasks (resolved in the task DB but not in the search DB): `SELECT t.id, t.status, f.isClosed FROM phabricator_maniphest.maniphes... [11:54:21] 10Phabricator (Upstream), 07Upstream: Searching shorter terms ("gerrit", "cors") in global Phabricator search leads to upstream request timeout (too many results?) - https://phabricator.wikimedia.org/T258803#10958251 (10Aklapper) >>! In T258803#9312008, @Aklapper wrote: > I'd say the error message should be ma... [11:54:37] 10Phabricator (Search), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Have a more scope-restricted default global (primary) search - https://phabricator.wikimedia.org/T397558#10958252 (10A_smart_kitten) I think it might be different -- reading that task, it sounds like it's proposing to add an 'application-... [11:59:07] 10Phabricator (Search), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Have a more scope-restricted default global (primary) search - https://phabricator.wikimedia.org/T397558#10958266 (10Aklapper) > whereas my wild idea would be to redirect that search bar to the Maniphest advanced search in all cases/by de... [12:16:29] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Re-create GitLab test instance - https://phabricator.wikimedia.org/T396622#10958322 (10Jelto) 05Openβ†’03Resolved I'll resolve this task, the test instance is up and running again and I simplified the setup a bit and fixed... [12:18:04] 10GitLab (Infrastructure), 06Release-Engineering-Team, 06collaboration-services: Upgrade GitLab to major version 18 - https://phabricator.wikimedia.org/T394382#10958326 (10Jelto) a:03Jelto [12:25:20] (03CR) 10Hashar: [C:03+2] Move VipsScaler to other [integration/config] - 10https://gerrit.wikimedia.org/r/1164233 (https://phabricator.wikimedia.org/T290759) (owner: 10Arlolra) [12:27:20] (03Merged) 10jenkins-bot: Move VipsScaler to other [integration/config] - 10https://gerrit.wikimedia.org/r/1164233 (https://phabricator.wikimedia.org/T290759) (owner: 10Arlolra) [12:45:49] 10Phabricator (Search), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Exception in "./bin/search index --force": "strlen() expects parameter 1 to be string, array given" - https://phabricator.wikimedia.org/T398197 (10Aklapper) 03NEW p:05Triageβ†’03Medium [13:06:18] 10Phabricator (Search), 10Release-Engineering-Team (Priority Backlog πŸ“₯): Exception in "./bin/search index --force": "strlen() expects parameter 1 to be string, array given" - https://phabricator.wikimedia.org/T398197#10958525 (10Aklapper) No clue where that `strlen()` call is even located; none of the files li... [13:16:11] 10Phabricator, 10Bitu, 06cloud-services-team, 06Infrastructure-Foundations, 10Striker: Inconsistent mapping of Developer accounts and SUL accounts across Phabricator, Bitu, and Striker - https://phabricator.wikimedia.org/T388498#10958558 (10Arendpieter) [13:18:50] 10Phabricator, 10Bitu, 06cloud-services-team, 06Infrastructure-Foundations, 10Striker: Inconsistent mapping of Developer accounts and SUL accounts across Phabricator, Bitu, and Striker - https://phabricator.wikimedia.org/T388498#10958568 (10Arendpieter) [13:28:33] !log gerrit: deleted videojs-resolution-switcher and videojs-responsive-layout , forks of other projects with no local modifications/changes. [13:28:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:31:58] !log gerrit: change performance/* submit strategy to "Rebase if Necessary" and "Allow content merge" | T390719 [13:32:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:32:00] T390719: Change Gerrit default submit strategy to 'Rebase if Necessary' and allowing content merge - https://phabricator.wikimedia.org/T390719 [13:37:45] !log gerrit: change mediawiki/libs/* submit strategy to "Rebase if Necessary" and "Allow content merge" | T390719 [13:37:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:37:47] T390719: Change Gerrit default submit strategy to 'Rebase if Necessary' and allowing content merge - https://phabricator.wikimedia.org/T390719 [13:46:58] 10Phabricator, 10Release-Engineering-Team (Priority Backlog πŸ“₯): Update to Phorge upstream 202x.xx release (the one after 2025.18) - https://phabricator.wikimedia.org/T393840#10958679 (10Aklapper) [13:46:58] 10Phabricator (Upstream), 10Release-Engineering-Team (Doing 😎), 07Performance Issue, 07Upstream: Ferret Fulltext search queries all apps' databases (also those of uninstalled apps) - https://phabricator.wikimedia.org/T397499#10958678 (10Aklapper) [13:57:08] !log gerrit: change labs/* submit strategy to "Rebase if Necessary" and "Allow content merge" | T390719 [13:57:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:57:12] T390719: Change Gerrit default submit strategy to 'Rebase if Necessary' and allowing content merge - https://phabricator.wikimedia.org/T390719 [13:58:59] 10Phabricator (Upstream), 07Upstream: Make it possible to change the 'default' column of a Phabricator workboard - https://phabricator.wikimedia.org/T397862#10958708 (10Aklapper) > One is an 'unplanned backlog' column; which contains valid tasks, but ones which the project's maintainers don't have any current... [14:09:05] 10Gerrit, 10CAS-SSO, 06Infrastructure-Foundations: gerrit.wikimedia.org should use IDP for login - https://phabricator.wikimedia.org/T147864#10958761 (10Arendpieter) [14:18:53] 10Scap, 10MediaWiki-Release-Tools, 13Patch-For-Review: PHP Fatal error: Cannot redeclare wikidiff2_do_diff() - https://phabricator.wikimedia.org/T340862#10958806 (10Reedy) (This is still an issue) [14:20:13] (03update) 10dancy: lint.py: Don't lint phan's internal stubs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/754 (https://phabricator.wikimedia.org/T340862) (owner: 10reedy) [14:20:33] (03approved) 10dancy: lint.py: Don't lint phan's internal stubs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/754 (https://phabricator.wikimedia.org/T340862) (owner: 10reedy) [14:23:57] (03unapproved) 10dancy: lint.py: Don't lint phan's internal stubs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/754 (https://phabricator.wikimedia.org/T340862) (owner: 10reedy) [14:25:17] dancy: heh [14:25:44] (03update) 10taavi: make-release: Ignore zip files as well [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/169 [14:50:31] 10Gerrit: Slow connection to gerrit via ssh - https://phabricator.wikimedia.org/T398030#10958948 (10Osnard) Thank you very much! Yes, "56.2 KBytes/seconds" looks like the speed I faced recently. Today I didn't experience this anymore. At least I now know I didn't cause this by my automation. Again, thank you. [15:09:41] 06Release-Engineering-Team, 07Epic, 05MW-1.43-release, 05MW-1.44-notes, and 4 others: [REST Sandbox] Remove SwaggerUI from MediaWiki Releases - https://phabricator.wikimedia.org/T397470#10959051 (10Reedy) [15:10:10] 06Release-Engineering-Team, 07Epic, 05MW-1.43-release, 05MW-1.44-notes, and 4 others: [REST Sandbox] Remove SwaggerUI from MediaWiki Releases - https://phabricator.wikimedia.org/T397470#10959052 (10Reedy) [15:13:01] 10Continuous-Integration-Infrastructure (Zuul upgrade), 07Browser-Tests, 07ci-test-error: Webdriver.io can't start Chrome (zuul upgrade) - https://phabricator.wikimedia.org/T397215#10959085 (10Peter) I'm thinking in CI we should setup [[ https://peter.sh/experiments/chromium-command-line-switches/#crash-dump... [15:14:35] 10Phabricator (Upstream), 07Upstream: Make it possible to change the 'default' column of a Phabricator workboard - https://phabricator.wikimedia.org/T397862#10959099 (10CCiufo-WMF) I think the generic situation where switching the default column is helpful is any time when a column or workboard's purpose chang... [15:15:16] 06Release-Engineering-Team, 07Epic, 05MW-1.43-release, 05MW-1.44-notes, and 4 others: [REST Sandbox] Remove SwaggerUI from MediaWiki Releases - https://phabricator.wikimedia.org/T397470#10959111 (10OWresch-WMF) Will also make the changes for 1.43. [15:15:37] (03open) 10bd808: tofu: Bump terraform-cloudvps to v0.3.1 [repos/releng/zuul/tofu-provisioning] - 10https://gitlab.wikimedia.org/repos/releng/zuul/tofu-provisioning/-/merge_requests/14 (https://phabricator.wikimedia.org/T398117) [15:18:30] (03merge) 10bd808: tofu: Bump terraform-cloudvps to v0.3.1 [repos/releng/zuul/tofu-provisioning] - 10https://gitlab.wikimedia.org/repos/releng/zuul/tofu-provisioning/-/merge_requests/14 (https://phabricator.wikimedia.org/T398117) [15:20:04] Project beta-update-databases-eqiad build #85837: 04FAILURE in 3.7 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/85837/ [15:25:18] https://gerrit.wikimedia.org/r/1162643 broke it [15:26:40] (03merge) 10reedy: make-release: Ignore zip files as well [repos/releng/release] - 10https://gitlab.wikimedia.org/repos/releng/release/-/merge_requests/169 (owner: 10taavi) [15:27:59] 10Beta-Cluster-Infrastructure: CampaignEvents: Error 1091: Can't DROP COLUMN `event_type`; check that it exists - https://phabricator.wikimedia.org/T394418#10959214 (10Daimona) This [[https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/85837/console | happened again ]] for [[https://gerrit.w... [15:29:33] hashar: I remember you talking about building debs inside containers recently. You might be interested in the things that WMCS are doing with https://gitlab.wikimedia.org/repos/cloud/cicd/gitlab-ci/-/tree/main/debian-builder-bullseye (also related to T212291) [15:29:33] T212291: debian packaging: create common guidelines and workflow - https://phabricator.wikimedia.org/T212291 [15:47:28] 10WikimediaDebug, 10Arc-Lamp, 10observability, 07Regression, 10SRE Observability (FY2025/2026-Q1): https://performance.wikimedia.org/php-profiling/ leads to 404 for all listed sources - https://phabricator.wikimedia.org/T391516#10959395 (10tappof) I took a look at the git log for the file modules/arclamp... [15:52:56] 10GitLab (Auth & Access), 06cloud-services-team, 10Cloud-VPS, 10Toolforge: Sync WMCS GitLab group membership from LDAP - https://phabricator.wikimedia.org/T398217#10959425 (10taavi) [16:06:02] (03open) 10dancy: api.js: Remove unused pinia state [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/866 [16:06:06] (03update) 10dancy: api.js: Remove unused pinia state [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/866 [16:08:24] (03merge) 10dancy: api.js: Remove unused pinia state [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/866 [16:15:32] (03open) 10dancy: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 [16:15:35] (03update) 10dancy: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 [16:16:04] (03update) 10dancy: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 [16:16:09] (03update) 10dancy: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 [16:16:52] (03update) 10dancy: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 [16:22:54] Yippee, build fixed! [16:22:54] Project beta-update-databases-eqiad build #85838: 09FIXED in 2 min 53 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/85838/ [16:28:19] (03open) 10dancy: api.py: Set Secure flag on session cookie [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/868 [16:28:22] (03update) 10dancy: api.py: Set Secure flag on session cookie [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/868 [16:30:32] (03merge) 10dancy: api.py: Set Secure flag on session cookie [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/868 [16:35:33] (03open) 10dancy: spiderpig/api.py: Replace some print()s with logging.info() calls [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/869 [16:35:36] (03update) 10dancy: spiderpig/api.py: Replace some print()s with logging.info() calls [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/869 [16:37:27] (03update) 10dancy: spiderpig/api.py: Replace some print()s with logging.info() calls [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/869 [16:37:55] !log gerrit: change wikimedia/fundraising/* submit strategy to "Rebase if Necessary" and "Allow content merge" | T390719 [16:37:59] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:38:03] T390719: Change Gerrit default submit strategy to 'Rebase if Necessary' and allowing content merge - https://phabricator.wikimedia.org/T390719 [16:38:26] bd808: thx for the link about building Debian package. I guess I will come to that when we reach migrating operations/debs/* repos :) [16:39:34] (03merge) 10dancy: spiderpig/api.py: Replace some print()s with logging.info() calls [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/869 [16:54:19] 10Continuous-Integration-Config: CI Failing when trying to add CLDR extension on CampaignEvents extension - https://phabricator.wikimedia.org/T398224 (10cmelo) 03NEW [17:09:34] 10GitLab (Auth & Access), 06cloud-services-team, 10Cloud-VPS, 10Toolforge: Sync WMCS GitLab group membership from LDAP - https://phabricator.wikimedia.org/T398217#10959885 (10bd808) I think this would have to be done with a bot or runbook. LDAP based group membership management is a closed extension to the... [17:19:38] 10GitLab (Auth & Access), 06cloud-services-team, 10Cloud-VPS, 10Toolforge: Sync WMCS GitLab group membership from LDAP - https://phabricator.wikimedia.org/T398217#10959948 (10thcipriani) FWIW, there are utilities that run in systemd (managed by puppet) to manage ldap -> gitlab group sync for a few groups:... [17:20:08] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10959955 (10bd808) 05Openβ†’03Invalid Fixed itself on the next run apparently [17:27:57] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960002 (10taavi) 05Invalidβ†’03Open The diff was to HAProxy config, which is now failing to reload: `lines=15,counterexample Jun 30 09:09:5... [17:31:13] (03update) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [17:31:15] (03update) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [17:31:16] !log gerrit: marked read-only all operations/debs/contenttranslation/apertium* repositories. Untouched since 2020. [17:31:17] (03update) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [17:31:17] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:31:50] (03update) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [17:33:38] 10Continuous-Integration-Config: CI stuck on "Waiting for Post-dependency install, pre-database dependent steps" when trying to add CLDR extension as a dependency of CampaignEvents - https://phabricator.wikimedia.org/T398224#10960027 (10Daimona) [17:33:47] 10Continuous-Integration-Config, 10Quibble: CI stuck on "Waiting for Post-dependency install, pre-database dependent steps" when trying to add CLDR extension as a dependency of CampaignEvents - https://phabricator.wikimedia.org/T398224#10960029 (10Daimona) [17:34:12] 10Continuous-Integration-Config, 10Quibble: CI stuck on "Waiting for Post-dependency install, pre-database dependent steps" when trying to add CLDR extension as a dependency of CampaignEvents - https://phabricator.wikimedia.org/T398224#10960033 (10Daimona) p:05Triageβ†’03High [17:34:32] (03update) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [17:34:32] (03update) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [17:34:35] (03update) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [17:46:25] hi Releng, I can't seem to change visibility of an existing phab task now [17:47:14] 10Continuous-Integration-Config, 10Quibble: CI stuck on "Waiting for Post-dependency install, pre-database dependent steps" when trying to add CLDR extension as a dependency of CampaignEvents - https://phabricator.wikimedia.org/T398224#10960088 (10Daimona) A few things to note: - The CLDR extension is really c... [17:51:58] Is there a new way to restrict visibility of an existing task short of using the button to escalate it to the security team? [17:56:19] ejegg: the annoying answer is that whether or not the policy controls appear on the edit form depends on which edit form phabricator chooses, which in turn depends on the task subtype as well as your groups [18:01:15] oh huh [18:01:37] I guess because wfan initially reported it as a 'bug report' it chose a different form [18:01:54] I was trying to get a different edit form but the form chooser only seemed to let me file new tasks [18:03:16] yeah, usually you can use the add action -> change subtype menu to change it to a "normal" task which has the advanced form with a relatively wide acl [18:03:36] although if you're in a hurry the escalate to secteam button works as well, unless you need an even more restrictive acl [18:12:05] Access Denied: Bug Report Editing Form, I am not able to update the bug report form [18:12:36] Could you add this field for us taavi, seems like we need to see the visible for our bug report, thanks! [18:24:26] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960214 (10bd808) https://gerrit.wikimedia.org/r/c/operations/puppet/+/1152083 looks to have been the most recent change to touch the `filter... [18:26:28] FIRING: PuppetAgentFailure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [18:26:36] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960215 (10wmcs-alerts) [18:31:28] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960242 (10taavi) deployment-prep might be behind in haproxy versions? `deployment-cache-upload08` is running 2.6, while the Puppet code defau... [18:55:53] (03approved) 10dancy: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 (owner: 10dduvall) [20:02:26] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960720 (10bd808) >>! In T398176#10960242, @taavi wrote: > deployment-prep might be behind in haproxy versions? `deployment-cache-upload08` is... [20:03:29] !log Remove `profile::cache::haproxy::version: haproxy26` from deployment-cache Prefix Puppet (T398176) [20:03:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:03:31] T398176: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176 [20:03:40] (03merge) 10dduvall: ci: Switch to strict semver tag format and create a release [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/136 [20:06:38] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960742 (10bd808) `lang=shell-session,counterexample bd808@deployment-cache-upload08:~$ sudo -i puppet agent -tv Info: Using environment 'prod... [20:09:44] (03PS1) 10Reedy: zuul: Drop CI for REL1_42 [integration/config] - 10https://gerrit.wikimedia.org/r/1165147 (https://phabricator.wikimedia.org/T389313) [20:10:03] (03CR) 10Reedy: [C:04-2] "Not quite yet" [integration/config] - 10https://gerrit.wikimedia.org/r/1165147 (https://phabricator.wikimedia.org/T389313) (owner: 10Reedy) [20:11:18] (03update) 10dduvall: version: 2.7.0 [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/137 [20:11:25] (03open) 10dduvall: version: 2.7.0 [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/137 [20:11:28] RESOLVED: PuppetAgentFailure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentFailure [20:11:36] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960778 (10wmcs-alerts) [20:11:49] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960780 (10bd808) Puppet didn't actually update the package. `lang=shell-session bd808@deployment-cache-upload08:~$ sudo apt-get upgrade hapro... [20:12:59] !log Upgraded haproxy to 2.8.14-1~bpo11+1 on deployment-cache-upload08 (T398176) [20:13:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:13:01] T398176: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176 [20:15:09] (03merge) 10dduvall: version: 2.7.0 [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/137 [20:17:11] !log Upgraded haproxy to 2.8.14-1~bpo11+1 on deployment-cache-text08 (T398176) [20:17:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:17:55] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960843 (10bd808) `lang=shell-session bd808@deployment-cache-text08:~$ sudo -i puppet agent -tv Info: Using environment 'production' Info: Ret... [20:21:28] 10Beta-Cluster-Infrastructure: Puppet agent failure detected on instance deployment-cache-upload08 in project deployment-prep - https://phabricator.wikimedia.org/T398176#10960857 (10bd808) 05Openβ†’03Resolved a:03bd808 >>! In T398176#10960242, @taavi wrote: > deployment-prep might be behind in haproxy ve... [20:27:50] (03open) 10dduvall: ci: Remove asset link for image [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/138 [20:30:47] (03merge) 10dduvall: ci: Remove asset link for image [repos/releng/kokkuri] - 10https://gitlab.wikimedia.org/repos/releng/kokkuri/-/merge_requests/138 [21:21:30] (03approved) 10thcipriani: scap php: New subcommand [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/859 (owner: 10dancy) [21:40:09] !log Unblocked 46.28.80.0/21 at CDN edge (T398124) [21:40:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:41:03] (03approved) 10brennen: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 (owner: 10dancy) [21:45:12] (03update) 10dancy: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 [21:46:23] 10Beta-Cluster-Infrastructure: Unblock 188.214.8.0/21 for Beta Cluster - https://phabricator.wikimedia.org/T395709#10961113 (10bd808) [21:47:25] (03merge) 10dancy: Log.vue: Clear intervalTimer on unmount [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/867 [22:09:39] !log Blocked 4 Class C networks with >1000 hits in the last 100,000 Beta Cluster requests [22:09:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:30:28] bd808: just a note that Class C ceased to exist in 1993 and looking it up that still makes it in training material :] [22:31:12] that tech debt has existed for longer (32 years) than it had been in action (1981-1993 or 12 years) [22:31:15] anyway [22:31:22] thanks for taking care of beta! [22:32:05] Are you trying to tell me that the invention of CIRD invalidated the concept of Class A, B & C networks? [22:32:11] *CIDR [22:32:19] no clue about the CIDR notation [22:32:31] but classes A, B, C, D, E were the classfull system [22:32:44] https://en.wikipedia.org/wiki/Classful_network still exists [22:32:44] which is how some large US companies ended up having a full /8 [22:33:08] and universities and states [22:33:19] yeh [22:33:21] yeah [22:33:28] I think the state of Idaho still has a Class A [22:33:49] or maybe it was a B. anyway [22:33:54] when I worked on the IPv6 address plan for the french postal services, I went to dig in the archive to get details about how they went to do the previous addressing plan (based on IPv4 [22:34:18] Class C == /24 in CIDR parlance [22:34:22] there was a memo explaining they asked for a Class A which got denied, and given the number of equipment they had they had no choice but to use the FULL address plan [22:34:47] with proxies in between to do the mapping. Fun times [22:35:25] nested NATs are how a lot of the internet still works sadly [22:35:38] sad? [22:35:48] that is the most robust model: "if it works, don't fix it!" [22:35:54] and because cgnat isn't enough... [22:36:04] https://www.ispreview.co.uk/index.php/2024/06/isp-sky-broadband-uk-deploying-ip-address-sharing-via-map-t.html [22:36:18] my ISP is sharing IPv4 [22:36:35] I have a range of port allocated, other ranges are for different customers [22:36:40] the "temporary" hack of NAT that was supposed to be replaced with IPv6 instead got replaced with more NAT [22:36:49] hey [22:36:53] you can do NAT with IPv6! [22:37:03] you can, but you don't need to [22:37:29] NAT as security barrier is a different sad thing to think about [22:39:43] I think the idea is a class C was assigned to a single entity [22:39:53] Reedy: so MAP-T is a 6to4 gateway I guess? [22:40:17] and with CIDR the IP spaces went further fragmented, though larger blocks were assigned to each regional registries (ARIN, APNIC, RIPE etc) [22:40:21] seemingly [22:40:22] >Unlike CGNAT, which is more of a stateful solution (i.e. it has to track the state of every single lease, session, IP address assignment and more), MAP-T is primarily stateless. Put another way, it can do what CGNAT does (sharing addresses), albeit without the costly overheads (CPU, database and memory requirements) because it doesn’t have to keep track of every single lease, session etc. This provides v4-v4 connectivity over a v6 domain ( [22:40:22] explained more below). [22:40:46] "MAP uses the extra bits available in the IPv6 address to contain the extra port range identifier bits of the A+P addressing pair that cannot be encoded directly into the IPv4 address, thus eliminating the need for "port routing" within the carrier network by leveraging the provider's own IPv6 rollout." [22:40:54] https://en.wikipedia.org/wiki/Mapping_of_Address_and_Port [22:41:07] nowadays I have no idea, there can be a /28 in the USA while the next contiguous one could have been landed to RIPE for an europoean customer. My understanding is that the IPv4 space is now heavily fragmented [22:42:56] Things are fragmented for sure, but I think it is still relatively rare for an allocation of less than a /24 to be made outside of a shared hosting/ISP setup with a shared egress [22:43:22] Like RIPE will give out a /24 and the ISP that gets it can carve it up smaller [22:45:59] anyway, I will take the note that kids these days might not believe that Class C == /24. [22:46:29] in general you cannot advertise a v4 subnet smaller than a /24 to the DFZ [22:46:49] s/subnet/route/ [22:56:10] 10Continuous-Integration-Config, 10Quibble: Unable to specify CLDR as an extension dependency in CI - https://phabricator.wikimedia.org/T398256 (10Daimona) 03NEW [23:38:35] (03CR) 10Reedy: zuul: Drop CI for REL1_42 [integration/config] - 10https://gerrit.wikimedia.org/r/1165147 (https://phabricator.wikimedia.org/T389313) (owner: 10Reedy) [23:53:08] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 07Essential-Work, 05Release, 05Train Deployments: 1.45.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T392178#10961421 (10SecurityPatchBot)