[07:35:52] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 (10hashar) For the repro: ` $ git clone https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm $ docker run... [07:49:40] (03PS1) 10Hashar: dockerfiles: civicrm: install db with root auth [integration/config] - 10https://gerrit.wikimedia.org/r/957237 (https://phabricator.wikimedia.org/T307178) [07:49:56] (03CR) 10Hashar: [C: 03+2] dockerfiles: civicrm: install db with root auth [integration/config] - 10https://gerrit.wikimedia.org/r/957237 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [07:50:41] (03CR) 10Hashar: "I have done the same for civicrm https://gerrit.wikimedia.org/r/c/integration/config/+/957237 for T307178" [integration/quibble] - 10https://gerrit.wikimedia.org/r/731438 (owner: 10Hashar) [07:51:08] (03Merged) 10jenkins-bot: dockerfiles: civicrm: install db with root auth [integration/config] - 10https://gerrit.wikimedia.org/r/957237 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [07:55:29] !log Built docker-registry.wikimedia.org/releng/civicrm:0.3.0-s4 image for T307178 [07:55:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:55:32] T307178: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 [08:00:18] 10Gerrit, 10Wikidata, 10Wikidata Query UI, 10wdwb-tech, 10Wikidata Dev Team (Sprint-โˆž): wikidata-query-gui-build doesnโ€™t work when latest commit is by dependabot (commit-msg hook adds Change-Id in wrong place) - https://phabricator.wikimedia.org/T295601 (10ItamarWMDE) @HasanAkgun_WMDE this seems to be ap... [08:04:17] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata Query Builder, 10Wikidata Query UI, and 6 others: [QB] [WDQS-GUI] Move build scripts from CI to the repository - https://phabricator.wikimedia.org/T328543 (10ItamarWMDE) 05Openโ†’03Stalled @hashar sorry to ping you, but I think we need you for the r... [08:36:11] (03PS1) 10Hashar: dockerfiles: civicrm: create db from source repo script [integration/config] - 10https://gerrit.wikimedia.org/r/957243 (https://phabricator.wikimedia.org/T307178) [08:38:51] 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T343728 (10jnuche) [08:40:29] (03CR) 10Hashar: [C: 03+2] "I am building the Bullseye image so I can try it out against https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/787694" [integration/config] - 10https://gerrit.wikimedia.org/r/957243 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [08:41:34] (03Merged) 10jenkins-bot: dockerfiles: civicrm: create db from source repo script [integration/config] - 10https://gerrit.wikimedia.org/r/957243 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [08:43:55] (03PS1) 10Hashar: jjb: update image for wikimedia-fundraising-civicrm-bullseye-docker [integration/config] - 10https://gerrit.wikimedia.org/r/957244 (https://phabricator.wikimedia.org/T307178) [08:44:23] (03CR) 10Hashar: [C: 03+2] jjb: update image for wikimedia-fundraising-civicrm-bullseye-docker [integration/config] - 10https://gerrit.wikimedia.org/r/957244 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [08:45:38] (03Merged) 10jenkins-bot: jjb: update image for wikimedia-fundraising-civicrm-bullseye-docker [integration/config] - 10https://gerrit.wikimedia.org/r/957244 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [08:54:01] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Migrate all CI jobs from stretch to buster or later and drop stretch testing support - https://phabricator.wikimedia.org/T278203 (10hashar) The CiviCRM repository is still tested with Stretch, that is being addressed by T307178 [08:58:29] (03PS1) 10Hashar: zuul: move wikimedia/fundraising/crm to Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) [08:59:32] (03CR) 10CI reject: [V: 04-1] zuul: move wikimedia/fundraising/crm to Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [09:06:05] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 (10hashar) @Ejegg here is the summary for this morning hacking session: I got crm to pass with the Bullseye... [09:06:44] ejegg: I finally managed to do the bits needed to migrate the Civi CRM to Bullseye and wrote the summary at https://phabricator.wikimedia.org/T307178#9162722 . Please poke me when you show up :-] [10:14:19] 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T343728 (10Aklapper) After deployment of `1.41.0-wmf.26` to group1, we had a spike from T345856 (as warned about in T343728#9152385) so... [10:14:40] 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10Patch-For-Review, 10Release, 10Train Deployments: 1.41.0-wmf.26 deployment blockers - https://phabricator.wikimedia.org/T343728 (10Aklapper) [10:28:15] 10Beta-Cluster-Infrastructure, 10Dumps-Generation, 10MediaWiki-ContentHandler: XMLDumps broken on deployment-mwmaint02 due to Jade Extension related content - https://phabricator.wikimedia.org/T345874 (10Ladsgroup) The content handler is registered from what I'm seeing ` ladsgroup@deployment-deploy03:~$ mwsc... [10:30:51] 10Beta-Cluster-Infrastructure, 10Dumps-Generation, 10MediaWiki-ContentHandler: XMLDumps broken on deployment-mwmaint02 due to Jade Extension related content - https://phabricator.wikimedia.org/T345874 (10Ladsgroup) I think `onContentHandlerForModelID` hook is doing something unbecoming. [10:40:28] 10Beta-Cluster-Infrastructure, 10Dumps-Generation, 10MediaWiki-ContentHandler: XMLDumps broken on deployment-mwmaint02 due to Jade Extension related content - https://phabricator.wikimedia.org/T345874 (10Ladsgroup) That was a red herring but I found the problem ` ladsgroup@deployment-deploy03:~$ mwscript eva... [10:45:52] 10Beta-Cluster-Infrastructure, 10Dumps-Generation, 10MediaWiki-ContentHandler, 10Patch-For-Review: XMLDumps broken on deployment-mwmaint02 due to Jade Extension related content - https://phabricator.wikimedia.org/T345874 (10Ladsgroup) Any review of this would be very welcome ^ [11:34:37] (03CR) 10Hashar: [C: 03+2] "I have deleted the Jenkins jobs :)" [integration/config] - 10https://gerrit.wikimedia.org/r/956935 (https://phabricator.wikimedia.org/T346176) (owner: 10Hashar) [11:36:11] (03Merged) 10jenkins-bot: Archive wikimedia/discovery/analytics [integration/config] - 10https://gerrit.wikimedia.org/r/956935 (https://phabricator.wikimedia.org/T346176) (owner: 10Hashar) [11:36:15] (03Merged) 10jenkins-bot: jjb: remove {name}-tox-docker-with-sonar [integration/config] - 10https://gerrit.wikimedia.org/r/956936 (https://phabricator.wikimedia.org/T346176) (owner: 10Hashar) [11:54:03] 10Phabricator, 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10User-brennen: Unify gitlab migration scripts - https://phabricator.wikimedia.org/T346191 (10Aklapper) > Let's make one script that includes the work we have already done and updates the repo uri in diffusion. If there was a mapping between re... [11:56:57] (03PS1) 10Hashar: jjb: upgrade pywikibot repos to tox 4.8.0 [integration/config] - 10https://gerrit.wikimedia.org/r/957277 (https://phabricator.wikimedia.org/T345695) [11:59:00] Hello. I'm starting to bump up against a 20 minute execution timeout for a job in GitLab-CI, building a Spark distribution. e.g. https://gitlab.wikimedia.org/repos/data-engineering/spark/-/jobs/142052 [12:00:01] What are my options? Ultimately, I'm looking to build artifacts for production, but I haven't requested access to a trusted runner for this project yet. [12:01:00] btullis: can't you set a timeout in the .gitlab-ci.yaml stages ( https://docs.gitlab.com/ee/ci/yaml/#timeout ) [12:01:46] then in reality I don't know how the build times out are set/enforced [12:02:00] Oh great, thanks. I will try that. I'd assumed that I was limited by a runner-specific timeout: https://docs.gitlab.com/ee/ci/runners/configure_runners.html#set-maximum-job-timeout-for-a-runner [12:04:07] Trying again with a 2h timeout. We'll know in 20 minutes whether it works, I suppose. Thanks again hashar [12:08:53] (03PS1) 10Hashar: Archive some pywikibot repositories [integration/config] - 10https://gerrit.wikimedia.org/r/957280 [12:10:26] btullis: else maybe the timeout can be configured on a per repo basis in the Gitlab WEB UI (there is a CI/CD configuration pannel) [12:10:49] btullis: or maybe the 20 minutes is enforced globally / at the runner level which would need an admin to change it or allow a repo to change it [12:11:01] (03CR) 10Hashar: [C: 03+2] Archive some pywikibot repositories [integration/config] - 10https://gerrit.wikimedia.org/r/957280 (owner: 10Hashar) [12:12:07] (03PS2) 10Hashar: Archive some pywikibot repositories [integration/config] - 10https://gerrit.wikimedia.org/r/957280 [12:12:09] (03PS2) 10Hashar: jjb: upgrade pywikibot i18 and xqbot tox 4.8.0 [integration/config] - 10https://gerrit.wikimedia.org/r/957277 (https://phabricator.wikimedia.org/T345695) [12:20:10] (03CR) 10Hashar: [C: 03+2] "Passed on https://gerrit.wikimedia.org/r/c/pywikibot/i18n/+/957278 and https://gerrit.wikimedia.org/r/c/pywikibot/bots/xqbot/+/957279" [integration/config] - 10https://gerrit.wikimedia.org/r/957277 (https://phabricator.wikimedia.org/T345695) (owner: 10Hashar) [12:21:32] (03Merged) 10jenkins-bot: Archive some pywikibot repositories [integration/config] - 10https://gerrit.wikimedia.org/r/957280 (owner: 10Hashar) [12:21:34] (03Merged) 10jenkins-bot: jjb: upgrade pywikibot i18 and xqbot tox 4.8.0 [integration/config] - 10https://gerrit.wikimedia.org/r/957277 (https://phabricator.wikimedia.org/T345695) (owner: 10Hashar) [12:39:34] 10Continuous-Integration-Config, 10VisualEditor, 10banana-checker, 10Patch-For-Review: banana-checker doesn't complain about empty strings in VisualEditor - https://phabricator.wikimedia.org/T346170 (10CodeReviewBot) esanders opened https://gitlab.wikimedia.org/repos/ci-tools/banana-checker/-/merge_request... [12:39:45] 10Continuous-Integration-Config, 10VisualEditor, 10banana-checker, 10Patch-For-Review: banana-checker doesn't complain about empty strings in VisualEditor - https://phabricator.wikimedia.org/T346170 (10Esanders) Upstream fix: https://gitlab.wikimedia.org/repos/ci-tools/banana-checker/-/merge_requests/7 [12:46:02] Project beta-scap-sync-world build #120472: 04FAILURE in 52 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/120472/ [12:49:25] hashar: Yes, it failed again after 20 minutes. I think it's configured at the runner level. I'll have a look and see what the timeout is for the trusted runners, if I can. Otherwise, we have one of our own runners that I might be able to use for this bit of testing. [12:56:17] Yippee, build fixed! [12:56:17] Project beta-scap-sync-world build #120473: 09FIXED in 1 min 12 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/120473/ [13:07:05] woohoo, thanks for all that work, hashar! [13:07:32] looks like there's just one test referring to the old job name causing the tox fail on https://gerrit.wikimedia.org/r/c/integration/config/+/957249 [13:07:37] I can tweak that if you want [13:09:22] oh i see, that test must just loop over all the stuff from layout.yaml [13:09:32] then I'll just delete the old job [13:09:47] (03PS2) 10Ejegg: zuul: move wikimedia/fundraising/crm to Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [13:10:22] ah, the vendor subrepo was also being tested against the old job [13:10:55] (03CR) 10Ejegg: "Thanks so much, hashar! PS2 just removes the last vestiges of the old job name." [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [13:12:43] 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10Pywikibot, 10Upstream: tox v4 with skipsdist=true does not recognize use_develop=true - https://phabricator.wikimedia.org/T346238 (10hashar) [13:13:25] ejegg: good morning! [13:14:19] so yeah essentially there were two issues [13:14:53] one with the authentication being changed in newer mariadb which is solved by using `--auth-root-authentication-method=normal` (that allows a non root user to connect as root with no password) [13:15:10] and the other were the GLOBAL being set for innodb which I think were ineffective previously [13:15:33] and to allow you to change the database setting however you want I have moved the mysql_install_db from the CI image toward the crm repo [13:15:53] the build passes, but I hvae no idea whether it is doing the right thing :) [13:16:18] hashar ok, thanks! So actually, all of those innodb_ settings are the defaults as of mariadb 10.5 (which CI is on) [13:16:38] we only ever needed to touch them so we could use utf8mb4 on an earlier version [13:16:49] if it's simpler we can just remove those settings entirely [13:16:58] 10Continuous-Integration-Config, 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10Patch-For-Review: Deal with tox 4 upgrading - https://phabricator.wikimedia.org/T345695 (10hashar) [13:17:03] but I'm fine leaving them there since the build passes [13:17:10] Ahhh [13:17:26] so potentially we could even remove that ci-create-dbs.sh script entirely [13:17:34] I'm so glad you had the chance to look at it - it would have taken me a lot longer to find that --auth-root-authentication-method=normal switch! [13:17:35] and bring back the command to the image [13:17:46] hashar yes indeed, let's remove it! [13:17:50] the cleaner the better [13:17:52] then that cuts you from the opportunity to tune the settings later on [13:18:03] I doubt we'll want to [13:18:51] utf8mb4 ought to be enough codepoints for anybody [13:18:52] :) [13:19:31] https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/957290 Drop innodb settings from ci-create-dbs.sh [13:19:39] I am trying without any innodb settings [13:20:07] since we want to mirror production, and since our ops team is very reluctant to spend another lots of hours changing storage formats, there's a lot of inertia keeping us on the current settings [13:20:14] if that works I will rebuild the CI image back to use the mysql_install_db and stop it running bin/ci-create-dbs.sh [13:20:50] and in the parent change I did this morning ( https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/787694 ) remove the ci-create-dbs.sh script [13:21:00] :) [13:21:10] for Quibble/MediaWiki I got Mariadb to use binary mode [13:21:51] cause that is what we use for MediaWiki in prod, but it is a different beast [13:22:03] I think it is because at the time MySQL did not support all of unicode [13:22:15] and probably still does not support the whole plane [13:23:15] well something explodes https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-docker/11196/console :/ [13:23:38] ah, but that's on the old stretch image [13:23:41] to be expected [13:23:50] ahaha [13:23:52] the experimental bullseye run is looking good so far [13:23:53] https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-bullseye-docker/8/console [13:24:33] we will see :] [13:27:48] (03PS1) 10Hashar: Revert "dockerfiles: civicrm: create db from source repo script" [integration/config] - 10https://gerrit.wikimedia.org/r/956821 [13:29:24] (03CR) 10Hashar: "I will rebase" [integration/config] - 10https://gerrit.wikimedia.org/r/956821 (owner: 10Hashar) [13:32:05] 10WikimediaDebug, 10MediaWiki-Platform-Team, 10Observability-Tracing: Add support for request tracing to WikimediaDebug browser extension - https://phabricator.wikimedia.org/T340573 (10pmiazga) @CDanis could you elaborate a little bit on this one? What do you mean "force a distributed tracing" ? Currently t... [13:33:15] (03PS2) 10Hashar: dockerfiles: civicrm [integration/config] - 10https://gerrit.wikimedia.org/r/956821 (https://phabricator.wikimedia.org/T307178) [13:33:55] ejegg: so for the CI image I have send https://gerrit.wikimedia.org/r/c/integration/config/+/956821/2/dockerfiles/civicrm/run-with-mysqld.sh [13:34:14] which removes the call to bin/ci-create-dbs.sh and restore the creation of the mariadb data dir with whatever defaults are shipped with Bullseye [13:34:30] aka I revert the change I made this morning AND remove the call to ci-create-dbs.sh [13:35:13] (03CR) 10Hashar: [C: 03+2] dockerfiles: civicrm [integration/config] - 10https://gerrit.wikimedia.org/r/956821 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [13:35:43] I am building it, bumping the bullseye job to the new image then will amend the crm change to remove the script and check experimental again [13:36:22] (03Merged) 10jenkins-bot: dockerfiles: civicrm [integration/config] - 10https://gerrit.wikimedia.org/r/956821 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [13:38:32] (03PS1) 10Hashar: jjb: update image for wikimedia-fundraising-civicrm-bullseye-docker [integration/config] - 10https://gerrit.wikimedia.org/r/957291 (https://phabricator.wikimedia.org/T307178) [13:38:55] (03CR) 10Hashar: [C: 03+2] jjb: update image for wikimedia-fundraising-civicrm-bullseye-docker [integration/config] - 10https://gerrit.wikimedia.org/r/957291 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [13:40:04] (03Merged) 10jenkins-bot: jjb: update image for wikimedia-fundraising-civicrm-bullseye-docker [integration/config] - 10https://gerrit.wikimedia.org/r/957291 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [13:44:33] (03PS3) 10Hashar: zuul: move wikimedia/fundraising/crm to Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) [13:45:05] what a spam machine I am :-\ [13:45:46] https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-bullseye-docker/9/console [13:46:12] 10GitLab (CI & Job Runners), 10Data-Platform-SRE: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10BTullis) [13:46:31] 10GitLab (CI & Job Runners), 10Data-Platform-SRE: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10BTullis) a:03BTullis [13:49:57] 10Phabricator, 10Release-Engineering-Team, 10collaboration-services, 10User-AKlapper: Establish a workflow that scales for requesting Phab 2FA resets - https://phabricator.wikimedia.org/T306708 (10Aklapper) a:05Aklapperโ†’03None Removing myself as assignee. Not sure who could drive this to a resolution t... [13:50:17] 10Release-Engineering-Team (Priority Backlog ๐Ÿ“ฅ), 10Wikimedia-Phabricator-Extensions, 10Patch-For-Review: Make error text clearer when blocked MediaWiki account tries to log into Phabricator - https://phabricator.wikimedia.org/T344195 (10Aklapper) [13:59:11] I've made a request for access to the trusted runners for my project (T346244) - Following the instructions here: https://wikitech.wikimedia.org/wiki/GitLab/Gitlab_Runner/Trusted_Runners#Request_access_to_Trusted_Runners [13:59:12] T346244: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 [14:00:05] ejegg: it passed! [14:00:19] hooray! [14:00:22] However, I've made a patch to https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner as instructed, but I can't push it. [14:00:41] I'll C+2 whatever needs it [14:01:06] (03CR) 10Ejegg: [C: 03+1] "Thanks, this looks great!" [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [14:01:25] I am switching CI to the bullseye job [14:02:16] (03CR) 10Hashar: [C: 03+2] "And the experimental Bullseye job passed on https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/787694 !! :)" [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [14:02:16] Nice, I have a big ol' chunk of library updates just waiting for the PHP7.4 bump [14:03:31] (03Merged) 10jenkins-bot: zuul: move wikimedia/fundraising/crm to Bullseye [integration/config] - 10https://gerrit.wikimedia.org/r/957249 (https://phabricator.wikimedia.org/T307178) (owner: 10Hashar) [14:04:00] deployed [14:04:15] ejegg: you can CR+2 https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/787694 :) [14:04:37] I am not sure why I bailed out on the original change ~ 18 months ago [14:05:04] I think maybe we weren't ready - we only migrated production a couple of months ago [14:05:24] it was the last box in the fundraising cluster still on buster [14:09:46] I don't know which php version is shipped though [14:11:24] buster was still PHP 7.3, so production just got on to 7.4 in July [14:13:04] and you use the Debian stock package? [14:14:05] I have checked the civicrm image has php 7.3 from Bullseye [14:15:39] ah we can get rid of composer-php73-docker too [14:15:51] oh weird, 7.3 on bullseye? [14:17:20] shoot, we need 7.4 [14:17:43] Let me see where they're getting that from on prod - I was pretty sure it was straight from debian [14:19:02] how do you install php 7.4 under bullseye? Maybe SRE provides it via an apt component/php74 [14:19:27] trying to remember the apt command to query source of a package [14:19:33] do you have that top of mind? [14:19:37] apt-cache policy php-cli [14:19:46] or: apt-cache policy php7.4-cli [14:19:52] http://debian.csail.mit.edu/debian bullseye/main amd64 Packages [14:20:14] Installed: 2:7.4+76 [14:20:25] oh I got confused cause bullseye comes with 7.4 https://packages.debian.org/search?keywords=php-cli [14:20:27] so that seems pretty standard [14:20:50] hmm, so it there a bit in the CI image that explicitly requests 7.3 ? [14:21:24] $ docker run --rm -it --entrypoint=php docker-registry.wikimedia.org/releng/civicrm:0.3.0-s6 --version [14:21:24] PHP 7.3.33-14+0~20230902.114+debian11~1.gbp764b27 (cli) (built: Sep 2 2023 07:10:18) ( NTS ) [14:21:27] Those are in the dev-images repo on gitlab, right? [14:22:02] nop the CI images from integration/config [14:22:08] ah ok [14:22:31] I am a large fan of the movie Inception, but not so much a fan of the amount of layers in our stack :D [14:22:36] hehe [14:22:44] so [14:23:01] the civicrm image is build from an image named php7.3 which is based upon Bullseye but borrows php 7.3 from sury.org [14:23:21] ah I see [14:23:25] sury.org is a site maintained by the Debian developer doing the php package and he is backporting/forward porting various php versions [14:23:40] right, let's just replace those 7.3 lines with 7.4 [14:23:53] and it should pull from the standard 7.4 sources, I think? [14:24:15] we can go with the releng image php 7.4 but that will gives the version we use for MEdiaWiki production, not the one from Debian [14:24:32] ok, I think that should be close enough [14:24:52] and it is based on Buster [14:24:58] ah dang [14:26:13] We have a dockerfile for civicrm locally that uses php7.4 on bullseye. Would it help to start with that? [14:26:37] I am crafting the patch :) [14:26:41] https://gitlab.wikimedia.org/repos/releng/dev-images/-/tree/main/dockerfiles/fundraising-civicrm-bullseye-php74-apache2 [14:30:20] (03PS1) 10Hashar: dockerfiles: civicrm: upgrade to php7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957305 [14:30:45] ejegg: yeah CI doesn't use that image :/ [14:31:17] (03PS2) 10Hashar: dockerfiles: civicrm: upgrade to php7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957305 [14:31:28] that might do it [14:32:28] and since the build failed locally due to some issue with AMPHOME I guess I will switch the Jenkins job and send a dummy patch [14:33:09] (03CR) 10Hashar: [C: 03+2] "That got build locally" [integration/config] - 10https://gerrit.wikimedia.org/r/957305 (owner: 10Hashar) [14:33:13] ah cool [14:33:53] I'm looking at our dev images and seeing a fairly long list of extensions, but I guess whatever the previous CI image had is probably enough! [14:33:56] https://gitlab.wikimedia.org/repos/releng/dev-images/-/blob/main/dockerfiles/fundraising-bullseye-php74/Dockerfile.template [14:34:15] (03PS1) 10Hashar: jjb: civicrm: upgrade php from 7.3 to 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957306 [14:34:39] yeah I am not sure about the php extensions that are required :-\ [14:35:15] (03Merged) 10jenkins-bot: dockerfiles: civicrm: upgrade to php7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957305 (owner: 10Hashar) [14:35:24] building it [14:38:26] I hvae updated the job and I have send the dummy commit [14:38:27] https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-bullseye-docker/12/console [14:40:02] it is missing gd [14:40:23] and there are some more: [14:40:24] 00:00:54.715 WARNING: Failed to find recommended PHP extension "gd". [14:40:24] 00:00:54.817 WARNING: Failed to find recommended PHP extension "imap". [14:40:24] 00:00:55.011 WARNING: Failed to find recommended PHP extension "zip". [14:45:25] (03PS2) 10Hashar: jjb: civicrm: upgrade php from 7.3 to 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957306 [14:45:27] (03PS1) 10Hashar: dockerfiles: civicrm: add gd, imap, zip php extensions [integration/config] - 10https://gerrit.wikimedia.org/r/957309 [14:45:37] (03CR) 10Hashar: [C: 03+2] dockerfiles: civicrm: add gd, imap, zip php extensions [integration/config] - 10https://gerrit.wikimedia.org/r/957309 (owner: 10Hashar) [14:46:45] (03Merged) 10jenkins-bot: dockerfiles: civicrm: add gd, imap, zip php extensions [integration/config] - 10https://gerrit.wikimedia.org/r/957309 (owner: 10Hashar) [14:49:47] hehe, always something more [14:50:05] 10GitLab (CI & Job Runners), 10Data-Platform-SRE: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10BTullis) I've been following the instruction here: https://wikitech.wikimedia.org/wiki/GitLab/Gitlab_Runner/Trusted_Runners#Request_access_to_Trusted_Run... [14:50:21] yeah I am used to it :) [14:50:43] as long as we end up in a better shape than ($day - 1), I am happy about it [14:50:52] 10GitLab (CI & Job Runners), 10Data-Platform-SRE: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10BTullis) [14:51:46] hehe, yep, I can feel the progress being made [14:51:54] https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-bullseye-docker/13/console [14:52:06] and we can drop the composer php73 job can't we? [14:52:11] yes, that too [14:52:45] (03PS3) 10Hashar: jjb: civicrm: upgrade php from 7.3 to 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957306 [14:53:22] * hashar waits for CI https://xkcd.com/303/ [14:53:24] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Platform-SRE, 10serviceops: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10BTullis) a:05BTullisโ†’03None Removing myself as the assignee, since it appears that I will need assistan... [14:54:04] ejegg: i guess you can +1 that last change [14:54:09] and if the build passes I will +2 it [14:54:16] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Platform-SRE, 10serviceops: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10dancy) >>! In T346244#9164100, @BTullis wrote: > It appears that I do not have the required rights to creat... [14:54:20] and we would have migrated to Bullseye + php 7.4 \o/ [14:54:39] (03CR) 10Ejegg: [C: 03+1] "Looks good to me!" [integration/config] - 10https://gerrit.wikimedia.org/r/957306 (owner: 10Hashar) [14:56:20] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Platform-SRE, 10serviceops: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10BTullis) >>! In T346244#9164113, @dancy wrote: >>>! In T346244#9164100, @BTullis wrote: >> It appears that... [14:56:30] success [14:58:19] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Platform-SRE, 10serviceops, 10Patch-For-Review: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10CodeReviewBot) dancy opened https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-... [14:58:25] hmm no I wasn't looking at the proper build [14:58:29] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Platform-SRE, 10serviceops, 10Patch-For-Review: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10CodeReviewBot) dancy merged https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-... [14:59:03] 00:07:16.710 .EEEEEEEEEEEEEEEEEEEEEEEEEEEEE........EEE...................... 63 / 632 ( 9%) [14:59:04] :( [14:59:16] dolphin.town approves [15:01:32] (03CR) 10Hashar: [C: 04-1] "I gave it a try on a dummy change but the build fails under php 7.4: https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicr" [integration/config] - 10https://gerrit.wikimedia.org/r/957306 (owner: 10Hashar) [15:01:37] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Platform-SRE, 10serviceops, 10Patch-For-Review: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10dancy) 05Openโ†’03Resolved a:03dancy You're all set. [15:02:03] ejegg: so yeah with php 7.4 phpunit has some failures https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-bullseye-docker/13/console the build hasn't completed yet though [15:02:27] PDOException: could not find driver [15:03:12] ahh, just needs php7.4-mysql then? [15:03:19] I thought you had that though [15:03:40] I thought as well [15:03:51] weird, definitely in the dockerfile [15:04:06] $ php -m|grep -i mysql [15:04:06] mysqli [15:04:06] mysqlnd [15:04:06] pdo_mysql [15:04:22] but sqlite is not present [15:04:32] I removed it compared to the other image [15:04:38] so maybe smashpig relies on sqlite [15:07:51] (03PS1) 10Hashar: dockerfiles: civicrm: add sqlite php extension [integration/config] - 10https://gerrit.wikimedia.org/r/957312 [15:08:12] aha, that could be it [15:09:06] (03PS2) 10Hashar: dockerfiles: civicrm: add sqlite php extension [integration/config] - 10https://gerrit.wikimedia.org/r/957312 [15:09:31] brennen: Hi. If you can access https://phabricator.wikimedia.org/phame/post/view/307/from_phabricator_to_phorge/ (Phab blog), could you give that short draft post about Moving to Phorge a quick read (and your okay)? TIA! [15:09:57] andre: yep, one moment [15:10:33] andre: LGTM! [15:10:45] yay [15:13:40] 10GitLab (Pipeline Services Migration๐Ÿค), 10Metrics Platform Backlog, 10Data Products (Sprint 01): Migrate metrics-platform repo to GitLab - https://phabricator.wikimedia.org/T344733 (10phuedx) [15:14:05] (03PS4) 10Hashar: jjb: civicrm: upgrade php from 7.3 to 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957306 [15:15:04] (03CR) 10Hashar: [C: 03+2] "That will give us the sqlite3 php extension which might fix smashpig." [integration/config] - 10https://gerrit.wikimedia.org/r/957312 (owner: 10Hashar) [15:16:11] (03Merged) 10jenkins-bot: dockerfiles: civicrm: add sqlite php extension [integration/config] - 10https://gerrit.wikimedia.org/r/957312 (owner: 10Hashar) [15:18:40] 10Phabricator, 10Release-Engineering-Team: New post for "Phabricating Phabricator" blog to celebrate migration to Phorge - https://phabricator.wikimedia.org/T340223 (10Aklapper) 05Openโ†’03Resolved Published: https://phabricator.wikimedia.org/phame/post/view/307/from_phabricator_to_phorge/ [15:29:28] ejegg: that was the lack of php-sqlite3 apparently [15:30:38] the thing is that I did notice a bunch of extensions in the orogiinal sury.org based images and felt like they were not needed [15:30:47] given the work on mysql earlier today, I felt like sqlite3 was not needed [15:30:48] bah [15:30:49] :) [15:36:24] (03CR) 10Hashar: [C: 03+2] jjb: civicrm: upgrade php from 7.3 to 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957306 (owner: 10Hashar) [15:37:04] ejegg: Civi CRM passed with php 7.4 / Bullseye and MariaDB stock defaults. I will let you announce the good news to the team :] [15:37:16] thank you for all the assistance! [15:38:46] (03Merged) 10jenkins-bot: jjb: civicrm: upgrade php from 7.3 to 7.4 [integration/config] - 10https://gerrit.wikimedia.org/r/957306 (owner: 10Hashar) [15:47:30] hooray hashar ! And thank you very VERY much [15:49:02] ๐Ÿ† [15:53:28] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Migrate all CI jobs from stretch to buster or later and drop stretch testing support - https://phabricator.wikimedia.org/T278203 (10Ejegg) [15:53:33] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 (10Ejegg) 05Openโ†’03Resolved a:03Ejegg It's working! [15:54:50] 10Continuous-Integration-Infrastructure, 10Wikimedia-Fundraising-CiviCRM: CiviCRM CI jobs fails when migrating from Stretch to Bullseye - https://phabricator.wikimedia.org/T307178 (10hashar) antoine-approve [17:19:58] hi releng, I'm not sure why composer-php73-docker is still running against the fundraising/crm repo [17:20:07] could the layout just need a reload? [17:20:26] looks like hashar removed it in https://gerrit.wikimedia.org/r/c/integration/config/+/957306/4/zuul/layout.yaml [17:20:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Fix Puppet agent provisioning on Jenkins agent instances - https://phabricator.wikimedia.org/T341051 (10dancy) @hashar Note that you still have a local cherry-pick related to this ticket in `integration-puppetmaster-02.integration.eqiad.wmfla... [17:20:42] but I'm seeing it run still on e.g. https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/956480/ [17:24:55] huh, still doing it on a much smaller library update patch: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/957328/ [17:34:48] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Platform-SRE, 10serviceops: Request access to trusted runners for data-engineering/spark - https://phabricator.wikimedia.org/T346244 (10dancy) >>! In T346244#9164100, @BTullis wrote: > I've been following the instruction here: https://wikitech... [18:04:06] 10GitLab (Project Migration), 10Release-Engineering-Team (Escape Goats๐Ÿ), 10collaboration-services, 10Patch-For-Review: Sync ldap/ops users to group on GitLab and define permissions for repos/sre - https://phabricator.wikimedia.org/T343035 (10dancy) [18:12:07] 10GitLab (Project Migration), 10Release-Engineering-Team (Escape Goats๐Ÿ), 10collaboration-services, 10Patch-For-Review: Sync ldap/ops users to group on GitLab and define permissions for repos/sre - https://phabricator.wikimedia.org/T343035 (10dancy) The initial sync of ldap/ops into https://gitlab.wikimedi... [18:12:24] 10GitLab (Project Migration), 10Release-Engineering-Team (Escape Goats๐Ÿ), 10collaboration-services, 10Patch-For-Review: Sync ldap/ops users to group on GitLab and define permissions for repos/sre - https://phabricator.wikimedia.org/T343035 (10dancy) p:05Triageโ†’03Low [18:32:29] 10Phabricator, 10Release-Engineering-Team, 10collaboration-services, 10User-brennen: Schedule a routine Phabricator deployment window with downtimed alerting - https://phabricator.wikimedia.org/T346266 (10brennen) [18:33:31] ejegg: I guess i forgot to reload Zuul :/ [18:34:30] 10Phabricator, 10Release-Engineering-Team, 10collaboration-services, 10Patch-For-Review, 10User-brennen: Schedule a routine Phabricator deployment window with downtimed alerting - https://phabricator.wikimedia.org/T346266 (10CodeReviewBot) brennen opened https://gitlab.wikimedia.org/repos/releng/release/... [18:34:39] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/957306 | civicrm: upgrade php from 7.3 to 7.4 [18:34:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:34:54] I multitasked to much at the end of my day [18:35:47] thanks again hashar [18:36:32] the job is gone :] (I rechecked your patch above) [18:36:32] we may have an issue with some geocoder tests but that's something we can figure out without needing to pester releng more, i hope! [18:36:47] well I am happy to be pestered [18:37:14] but long term we should phase out that job in favor of PipelineLib/Blubber which will lets you define the environment directly from the source repo [18:37:27] oho? [18:37:34] instead of relying on a CI image and workflow (zuul) behind guards [18:37:34] I should read up on that [18:38:01] greg surely knows about and others from releng knows way more about it than me [18:39:49] oh interesting, it automatically builds production docker images on merge! [18:40:10] yeahhh [18:40:23] the doc page is https://wikitech.wikimedia.org/wiki/PipelineLib [18:40:50] https://wikitech.wikimedia.org/wiki/Blubber is the home made tool to define a docker image using a custom yaml file (but I think that pages might require some updates) [18:41:21] 10Phabricator, 10Release-Engineering-Team, 10collaboration-services, 10Patch-For-Review, 10User-brennen: Schedule a routine Phabricator deployment window with downtimed alerting - https://phabricator.wikimedia.org/T346266 (10brennen) [18:41:24] and PipelineLib (implemented in Jenkins) is replaced by https://gitlab.wikimedia.org/repos/releng/kokkuri (which runs on Gitlab) [18:42:03] anyway I am off for real :] [18:49:11] 10Phabricator, 10Release-Engineering-Team, 10collaboration-services, 10Patch-For-Review, 10User-brennen: Schedule a routine Phabricator deployment window with downtimed alerting - https://phabricator.wikimedia.org/T346266 (10brennen) To update once a window is set: https://www.mediawiki.org/wiki/Phabrica... [19:18:11] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10SDunlap) Hello, I'm on the QTE team, and I'll be leading some small sprints for a tools to manage test environments for Mediawiki projects. Could I please have access to create new p... [19:23:08] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10Ladsgroup) >>! In T706#9164845, @SDunlap wrote: > Hello, I'm on the QTE team, and I'll be leading some small sprints for a tools to manage test environments for Mediawiki projects. C... [19:39:08] 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: - https://phabricator.wikimedia.org/T346276 (10SDunlap) [19:39:34] 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: qte - https://phabricator.wikimedia.org/T346276 (10SDunlap) [19:40:10] 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: qte - https://phabricator.wikimedia.org/T346276 (10brennen) 05Openโ†’03In progress a:03brennen [19:45:22] 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: qte - https://phabricator.wikimedia.org/T346276 (10brennen) 05In progressโ†’03Resolved Created. Added everyone I could find with an existing GitLab account as an owner. [19:47:18] !log gitlab: created repos/qte per T346276 [19:47:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:47:21] T346276: Create new GitLab project group: qte - https://phabricator.wikimedia.org/T346276 [20:26:47] a heads up, I do not know if I will be well enough to be at tomorrow's deployment/training window (cc thcipriani ), migraines... [20:30:45] hope you feel better soon! [22:14:02] apergos: thank you for notifying me! Hope you feel better soon <3