[03:04:10] 10Release-Engineering-Team, 10Fundraising-Backlog, 10MediaWiki-extensions-FundraiserLandingPage, 10donate.wikimedia.org, 10MW-1.42-notes (1.42.0-wmf.5; 2023-11-14): donate.wikimedia.org landing pages don't have descriptions in search engine results - https://phabricator.wikimedia.org/T254808 (10Ejegg) Th... [03:28:18] Yippee, build fixed! [03:28:18] Project mwcore-phpunit-coverage-master build #3256: 09FIXED in 28 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/3256/ [04:15:53] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T350083 (10ssastry) @hashar https://gerrit.wikimedia.org/r/c/mediawiki/core/+/977625 should be merged and backported before rolling out the train. [04:57:31] (Queue (Jenkins jobs + Zuul functions) alert) firing: - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [05:09:10] PROBLEM - Work requests waiting in Zuul Gearman server on contint2002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [05:17:32] (Queue (Jenkins jobs + Zuul functions) alert) resolved: - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [06:25:34] RECOVERY - Work requests waiting in Zuul Gearman server on contint2002 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [08:50:45] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T350083 (10hashar) >>! In T350083#9362310, @ssastry wrote: > @hashar https://gerrit.wikimedia.org/r/c/mediawiki/core/+/977625 should be merged and backported b... [09:21:35] 10Phabricator, 10DBA: Switchover m3 master db1159 -> db1119 - https://phabricator.wikimedia.org/T352149 (10Marostegui) [09:25:28] 10Phabricator, 10DBA, 10collaboration-services: Switchover m3 master db1159 -> db1119 - https://phabricator.wikimedia.org/T352149 (10Marostegui) [09:37:57] 10Phabricator, 10Release-Engineering-Team, 10DBA, 10collaboration-services: Switchover m3 master db1159 -> db1119 - https://phabricator.wikimedia.org/T352149 (10Marostegui) Just FYI, I will be putting phabricator in RO for a few seconds during this week (early in a European morning) to switchover the master. [09:46:13] 10Phabricator (Upstream), 10Release-Engineering-Team (Radar), 10Developer Productivity, 10Upstream, 10User-brennen: Add Open Graph support to Phabricator Maniphest Tasks to have link preview on Telegram, Slack, and other messaging apps - https://phabricator.wikimedia.org/T288117 (10hashar) >>! In T288117... [09:52:20] 10Release-Engineering-Team, 10MW-on-K8s, 10serviceops, 10Wikimedia-production-error: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10hashar) p:05Triage→03Unbreak! [09:59:22] 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Release, 10Train Deployments: 1.42.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T350083 (10hashar) Since yesterday we get a bunch of `Required database not available at /usr/share/GeoIP/GeoIP.dat.` which I have filed... [10:01:30] 10Release-Engineering-Team, 10MW-on-K8s, 10serviceops, 10Wikimedia-production-error: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10hashar) [10:51:00] 10GitLab (Infrastructure), 10collaboration-services: git over ssh is not working on GitLab test instance - https://phabricator.wikimedia.org/T351928 (10Jelto) 05Open→03Resolved a:03Jelto As discussed in our last Monday meeting, I double checked bastion and ssh config. The test instance uses a public addr... [11:18:18] 10Release-Engineering-Team, 10MW-on-K8s, 10serviceops, 10Wikimedia-production-error: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10Joe) The problem is that the new kubernetes nodes don't have a copy o... [11:24:07] 10Release-Engineering-Team, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, 10serviceops, 10Wikimedia-production-error: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10taavi) [11:25:26] 10Release-Engineering-Team, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, 10serviceops, 10Wikimedia-production-error: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10Joe) codesearch also says t... [12:14:55] 10Release-Engineering-Team, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, 10serviceops, and 2 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10jijiki) @jbond @Muehlenhoff It appears that w... [12:22:11] 10Release-Engineering-Team, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, 10serviceops, and 2 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10jbond) >>! In T352156#9363098, @jijiki wrote:... [12:30:40] 10Beta-Cluster-Infrastructure: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) [12:30:58] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) [12:31:07] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) [12:34:12] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) [12:38:03] 10Project-Admins: Create project tag for PlaceNewSection extension - https://phabricator.wikimedia.org/T351776 (10Ernstkm) Hi, all. Thanks for your patience. I seem to have become the //de facto// maintainer of this extension by posting [[https://www.mediawiki.org/w/index.php?title=Topic:Wfa1kbk1f2gtalfh&topic_... [12:42:56] 10Phabricator: "Call to phutil_nonempty_string() expected null or a string, got: bool" rendering alt text of image file - https://phabricator.wikimedia.org/T352170 (10Aklapper) p:05Triage→03Low [12:43:04] 10Phabricator: "Call to phutil_nonempty_string() expected null or a string, got: bool" rendering alt text of image file - https://phabricator.wikimedia.org/T352170 (10Aklapper) [13:02:57] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10Aklapper) 1.42.0-wmf.13 is six weeks away so if this should block deployment then it should be the next one instead. We... [13:03:12] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10Aklapper) [13:03:18] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.7 deployment blockers - https://phabricator.wikimedia.org/T350083 (10Aklapper) [13:13:06] 10Beta-Cluster-Infrastructure, 10Traffic, 10Beta-Cluster-reproducible: HTTP 504 connection timeout error accessing MW API on Beta cluster - https://phabricator.wikimedia.org/T351930 (10Jgiannelos) I am getting test failures on restbase because of enwiki beta Action API returning 504. I think I first encounte... [13:17:22] 10Release-Engineering-Team, 10LandingCheck, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, and 3 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10hashar) [13:18:47] 10Release-Engineering-Team, 10LandingCheck, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, and 3 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10hashar) >>! In T352156#9363024, @Joe wrote:... [13:25:30] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10hashar) I do not know what to do with this task since that involves #beta-cluster-infrastructure which might be subtly b... [13:33:00] 10Project-Admins: Create project tag for PlaceNewSection extension - https://phabricator.wikimedia.org/T351776 (10Iniquity) >>! In T351776#9363200, @Ernstkm wrote: > That said, I'm amenable to doing whatever is necessary to ensure the health and longevity of this extension. If that means moving it from GitHub to... [13:42:33] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10hashar) Support for source map in resource loader has long been requested and got enabled on October 2023 resolving T475... [13:42:39] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10hashar) [14:23:27] (03CR) 10Subramanya Sastry: [C: 03+2] "I am self merging this to pare down the review queue for everyone. This is only a testing related patch that runs on a labs vm." [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/977752 (owner: 10Subramanya Sastry) [14:24:11] (03Merged) 10jenkins-bot: Make self-link handling deal with absolute links as well [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/977752 (owner: 10Subramanya Sastry) [14:29:33] (DatasourceError) firing: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [14:38:50] 10Gitlab-Application-Security-Pipeline, 10Security-Team, 10SecTeam-Processed, 10user-sbassett: Pin OSV to v1.4.2 (not v1.4.3) until v1.5.0 comes out? - https://phabricator.wikimedia.org/T351263 (10Jdforrester-WMF) >>! In T351263#9361483, @sbassett wrote: >>>! In T351263#9360995, @Jdforrester-WMF wrote: >>... [14:39:33] (DatasourceError) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [14:40:39] 10Release-Engineering-Team, 10LandingCheck, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, and 3 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10Joe) >>! In T352156#9363316, @hashar wrote:... [14:42:21] 10Release-Engineering-Team, 10LandingCheck, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, and 3 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10kostajh) https://gerrit.wikimedia.org/r/c/m... [14:44:28] 10Gitlab-Application-Security-Pipeline, 10Security-Team, 10SecTeam-Processed, 10user-sbassett: Error from using ASP v0.1.3; attempted `apt-get update` in 'step_script' fails - https://phabricator.wikimedia.org/T352089 (10Jdforrester-WMF) Thanks! Not sure who else is using, and if any docs need updating, bu... [14:49:33] We get 503s on various wikis [14:49:38] (DatasourceError) firing: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [14:50:33] ah, works again [14:53:21] 10Phabricator: Phabricator documentation for Arcanist should have links to the source code - https://phabricator.wikimedia.org/T351581 (10Aklapper) I'd love to have a bit more context / an example link here. If possible - TIA :) [14:54:38] (DatasourceError) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [15:14:50] brennen: If I self-merged a bunch of Phab changes, would you at some point have time to deploy them? [15:28:39] andre: we are in the middle of an incident https://www.wikimediastatus.net/ [15:29:26] 10Phabricator: Phabricator documentation for Arcanist should have links to the source code - https://phabricator.wikimedia.org/T351581 (10Pppery) See https://translatewiki.net/wiki/Phabricator:arcanist-core-00003a701ef56dd7/qqq for an example - it says "ArcanistDiffWorkflow.php:2568" with no link. This task is s... [15:30:39] eh I know, I didn't mean "right now", sorry :) [15:30:59] 10Release-Engineering-Team, 10LandingCheck, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, and 3 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10jijiki) 05Open→03Resolved a:03jijiki... [15:31:22] andre: I actually was just answering about the 503s :D [15:35:11] heh :D [15:43:47] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) >>! In T352167#9363247, @Aklapper wrote: > 1.42.0-wmf.13 is six weeks away so if this should block deploymen... [16:14:08] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) https://commons.m.wikimedia.beta.wmflabs.org/wiki/Main%20Page?uselang=pt ` lines=5 Source map error: Error:... [16:15:34] andre: happy to sort that out, yes. [16:18:07] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review: Reports: "Older" bar link in Histogram has wrong task query URL - https://phabricator.wikimedia.org/T336175 (10CodeReviewBot) aklapper merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/5 Repo... [16:20:43] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review: Filter default Phabricator calendar view to show items tagged with #events only - https://phabricator.wikimedia.org/T343258 (10CodeReviewBot) aklapper merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_... [16:22:53] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review: Update custom allowlist in src/applications/phurl/storage/PhabricatorPhurlURL.php - https://phabricator.wikimedia.org/T343267 (10CodeReviewBot) aklapper merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merg... [16:25:01] 10Phabricator, 10Release-Engineering-Team (Quid Pro Crow 🦃), 10Patch-For-Review: Remove broken custom code in DifferentialRevisionSearchEngine.php - https://phabricator.wikimedia.org/T351957 (10CodeReviewBot) aklapper merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/22 Remo... [16:27:52] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) https://commons.m.wikimedia.beta.wmflabs.org/wiki/Main%20Page?uselang=de&useskin=vector ` lines=5 Source map... [16:34:15] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Technical-Debt: Revert unused downstream Phab code to suppress triggering mail (predates silent batch-editing) - https://phabricator.wikimedia.org/T343264 (10Aklapper) [16:36:08] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Technical-Debt: Revert unused downstream Phab code to suppress triggering mail (predates silent batch-editing) - https://phabricator.wikimedia.org/T343264 (10CodeReviewBot) aklapper merged https://gitlab.wikimedia.org/rep... [16:36:13] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) https://test.wikipedia.org/wiki/Main_Page?uselang=es ` Source map error: Error: request failed with status 4... [16:43:12] 10Phabricator, 10Release-Engineering-Team (Quid Pro Crow 🦃), 10Patch-For-Review: Remove unused custom "Add more repository details to harbormaster build steps" - https://phabricator.wikimedia.org/T351958 (10CodeReviewBot) aklapper merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requ... [16:59:07] 10Release-Engineering-Team, 10Scap, 10MW-on-K8s, 10serviceops-radar: Add a scap way to deploy mw-on-k8s without an image rebuild - https://phabricator.wikimedia.org/T351880 (10dancy) `scap sync-world --k8s-only -Dbuild_mw_container_image:False` should cover this need. [16:59:25] 10Release-Engineering-Team, 10Scap, 10MW-on-K8s: Scap Mediawiki K8s deployments - https://phabricator.wikimedia.org/T318536 (10dancy) [17:24:21] 10Release-Engineering-Team (Priority Backlog 📥), 10dev-images, 10docker-pkg, 10User-brennen: Update dev-images to deal with breaking removal of seed_image in docker-pkg - https://phabricator.wikimedia.org/T352200 (10brennen) [17:55:56] 10Continuous-Integration-Config, 10Release-Engineering-Team (Radar), 10MediaWiki-Vendor, 10Parsoid (Tracking), 10ci-test-error (WMF-deployed Build Failure): `mediawiki-core-php72-phan-docker` job runs `composer install` instead of using packages from mediawiki/v... - https://phabricator.wikimedia.org/T287419 [18:07:44] 10Gitlab-Application-Security-Pipeline, 10Security Team AppSec, 10Security-Team, 10Patch-For-Review, and 2 others: Address issues within certain Gitlab CI security templates - https://phabricator.wikimedia.org/T338034 (10sbassett) For the last checkbox here: ` Investigate additional tools for validating gi... [18:11:45] (03Abandoned) 10Jbond: tox-ci: Improve abbility to run localy [integration/config] - 10https://gerrit.wikimedia.org/r/665133 (https://phabricator.wikimedia.org/T274338) (owner: 10Jbond) [18:12:16] 10Gitlab-Application-Security-Pipeline, 10Security-Team, 10SecTeam-Processed, 10user-sbassett: Error from using ASP v0.1.3; attempted `apt-get update` in 'step_script' fails - https://phabricator.wikimedia.org/T352089 (10sbassett) >>! In T352089#9363648, @Jdforrester-WMF wrote: > Thanks! Not sure who else... [18:46:32] 10Beta-Cluster-Infrastructure: "PHP Notice: Undefined variable: wmgCentralAuthCookieDomain in /srv/mediawiki/wmf-config/CommonSettings.php on line 1870" in beta - https://phabricator.wikimedia.org/T352210 (10Daimona) [20:24:48] 10Beta-Cluster-Infrastructure: "PHP Notice: Undefined variable: wmgCentralAuthCookieDomain in /srv/mediawiki/wmf-config/CommonSettings.php on line 1870" in beta - https://phabricator.wikimedia.org/T352210 (10Jdforrester-WMF) Checking manually it seems set as expected: ` jforrester@deployment-deploy03:/srv/med... [20:28:45] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: "PHP Notice: Undefined variable: wmgCentralAuthCookieDomain in /srv/mediawiki/wmf-config/CommonSettings.php on line 1870" in beta - https://phabricator.wikimedia.org/T352210 (10Jdforrester-WMF) Prod config sets a default of '' which BC's `-` operator over-rid... [20:32:48] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10hashar) >>! In T352167#9363925, @AlexisJazz wrote: > ... > In my experience gadgets are cached longer than userscripts,... [20:42:26] 10Release-Engineering-Team (Radar), 10MediaWiki-extensions-VipsScaler, 10Structured Data Engineering, 10Structured-Data-Backlog, 10Wikimedia-Extension-setup: Undeploy VipsScaler from Wikimedia wikis - https://phabricator.wikimedia.org/T290759 (10Ladsgroup) This doesn't fail anymore https://test.wikipedia... [20:43:40] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader, 10Wikimedia-production-error: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) [21:12:45] 10Beta-Cluster-Infrastructure, 10MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader, 10Wikimedia-production-error: Source map error: Error: request failed with status 404 - https://phabricator.wikimedia.org/T352167 (10AlexisJazz) >>! In T352167#9365090, @hashar wrote: >>>! In T352167#9363925, @AlexisJazz... [21:14:00] 10Release-Engineering-Team, 10LandingCheck, 10MW-on-K8s, 10MediaWiki-extensions-WikimediaEvents, and 4 others: PHP Warning: geoip_country_code_by_name(): Required database not available at /usr/share/GeoIP/GeoIP.dat. - https://phabricator.wikimedia.org/T352156 (10hashar) >>! In T352156#9363645, @kostajh wr... [21:16:38] 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10Data-Engineering: Unblock Dockerfile syntax to build images with Gitlab trusted runner - https://phabricator.wikimedia.org/T351792 (10thcipriani) > Looks like the root problem is this configuration: > https://gerrit.wikimedia.org/r/plugins/gitiles/op... [21:59:47] 10Scap: scap fails to bootstrap itself - https://phabricator.wikimedia.org/T352223 (10Dzahn) [22:04:12] 10Scap: scap fails to bootstrap itself - https://phabricator.wikimedia.org/T352223 (10Dzahn) scap::target does an "include scap" and that has "enable_bootstrapping = true". The /etc/scap.cfg and /usr/local/bin/bootstrap-scap-target.sh gets installed. Inside the bootstrap script there is an rsync command to rsy... [22:07:10] 10Scap: scap fails to bootstrap itself - https://phabricator.wikimedia.org/T352223 (10Dzahn) ` root@deploy-1004:/var/lib/scap# file scap-wheels scap-wheels: cannot open `scap-wheels' (No such file or directory) ` [22:08:04] 10Phabricator, 10Release-Engineering-Team (Radar), 10collaboration-services: upgrade phab (phorge) hosts to bullseye - https://phabricator.wikimedia.org/T334519 (10Dzahn) current issue: scap fails to boostrap itself from the local deployment server: -> T352223 [22:14:41] 10Scap: scap fails to bootstrap itself - https://phabricator.wikimedia.org/T352223 (10Dzahn) The scap-wheels directory exists on a production deployment server but was only created on September 27. Does not exist in deployment server in cloud project. So seems like things were changed on that date but only in p... [22:19:50] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: "PHP Notice: Undefined variable: wmgCentralAuthCookieDomain in /srv/mediawiki/wmf-config/CommonSettings.php on line 1870" in beta - https://phabricator.wikimedia.org/T352210 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF Appears fixed: {F415... [22:23:20] 10Continuous-Integration-Config, 10PHP 8.3 support: Add experimental PHP 8.3 jobs - https://phabricator.wikimedia.org/T339349 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF