[00:15:04] 10wikitech.wikimedia.org, 10MediaWiki-extensions-OATHAuth: Wikitech users being unexpectedly prompted for 2FA tokens - https://phabricator.wikimedia.org/T379398#10306004 (10bd808) >>! In T379398#10304997, @bd808 wrote: > The account does not have global 2FA following unification with [[https://meta.wikimedia.o... [00:20:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [00:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [00:53:43] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306066 (10Hakimi97) >>! In T379340#10305075, @LucasWerkmeister wrote: > If you want to reimplement the tool (my very naive guess, without ever having seen the tool in action, is that it seems like a manag... [03:20:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [03:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [06:50:17] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306182 (10Chlod) I was under the impression that the `jawi` name would be unusable, since [usurpation](https://wikitech.wikimedia.org/wiki/Help:Toolforge/Abandoned_tool_policy#Usurpation) lists adoption a... [07:09:28] 10Tool-inteGraality, 07Documentation, 07good first task: Add information to InteGraality user guide from Wikipedia Workbook for Cultural Institutions - https://phabricator.wikimedia.org/T376902#10306189 (10Ibom99) Hi there, I'd like to contribute by working on this if that's okay. [08:11:01] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452 (10JeanFred) 03NEW [08:11:58] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306235 (10JeanFred) The UWSGI logs are a vast loop of ` *** Starting uWSGI 2.0.19.1-debian (64bit) on [Mon Nov 4 14:27:45 2024] *** *** Starting uWSGI 2.0.19.1-debian (64bit) on [Mon Nov 4 14:27:58 2024] *... [08:12:21] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306236 (10JeanFred) Two days ago, I ran `webservice restart` a few times, which did not help [08:13:07] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306237 (10JeanFred) Yesterday, just in case, I recreated the virtual-environment using ` toolforge webservice python3.9 shell webservice-python-bootstrap --fresh ` which did not help. [08:14:58] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306239 (10JeanFred) Asking for help on Telegram, I was told this might be T362867#10292196 [08:17:15] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306241 (10JeanFred) @dhinus on IRC took some steps some 8 hours ago: the pod is indeed in CrashLoopBackOff, and has already restarted 44 times kubectl describe pod shows "Back-off restartin... [08:23:23] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306257 (10JeanFred) The service was going in 403 forbidden (at least something different!) When I tried to stop/start, it started as a PHP service ; I then ran the command logged above in SAL ; and edited `w... [09:07:23] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306276 (10Hakimi97) >>! In T379340#10306182, @Chlod wrote: > @Hakimi97: If you or others from the Malay community need some help on this part, I can provide some engineering support. I think the hardest p... [09:35:30] 06cloud-services-team, 10Toolforge: mysqldump Command Not Found on tools Bastion - https://phabricator.wikimedia.org/T378882#10306299 (10taavi) a:05taavi→03None [10:10:41] 06cloud-services-team, 10Cloud-VPS: maps project wildcard domains - https://phabricator.wikimedia.org/T299775#10306312 (10taavi) 05Open→03Resolved a:03taavi Closing. Please create a new task if/when the wmflabs.org names can be dropped. [10:32:14] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306350 (10fnegri) Weird, I would try stopping and starting a couple more times, if that doesn't work, is there a way to add some extra debug logs in the app itself, to understand better where exactly it gets... [11:08:44] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306359 (10Hakimi97) >>! In T379340#10304722, @JJMC89 wrote: > The tool's software is not in compliance with the Cloud Services Terms of use as it is not [[https://wikitech.wikimedia.org/wiki/Wikitech:Clou... [11:20:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [11:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [11:39:17] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306370 (10Aviator) Aviator here. The tool is indeed licensed under GPL. As for the binary, it was compiled from my own code that is written in C++ or Rust (I can't remember but I can dig my old files). I... [12:20:51] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306388 (10Hakimi97) >>! In T379340#10306370, @Aviator wrote: > Aviator here. The tool is indeed licensed under GPL. As for the binary, it was compiled from my own code that is written in C++ or Rust (I ca... [12:37:46] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306395 (10LucasWerkmeister) `kubectl describe pods` showed ` Last State: Terminated Reason: OOMKilled ` so I tried bumping the memory limit. (Bit annoying that the OOMKilled wasn’t shown in `kubect... [12:39:53] 10Tool-spacemedia, 03Wikimania-Hackathon-2024: Include Flickr accounts of U.S. Government into Spacemedia tool - https://phabricator.wikimedia.org/T372192#10306398 (10Don-vip) 05In progress→03Resolved [12:40:24] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306400 (10LucasWerkmeister) >>! In T379452#10306395, @LucasWerkmeister wrote: > https://wikiloves.toolforge.org/ just shows me a 403, I don’t know if that’s correct or not. Turns out it was defaulting to th... [13:11:11] (03open) 10lucaswerkmeister: Warn about unknown keys in service.template [repos/cloud/toolforge/tools-webservice] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/tools-webservice/-/merge_requests/59 [13:12:23] 10Tool-wikiloves: wikiloves inaccessible since November 4th - https://phabricator.wikimedia.org/T379452#10306419 (10LucasWerkmeister) >>! In T379452#10306396, @Stashbot wrote: > {nav icon=file, name=Mentioned in SAL (#wikimedia-cloud), href=https://sal.toolforge.org/log/-CzvEJMBFFSCpsJzFQvR} [2024-11-09T12:38:43... [13:50:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [13:52:41] FIRING: [2x] PuppetCertificateAboutToExpire: Puppet CA certificate deployment-poolcounter06.deployment-prep.eqiad.wmflabs is about to expire in 18d 23h 58m 30s - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetCertificateAboutToExpire - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetCertificateAboutToExpire [14:00:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [15:21:26] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306568 (10Hakimi97) 05Declined→03Open [15:41:02] 06Toolforge-standards-committee: Adoption request for jawi - https://phabricator.wikimedia.org/T379340#10306571 (10LucasWerkmeister) >>! In T379340#10306370, @Aviator wrote: > Aviator here. The tool is indeed licensed under GPL. As for the binary, it was compiled from my own code that is written in C++ or Rust (... [16:44:08] (03CR) 10Umherirrender: "codex-php is not listed under https://codesearch.wmcloud.org/libraries/?action=repos" [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1079381 (https://phabricator.wikimedia.org/T373708) (owner: 10Krinkle) [17:20:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [17:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [23:20:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [23:30:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [23:35:20] 10Tools, 06Commons: Intermittent failure of FlickreviewR 2 bot - https://phabricator.wikimedia.org/T379459#10306873 (10Peachey88) I've added all the other users with access to the project per https://toolsadmin.wikimedia.org/tools/id/yifeibot, It looks like @zhuyifei1999 was last active less than a month ago (...