[01:22:36] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [02:35:52] 06cloud-services-team, 10Toolforge: mysqldump Command Not Found on tools Bastion - https://phabricator.wikimedia.org/T378882#10286614 (10Legoktm) If you use the job framework (see https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework), there's a mariadb image which contains mysqldump. [03:37:38] 10Tools: I keep getting Toolforge messages about rustup - https://phabricator.wikimedia.org/T378437#10286651 (10Legoktm) a:03Legoktm Sorry Magnus! I thought I had fixed this but guess not. If you want, I can remove you as a maintainer, this is just the shared rustup installation on Toolforge. [05:04:50] 10Tools: I keep getting Toolforge messages about rustup - https://phabricator.wikimedia.org/T378437#10286657 (10Legoktm) 05Open→03Resolved Should be fixed now, I bumped up the memory: https://wikitech.wikimedia.org/w/index.php?title=Tool%3ARustup&diff=2240867&oldid=2236350 [05:22:36] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [09:22:36] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [10:38:30] FIRING: PuppetAgentNoResources: No Puppet resources found on instance cloudinfra-idp-1 on project cloudinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [11:15:56] (03PS1) 10Ladsgroup: Index operations/dumps [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1086439 (https://phabricator.wikimedia.org/T378863) [11:17:53] (03CR) 10Ladsgroup: [C:03+2] Index operations/dumps [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1086439 (https://phabricator.wikimedia.org/T378863) (owner: 10Ladsgroup) [11:19:31] (03Merged) 10jenkins-bot: Index operations/dumps [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1086439 (https://phabricator.wikimedia.org/T378863) (owner: 10Ladsgroup) [11:23:43] 10VPS-project-Codesearch, 06Data-Platform, 10Dumps-Generation, 13Patch-For-Review: Index operations/dumps - https://phabricator.wikimedia.org/T378863#10286788 (10Ladsgroup) 05Open→03Resolved a:03Ladsgroup It'll be there in a day. [13:22:36] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [13:28:12] RESOLVED: PuppetAgentStaleLastRun: Last Puppet run was over 24 hours ago on instance toolsbeta-test-k8s-haproxy-6 in project toolsbeta - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentStaleLastRun [13:52:39] FIRING: [2x] PuppetCertificateAboutToExpire: Puppet CA certificate deployment-poolcounter06.deployment-prep.eqiad.wmflabs is about to expire in 24d 23h 58m 30s - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetCertificateAboutToExpire - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetCertificateAboutToExpire [14:49:01] 10wikitech.wikimedia.org: Can't log into Wikitech to unify - https://phabricator.wikimedia.org/T378897 (10Miniapolis) 03NEW [15:01:57] 10wikitech.wikimedia.org: Can't log into Wikitech to unify - https://phabricator.wikimedia.org/T378897#10286892 (10Miniapolis) I was able to log in by changing my password to my SUL, and hope that's not a problem. [15:03:06] 06cloud-services-team, 10Toolforge: mysqldump Command Not Found on tools Bastion - https://phabricator.wikimedia.org/T378882#10286889 (10taavi) 05Open→03Resolved a:03taavi I updated the docs at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#ToolsDB_Backups to use the image, closing. [15:25:58] 10Tool-toolwatch: Implementing alert system to notify maintainers of downtime - https://phabricator.wikimedia.org/T368816#10286924 (10Tacsipacsi) >>! In T368816#10286507, @Gopavasanth wrote: > The tool now exclusively tracks web-based tools, screenshot attached with new latest stats: Actually, it isn’t //[[http... [15:30:49] 10Tool-toolwatch, 06Toolforge-standards-committee: toolwatch loads third party resources - https://phabricator.wikimedia.org/T378901 (10taavi) 03NEW [15:42:02] 10wikitech.wikimedia.org, 10Wikidata, 10Wikimedia-Interwiki-links: Enable interwiki links to/from Wikitech - https://phabricator.wikimedia.org/T290147#10286938 (10Pppery) [16:24:58] 10Tool-toolwatch, 06Toolforge-standards-committee: toolwatch loads third party resources - https://phabricator.wikimedia.org/T378901#10286948 (10JJMC89) [16:25:07] 06Toolforge-standards-committee, 10Tools, 06Privacy Engineering, 07Privacy: Hunt for Toolforge tools that load resources from third party sites - https://phabricator.wikimedia.org/T172065#10286949 (10JJMC89) [16:25:29] 10Tool-toolwatch, 06Toolforge-standards-committee, 07Privacy: toolwatch loads third party resources - https://phabricator.wikimedia.org/T378901#10286950 (10JJMC89) [17:18:28] 06Toolforge-standards-committee: Adoption request for Yapperbot - https://phabricator.wikimedia.org/T361426#10287021 (10JJMC89) 05In progress→03Declined Adopting the tool would result in credentials being removed and therefore the bot no longer functioning. Forking should be used instead. [17:18:59] 06Toolforge-standards-committee: Adoption request for geograph2commons - https://phabricator.wikimedia.org/T345707#10287027 (10JJMC89) 05Open→03Declined >>! In T345707#10053246, @bjh21 wrote: > My intention in adopting this tool was not primarily to resurrect it, but simply to stop people trying to use it. [17:22:36] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [17:24:11] FIRING: Temperature: Inlet Temp issue on clouddumps1001:9290 - https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook - https://grafana.wikimedia.org/d/ZA1I-IB4z/ipmi-sensor-state?orgId=1&viewPanel=92&var-server=clouddumps1001 - https://alerts.wikimedia.org/?q=alertname%3DTemperature [17:25:54] 10Toolforge-standards-committee (Maintainer needed), 10Tools: Snottywong/JackPotte tools.wmflabs.org scripts are broken and unmaintained - https://phabricator.wikimedia.org/T217789#10287031 (10JJMC89) [17:27:12] 10Toolforge-standards-committee (Maintainer needed), 10Tools, 06Commons: Steinsplitter's tools need a new maintainer - https://phabricator.wikimedia.org/T350953#10287034 (10JJMC89) [17:28:19] 10Toolforge-standards-committee (Maintainer needed), 10Tools: wikistream.toolforge.org needs new maintainers - https://phabricator.wikimedia.org/T251555#10287047 (10JJMC89) [17:29:11] RESOLVED: Temperature: Inlet Temp issue on clouddumps1001:9290 - https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook - https://grafana.wikimedia.org/d/ZA1I-IB4z/ipmi-sensor-state?orgId=1&viewPanel=92&var-server=clouddumps1001 - https://alerts.wikimedia.org/?q=alertname%3DTemperature [17:34:55] 10Toolforge-standards-committee (Maintainer needed), 10Tools: Co-maintainers needed for the tool Templatetransclusioncheck - https://phabricator.wikimedia.org/T223917#10287054 (10JJMC89) @Adithyak1997: Are you still looking for maintainers? You have not responded here or on your talk page. [17:41:15] 10Toolforge-standards-committee (Maintainer needed), 10Tools: Bring the ppp-sparql tool back to life - https://phabricator.wikimedia.org/T223859#10287058 (10JJMC89) 05Open→03Invalid The tool was deleted in T216427. [17:41:37] 10Toolforge-standards-committee (Maintainer needed), 10Tools: Bring Bene's sparql tool back to life - https://phabricator.wikimedia.org/T223858#10287067 (10JJMC89) 05Open→03Declined [17:44:53] 10Tools: Improve interface to KMLexport - https://phabricator.wikimedia.org/T94575#10287072 (10JJMC89) [17:46:12] 06cloud-services-team, 10Toolforge: Review SWViewer tool for compliance with ToS - https://phabricator.wikimedia.org/T241937#10287073 (10JJMC89) [17:59:58] 06cloud-services-team, 10Toolforge: Review SWViewer tool for compliance with ToS - https://phabricator.wikimedia.org/T241937#10287074 (10Iluvatar) 05Open→03Invalid The ticket was opened 5 years ago :) [21:18:12] FIRING: PuppetAgentNoResources: No Puppet resources found on instance toolsbeta-harbor-1 on project toolsbeta - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [21:22:37] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks