[03:14:46] <jinxer-wm>	 (SystemdUnitFailed) firing: (2) wmf_auto_restart_uwsgi-puppetboard.service Failed on puppetboard1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[07:14:46] <jinxer-wm>	 (SystemdUnitFailed) firing: (2) wmf_auto_restart_uwsgi-puppetboard.service Failed on puppetboard1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[07:43:57] <slyngs>	 volans: Our weird irc.wikimedia.org server isn't really own by any team right? 
[07:45:51] <volans>	 slyngs: yes and no... the service itself it a bit unowned, the servers are usually rebooted by us when needed. In the past faidon worked on a software replacement...
[07:47:02] <slyngs>	 I'm trying to triage this: https://phabricator.wikimedia.org/T341097 ... Sadly that seems to go in our stack of tasks then
[07:50:18] <volans>	 TIL, I didn't know about it
[07:50:36] <volans>	 the rewrite was putting in place an irc server that allowed to read only and not write at all...
[07:57:58] <slyngs>	 Ideally we'd complete the rewrite ... Hmm...
[08:02:51] <volans>	 ideally...
[08:09:43] <slyngs>	 A fun project for someone with to much time on their hands and a tolerance for breaking important stuff. I'll tag us and we'll deal with it when there's two Thursdays in a week.  
[08:10:07] <volans>	 :D
[08:14:46] <jinxer-wm>	 (SystemdUnitFailed) resolved: (2) wmf_auto_restart_uwsgi-puppetboard.service Failed on puppetboard1003:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[09:29:46] <jinxer-wm>	 (SystemdUnitFailed) firing: ifup@ens13.service Failed on netflow1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[09:34:46] <jinxer-wm>	 (SystemdUnitFailed) resolved: ifup@ens13.service Failed on netflow1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed
[11:43:39] <wikibugs>	 10Puppet, 10Infrastructure-Foundations, 10Puppet-Core, 10SRE, and 2 others: Convert all of our site.pp/roles to the role/profile paradigm - https://phabricator.wikimedia.org/T159412 (10MoritzMuehlenhoff) >>! In T159412#8986264, @jbond wrote: > I personally dont think its to bad to include e.g. role::mediaw...
[13:05:00] <wikibugs>	 10CAS-SSO, 10Infrastructure-Foundations, 10SRE, 10serviceops-collab, and 4 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10Jelto) GitLab replicas (e.g. https://gitlab-replica.wikimedia.org) use oidc as the default login method now. Login (normal user login a...
[14:31:43] <wikibugs>	 10CFSSL-PKI, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review, 10Puppet (Puppet 7.0): PKI: add the new puppet CA to the pki infrastructre - https://phabricator.wikimedia.org/T340557 (10jhathaway) >>! In T340557#8985560, @jbond wrote: > And use this for talking to the pki infrastructure.  This is a...
[14:40:36] <wikibugs>	 10CFSSL-PKI, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review, 10Puppet (Puppet 7.0): PKI: add the new puppet CA to the pki infrastructre - https://phabricator.wikimedia.org/T340557 (10jbond) >>! In T340557#8990830, @jhathaway wrote: >>>! In T340557#8985560, @jbond wrote: >> And use this for talki...
[15:11:54] <wikibugs>	 10CAS-SSO, 10Infrastructure-Foundations, 10SRE, 10serviceops-collab, and 4 others: migrate gitlab away from the CAS protocol - https://phabricator.wikimedia.org/T320390 (10Arnoldokoth) I ran into a 422 error trying to login to gitlab-replica.  {F37129701}
[15:38:50] <wikibugs>	 10Puppet, 10Infrastructure-Foundations, 10Puppet-Core, 10SRE, and 2 others: Convert all of our site.pp/roles to the role/profile paradigm - https://phabricator.wikimedia.org/T159412 (10Dzahn) This should be discussed with Joe, since he is the original task creator, which said everything should follow the r...
[17:48:59] <wikibugs>	 10SRE-tools, 10Infrastructure-Foundations, 10Patch-For-Review, 10cloud-services-team (FY2022/2023-Q4): Allow wmcs cookbooks running on cloudcuminXXXX to write to the SAL - https://phabricator.wikimedia.org/T325756 (10fnegri) I discussed this with @Volans today and we agreed it would be nice to keep Spicera...