[07:22:04] volans: thanks a lot for https://gerrit.wikimedia.org/r/c/operations/homer/public/+/1180562 What do you think of running black on those files ? Maybe like the `py3-format` from the spicerack repo? [08:44:08] XioNoX: sure, you can follow https://doc.wikimedia.org/spicerack/master/development.html#code-style [08:45:14] volans: are git hooks shared with everybody or we all have to manually configure it? [08:46:02] the latter, also because it's a personal choice to set it up in the editor/ide or at git commit time or at git review time [08:46:27] for example I have it in pre-review [08:46:33] for spicerack [08:46:48] I've just added it to pre-commit [08:48:41] or if you want to be bleeding edge you can follow what we've done with pywmflib :-P [08:49:30] remember it's not only py3-format but also py3-style [08:49:40] yeah I was looking at the two files [08:49:48] I guess it should just about a copy paste [08:50:14] and then add to the git blame ignore the black commits [08:50:21] what's the bleeding edge? [08:57:18] see pywmflib :-P [08:57:35] ruff? [08:57:58] anyway, no time for bleeding edge right now [08:58:31] yep [15:30:23] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 06SRE: ssw1-f1-eqiad: Fan Spinning Upgraded - https://phabricator.wikimedia.org/T400783#11119711 (10Jclark-ctr) a:05Jclark-ctr→03None [15:43:48] FIRING: PuppetFailure: Puppet has failed on cumin1003:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [15:48:48] FIRING: [2x] PuppetFailure: Puppet has failed on cumin1003:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [15:53:49] FIRING: [3x] PuppetFailure: Puppet has failed on cumin1002:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [15:54:55] topranks, XioNoX: did you modify tests/generate_schema_docs.py in homer's public repo on cumin1003? [15:55:01] modified: tests/generate_schema_docs.py [15:55:06] diff is [15:55:07] old mode 100755 [15:55:07] new mode 100644 [15:55:09] hmm, no [15:56:43] can I revert so that puppet get unbroken? [15:56:49] yup [15:57:03] {done} [15:57:54] it was on all cumin hosts, fixing [15:59:31] volans: fyi self merged https://gerrit.wikimedia.org/r/c/operations/homer/public/+/1182164?usp=email [16:00:10] +1 [16:03:48] FIRING: [3x] PuppetFailure: Puppet has failed on cumin1002:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [16:23:48] FIRING: [2x] PuppetFailure: Puppet has failed on cumin1003:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [16:28:48] RESOLVED: [2x] PuppetFailure: Puppet has failed on cumin1003:9100 - https://puppetboard.wikimedia.org/nodes?status=failed - https://grafana.wikimedia.org/d/yOxVDGvWk/puppet - https://alerts.wikimedia.org/?q=alertname%3DPuppetFailure [18:35:56] FIRING: MaxConntrack: Max conntrack at 82.65% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [18:40:56] RESOLVED: MaxConntrack: Max conntrack at 82.65% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [19:23:25] 10netops, 06Infrastructure-Foundations, 06SRE: Investigate using BGP addpath for unicast IBGP spine/leaf pods - https://phabricator.wikimedia.org/T402640#11121128 (10ayounsi) If I understand correctly we currently get some "per rack" load balancing, where `E3` might randomly prefer `E1` but servers in `E4` m... [23:04:25] FIRING: MirrorHighLag: Mirrors - /srv/mirrors/debian synchronization lag - https://wikitech.wikimedia.org/wiki/Mirrors - https://grafana.wikimedia.org/d/dbd8a904-eab2-48d1-a3b9-fa1851ef3ed2/mirrors?orgId=1 - https://alerts.wikimedia.org/?q=alertname%3DMirrorHighLag [23:11:32] 07Puppet, 06SRE, 10Readers Essential Work 2025 (Simplify MobileFrontend): Certain mobile devices are (possibly) not being redirected to our mobile site - https://phabricator.wikimedia.org/T388032#11121841 (10Jdlrobson-WMF)