[05:08:33] 10Puppet, 10Continuous-Integration-Infrastructure, 10Infrastructure-Foundations: Puppet failure on integration-puppetmaster-02.integration.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T290422 (10hashar) Who knows? :] Thank you for the certificates regeneration! [09:43:52] topranks: in relation to statograph i was just building a version with your fix, ready to deploy. however i notice that things seem to be working now. Is this something you changed or did i just get lucky and hit a time where we have no big floats? [10:11:54] jbond: thanks! Was just reviewing the packaging chat here (no harm anyway) so you’ve saved me some trouble [10:12:25] I don’t know if it’s simply a matter of no big floats. Perhaps stausgraph.io fixed it their side also. [10:13:19] Just running an errand right now, I could test again from my laptop when I get back and verify. [10:13:56] Or perhaps it’s something we’d want anyway, just to be safe (even if it is seemingly fixed by them?) [10:17:47] topranks: sorry missed the packaging chat, didn;t meant to step on toes :). i guess we are either getting lucky or statuspage.io made an update either way i would say its safest to deploy [10:22:50] My guess based on yesterday is it’s probably a fix on their part - as we’d be unlikely to magically get lucky so much. But can’t say. Agree it is probably safest to deploy though, I don’t believe we need massive precision. [10:24:12] Not stepping on toes at all, if you’re in the process by all means continue, or otherwise I’ll try to look when I’m back. [10:24:36] Thanks! [10:26:35] ack ill deploy now cheers [13:50:10] 10Puppet, 10Infrastructure-Foundations, 10Patch-For-Review, 10User-jbond: puppetdb seems to be slow on host reimage - https://phabricator.wikimedia.org/T263578 (10jbond) After doing the work to compact this again the database as a whole is most tables have a row count of either equal to the number of hosts... [14:47:26] volans: jbond: topranks: wow really nice catch on T290425 [14:47:27] T290425: statograph_post service fail on alert hosts - https://phabricator.wikimedia.org/T290425 [14:50:48] Hey Chris! Hope all is good. [14:50:52] Yeah bit of an odd one alright. [15:29:54] 10Mail, 10Infrastructure-Foundations, 10Patch-For-Review: Upgrade MXes to Bullseye - https://phabricator.wikimedia.org/T286911 (10herron) Seeing errors like this in the paniclog unfortunately ` mx2002:~# zcat /var/log/exim4/paniclog*.gz 2021-08-29 00:00:01 1mK8Ez-0095xJ-IE Tainted filename for search '/etc/...