[07:44:33] greetings [08:36:01] morning [12:53:23] JFYI I resolved https://portal.victorops.com/ui/wikimedia/incident/8249/details which was long-standing ack'd [12:53:48] I'm assuming because in the meantime I fixed the alert expression itself, the resolve never went out, just guessing tho [13:14:39] is someone doing something on cloudcephosd1045? [13:15:59] not afaik, the cloudcephosd work I'm aware of is https://phabricator.wikimedia.org/T419892 [13:16:33] ah, probably https://phabricator.wikimedia.org/T429387#12285768 [13:16:35] I am [13:16:43] firmware downgrade [13:18:21] ...or maybe not [13:22:18] guess I'm waiting for dcops for this [13:23:53] FYI, I'm rolling out pdns-rec security updates on cloudservices* (one at a time) [13:24:40] thx moritzm [13:30:55] is etherpad now responding for anyone else? [13:30:57] not* [13:31:27] likewise for pdns itself [13:36:31] taavi: ep is working for me... but maybe for you too by now? [13:36:42] * taavi checks [13:36:48] yeah, works again now [13:36:55] thanks for checking though [16:59:41] I'm trying fw updates again, brace for osd alerts [17:13:00] * dhinus off