[09:09:49] <dcaro>	 thanks!
[10:35:08] <dcaro>	 oh, I think mysql in the replica crashed again
[10:40:50] <dhinus>	 ouch, looking
[10:43:17] <dhinus>	 yep, SEGV in clouddb1022@s3
[10:43:34] <dhinus>	 I will depool and add notes to T420177
[10:43:34] <stashbot>	 T420177: clouddb1013 crashed after the upgrade to mariadb 10.11.16 - https://phabricator.wikimedia.org/T420177
[10:54:08] <volans>	 was reverting mariadb to the previous version considered?
[11:04:53] <dhinus>	 volans: I suggested it in the task, but it's still unclear if the version is the problem, or data corruption. clouddb1022 was cloned from another host, maybe from clouddb1013
[11:05:18] <dhinus>	 I will wait for manuel to comment in the task, he did the cloning
[11:11:01] <volans>	 k
[13:01:36] <taavi>	 fwiw I merged the main T411102 patch earlier today, which we'll need for nftables support in cloud vps
[13:01:37] <stashbot>	 T411102: Puppet types causing issues around nftables::service - https://phabricator.wikimedia.org/T411102
[13:02:31] <taavi>	 actually merging the 'fix firewall::service' patch still needs some work, since it turns out firewall::service validated its parameters differently depending on whether the host was using ferm or nftables, so there's some uses that need to be updated to pass the stricter validation first :/
[18:03:22] * dcaro off
[18:03:24] <dcaro>	 cya tomorrow!
[18:55:17] <andrewbogott>	 dhinus, still working?
[19:00:04] <dhinus>	 nope sorry, can I help?
[19:00:15] <andrewbogott>	 not urgent, I'll ask tomorrow if I'm still puzzled :)
[19:00:56] <dhinus>	 ok!