[15:56:02] db2166 was maintenance, or did it crash? [15:58:10] mysqld (initiated by: unknown): Normal shutdown [15:59:32] ^ marostegui [16:00:09] jynus: I depool it [16:00:19] you didn't stop it, I get? [16:00:24] no [16:03:10] mw is erroring since 14:45, but only recently mysql alerted :-( [16:03:29] (no user impacting errors, probably just log spam) [16:05:50] actually, it creashed several times [16:06:01] and this is 10.4 [16:06:47] the delay was probably because it flapped several times on -> off -> on -> off [16:07:30] I will help by creating a task and later it can be taken from there more calmly [16:08:32] mw is happy now [16:10:20] that's weird [16:10:33] index corruption, will finish creating the task soon [16:11:06] Corruption of an index tree: table `wikidatawiki`.`wbt_item_terms` index `wbt_item_terms_item_id`, father ptr pa [16:17:36] https://phabricator.wikimedia.org/T323040 [16:20:17] I made a dump NDA as I just copied and pasted, but maybe with more detailed check can be made public [16:20:32] *the crash dump [16:31:07] Thanks I am taking a look [16:32:46] To be honest I don't think we can figure much out, and maybe we should just reclone it [16:33:05] The other recent crash was a different table and a different section [16:33:24] +1 [16:33:57] the new stuff is that I think it started crashing and starting up many times