[05:58:09] marostegui: do you know what was the script that made the depool timeout? do you think making the script retry after the batch is done would be useful? [05:58:33] I made most of my scripts reload the config every ten minutes [05:58:57] Amir1: I can tell you what is connected now, but hard to know what was running at the time [05:59:20] ok [05:59:24] www-data 18078 2.7 0.0 220280 65488 pts/11 S+ Jan07 131:53 php -ddisplay_errors=On /srv/mediawiki/multiversion/MWScript.php migrateRevisionActorTemp.php --wiki=dewiki --sleep=2 [05:59:27] likely to be that one [05:59:36] I guess [05:59:54] that stops and reload every ten minutes [06:00:04] then it must be something else :) [06:00:47] ugh, I take it back, most sections are done but not s5 [06:00:48] dewiki: ... rev_id=28455209, updated 25005956 [06:01:17] I think it's because the list of things to skip would be large [06:01:30] so whatever it is, it was coming from mwmaint [06:01:57] right now I am getting stopped on s8 because of xml dumps [06:02:11] Which accessing a host NOT in vslow,dump group [06:02:14] Going to report that again [07:05:17] going to failover m2 proxy [09:13:23] jynus: I have reimaged db2078 (misc codfw host, used also for dumps) to Bullseye, the dumps today finished, but if for the next run you see something weird let me know. Mysql version has changed, "only" the OS [09:13:57] ok [11:40:56] marostegui: i feel so honoured [11:41:02] I know you do [11:41:16] I basically made your day, I know [11:41:24] you have no idea [12:55:05] "LSobanski renamed this task from 'Send additional Phabricator notifications to #wikimedia-data-persistence-feed' to 'Send additional Phabricator notifications to #wikimedia-data-persistence-feed to #wikimedia-data-persistence-feed'." ? [12:55:45] -_- [12:56:35] * sobanski fails at Phabricator... and text field recognition [12:56:48] jynus: thanks, fixed [12:56:51] did it rename the tags on the title on its own? [12:57:34] Nope, I just copy-pasted the text from the title into the title (again) instead of the description (which I Intended) [12:57:41] ah, ok [12:58:59] it has happened to me several times that IRC channels get recognized as phab tags and get renamed automatically [14:55:09] marostegui: when it's okay from you to deploy the patch for T260297? [14:55:09] T260297: db-eqiad and db-codfw sectionsByLoad can get out of sync - https://phabricator.wikimedia.org/T260297 [14:55:12] https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/702421 [14:55:35] Amir1: Ah that one [14:55:50] Amir1: Sure, I am going to go offline in a sec, but if you will be around to monitor things, sure :) [14:56:17] yeah, I feel a bit tired to do risky stuff like this [14:56:22] probably tomorrow [14:56:27] +1 [14:58:38] Amir1: Are the actor things still running? [14:58:55] Reedy: yes, I think it'll be for the next three to five months [14:59:01] Is it running against enwiki? [14:59:05] yes [14:59:05] https://en.wikipedia.org/w/index.php?title=Michael_Ching&oldid=prev&diff=998810569 [14:59:12] But if you look at the row... rev_actor is 0 [14:59:51] (from an email to security@) [14:59:55] hmm, it definitely has not reached that point [15:00:04] but it should be on write both [15:00:11] so it's a bug in write both I think [15:00:20] not the script [15:00:28] what is the value in the temp table? [15:00:59] non zero [15:01:15] The user also reports it's not in their contributions https://en.wikipedia.org/wiki/Special:Contributions/Dnllnd [15:01:29] Which would make some sense... but surely that should take note? [15:01:42] They're also claiming they didn't make the edit [15:02:21] select * from revision_actor_temp where revactor_rev = 998810569; [15:02:25] select * from actor where actor.actor_id = 666071; [15:02:29] that matches up to their username [15:02:43] so the read is still on the temp table [15:03:52] I haven't checked just yet to see if the edit was seemingly made by them (according to CU) [15:04:59] yeah and check rc too I gues [15:05:38] <_joe_> Reedy: yeah I was about to suggest to look at CU [15:06:18] Though jynus just replied... [15:06:48] oh, sorry, I didn't know you were discussing it here [15:07:01] :) [15:07:16] I just made a reference that their contributions were indeed in their contributions page [15:07:30] but said that if still that was weird to return back [15:07:31] I didn't even click that it's no longer 2021 [15:07:42] But if it's an edit from Jan 2021... No RC or CU [15:07:55] yeah, there is a chance it is just a date confusion, so I wanted to mention that first [15:07:58] But also explains why it's not on the contributions pages [15:08:05] I think you're probably right :) [15:08:17] and if it is not, there is not much we can do, most private stuff gets deleted after 3 months [15:08:38] but still, we want to encourage users to report strange things [16:19:47] Reedy: :-)