[03:04:29] musikanimal: when you have time, can you check ipcheck.toolforge? times out for me :-p [03:05:08] not an emergency of any sort and I am dealing with my own IRL emergency so just letting you know for ref :-p [03:08:42] revi: I've restarted it [03:08:56] thx :-) [03:11:12] np [03:14:02] 'no healthy upstream' [03:14:53] musikanimal: ^ [03:20:30] I've restarted again. Not sure what the issue is. The webservice was stuck with a `CrashLoopBackOff` status [03:37:48] okay, maybe it's better now. Did a hard restart among a few other tiny things [09:01:06] Hi folks, I need to update the URL of librenms/librenms on this repo as we migrated it to GitLab but it says it's read only, what's the propper way to update it?? https://gerrit.wikimedia.org/g/cloud/instance-puppet/+/e9780723d27231886eab652f76b80c477efbc48e/deployment-prep/_.yaml#2919 [09:13:42] denisse: from horizon's puppet menu [09:14:35] denisse: https://horizon.wikimedia.org/project/puppet/ [09:14:36] RhinosF1: Yeah, I tried that but it was empty, I just noticed I selected the wrong project, thnx!! [09:14:55] Yw! [09:17:38] Weird... I keep getting Unrecognized schema in response body. (HTTP 403). 🤔 [09:19:15] Ah, it seems I'm only reader of deployment-prep... [09:24:11] Could someone please add me?? My username there is "Andrea Denisse". [12:00:59] denisse: done! [16:12:10] !log deployment-prep decommission deployment-docker-mathoid02 - T436465 [16:12:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [16:12:15] T436465: Replace deployment-docker-mathoid02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436465 [16:32:26] !log deployment-prep adjust cpjobqueue config to temporarily send jobs to deployment-jobrunner06 (PHP 8.5) - T435393 [16:32:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [16:32:32] T435393: Beta Cluster: Switch to PHP 8.5, includes writing Puppet changes to support PHP 8.5 on baremetal - https://phabricator.wikimedia.org/T435393 [16:37:30] thank you for all your work on deployment-prep, Southparkfan [16:39:30] Lovely to hear that! And thank *you* for all assistance :-) [16:44:14] * dcaro off [16:53:32] !log deployment-prep cpjobqueue: switch back to deployment-jobrunner05 (PHP 8.3) - T435393 [16:53:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [16:53:37] T435393: Beta Cluster: Switch to PHP 8.5, includes writing Puppet changes to support PHP 8.5 on baremetal - https://phabricator.wikimedia.org/T435393 [18:37:55] Southparkfan: do you know anything about the deployment-wikikube-v127 cluster in deployment-prep? Is that strictly a bd.808 project? [20:17:19] andrewbogott: I haven't interacted with that cluster yet, I think you're right [20:17:31] Not sure what the status of that cluster is [20:46:13] probably 'waiting for bryan to come back from sabbatical' [20:52:42] My RMCD bot is sometimes getting logged off before it updates its reports (assertuserfailed after logged in for less than 20 minutes). This feels like relatively new behavior. Any insights? reply here or at https://en.wikipedia.org/wiki/User_talk:RMCD_bot#The_bot_is_sometimes_getting_logged_off_before_it_updates_its_reports [21:44:05] what login method are you using? [21:46:59] andrewbogott: I have managed to gain access to the deployment-wikikube-v127-ytqt4eamhuq3-master-0 - this cluster is rather... empty. [21:48:33] Oh, right. I meant to mention that. action=login with Special:BotPasswords (re @AntiComposite: what login method are you using?) [21:50:38] Southparkfan: that's probably good... I kind of need it rebuilt. But if Bryan built it as a learning exercise we should save the rebuild for him. https://phabricator.wikimedia.org/T437500 [22:00:44] Would be my preferred as well. [22:00:51] err, preferred option [22:02:54] Bill if you can get the bot to log the request ID of the failing request (should be in an `X-Request-ID` response header, or perhaps `x-request-id` if you’re using a case-sensitive interface), that might help with looking into it [22:18:21] Hi! We were trying to send some events from beta cluster to beta cluster stream and we have seen that the last one is responding all the time with a 500 error. Is there a known issue? Should it work? [22:18:21] Thanks! [22:31:57] Lucas, I don't know how to get the request ID, it doesn't seem available via the API https://www.mediawiki.org/wiki/API:Edit [22:32:22] !log deployment-prep set role::logging::mediawiki::udp2log::monitor: false to unbreak T436469 [22:32:27] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [22:32:28] T436469: Replace deployment-mwlog02 (Bullseye deprecation) - https://phabricator.wikimedia.org/T436469 [22:36:17] !log deployment-prep point profile::rsyslog::udp_tee::destinations to both deployment-mwlog02 and deployment-mwlog03 8421/udp - T436469 [22:36:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [22:45:41] looks like botclasses and php-curl don’t make it easy but I think this might work: https://paste.toolforge.org/view/e9a9c968 (around here (https://en.wikipedia.org/w/index.php?title=User:RMCD_bot/botclasses.php&oldid=1372704733)) (re @Bill: Lucas, I don't know how to get the request ID, it doesn't seem available via the API [22:45:41] https://www.mediawiki.org/wiki/API:Edit)