[00:05:26] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 873 bytes in 0.231 second response time [00:05:34] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1944 bytes in 0.088 second response time [00:05:38] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1940 bytes in 0.092 second response time [00:05:50] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1928 bytes in 0.057 second response time [00:12:16] from /usr/lib/nagios/plugins/check_puppet_run:154:in `
' [00:14:36] RECOVERY - Phabricator on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 22044 bytes in 0.160 second response time [00:20:35] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5791 bytes in 0.133 second response time [00:20:37] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5551 bytes in 0.110 second response time [00:20:49] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 5534 bytes in 0.053 second response time [00:21:25] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5551 bytes in 0.219 second response time [00:33:34] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1938 bytes in 0.092 second response time [00:33:38] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1941 bytes in 0.091 second response time [00:33:50] RECOVERY - http on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 4149 bytes in 0.632 second response time [00:34:26] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 869 bytes in 0.266 second response time [00:41:16] RECOVERY - Puppet on tools2002 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [00:49:25] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5798 bytes in 0.249 second response time [00:49:49] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 5535 bytes in 0.053 second response time [00:50:35] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5560 bytes in 0.085 second response time [00:50:37] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5558 bytes in 0.080 second response time [02:29:34] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1936 bytes in 0.094 second response time [02:29:38] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 874 bytes in 0.226 second response time [02:29:44] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1950 bytes in 0.252 second response time [02:29:50] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1926 bytes in 0.070 second response time [02:31:34] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5560 bytes in 0.090 second response time [02:31:38] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5549 bytes in 0.126 second response time [02:31:42] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5549 bytes in 0.226 second response time [02:31:50] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 5542 bytes in 0.049 second response time [03:08:50] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 828 bytes in 0.068 second response time [03:09:24] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1906 bytes in 0.236 second response time [03:09:34] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 846 bytes in 0.443 second response time [03:09:38] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 848 bytes in 0.479 second response time [03:15:25] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5987 bytes in 0.838 second response time [03:15:35] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5983 bytes in 0.397 second response time [03:15:37] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5991 bytes in 0.402 second response time [03:15:51] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 5965 bytes in 0.373 second response time [13:49:00] $log forcing main site through tools2002 again for more testing [13:49:08] Saved item "forcing main site through tools2002 again for more testing" [13:50:25] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 846 bytes in 0.205 second response time [13:50:35] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1898 bytes in 0.104 second response time [13:50:37] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 2138 bytes in 0.124 second response time [13:50:49] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1886 bytes in 0.079 second response time [14:06:22] $log rolled back [14:06:24] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5565 bytes in 0.080 second response time [14:06:30] Saved item "rolled back" [14:06:34] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5821 bytes in 0.242 second response time [14:06:36] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5807 bytes in 0.131 second response time [14:06:50] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 5553 bytes in 0.054 second response time [14:40:17] [02puppet] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/0cb01fff1040...9e3987999644 [14:40:19] [02puppet] 07MacFan4000 039e39879 - loglevel trace8 [14:40:58] $log undo rollback [14:41:16] Saved item "undo rollback" [14:42:25] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 Origin Down - 1898 bytes in 0.222 second response time [14:42:35] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 848 bytes in 0.491 second response time [14:42:37] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 844 bytes in 0.278 second response time [14:42:51] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 521 - 830 bytes in 0.431 second response time [15:04:24] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5987 bytes in 0.559 second response time [15:04:34] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5985 bytes in 0.355 second response time [15:04:38] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5985 bytes in 0.544 second response time [15:04:50] RECOVERY - Flask-site on tools1 is OK: HTTP OK: HTTP/1.1 200 OK - 5973 bytes in 0.692 second response time [15:11:51] PROBLEM - Flask-site on tools1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2341 bytes in 0.185 second response time [15:12:25] PROBLEM - Flask-site on bots1 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2357 bytes in 0.554 second response time [15:12:30] $log added bots2002 to the pool and verified that everything is working [15:12:35] PROBLEM - Flask-site on bots2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2357 bytes in 0.224 second response time [15:12:37] PROBLEM - Flask-site on tools2002 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2357 bytes in 0.268 second response time [15:13:01] Saved item "added bots2002 to the pool and verified that everything is working" [15:14:25] RECOVERY - Flask-site on bots1 is OK: HTTP OK: HTTP/1.1 200 OK - 5575 bytes in 0.085 second response time [15:14:35] RECOVERY - Flask-site on bots2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5567 bytes in 0.148 second response time [15:14:37] RECOVERY - Flask-site on tools2002 is OK: HTTP OK: HTTP/1.1 200 OK - 5563 bytes in 0.119 second response time [15:30:44] [02puppet] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/9e3987999644...ed177973392e [15:30:46] [02puppet] 07MacFan4000 03ed17797 - reset loglevel to normal [15:47:59] $log added new servers to hund [15:48:10] Saved item "added new servers to hund" [17:01:39] [02puppet] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/ed177973392e...f58af5a8e785 [17:01:40] [02puppet] 07MacFan4000 03f58af5a - streambot: force ipv6 [17:03:36] PROBLEM - StreamBot on bots1 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'python3' [17:22:54] [02snitchbot] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/snitchbot/compare/b8dd549235a6...386b96762ecb [17:22:56] [02snitchbot] 07MacFan4000 03386b967 - bind to ipv6 [17:38:43] MacFan4000: status [17:39:18] trying to fix ipv6 compat for streambot, flask is going the new servers without any issue [17:39:27] Cool [17:39:45] MacFan4000: can't you just use the ipv6 only rotation [17:45:37] RECOVERY - StreamBot on bots1 is OK: PROCS OK: 1 process with command name 'python3' [17:47:38] PROBLEM - StreamBot on bots1 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'python3' [17:50:06] [02snitchbot] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/snitchbot/compare/386b96762ecb...ea6fa1199d07 [17:50:08] [02snitchbot] 07MacFan4000 03ea6fa11 - Update snitchbot.py [17:52:38] RECOVERY - StreamBot on bots1 is OK: PROCS OK: 1 process with command name 'python3' [17:53:30] [02snitchbot] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/snitchbot/compare/ea6fa1199d07...8f22303d4ee6 [17:53:32] [02snitchbot] 07MacFan4000 038f22303 - import ssl [17:54:38] PROBLEM - StreamBot on bots1 is CRITICAL: PROCS CRITICAL: 0 processes with command name 'python3' [18:51:09] [02puppet] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/puppet/compare/f58af5a8e785...1e04fabec58e [18:51:10] [02puppet] 07MacFan4000 031e04fab - rollback for now [18:52:38] RECOVERY - StreamBot on bots1 is OK: PROCS OK: 1 process with command name 'python3' [19:06:45] $log Restarted StreamBotMH [19:06:45] [tell] dmehus: 2022-01-27 - 23:50:04UTC tell dmehus that he still has a lot of backlog of CU investigations to look at when he is back this weekend. Including several other wiki requests that I have set aside for him. [19:06:52] Saved item "Restarted StreamBotMH" [19:07:13] dmehusL please don't touch it during the migration [19:07:23] Oh, whay migration? [19:07:24] i continually making changes to it [19:07:32] what* migration*? [19:07:45] we are migrating to new debian bullseye servers [19:07:51] Oh, cool :) [19:07:59] sorry, didn't know that was already underway :) [19:08:26] issue is they are ipv6 only and streambot doesn't like that for some reason' [19:08:35] oh [19:09:09] yeah, Miraheze had to fix something similar related to the MirahezeRC bot, I think [19:09:41] should we shut StreamBotMH down then? [19:09:43] [02snitchbot] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/MirahezeBots/snitchbot/compare/8f22303d4ee6...aaa20a72d039 [19:09:44] [02snitchbot] 07MacFan4000 03aaa20a7 - Update snitchbot.py [19:10:15] i dont want too much downtime [19:10:27] oh ok [19:12:47] well, i did get it to use ssl which is good [21:37:22] $log disable https://phab.mirahezebots.org/people/manage/25/#54 [21:37:23] [url] ♟ Manage User · Bukkit | phab.mirahezebots.org [21:37:30] Saved item "disable https://phab.mirahezebots.org/people/manage/25/#54" [21:37:31] [url] ♟ Manage User · Bukkit | phab.mirahezebots.org [21:37:41] Sario: if you free, you can remove bots from wikijs's channels [21:37:53] RhinosF1: sure [21:38:11] Or any of there 50 accounts [21:39:48] RhinosF1: do we have a channel list to work from? [21:40:07] Sario: in the various files [21:40:13] I think most are StreamBotMH [21:40:21] RhinosF1: ack [21:41:22] On botswiki RhinosF1 blocked [[User:Bukkit]] with an expiration time of indefinite (account creation disabled, email disabled, cannot edit own talk page): Abuse of multiple accounts on partner platforms; https://bots.miraheze.org/wiki/Special:Log/block [21:43:18] RhinosF1, I think we should just block 29TB but not opposed to blocking the master. Would you restore the user's talk page access though? [21:43:43] dmehus: while he's locked I'm leaving it that way [21:44:01] If he gets unlocked, he's welcome to talk to me [21:45:29] RhinosF1, well it's a self-requested lock, so he could unlock himself at any time. I am not opposed to him being blocked locally, even if unlocked, but do prefer for user talk access not to be revoked unless absolutely necessary [21:45:53] dmehus: if he gets unlocked, I'll restore TP [21:46:24] RhinosF1, fine, and no objections to me restoring TP if you're not around and he's unlocked? [21:46:36] No [21:46:42] okay, great, thanks :) [21:46:47] that's fine then [21:51:26] $log restart Streambot [21:51:32] Saved item "restart Streambot" [22:02:10] RhinosF1: dmehus: Do either of yall know if wikijs/bukkit is responsible for the #miraheze-ballmedia-* channels? [22:04:13] They are agent [22:05:10] Ok [22:09:29] Sario, what do you mean? [22:10:39] Those channels were created by Agent, and I added StreamBotMH to the channels, afaik [22:11:12] That's what I needed to know, so I would know if I needed to remove our bots from those channels [22:18:38] Sario, ah, okay great :) [22:25:04] bukkit did the buffalonas channels i belive [22:28:29] oh, nvm that was Pokey [22:28:51] Heyo [22:29:07] what did I break? [22:29:30] nothing [22:29:48] The only ones I've found so far are the gratisdata and gratispaidiea wikis [22:30:29] i haven't seen anything about them loosing their mh cloak, so i guess they are keeping it [22:30:38] There was nobody in either of those channels except Streambot [22:31:36] Sario: according to chanserv those channels are registered to Ugochimobi [22:31:57] gratispedia and such [23:00:05] yeah those are likely Ugochimobi's related to his famepedia and famedata spinoff wikis [23:00:12] and hey Pokey :) [23:00:39] hey Doug! [23:02:42] how're you? [23:04:35] oh, fine, thanks :) [23:04:41] you? [23:07:00] good ty!