[00:23:28] PROBLEM - Check systemd state on doc1001 is CRITICAL: CRITICAL - degraded: The following units failed: rsync-doc-doc1002.eqiad.wmnet.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [01:20:25] RECOVERY - Check systemd state on doc1001 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [01:53:50] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.38.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T300200 (10Zabe) [02:29:11] 10Release-Engineering-Team (Next), 10Release, 10Train Deployments: 1.38.0-wmf.24 deployment blockers - https://phabricator.wikimedia.org/T300200 (10ppelberg) [13:19:51] !log apply T302659 on deployment-prep centralauth databases [13:19:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:19:55] T302659: Adjust the field type of localuser.lu_attached_timestamp to fixed binary on wmf wikis - https://phabricator.wikimedia.org/T302659 [13:24:48] !log apply T302660 on deployment-prep centralauth databases [13:24:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:24:51] T302660: Adjust the field type of renameuser_queue timestamp columns to fixed binary on wmf wikis - https://phabricator.wikimedia.org/T302660 [13:48:50] ^ centralauth does not support update.php, thus doing it manually [14:39:34] hmm, beta-code-update-eqiad runs but no changes are actually fetched onto deployment-deploy03 [16:43:55] zabe: Please email me if there continues to be a problem with that job. [20:05:17] !log apply T302658 on deployment-prep centralauth databases [20:05:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:05:21] T302658: Adjust the field type of globaluser timestamp columns to fixed binary on wmf wikis - https://phabricator.wikimedia.org/T302658 [20:09:39] dancy, seems to have fixed itself