[02:07:23] PROBLEM - IPv6 ping to ulsfo on ripe-atlas-ulsfo IPv6 is CRITICAL: CRITICAL - failed 177 probes of 623 (alerts on 65) - https://atlas.ripe.net/measurements/1791309/#!map https://wikitech.wikimedia.org/wiki/Network_monitoring%23Atlas_alerts https://grafana.wikimedia.org/d/K1qm1j-Wz/ripe-atlas [02:25:09] RECOVERY - IPv6 ping to ulsfo on ripe-atlas-ulsfo IPv6 is OK: OK - failed 43 probes of 623 (alerts on 65) - https://atlas.ripe.net/measurements/1791309/#!map https://wikitech.wikimedia.org/wiki/Network_monitoring%23Atlas_alerts https://grafana.wikimedia.org/d/K1qm1j-Wz/ripe-atlas [12:31:55] (03CR) 10Reedy: [C: 03+2] Use correct load balancers for remote databases [extensions/SecurePoll] (wmf/1.37.0-wmf.16) - 10https://gerrit.wikimedia.org/r/708994 (https://phabricator.wikimedia.org/T287780) (owner: 10Reedy) [12:31:58] (03CR) 10Reedy: [C: 03+2] Pass an actual user instance to $page->newPageUpdater() [extensions/SecurePoll] (wmf/1.37.0-wmf.16) - 10https://gerrit.wikimedia.org/r/708995 (https://phabricator.wikimedia.org/T287782) (owner: 10Reedy) [12:36:03] (03Merged) 10jenkins-bot: Use correct load balancers for remote databases [extensions/SecurePoll] (wmf/1.37.0-wmf.16) - 10https://gerrit.wikimedia.org/r/708994 (https://phabricator.wikimedia.org/T287780) (owner: 10Reedy) [12:36:51] (03Merged) 10jenkins-bot: Pass an actual user instance to $page->newPageUpdater() [extensions/SecurePoll] (wmf/1.37.0-wmf.16) - 10https://gerrit.wikimedia.org/r/708995 (https://phabricator.wikimedia.org/T287782) (owner: 10Reedy) [12:37:36] That was... quick [12:40:01] !log reedy@deploy1002 Synchronized php-1.37.0-wmf.16/extensions/SecurePoll/: T287780 T287782 (duration: 00m 58s) [12:40:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [12:40:10] T287782: Can't add or edit translations in SecurePoll - https://phabricator.wikimedia.org/T287782 [12:40:10] T287780: SecurePoll CreatePage can no longer correctly select "remote" wiki databases that aren't in the same cluster - https://phabricator.wikimedia.org/T287780 [13:09:14] (03PS1) 10Zabe: Add rollbacker group to kswiki [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709214 (https://phabricator.wikimedia.org/T286789) [16:07:15] (03PS2) 10Zabe: Add rollbacker group for kswiki [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709214 (https://phabricator.wikimedia.org/T286789) [18:27:59] PROBLEM - Varnish traffic drop between 30min ago and now at eqsin on alert1001 is CRITICAL: 35 le 60 https://wikitech.wikimedia.org/wiki/Varnish%23Diagnosing_Varnish_alerts https://grafana.wikimedia.org/dashboard/db/varnish-http-requests?panelId=6&fullscreen&orgId=1 [18:29:01] PROBLEM - Varnish traffic drop between 30min ago and now at eqiad on alert1001 is CRITICAL: 57.64 le 60 https://wikitech.wikimedia.org/wiki/Varnish%23Diagnosing_Varnish_alerts https://grafana.wikimedia.org/dashboard/db/varnish-http-requests?panelId=6&fullscreen&orgId=1 [18:32:51] RECOVERY - Varnish traffic drop between 30min ago and now at eqiad on alert1001 is OK: (C)60 le (W)70 le 94.77 https://wikitech.wikimedia.org/wiki/Varnish%23Diagnosing_Varnish_alerts https://grafana.wikimedia.org/dashboard/db/varnish-http-requests?panelId=6&fullscreen&orgId=1 [18:33:45] RECOVERY - Varnish traffic drop between 30min ago and now at eqsin on alert1001 is OK: All metrics within thresholds. https://wikitech.wikimedia.org/wiki/Varnish%23Diagnosing_Varnish_alerts https://grafana.wikimedia.org/dashboard/db/varnish-http-requests?panelId=6&fullscreen&orgId=1 [22:56:13] (03PS1) 10Zabe: Remove unused celebration logos [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709231 (https://phabricator.wikimedia.org/T272108) [22:58:22] (03PS2) 10Zabe: Remove unused enwiki celebration logos [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709231 (https://phabricator.wikimedia.org/T272108) [22:59:44] (03PS1) 10Zabe: Remove unused eswiki celebration logos [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709232 (https://phabricator.wikimedia.org/T280908) [23:04:48] (03PS1) 10Zabe: Add media.defense.gov to the wgCopyUploadsDomains allowlist of Wikimedia Commons [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709233 (https://phabricator.wikimedia.org/T287264) [23:08:49] PROBLEM - SSH on cp5005.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [23:12:59] (03PS1) 10Zabe: Add tewikisource as import source for tewikibooks [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709234 (https://phabricator.wikimedia.org/T286978) [23:24:30] (03PS1) 10Zabe: Enable SUL autologin for wikimania.wikimedia.org [mediawiki-config] - 10https://gerrit.wikimedia.org/r/709235 (https://phabricator.wikimedia.org/T285197)