[06:28:55] I am going to switch phabricator master, expect around 1 minute of RO time https://phabricator.wikimedia.org/T398818 [06:32:00] Done, RO was 30 seconds [06:55:06] thank you! [07:22:35] hnowlan nemo-yiannis can we revert or dial down the mobileapps 10k logs/s "outbound request" firehose? I believe that's https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1167160 [07:22:59] we are rate limiting ingestion on the logstash side with https://gerrit.wikimedia.org/r/c/operations/puppet/+/1167287 [07:23:05] I think hnowlan turned it off yesterday [07:23:08] let me revert [07:23:19] thank you nemo-yiannis [07:24:46] unfortunately codfw kafka consumers can't keep up with the load of consuming from eqiad [07:24:59] otherwise the rate limit on the logstash side would be enough [07:26:25] https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1167453 [07:26:44] LGTM thank you nemo-yiannis [07:28:30] thanks godog, waiting for CI and will deploy [07:28:59] cheers [08:00:13] FYI; I need to reboot puppetserver1001, Puppet will be disabled on eqiad and the edges for about 15 minutes starting in a few minutes (no puppet merges will be possible during that period either) [08:02:47] actually, no. I'll wait until the current mediawiki deployment window has completed [08:14:25] I will be rolling out a minor Netbox update in a few minutes. See: https://phabricator.wikimedia.org/T397300 [09:04:32] FYI; I need to reboot puppetserver1001, Puppet will be disabled on eqiad and the edges for about 15 minutes starting in a few minutes (no puppet merges will be possible during that period either) [09:20:40] reboot is complete and Puppet has been re-enabled