[15:17:40] (LogstashKafkaConsumerLag) firing: Too many messages in logging-eqiad for group logstash7-codfw - https://wikitech.wikimedia.org/wiki/Logstash#Kafka_consumer_lag - https://grafana.wikimedia.org/d/000000484/kafka-consumer-lag?var-cluster=logging-eqiad&var-datasource=eqiad%20prometheus/ops - https://alerts.wikimedia.org/?q=alertname%3DLogstashKafkaConsumerLag [15:43:28] hey folks, as mentioned in the SRE meeting yesterday we are planning to do a network move for hosts in codfw rack B5 on Thursday starting 16:00 UTC [15:43:44] That rack has centrallog2002, graphite2004 and prometheus2005 in it [15:44:05] Outage for each should be less than 60 seconds, just moving the uplink from old switch to new one [15:44:48] Thanks for the heads up! [15:44:53] We're tracking in T355549, and/or detailing specific actions in the below google sheet [15:44:53] T355549: Migrate servers in codfw rack B5 from asw-b5-codfw to lsw1-b5-codfw - https://phabricator.wikimedia.org/T355549 [15:44:58] https://docs.google.com/spreadsheets/d/1PlGGLclKFYR9XaqjOLibhiwwny0fOD8gLMwsNhIzGRo/edit#gid=2011030091 [15:45:48] lmata: no probs, and sorry for the short notice, the rest are a few more weeks out so hopefully easier [15:46:06] in terms of specific actions for those hosts, depooling or similar, does anything need to be done? [15:46:56] I’ll follow up on that. [15:47:45] lmata: appreciate it - ping me if I can help with anything [15:48:12] Will do, thanks again for the notice. [19:17:41] (LogstashKafkaConsumerLag) firing: Too many messages in logging-eqiad for group logstash7-codfw - https://wikitech.wikimedia.org/wiki/Logstash#Kafka_consumer_lag - https://grafana.wikimedia.org/d/000000484/kafka-consumer-lag?var-cluster=logging-eqiad&var-datasource=eqiad%20prometheus/ops - https://alerts.wikimedia.org/?q=alertname%3DLogstashKafkaConsumerLag [23:17:57] (LogstashKafkaConsumerLag) firing: Too many messages in logging-eqiad for group logstash7-codfw - https://wikitech.wikimedia.org/wiki/Logstash#Kafka_consumer_lag - https://grafana.wikimedia.org/d/000000484/kafka-consumer-lag?var-cluster=logging-eqiad&var-datasource=eqiad%20prometheus/ops - https://alerts.wikimedia.org/?q=alertname%3DLogstashKafkaConsumerLag