[08:58:21] 06serviceops: kafka2001 seems out of sync with the rest of the cluster - https://phabricator.wikimedia.org/T370574 (10elukey) 03NEW [09:52:40] 06serviceops: kafka2001 seems out of sync with the rest of the cluster - https://phabricator.wikimedia.org/T370574#10000059 (10elukey) Current status under replicated topics: ` elukey@kafka-main2001:~$ kafka topics --describe | grep Isr | egrep 'Isr: 200[0-9],200[0-9]$' Topic: __consumer_offsets Partition: 0 L... [13:19:28] 06serviceops: kafka2001 seems out of sync with the rest of the cluster - https://phabricator.wikimedia.org/T370574#10000190 (10elukey) I checked the IRC logs and I forgot that around the time of the errors we had a codfw network outage in some rows. [13:27:15] 06serviceops: kafka2001 seems out of sync with the rest of the cluster - https://phabricator.wikimedia.org/T370574#10000193 (10elukey) This may be an issue of some partition data getting corrupted, and in most of the other occurrences of similar issues found on the Internet it seems that Kafka should overcome it...