[06:35:51] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 3 others: Q3:(Need By: TBD) rack/setup/install 2 new labstore hosts - https://phabricator.wikimedia.org/T302981 (10ArielGlenn) Hey @Ottomata it's great to see these hosts moving loser to being in production! One thing I noticed, they are picking...
[12:13:04] <wikibugs>	 10Puppet, 10Infrastructure-Foundations: Ferm unloads all iptables rules when it hits a parsing error - https://phabricator.wikimedia.org/T315305 (10jbond)
[12:15:40] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 3 others: Q3:(Need By: TBD) rack/setup/install 2 new labstore hosts - https://phabricator.wikimedia.org/T302981 (10Ottomata) Wrong andrew, I think you meant to ping @Andrew ?
[12:20:04] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 3 others: Q3:(Need By: TBD) rack/setup/install 2 new labstore hosts - https://phabricator.wikimedia.org/T302981 (10ArielGlenn) >>! In T302981#8157062, @Ottomata wrote: > Wrong andrew, I think you meant to ping @Andrew ?  Bah, yes I did. Thank you!
[12:24:00] <wikibugs>	 10netops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10SRE, 10ops-codfw: DRAC firmware upgrades codfw (was: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL ))) - https://phabricator.wikimedia.org/T283582 (10hashar)
[12:50:26] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 3 others: Q3:(Need By: TBD) rack/setup/install 2 new labstore hosts - https://phabricator.wikimedia.org/T302981 (10Andrew) Thanks for the suggestion @ArielGlenn. Those hosts are really not working at all right now (something awful is happening wi...
[13:39:51] <wikibugs>	 10Puppet, 10Infrastructure-Foundations, 10Documentation: update puppet documentation - https://phabricator.wikimedia.org/T315317 (10Aklapper)
[14:01:06] <wikibugs>	 10Puppet, 10Infrastructure-Foundations, 10Patch-For-Review: Ferm unloads all iptables rules when it hits a parsing error - https://phabricator.wikimedia.org/T315305 (10jbond) p:05Triage→03Medium
[15:25:18] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Link from lsw1-e1-eqiad to lsw1-f2-eqiad down - https://phabricator.wikimedia.org/T315052 (10Cmjohnson) @cmooney The QSFP28 module for et-o/o/54 on lsw1-f3-eqiad has been replaced.
[20:56:46] <wikibugs>	 10Puppet, 10Beta-Cluster-Infrastructure, 10Infrastructure-Foundations, 10Traffic: Evaluation Error on deployment-cache-text06 puppet run - https://phabricator.wikimedia.org/T315351 (10RhinosF1) p:05Triage→03Unbreak! Hi Traffic, this might be stopping beta coming back up (or a false alarm). Can you take...
[22:36:54] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10Observability-Metrics, 10SRE, and 2 others: LibreNMS seemingly not collecting data for many ports after migration to netmon1003 - https://phabricator.wikimedia.org/T314972 (10andrea.denisse) Hello team, I submitted the following patches for this issue:  1. [[ https...
[22:37:43] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10Observability-Metrics, 10SRE, and 2 others: LibreNMS seemingly not collecting data for many ports after migration to netmon1003 - https://phabricator.wikimedia.org/T314972 (10andrea.denisse)
[23:28:50] <wikibugs>	 10Puppet, 10Beta-Cluster-Infrastructure, 10Infrastructure-Foundations, 10SRE, 10Traffic: Evaluation Error on deployment-cache-text06 puppet run - https://phabricator.wikimedia.org/T315351 (10TheresNoTime) Introduced by https://gerrit.wikimedia.org/r/c/operations/puppet/+/816806 ?  `lang=diff diff --git a...