[00:34:00] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760024 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host lvs7001.magru.wmnet with OS bullseye [00:44:50] 06Traffic, 06Infrastructure-Foundations, 10vm-requests: Site: (2) VMs for ncredir - https://phabricator.wikimedia.org/T363881#9760045 (10BCornwall) 05Open→03In progress a:03BCornwall [00:45:36] 06Traffic, 06Infrastructure-Foundations, 10vm-requests: magru: (2) VMs for ncredir - https://phabricator.wikimedia.org/T363881#9760048 (10BCornwall) [00:46:54] 06Traffic, 06Infrastructure-Foundations, 10vm-requests: magru: (2) VMs for ncredir - https://phabricator.wikimedia.org/T363881#9760056 (10BCornwall) p:05Triage→03High [00:46:58] 06Traffic, 06Infrastructure-Foundations, 10vm-requests: magru: (2) VMs for ncredir - https://phabricator.wikimedia.org/T363881#9760054 (10BCornwall) 05In progress→03Stalled Waiting for magru to get Ganeti set up. [01:26:55] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760118 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host lvs7001.magru.wmnet with OS bullseye compl... [01:37:49] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760148 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host lvs7002.magru.wmnet with OS bullseye [01:39:27] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760151 (10ssingh) [02:29:02] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760193 (10ssingh) [02:31:07] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760197 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host lvs7002.magru.wmnet with OS bullseye compl... [07:44:05] 10netops, 06Infrastructure-Foundations, 07sre-alert-triage: Alert in need of triage: BGP status (instance cr2-eqord) - https://phabricator.wikimedia.org/T363895 (10LSobanski) 03NEW [09:59:58] 10netops, 06Infrastructure-Foundations, 07sre-alert-triage: Alert in need of triage: BGP status (instance cr2-eqord) - https://phabricator.wikimedia.org/T363895#9760669 (10cmooney) p:05Triage→03Low These are direct peerings to Equinix tehmselves over their own exchange. We are waiting on them to complet... [10:01:40] 10netops, 06Infrastructure-Foundations: Alert in need of triage: BGP status (instance cr2-eqord) - https://phabricator.wikimedia.org/T363895#9760673 (10cmooney) [10:02:26] 10netops, 06Infrastructure-Foundations: BGP status (instance cr2-eqord) - April 2024 - Equinix peering AS15830 - https://phabricator.wikimedia.org/T363895#9760676 (10cmooney) [10:22:13] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760695 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host lvs7003.magru.wmnet with OS bullseye [10:30:49] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760717 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host lvs7003.magru.wmnet with OS bullseye execu... [10:30:59] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760718 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host lvs7003.magru.wmnet with OS bullseye [11:19:12] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760772 (10ssingh) [11:24:18] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760774 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host lvs7003.magru.wmnet with OS bullseye compl... [13:30:03] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9760937 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host dns7001.wikimedia.org with OS bookworm [14:53:05] sukhe: fabfur: btw I've reviewed the data analysis scripts for probenet and I'm pretty ready to grab numbers as soon as we turn it on :) [14:53:53] cdanis: thanks, just ironing out some DNS box issues today but lvs and cp are "ready" :) [14:54:00] wooo! [14:54:05] happy to help with the backport for the extension as well [15:15:18] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9761225 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host dns7001.wikimedia.org with OS bookworm exe... [15:49:39] thanks cdanis ! [17:12:43] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9761576 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host dns7001.wikimedia.org with OS bookworm [18:15:33] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9761743 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host dns7001.wikimedia.org with OS bookworm com... [18:16:27] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9761747 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host dns7002.wikimedia.org with OS bookworm [18:31:04] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9761762 (10ssingh) [18:36:35] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9761770 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host dns7002.wikimedia.org with OS bookworm exe... [18:36:43] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9761774 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by sukhe@cumin1002 for host dns7002.wikimedia.org with OS bookworm [19:40:50] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9762008 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by sukhe@cumin1002 for host dns7002.wikimedia.org with OS bookworm com... [19:42:23] 10netops, 06Traffic, 06DC-Ops, 06Infrastructure-Foundations, 10ops-magru: Q4:rack/setup/install magru misc servers - https://phabricator.wikimedia.org/T362730#9762017 (10ssingh)