[03:06:33] 06Traffic, 06Data-Persistence, 06SRE, 10SRE-swift-storage, and 5 others: Change default image thumbnail size - https://phabricator.wikimedia.org/T355914#9636507 (10Ganesha811) @Jdlrobson, do you know if there has been any further progress on this? Has this change been added to the schedule or work plan of... [07:19:51] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9636601 (10ayounsi) FYI it's alerting for one of its PSU being down, but we don't really care anymore : > asw-a-codfw> show system alarms > 1 alarms currently active > Ala... [11:13:05] 06Traffic, 06Data Products, 06Data-Engineering, 10Observability-Logging, 13Patch-For-Review: 14Change HAProxy log-format to support missing information - 14https://phabricator.wikimedia.org/T358105#9637079 (10Fabfur) 05Open→03Resolved 14This can be closed as the new log format is tested and read... [11:13:14] 06Traffic, 06Data Products, 06Data-Engineering, 10Observability-Logging, 13Patch-For-Review: Move analytics log from Varnish to HAProxy - https://phabricator.wikimedia.org/T351117#9637081 (10Fabfur) [11:31:27] 10netops, 06Infrastructure-Foundations, 10netbox, 13Patch-For-Review: Netbox: bug preventing removing a parent bridge in custom script automation - https://phabricator.wikimedia.org/T359629#9637159 (10Aklapper) [12:10:39] 10netops, 06DC-Ops, 06Infrastructure-Foundations: Take advantage of 10Gb NICs in the new network stack - https://phabricator.wikimedia.org/T360297#9637345 (10ayounsi) p:05Triage→03Low Thanks for the task, nothing private in there. I think we should : 1/ filter out the hosts that are due for a refresh if... [14:11:01] hi :-) is it safe to just merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/1004082? [14:14:08] taavi: should be fine, the usual caveats about disabling Puppet on A:cp and proceeding on one host, etc :) [14:14:18] also, I see a reference to the /16 in hieradata/common/profile/dumps/distribution.yaml. should it be removed from there too? [14:16:59] yeah, I have a separate patch for that (https://gerrit.wikimedia.org/r/c/operations/puppet/+/1007889) [14:17:11] so proceeding then, thanks [14:17:15] please do +1 [14:24:18] works fine with cp3066 (text), cp3080 (upload), so re-enabling puppet [14:24:22] thanks [14:25:27] done, except on cp4037 which has puppet already disabled with a reference to T358105 [14:25:28] T358105: Change HAProxy log-format to support missing information - https://phabricator.wikimedia.org/T358105 [14:59:23] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9638240 (10Papaul) [15:29:33] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9638343 (10Papaul) [16:14:46] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE, 13Patch-For-Review: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9638559 (10Papaul) [16:18:26] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE, 13Patch-For-Review: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9638580 (10Papaul) [16:18:40] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE, 13Patch-For-Review: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9638581 (10Papaul) [16:19:43] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE, 13Patch-For-Review: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9638602 (10Papaul) [16:59:50] 06Traffic, 06Data Products, 06Data-Engineering, 10Observability-Logging, 13Patch-For-Review: Move analytics log from Varnish to HAProxy - https://phabricator.wikimedia.org/T351117#9638903 (10Fabfur) I've fixed some errors (now metadata field name should be correct) in the Benthos configuration. I've repo... [17:54:23] 10netops, 06Infrastructure-Foundations, 10ops-codfw, 06SRE: Decom asw-a-codfw switch stack - https://phabricator.wikimedia.org/T358244#9639269 (10Papaul) Zeroize done on asw-a1 setups: - delete the member from the master - Disconnect both cable going to asw-a2 and asw-a7 - while login into to console r... [18:21:22] 06Traffic, 06Data Products, 06Data-Engineering, 10Observability-Logging, 13Patch-For-Review: Benthos: better management for unparsable logs - https://phabricator.wikimedia.org/T359627#9639365 (10Fabfur) 05Open→03In progress After some second of live logging I can confirm that the `SSL handshake failu... [18:21:27] 06Traffic, 06Data Products, 06Data-Engineering, 10Observability-Logging, 13Patch-For-Review: Move analytics log from Varnish to HAProxy - https://phabricator.wikimedia.org/T351117#9639367 (10Fabfur) [20:23:25] 06Traffic, 06Data-Engineering, 10Observability-Logging, 13Patch-For-Review: Benthos: better management for unparsable logs - https://phabricator.wikimedia.org/T359627#9639900 (10VirginiaPoundstone) [20:27:10] 06Traffic, 06Data-Engineering, 10Observability-Logging: Check statsv and eventlogging VarnishKafka instances - https://phabricator.wikimedia.org/T359178#9639915 (10VirginiaPoundstone) [21:19:58] 06Traffic, 06Data-Engineering, 10Observability-Logging, 13Patch-For-Review: Install new Benthos instance on cp hosts - https://phabricator.wikimedia.org/T358109#9640061 (10VirginiaPoundstone) [22:25:58] 06Traffic, 06Data-Engineering, 10Observability-Logging: Check statsv and eventlogging VarnishKafka instances - https://phabricator.wikimedia.org/T359178#9640223 (10Ottomata) @Fabfur I'm really really hoping we can remove varnishkafka-eventlogging after we complete {T353817}, as part of {T238230}. This is re...