[06:25:49] <_joe_>	 bd808: I think we should move away from that image altogether
[06:26:12] <_joe_>	 but yes we'll fix it
[06:30:55] <wikibugs>	 10serviceops, 10Release Pipeline, 10Toolhub, 10Patch-For-Review: docker-registry.wikimedia.org/nodejs10-devel container after 0.0.3 does not include `npm` - https://phabricator.wikimedia.org/T286212 (10Joe) For the record, the pinning preferring wikimedia packages was fixed last year, causing this effect.
[06:31:57] <wikibugs>	 10serviceops, 10Release Pipeline: Production buster-nodejs10-devel image has npm 5.x, which is not actually compatible with node 10.x - https://phabricator.wikimedia.org/T284112 (10Joe) I think we should offer a consistent npm version in all node10 images.  So: on -devel images, we should use the packages from...
[06:37:31] <legoktm>	 !issync
[06:37:32] <ircservserv-wm>	 Syncing #wikimedia-serviceops (requested by legoktm)
[06:37:33] <ircservserv-wm>	 Set /cs flags #wikimedia-serviceops litharge +o
[07:30:05] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE: Evaluate contour as an ingress - https://phabricator.wikimedia.org/T286196 (10Joe) **Usage**  Basic usage can be obtained by just defining the Ingress properties in the charts:  ` apiVersion: networking.k8s.io/v1 kind: Ingress metadata:   name: lambdoid spec:   rules:   - ht...
[07:32:00] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE: Evaluate contour as an ingress - https://phabricator.wikimedia.org/T286196 (10Joe) Collection of metrics via prometheus is supported natively, both from envoy and contour.  Using configuration options is also possible to make contour / envoy log in json format, see https://p...
[07:33:20] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE: Evaluate contour as an ingress - https://phabricator.wikimedia.org/T286196 (10Joe) Overall, contour seems built the right way and looks like a promising ingress. I am just wary of using CRDs so much, and of the fact it needs yet another operator to work properly.  I think th...
[09:03:21] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE: Evaluate contour as an ingress - https://phabricator.wikimedia.org/T286196 (10Joe) 05Open→03Resolved p:05Triage→03High
[09:03:23] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE: Create a gateway in kubernetes for the execution of our "lambdas" - https://phabricator.wikimedia.org/T261277 (10Joe)
[09:33:20] <wikibugs>	 10serviceops, 10conftool, 10Datacenter-Switchover, 10User-jbond: DNS Discovery operations diffs incosistent (post DC-Switch) - https://phabricator.wikimedia.org/T286231 (10jbond)
[09:46:30] <wikibugs>	 10serviceops, 10conftool, 10Datacenter-Switchover, 10User-jbond: DNS Discovery operations diffs incosistent (post DC-Switch) - https://phabricator.wikimedia.org/T286231 (10jbond) While making the change here i notice that some services which would normally be active-active seem to be only running in codfw...
[09:59:48] <jbond>	 _joe_: if you get a sec can you take a loko at https://gerrit.wikimedia.org/r/c/operations/puppet/+/703390 looks related to DC-switch
[10:02:24] <_joe_>	 jbond: sigh yes, done
[10:02:34] <jbond>	 thanks
[10:03:12] <jbond>	 _joe_: just to double check its expected that services which are normally active active are only in codfw during the switch?
[10:03:23] <_joe_>	 yes
[10:03:27] <jbond>	 great thanks
[10:16:38] <wikibugs>	 10serviceops, 10conftool, 10Datacenter-Switchover, 10Patch-For-Review, 10User-jbond: DNS Discovery operations diffs incosistent (post DC-Switch) - https://phabricator.wikimedia.org/T286231 (10jbond) I have fixed the check however we should add something to the playbook/cookbooks to catch this change in t...
[21:40:14] <wikibugs>	 10serviceops, 10conftool, 10Datacenter-Switchover, 10User-jbond: DNS Discovery operations diffs incosistent (post DC-Switch) - https://phabricator.wikimedia.org/T286231 (10Legoktm) 05Open→03Resolved a:03jbond Not sure how I missed the icinga alert, but thanks for the ping. I have documented this as a...