[10:45:58] slyngs: hi! we are getting a bunch of puppet errors, that seem related to https://gerrit.wikimedia.org/r/q/Ie7d90e4e55486fc292a4c02cc0593f0e80002a08 [10:46:20] the error is " rejected: parameter 'enumerable' expects an Iterable value, got Undef (file: /etc/puppet/modules/ldap/manifests/client/utils.pp, line: 83, column: 54) on node cloudcontrol1006.wikimedia.org" [10:57:44] dcaro: Yep, on it [10:58:04] thanks! let me know if I can help anyhow :) [10:59:03] I have a patch for review in a little bit :-) [11:01:03] Hmm, different bug than the one I was fixing actually... [11:15:00] dcaro: Better? [11:16:19] slyngs: yep! thanks! that works :) [11:16:59] Sorry about that, I'll test with one of the cloudcontrol servers for the followup patches [11:20:22] no problem, stuff happens :), that means you are trying things ;), btw. labstore*, cloudservices*, cloudweb* were also failing if you want to use those too/instead [11:24:45] it might be affecting also some VMs: E: Unable to locate package python3-bitu-ldap [11:25:00] (from cvn-apache9.cvn.eqiad1.wikimedia.cloud) [11:25:17] Okay, ... I'll just remove the package for now. [11:28:07] this looks different (cloudcumin1001): Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, Failed to execute generator /usr/bin/systemd-analyze: Execution of '/usr/bin/systemd-analyze calendar *-*-0 09:42:00' returned 1: Failed to parse calendar specification '*-*-0 09:42:00': Invalid [11:28:07] argument (file: /etc/puppet/modules/systemd/manifests/timer.pp, line: 38, column: 13) (file: /etc/puppet/modules/systemd/manifests/timer/job.pp, line: 163) on node cloudcumin1001.eqiad.wmnet [11:29:15] I think this was volans ^xd [11:29:24] Wuhuu, not me :-) [11:29:40] or maybe not, but he might be able to help :) [11:31:18] (btw. no blame intended at all, just trying to fix things) [11:31:30] No no I know :-) [11:33:07] dcaro: whoops what did I do? O thought I had added all hiera for wmcs too [11:33:09] I think that the change was https://gerrit.wikimedia.org/r/c/operations/puppet/+/864729, looking [11:33:13] dcaro: https://gerrit.wikimedia.org/r/c/operations/puppet/+/867137 <- Patch for removing broken package [11:33:20] mmmh when zero should not install ot [11:33:22] *it [11:33:36] sorry fpe thw typos I'm on the move [11:34:10] np [11:34:52] probably the quickest thing is tp rwvert [11:34:55] I think that it might be trying to validate the parameter, even when 0 (that is, even when specified absert) [11:35:00] *absent [11:35:27] really? why... :( [11:35:48] the compiler was happy IIRC [11:36:20] it's a guess, I might be wrong [11:37:07] I'll just test it in the puppet debugger container, just a sec [11:37:07] that's https://gerrit.wikimedia.org/r/c/operations/puppet/+/864729 btw [11:37:26] no rush though, we are not using that machine yet [11:37:29] could you please revert it and I'll fix it properly asap [11:37:40] sorry foe the troible [11:37:54] πŸ‘ [11:39:02] volans: i think its because when you sert the resource to absetn the time comes out as "*-*-0 HH:M [11:39:17] and even though the resource is absent the validation still happens [11:39:26] and that is not a valid time stamp [11:40:37] jbond: so the fix is duplicaring the whole block? no better solution? [11:41:08] hmm... there's a stacked patch to it, so the revert is not as simple as clicking the button xd [11:41:28] you can revert both [11:41:33] if itcs easier [11:41:39] the 2 is a noop [11:42:15] I wonder why puppey run did not fail on cuminunpriv1001 though [11:42:35] jbond: ^^^ ? [11:42:48] I did run it manually was a noop [11:43:10] it should be disabled there too [11:43:45] not sure ill take a better look [11:43:55] You could do ($insetup_role_report_day == 0).bool2str('*', "${insetup_role_report_day}") [11:44:03] maybe different systemd version? [11:44:08] cuminunpriv is still buster [11:44:14] maybe that makes the difference somehow [11:47:58] slyngs: I think add-ldap-group should be included on ldap::management, not on ldap::client::utils [11:48:37] taavi: Yeah, there where a bunch of things wrong with those patches [11:49:19] the ldap puppetization in general is a mess. I've been hoping to do a bunch of clean up around there, once stretch and nscd is gone [11:50:14] I wanted to get my patches out, and then start working on move and removing as well :-) [12:30:50] Krinkle: indeed, my bad. I 've responded in the change, but here it is as well. Correct link was: https://gerrit.wikimedia.org/r/c/operations/puppet/+/858294 [13:50:39] akosiaris: thx [14:06:25] Emperor: do you see anything funny on thanos swift? [14:07:04] we are bumping up the traffic we send from codfw, things look ok so far [15:23:57] cwhite: I have 20 minutes until my next meeting, ok if I merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/865731? [15:24:50] (I'm also fine with you merging it whenever) [15:26:37] andrewbogott: yep, sounds good [15:29:03] cwhite: merged. In a few I'll also do https://gerrit.wikimedia.org/r/c/operations/puppet/+/865174 and https://gerrit.wikimedia.org/r/c/operations/puppet/+/865184 [15:29:17] of course all of the above are intended to be no-ops in prod :) [15:33:53] andrewbogott: I think 865174 still has an item to resolve. [15:34:34] oh, the 'Needs a default value' comment? I'll take care of that [16:10:26] effie: I'm not really here this pm, but it looks OK [16:38:50] cool [17:34:39] dcaro: sorry for have missed to circle back, the puppet failure on the cloudcumin should have been solved with this: https://gerrit.wikimedia.org/r/c/operations/puppet/+/867165 [17:36:47] πŸ‘ indeed it was fixed, thanks! [17:41:08] sorry again for the trouble! [17:45:16] chrome is making my computer want to die :( [17:45:33] well chrome + meet [18:06:45] lmata: no joke i stopped using chrome 2 weeks ago and shifted back to FF for meet [18:06:48] for that reason [18:06:53] my laptop sounded like a jet every gmeet [18:07:17] FF isn't great and won't blur my background like chrome but it only makes my laptop sound like a prop plane not a jet [18:10:33] Yeah it makes the gpu and cpu shoot up like it’s on fire Β―\_(ツ)_/Β―