[17:30:24] Hello Londoners! My puppet is not compiling with a weird pcc error: [17:30:24] unable to determine $_trusted_certname: from trusted ( or facts () (file: /srv/jenkins/puppet-compiler/37307/change/src/manifests/realm.pp, line: 29, column: 5) on node an-launcher1002.eqiad.wmnet [17:30:24] https://puppet-compiler.wmflabs.org/pcc-worker1003/37307/an-launcher1002.eqiad.wmnet/change.an-launcher1002.eqiad.wmnet.err [17:30:24] Does someone knows if it is some pb in my code or in the puppet state? Thanks ! [17:37:12] aqu: hi! not a londoner but let me see if I can help anyway :) this is for https://gerrit.wikimedia.org/r/833406, right? [17:47:45] I can't immediately see where this is coming from -- it might be the PCC facts being out of date, but I don't see what recent change they could be missing either [17:48:11] https://gerrit.wikimedia.org/r/c/operations/puppet/+/831230 could be involved, it's possible something in PCC needs updating wrt that change but I don't see the full story yet [17:49:06] aqu: if you haven't already, you might try making a separate no-op Puppet change (add a space to a comment or something) and run PCC on the same host, to verify you get the same error -- if you do, that would rule out your change as a possible cause (which already looks pretty unlikely) [17:56:36] rzl: thanks I am going to try that [18:46:43] rzl: So it must be a problem of pcc facts. Here is an example in isolation: [18:46:44] https://gerrit.wikimedia.org/r/c/operations/puppet/+/833842 [18:46:44] Leading to the same error: [18:46:44] with https://puppet-compiler.wmflabs.org/pcc-worker1002/37309/an-launcher1002.eqiad.wmnet/prod.an-launcher1002.eqiad.wmnet.err [18:46:44] Do you know how to fix that? [18:50:18] okay, good to confirm! I don't know offhand, no -- I could try running https://wikitech.wikimedia.org/wiki/Help:Puppet-compiler#Manually_update_production but it shouldn't make a difference as long as the automated process is working [18:51:43] I think the updates are happening but some fact needs updating, and that's past my very limited PCC expertise :) j.bond or _j.oe_ might be able to help, but it's late for both of them and given the SRE summit this week I'm not sure about their availability [18:52:13] let me try that manual update, just in case it unblocks you -- I think it won't make a difference, but it doesn't hurt [19:02:31] aqu: okay, done -- give that another try just to see [19:20:16] rzl: thx, I am on it [19:22:29] Same problem. Result is here: https://puppet-compiler.wmflabs.org/pcc-worker1003/37310/ [19:23:46] okay, good to know -- afraid that's probably as far as I can help you :( if you don't mind opening a phab task under #puppet-compiler, and tagging j.bond and j.oe, they might be able to take a look in between summit sessions tomorrow [19:30:08] Alright thx! [20:19:53] https://phabricator.wikimedia.org/T318281 [20:20:18] 👍