[00:00:14] not yet live in prod though [00:13:40] legoktm: can I ask you a question, in addition to this one? :) [00:13:51] PHP issue [00:13:55] Yep, what's up? [07:38:12] Hello folks, cross-posting in here too - I am going to reimage kafka-main1003 to buster in a bit if nobody opposes (nothing is on fire afais, the cluster should handle the stopped node gracefully as happened in codfw) [09:53:00] I'm rebooting grafana1002 in a few minutes, if that a problem to anyone, please speak up [09:55:16] or forever hold your peace [12:23:50] Does anyone know the process of becoming an "admin" user on Wikitech? [12:24:08] I have no strong need for this I think, but I went to correct a typo on this page and it's restricted: [12:24:09] https://wikitech.wikimedia.org/wiki/SRE/Clinic_Duty/Access_requests [12:24:19] So wondering what the best way forward is. [12:29:26] topranks: poke someone already with it [12:29:33] I don't think there is a real process [12:29:46] Probably one of the WMCS folks [12:30:25] RhinosF1: ok thanks I will see if I can find a victim :) [12:30:26] thanks. [12:30:43] Hehe [12:39:11] topranks: you want to find someone in this list: https://wikitech.wikimedia.org/wiki/Special:ListUsers/bureaucrat [12:40:22] taavi: cool thanks for that :) [15:42:10] hey all! N00b here trying to reimage via sre.hosts.reimage. It's failed twice with errors: https://phabricator.wikimedia.org/P18732 . Looks like a timeout to me, any suggestions? [15:45:41] inflatador: I think it means the server didn't reboot? [15:47:39] weird. I can get an SSH prompt, but it prompts for pw. I guess it never got to the puppet config step? [15:50:21] you can try install_console to get on the host and manually execute puppet agent to debug and/or continue [15:50:57] cumin1001:~$ sudo /usr/local/bin/install_console foo.eqiad.wmnet [15:51:11] (which will use the root key that's present pre-puppetization to get a remote root shell) [15:52:06] bblack thanks. I'm trying again from the cumin host in DFW (previous runs in IAD). Probably won't make a difference, but will try the console thing if it fails [15:56:31] it failed again, and the "install_console" command is prompting for a password, which is apparently not the mgmt password, any idea what it wants? [15:58:23] if install_console doesn't work, then it may have gotten past that phase and needs a real console root login [15:58:39] let me try ont he codfw host and see what I see before I relay more dead-end advice :) [16:00:16] yeah, so, depending exactly where a reimage halts in the process, different access methods will work [16:00:32] at the stage it's currently stuck at on elastic2051, the way to get into root on the host is: [16:00:42] ssh root@elastic2051.mgmt.codfw.wmnet [16:01:02] [use ssh key + management password from pws repo] [16:01:10] press enter to get a prompt [16:01:18] log in as root, using root_password from pws repo [16:01:55] gotta go afk but will be back in ~30 [16:02:44] probably from there you can try an agent run and see what's borked [16:47:22] back, will take a look [16:50:55] Is there an SRE around who is willing to +2 https://gerrit.wikimedia.org/r/c/operations/puppet/+/753780 ? I need to to make progress on scap work in beta. [17:28:31] dancy: done [17:28:35] Thanks! [17:33:35] hi all, i have created a new puppet functions which gives access to the parameteres passed to the currently scoped resource. This can be usefull for passing parameteres directly to a config file (https://gerrit.wikimedia.org/r/c/operations/puppet/+/753724) or passing parameteres between classes e.g. from profile::foo -> foo (https://gerrit.wikimedia.org/r/c/operations/puppet/+/753786). [17:34:13] currently the function is only used in cloud, the later cr above is would be the first use case in production but i would also like to use it as review for the underlining function [17:34:59] *as such i would like to use 753786 as a retrospective review of the underlining function [17:35:13] seeking volunteeres :) [17:50:18] jbond: my initial concern is that is makes are puppet more bespoke [17:50:36] s/are/our/ [17:51:49] * jbond resists saying too late [17:52:27] jhathaway: i have created a PR https://github.com/voxpupuli/puppet-extlib/pull/194 which is a bit like beta for stdlib. assuming it gets a warm reception there ill create a pr to stdlib [17:54:24] jbond: very cool, I'll take a look at that as well [17:54:37] great thanks <3