[05:30:40] I am going to put phabricator in RO for a few seconds to switch its database master [09:10:55] GitLab needs a short maintenance at 11:00 UTC [10:49:16] gitlab maintenance starting soon [10:51:32] k [10:58:38] done [13:56:56] okay, I made an oopsie I don't know how to clean it up, I somehow managed to created a file in my home directory in cumin called 'elect file_path, file_name, size, file_date from backup_files join backups on backups.id = backup_id where (backups.source like '\''dbstore100-'\'' or backups.source like '\''db1-'\'') and backups.section = '\''x1'\'' and file_name like '\''-.ibd'\'' and size > 1048576 limit 50;' (the 's are part of the file [13:56:56] name) [13:57:16] and now, rm doesn't work "File name too long" [13:57:32] neither mv, nor sudo rm, nor sudo rm -f [13:57:35] no worries [13:58:11] the easiest is rm -i so ir asks for permision [13:58:51] still gives filename too long I think [13:59:00] ladsgroup@cumin1001:~$ sudo rm -i "'elect file_path, file_name, size, file_date from backup_files join backups on backups.id = backup_id where (backups.source like '\''dbstore100-'\'' or backups.source like '\''db1-'\'') and backups.section = '\''x1'\'' and file_name like '\''-.ibd'\'' and size > 1048576 limit 50;'" [13:59:00] rm: cannot remove ''\''elect file_path, file_name, size, file_date from backup_files join backups on backups.id = backup_id where (backups.source like '\''\'\'''\''dbstore100-'\''\'\'''\'' or backups.source like '\''\'\'''\''db1-'\''\'\'''\'') and backups.section = '\''\'\'''\''x1'\''\'\'''\'' and file_name like '\''\'\'''\''-.ibd'\''\'\'''\'' and size > 1048576 limit 50;'\''': File name too long [13:59:27] Amir1: can I try? [13:59:32] oh definitely! [14:00:03] check DELETEME [14:00:07] I think you can handle that [14:00:22] :-P [14:00:28] ah, nice, mv didn't work for me [14:01:43] how did you do it? So I don't bother you :D [14:01:55] ls e* [14:02:07] to check it was the only file starting with e [14:02:24] then: [14:02:26] mv e* DELETEME [14:02:40] ah [14:02:50] clever [14:04:37] check also man rm (not joking) it has actually useful tips, like how to remove a file named -foo [14:05:04] that or the interactive flag are nice [14:05:13] or if not, find + exect + interactive [14:05:52] thanks [15:45:52] anyone tried to bootstrap a Pontoon stack recently? I'm getting this: [15:45:55] Error: Evaluation Error: Error while evaluating a Function Call, wrong number of arguments (given 2, expected 1) (file: /tmp/bootstrap/init/00realm.pp, line: 81, column: 21) on node cdanis-pontoon-puppet01.sre-sandbox.eqiad1.wikimedia.cloud [15:46:16] the line referenced in the generated 00realm file is: [15:46:18] $_nameservers = lookup('profile::resolving::nameservers') [15:46:51] sukhe: ^ ? [15:47:04] [he's been refactoring some related things] [15:47:28] we haven't touched this bit yet [15:47:42] oh ok [15:48:19] it's been quite a while since we touched this in T340479 [15:48:20] T340479: Reduce toil in provisioning and decommissioning of DNS/NTP servers by automating generation of resolv.conf and NTP peers - https://phabricator.wikimedia.org/T340479 [15:49:08] looking shortly if it can be that and we never hit because no one touched it! [15:53:34] the hiera looks correct [15:56:18] maybe it's a puppet7 thing? [15:56:57] possible but can't see how though, it's just a hiera lookup and I don't think anything has changed there [15:57:07] I think hiera lookups might be broken entirely on this host [15:57:13] but also that doesn't make much sense [15:57:15] cdanis: can you paste the full thing if possible? [15:57:36] that is the full thing sadly [15:57:50] I wonder if lookup() itself is non-defined, somehow [15:58:10] although you'd think the error message would be better [15:58:56] does "puppet lookup ..." work? [15:59:13] bblack: it does not, for any key, and reports a similar error [15:59:20] cdanis@cdanis-pontoon-puppet01:~$ sudo puppet lookup --compile --explain profile::base::certificates::puppet_ca_content [15:59:22] Error: Could not run: wrong number of arguments (given 2, expected 1) [15:59:25] one of many variations I've tried [15:59:34] if you look at the commit history as well for this [15:59:47] lookup()s were being removed [16:00:31] hmmm [16:00:46] # trusted facts are not always available with puppet master --compile (used by pcc) [16:00:49] # or puppet lookup --compile. As such we use the fqdn when the trusted facts are [16:00:52] # not available T248169 [16:00:54] T248169: puppet populate failing on some nodes - https://phabricator.wikimedia.org/T248169 [16:01:49] well it's the same error without --compile as well [16:03:40] ok so at least in this case I think you can simply use the anycast recursor address instead [16:03:44] cdanis: has puppet run on the client node and the master has the facts? [16:03:45] and skip the lookup [16:03:57] volans: yes [16:04:00] not the solution you were looking for but it will work :) [16:04:03] puppet lookup --compile --node {fqdn} {key} # works only if the puppemaster has the facts [16:04:17] "ns-recursor.openstack.eqiad1.wikimediacloud.org" [16:04:18] or at least, the agent has certainly run on the node [16:05:17] also matches [16:05:18] hieradata/cloud/eqiad1.yaml-profile::resolving::nameservers: [16:05:18] hieradata/cloud/eqiad1.yaml: - "ns-recursor.openstack.eqiad1.wikimediacloud.org" [17:17:25] Happy Friday! If anyone has time for a quick review: https://gerrit.wikimedia.org/r/c/operations/puppet/+/979388 [17:17:50] And to you. Looking now.