[12:40:13] !log lucaswerkmeister@tools-bastion-15 tools.wd-image-positions deployed 1d78cb4fd9 (l10n updates: nl) [12:40:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wd-image-positions/SAL [13:24:00] !log tools.dimastbkbot webservice python3.9 start (T428139) [13:24:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.dimastbkbot/SAL [13:24:04] T428139: [toolsdb] Transaction History Length growing too much - https://phabricator.wikimedia.org/T428139 [14:21:54] !log tools.curator reset mariadb password T428367 [14:21:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.curator/SAL [14:21:58] T428367: Rotate MariaDB SQL password for tool-curator - https://phabricator.wikimedia.org/T428367 [16:29:36] o/ deployment-cirrussearch[12|13|14].deployment-prep.eqiad1.wikimedia.cloud have their ssl cert expired, can I renew the certs myself as a member of the deployment-prep project? [16:30:41] dcausse: i assume that's because puppet has been broken there for a month and a half, T424104 T424100 [16:30:42] T424104: No Puppet resources found on instance deployment-cirrussearch12 on project deployment-prep - https://phabricator.wikimedia.org/T424104 [16:30:42] T424100: No Puppet resources found on instance deployment-cirrussearch14 on project deployment-prep - https://phabricator.wikimedia.org/T424100 [16:30:55] fixing that is very welcome [16:31:15] taavi: oh thanks, will dig into this [16:31:17] dcausse: I think inflatador is in the process of replacing those hosts? Puppet is busted there and acked in https://prometheus-alerts.wmcloud.org/?q=project%3Ddeployment-prep [16:32:06] bd808: yes I think he started migrating those to opensearch2, will check if we can still get opensearch1 working properly in the meantime [17:02:20] it's been a long time since I looked at puppet in deployment-prep, what's the recommendation to add stuff in puppet://hieradata/cloud/eqiad1/deployment-prep/common.yaml vs things like https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/instance-puppet/+log/master/deployment-prep/deployment-cirrussearch.yaml ? [17:05:32] if it's supposed to affect alll instance it should be in common.yaml - if it's only meaningful for cirrussearch then it should be specific to that file? [17:05:59] probably if the key starts with profile::opensearch::.. then it's cirrussearch-only ? [17:07:03] putting things in puppet.git has the downside that a much smaller set of people can edit it, so usually managing the hiera via horizon tends to be preferrable [17:07:23] ah I see, if say the "profile::opensearch::instances:" is only used by a role like role::cirrus::beta then it should not be in puppet://hieradata/cloud/eqiad1/deployment-prep/common.yaml [17:08:09] taavi: sounds good, do we know what will take precendence if I duplicate values there (horizon or the main puppet repo)? [17:08:34] https://wikitech.wikimedia.org/wiki/Puppet/Hiera#The_lookup_hierarchy [17:09:04] the caveat to that 'generally' is that having things set in one place is much much better than conflicting data in two [17:09:11] this might be missing some info about cloud and horizon [17:09:28] but horizon data overrides puppet.git data [17:09:57] thanks! [17:10:38] sure I don't plan to dup things but since I can't merge in puppet there might still be some duplication while I change things [17:13:35] would use Horizon for testing and once it's "stable" and unlikely to change a lot.. upload a patch to the repo [17:18:02] yes definitely [17:19:47] possibly I'd like remove as much as I can from puppet to hopefully have all things cirrus related in horizon [17:44:39] I would like to remove 100% of hiera settings for Beta Cluster from ops/puppet. There are approximately zero prod SREs available to fix hiera in Beta Cluster on an average day. The horizon system is more accessible. [17:45:42] My real dream there is to make it all gitops opentofu stuff in the https://gitlab.wikimedia.org/cloudvps-repos/deployment-prep/tofu-provisioning repo. [17:53:45] beta-logs.wmcloud.org has nothing to do with deployment-prep hiera values from puppet://hieradata/cloud/eqiad1/deployment-prep/common.yaml right? [17:56:06] bd808: I guess it wouldn't take too much effort to export the current data from puppet.git to enc, and then we could just drop it all [17:56:15] .. I just nerd-sniped myself there, didn't I? [17:56:48] dcausse: depends on what exactly you mean by that, but my understanding is that that opensearch instance lives in a different project so probably yes? [17:57:54] taavi: just trying to anticipate if https://gerrit.wikimedia.org/r/c/operations/puppet/+/1298864 could possibly break the opensearch instance behind beta-logs.wmcloud.org, but I suspect it's totally unrelated [17:58:39] yeah, seems likely [17:58:42] want me to merge that now? [17:58:56] taavi: if you can that'd be great! :) [17:59:21] thanks! [18:02:40] taavi: I lost track of where we were talking about it, but another thing I need to understand to move forward faster is how to import an existing ENC managed hiera collection into opentofu. [18:03:11] bd808: iirc the blocker with that is exposing the prefix IDs the import needs somewhere more user-friendly than raw ENC API calls [18:03:31] yeah, that sounds right [18:05:32] T423970 [18:05:33] T423970: [tofu-cloudvps] Add support for importing legacy cloudvps_puppet_prefix objects - https://phabricator.wikimedia.org/T423970 [19:14:30] Bit of a strange question; is there a way to determine ldap account age? (I don't see created at in ldapsearch). Would be late 2011/2012 so before current tooling/standards, can't find in sal, suspect before fab [19:20:04] https://ldap.toolforge.org/user/reedy [19:20:09] >Account created: 20101015004557Z [19:20:58] That's exactly what I was looking for - thanks