[09:46:30] jynus: this should be ready for the final round of reviews https://gerrit.wikimedia.org/r/c/operations/puppet/+/931880 [09:46:39] having a look [09:49:37] thanks! merged [09:50:28] run puppet on de hosts [09:50:33] and we can test the backup run [09:50:45] let me know a host to run it manually, I can show you [09:50:47] ok, doing it [09:53:29] did it create a /srv/backups ? (maybe it conflicted with other functionality) [09:54:13] Notice: /Stage[main]/Profile::Dbbackups::Mydumper/File[/srv/backups]/ensure: created [09:54:45] but this failed [09:54:46] Error: /Stage[main]/Profile::Dbbackups::Mydumper/File[/etc/wmfbackups/statistics.cnf]/ensure: change from 'absent' to 'present' failed: Could not set 'present' on ensure: No such file or directory - A directory component in /etc/wmfbackups/statistics.cnf20230623-934306-vnip2b.lock does not exist or is a dangling symbolic link (file: /etc/puppet/modules/profile/manifests/dbbackups/mydumper.pp, line: 109) [09:55:19] perhaps an ordering issue? running puppet again [09:55:38] I see the wmfbackups deb package was installed later in the run [09:55:46] no, it is not that [09:56:08] Notice: /Stage[main]/Profile::Dbbackups::Mydumper/File[/etc/wmfbackups/backups.cnf]/ensure: defined content as '{md5}9a6e53eb778b049bc2201c7c52743845' [09:56:08] or maybe it is [09:56:12] it worked this time [09:56:20] I see, race condition [09:56:38] no other errors? [09:56:44] no other errors [09:56:46] wow [09:56:51] I am surprised [09:56:54] File[/etc/wmfbackups/backups.cnf] should depend on Package[wmfbackups] [09:57:02] can you tell me the host? [09:57:16] there is a reason why that is not there, but I can fix it later [09:57:27] cloudservices2004-dev.codfw.wmnet, cloudservices[1004-1005].wikimedia.org [09:58:18] let's maybe go dm so we don't monopolize the channel [10:32:58] hnowlan: found the issue, I'm trying to see how I can fix it