[08:08:24] <dcausse>	 o/ I retried a couple times the job at https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/jobs/60209 without much luck, I wonder if I missed something and if there's something easy to fix? (I can file a task if nothing obvious comes to mind)
[08:37:10] <jelto>	 dcausse: You are using tags (like flink-1.16-rc1) right? If yes, you also have to "protect" these tags under https://gitlab.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater/-/settings/repository -> Protected tags. You also can use a wildcard like flink-*.
[08:37:55] <dcausse>	 jelto: oh did not know that, thanks! :)
[08:37:59] <jelto>	 There is a bug in gitlab if you protect a tag after the tag was created, it does not get picked up. So best would be to just bump the tag to something like flink-1.16-rc2 or something like that
[08:38:24] <dcausse>	 jelto: are there common practices on protected prefixes?
[08:38:50] <dcausse>	 I mean on naming them?
[08:39:26] <jelto>	 Protecting the main branch is common and the desired (and default) configuration. Tags is a bit special but they act the same in gitlab and you also have to protect them if you want to use tags on trusted runners
[08:39:36] <jelto>	 no I don't think so :)
[08:40:58] <dcausse>	 ok thanks! will try all this :)
[08:41:57] <jelto>	 let me know if this is still stuck.
[08:56:15] <dcausse>	 seems to work with tags \o/, haven't found the button to trigger the publish job on the main branch yet but I don't despair :) 
[08:58:07] <jelto>	 great happy to hear that! I'm going to update the docs to make sure users protect tags and branches
[08:58:59] <jelto>	 your job is configured to run on tags on protected (main) branch only:   rules: - if: $CI_COMMIT_TAG && $CI_COMMIT_REF_PROTECTED in your .gitlab-ci.yml
[08:59:35] <dcausse>	 got it, thanks! :)
[09:00:45] <dcausse>	 the job went well but image is not visible in https://docker-registry.wikimedia.org/
[09:22:52] <jelto>	 The webinterface is rendered periodically with some delay afaik. docker pull docker-registry.wikimedia.org/repos/search-platform/flink-rdf-streaming-updater:flink-1.16-rc1 works for me and it's pulling the image
[10:55:43] <wikibugs>	 10GitLab (CI & Job Runners), 10serviceops-collab, 10Patch-For-Review: Use dedicated volume for /var/lib/docker on Trusted Runners - https://phabricator.wikimedia.org/T329035 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by eoghan@cumin1001 for host gitlab-runner1003.eqiad.wmnet with...
[11:31:13] <wikibugs>	 10GitLab (CI & Job Runners), 10serviceops-collab: Use dedicated volume for /var/lib/docker on Trusted Runners - https://phabricator.wikimedia.org/T329035 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by eoghan@cumin1001 for host gitlab-runner1003.eqiad.wmnet with OS bullseye completed: -...
[13:12:33] <wikibugs>	 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10mwbot-rs, 10mwcli: GitLab CI jobs failing with "You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" - https://phabricator.wikimedia.org/T329216 (10Addshore)
[13:13:55] <wikibugs>	 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10mwbot-rs, 10mwcli: GitLab CI jobs failing with "You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" - https://phabricator.wikimedia.org/T329216 (10Addshore)...
[13:24:41] <wikibugs>	 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10mwbot-rs, 10mwcli: GitLab CI jobs failing with "You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" - https://phabricator.wikimedia.org/T329216 (10Addshore)...
[13:41:45] <wikibugs>	 10GitLab (CI & Job Runners), 10Release-Engineering-Team, 10mwbot-rs, 10mwcli: GitLab CI jobs failing with "You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit" - https://phabricator.wikimedia.org/T329216 (10Addshore)...
[14:05:40] <wikibugs>	 10GitLab (CI & Job Runners), 10serviceops-collab, 10Patch-For-Review: Use dedicated volume for /var/lib/docker on Trusted Runners - https://phabricator.wikimedia.org/T329035 (10eoghan)
[14:07:54] <wikibugs>	 10GitLab (CI & Job Runners), 10observability, 10serviceops-collab: node exporter for gitlab-runner hosts is missing metrics for /var/lib/docker - https://phabricator.wikimedia.org/T329286 (10Jelto)
[16:02:49] <wikibugs>	 10GitLab (CI & Job Runners), 10serviceops-collab, 10Patch-For-Review: Use dedicated volume for /var/lib/docker on Trusted Runners - https://phabricator.wikimedia.org/T329035 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by eoghan@cumin1001 for host gitlab-runner1004.eqiad.wmnet with...
[16:48:33] <wikibugs>	 10GitLab (CI & Job Runners), 10serviceops-collab: Use dedicated volume for /var/lib/docker on Trusted Runners - https://phabricator.wikimedia.org/T329035 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by eoghan@cumin1001 for host gitlab-runner1004.eqiad.wmnet with OS bullseye completed: -...
[17:06:08] <wikibugs>	 10GitLab (CI & Job Runners), 10serviceops-collab: Use dedicated volume for /var/lib/docker on Trusted Runners - https://phabricator.wikimedia.org/T329035 (10eoghan)
[17:55:14] <wikibugs>	 10GitLab (Infrastructure), 10serviceops-collab: Migrate gitlab-test instance to bullseye - https://phabricator.wikimedia.org/T318521 (10Jelto) a:05Dzahn→03Jelto I removed the `gitlab-bulleye.devtools.wmcloud.org` DNS entry which was pointing to `185.15.56.117` (and confusing me quite a bit).  I re-run the...