[07:23:47] anybody else received a mail from @gitlab.com with subject "Changes to your GitLab subscription | Wikimedia" containing an unsubscribe link that suggests you've landed on some mailing list of theirs? [07:35:58] I got that mail too. However I'm instance admin and subscribed to the releases newsletter. [07:35:58] But this message smells like unwanted sales mails. I'm happy to have more feedback from others. Sending this kind of mails to our users without consent is a no-go and highly confusing. [08:49:32] I have an account on Wikimedia gitlab and haven't had anything [08:51:32] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Jelto) >>! In T307142#7976964, @Jelto wrote: > ... > I'll check bacula tomorrow for new backups of `gitlab1004` and make sure the replicas synced prope... [08:52:02] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Jelto) [15:07:28] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10Patch-For-Review, 10User-brennen: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10Jelto) After migrating to new hosts (T307142) we got a bacula alert about backups on `gitlab1001` (the old production machine): ` PROB... [15:12:51] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Jelto) >>! In T307142#7978386, @Jelto wrote: > ... > One rsync job failed on the old production host `gitlab1001`. I'll take a closer look. I assume th... [15:30:41] 10GitLab (Infrastructure), 10Data-Persistence-Backup, 10serviceops, 10Patch-For-Review, 10User-brennen: Backups for GitLab - https://phabricator.wikimedia.org/T274463 (10jcrespo) Bacula is now happy: `Fresh: 116 jobs` Note on first setup, the backup check is quite sensitive, so anything that is not a su... [16:18:57] jayme, jelto: it's definitely salespam. i'm taking a call with them next week, mostly to tell them to knock it off. [16:20:05] brennen: thanks a lot! [16:21:02] brennen: maybe try to figure out where they got the mail adress from (like if there is some kind of telemetry involved) [16:21:24] i don't know whether they'll listen. full-time sales drones have a way of being relentless about this kind of thing. [16:21:35] there had extremely better not be telemetry involved. [17:05:00] Dez [18:14:46] :( [18:15:04] I merged a change that did nothing but change the time to keep backups [18:15:13] that triggered a gitlab reconfig [18:15:17] and now it's down :( [18:15:52] ah.. but it's back now :) [18:16:24] well. it was https://gerrit.wikimedia.org/r/c/operations/puppet/+/802622 causing an "Exec[Reconfigure GitLab]" [18:16:45] seems ok now though. no worries then [19:19:03] whew! [19:21:25] restarts take quite a while [19:37:15] (usually just about long enough to think "i wonder if this is going to come back up") [19:40:52] haha, yea, that was _exactly_ the time frame. just long enough to start worrying [19:41:09] but then ok shortly after a few reloads