[09:22:10] while debugging an unrelated issue I saw that ldap prometheus scrapping is failing on 50% of hosts. Does anyone know why? Could it be ongoing setup- in which case, maybe it could be acked for some weeks? https://grafana.wikimedia.org/goto/A-7PpOVSk?orgId=1 [09:24:37] I think it is WIP hosts indeed T331699 moritzm- maybe I can downtime the checks for some weeks? [09:24:37] T331699: Migrate the r/w LDAP servers to Bookworm and MDB storage - https://phabricator.wikimedia.org/T331699 [09:25:08] yeah, let me extend the downtime now [09:25:30] done [09:26:25] thank you a lot, moritzm! [10:38:54] 28 billion page views were served in october, I belive that is an all-time record [11:20:17] <_joe_> significantly more than what we're used to, right? [11:20:28] <_joe_> our usual figure is ~ 800M/day [11:25:03] is itt possible that we changed a bit what we count as views? [11:34:54] the page-analytics service (which powers pageviews instead of aqs1 now) was enabled on the 11th of October which I think wouldn't have changed anything as it uses the exact same logic [11:35:00] but just in case there's a notable pattern [11:42:00] the upwards trend, looking at it daily seems to start the 22-23 of october actually https://stats.wikimedia.org/#/en.wikipedia.org/reading/total-page-views/normal|bar|2023-02-27~2023-11-02|~total|daily [14:52:47] urandom, jhathaway: just a heads up, myself and Papaul are doing some planned maintenance in codfw starting in 2 hours time [14:53:04] moving uplinks from row A/B to new switches (T347191) [14:53:04] T347191: Bring codfw row A-B EVPN switches live and make them gateway for existing Vlans - https://phabricator.wikimedia.org/T347191 [14:53:05] thanks topranks [14:53:28] shouldn't be any downtime but it's a delicate move, ping me if any sign of problems (I'll keep an eye out myself anyway) [14:54:02] will do [15:07:44] topranks: cool, will do [17:04:28] jhathaway, urandom: kicking off our maintenance in codfw rows A/B now (T347191) [17:04:29] T347191: Bring codfw row A-B EVPN switches live and make them gateway for existing Vlans - https://phabricator.wikimedia.org/T347191 [17:04:55] roger roger, thanks topranks [17:05:05] topranks: 👍 [17:10:34] <_joe_> jhathaway: can you rejoin #-private? [17:10:45] yup! [19:58:39] Amir1: IIRC we decided some time ago to remove the SRE phab tag on tickets that were already assigned to teams. Would it be possible to update @Maintenance_bot to stop adding SRE to tickets with specific teams assigned? [20:03:18] brett: https://gitlab.wikimedia.org/ladsgroup/Phabricator-maintenance-bot/-/blob/master/project_grouper.py?ref_type=heads [20:03:27] Sending a MR [20:04:50] many thanks [20:05:19] brett: https://gitlab.wikimedia.org/ladsgroup/Phabricator-maintenance-bot/-/merge_requests/1 [20:06:05] urbanecm, Amir1: ^ [20:06:56] While I don't know where any docs describing the decision are, I believe that the idea was that "SRE" would be tagged only when no other tag is added. Anyone with a better memory care to chime in? [20:07:59] brett: MR looks good to me, but I'll let you confirmed whether you want it merged or not. [20:08:18] (Merged now i mean) [20:09:47] Don't have approve privs but yeah, +1 [20:12:47] urbanecm: merge away then [20:13:38] brett: I do, that's why I asked :) done [20:18:21] brett: that was the idea with the old clinic duty dashboard -- we had to remove the SRE tag so that already-triaged tasks wouldn't show up [20:18:44] the current dashboard just doesn't list tasks that have a subteam tagged on them, so removing that tag isn't necessary anymore [20:19:01] you can't find it documented because it's no longer something we do :)