[00:26:58] !log gitlab-runners gitlab-runners-puppetserver-01 cd /srv/git/operations/puppet ; sudo -u gitpuppet git pull | after deploying gerrit:1166262 & gerrit:1166263 | run puppet on runner-1033 which created /etc/ferm/conf.d/18_docker-allow-cloudinfra-puppet-enc, open firewall holes for T397591 [00:27:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Gitlab-runners/SAL [00:27:01] T397591: Unable to connect to Beta Cluster from WMCS GitLab CI runners - https://phabricator.wikimedia.org/T397591 [09:47:36] hi all! [09:47:37] i need to setup alerts for my cloudvps managed service.. i can check in the grafana we have live metrics..we need to use them and setup alerts accordingly in here [09:47:39] https://prometheus-alerts.wmcloud.org/?q=%40state%3Dactive&q=project%3Dvideocuttool [09:47:40] by default our project is not showing up here...what should i do? [09:51:24] see https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_managed_monitoring. there is no self service for any of this yet, you should file a phabricator task tagged with #cloud-vps detailing what changes you want to be made [11:27:48] Has anything changed with the wbt_item_terms table of wikibase? According to the doc, it's still in use, but it has apparently disappeared from the WD replica. Noticed with T398932, where it makes some XTools things crash. [11:27:48] T398932: Pageinfo crashing due to missing db table wbt_item_terms - https://phabricator.wikimedia.org/T398932 [11:28:32] Alien333: yes, see https://wikitech.wikimedia.org/wiki/News/2025_Wikidata_term_store_database_split [11:29:08] Ok, thank you very much. [12:28:41] hey @taavi [12:28:42] ``` [12:28:43] A base set of alerting rules is defined for each project. In addition, Cloud VPS admins can define additional rules for each project. The Cloud VPS admins can also route alerts for a specific project to a list of Libera.Chat IRC channels or email addresses``` [12:28:45] within that wiki i found this..and so i thought there would be some basic alerts defined for our project and was looking into them [12:28:46] how can we onboard to promotheus if our project is not listed in it. can't find that in this wiki (re @wmtelegram_bot: see https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_managed_monitoring. there is no self service for any of this yet,...) [12:36:20] if your project is not listed on prometheus-alerts.wmcloud.org, it means that none of those alerts are currently triggering on your project (which is a good thing, usually) [12:36:30] is there a specific problem you're trying to solve with all of this? [12:39:57] o/ FYI I'm getting a generic WMCS error page for PAWS when I try to access it: https://hub-paws.wmcloud.org/hub/login. the static public notebook links are working fine. [12:40:09] that's me [12:40:14] oh, I did not update the status, one sec [12:40:22] !status paws down, on it [12:40:27] oh excellent - thanks dcaro and no urgency [12:40:43] or better [12:40:46] !status paws down, T398912 [12:40:47] T398912: [2025-07-08] PAWS down - https://phabricator.wikimedia.org/T398912 [13:49:35] hi @taavi [13:49:36] yeah we would want to setup alerts for basic items like disk usage reaching 100% (and we would have to delete it) [13:49:37] these metrics are recorded in grafana.. its just that an alert for this could be helpful (re @wmtelegram_bot: is there a specific problem you're trying to solve with all of this?) [15:48:55] !log lucaswerkmeister@tools-bastion-13 tools.wmopbot toolforge jobs restart bot # left IRC 15:31 UTC for no reason I can make out [15:48:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wmopbot/SAL