[13:47:18] dancy: deploy1003 re-added in the masters list, first deploy was a bit slow, the next ones faster, I think we are good, but let me know if something comes up and I 'll remove it again. [14:02:46] Thanks Alex. Pinging dduvall know since he's on train duty [14:03:32] thanks! [14:04:11] big thanks to akosiaris for actioning that bot thing so quickly! [14:39:11] arnoldokoth: can I merge your Puppet change : Arnold Okoth: install: adjust vrts partition configs (131b2f66ba) ? [14:40:14] Yes, you can. My bad... Submitted then had to join a meeting. [14:40:20] no pb [19:45:00] Hello! I have a couple of Qs (more to come haha): [19:45:01] 1. Is there currently a way we can hook up our local environment(s) to LogStash Prod to test out local changes? [19:45:01] 2. My team would like to set up an alerting system for specific events or logs, what are some ways we can consider starting this? Do we use Prometheus Alertmanager for ex? [19:54:00] ecarg: re #2, yeah, that's the intent -- the there's some onboarding instructions and a user guide at https://wikitech.wikimedia.org/wiki/Alertmanager [23:43:01] !incidents [23:43:01] 4921 (RESOLVED) ATSBackendErrorsHigh cache_text sre (mw-web-ro.discovery.wmnet eqiad) [23:43:01] 4924 (RESOLVED) GatewayBackendErrorsHigh sre (wikifeeds_cluster rest-gateway eqiad) [23:43:02] 4916 (RESOLVED) db1249 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:02] 4920 (RESOLVED) db1242 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:02] 4918 (RESOLVED) db1243 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:02] 4919 (RESOLVED) db1241 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:02] 4915 (RESOLVED) db1244 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:03] 4914 (RESOLVED) db1221 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:03] 4917 (RESOLVED) db1248 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:04] 4913 (RESOLVED) db1160 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:04] 4910 (RESOLVED) db1199 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:05] 4911 (RESOLVED) db1247 (paged)/MariaDB Replica Lag: s4 (paged) [23:43:05] 4923 (RESOLVED) HaproxyUnavailable cache_text global sre (thanos-rule) [23:43:06] 4922 (RESOLVED) VarnishUnavailable global sre (varnish-text thanos-rule)