[14:52:21] (not sure if I am in correct channel, if I am not - please point me in correct direction, thanks!) I imagine you are aware about issues accessing upload.wikimedia.org from Russia given announcement on wikimediastatus.net. It does seem that Russian authorities banned IP of upload-lb.esams.wikimedia.org, however only esams IP is banned. [14:52:22] upload-lb.${anyotherdc}.wikimedia.org is accessible. So, my question is, would it be possible to direct Russian users via GeoDNS towards different DC [let's say drmrs] in order to resolve this issue even if temporarily? [14:55:00] simsalabim: we typically don't do that, as you can imagine. it's a game of cat and mouse at that stage, as we have seen it play out in the past. by that I mean that if this is indeed blocking, the blocking can just follow that IP change [14:55:35] sukhe: understandable, thanks [17:58:01] any Squid experts or people with general awareness here? [17:58:22] I am trying to debug a failing Pybal healthcheck and failing at that [17:58:26] https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/hieradata/common/service.yaml#4638 [17:58:43] I thought a 400 status code should be fine here to test against and that's also what the blackbox monitoring is doing [17:59:00] but this doesn't work. if I switch pybal to a TCP connection check instead, that's fine, so it's not a connectivity issue [17:59:17] I don't think 200 works here but I still tried that and it fails [17:59:58] there is also no other use of 400 http_status here so now I am thinking if Pybal doesn't support anything other than 3x and 2x [18:00:04] somehow. any thoughts? [18:01:20] or, I get Squid to just return a 3x and 2x somehow and check against that? [18:06:57] sukhe: yeah, pybal healthchecks indeed only work for 2xx or 3xx codes, no matter what you put in http-status [18:07:22] cdanis: interesting, how did you verify that? [18:07:38] I was trying to parse through https://twisted.org/documents/8.1.0/api/twisted.web.client.HTTPClientFactory.html [18:07:39] https://gitlab.wikimedia.org/repos/sre/pybal/-/blob/cd715ddb11a16150f7df66987cdf2c6b2c3d7342/pybal/monitors/proxyfetch.py#L81 [18:07:59] that winds up getting passed as the status arg to getProxyPage() [18:08:10] which does this https://gitlab.wikimedia.org/repos/sre/pybal/-/blob/cd715ddb11a16150f7df66987cdf2c6b2c3d7342/pybal/monitors/proxyfetch.py#L225 [18:08:33] ^ is the only place where the expected status is used at all [18:08:40] ^ [18:08:49] wow thanks ok that confirms what I am seeing then [18:09:01] https://codesearch.wmcloud.org/operations/?q=redirect&files=pybal&excludeFiles=&repos= [18:09:09] ok. thanks folks. so now I get Squid to somehow give me a 2x [18:09:09] it does not affect other twisted logic at all, and twisted defaults to considering status codes as failures in https://github.com/twisted/twisted/blob/twisted-18.9.0/src/twisted/web/client.py#L156 unless there's a specific handler for that status code to not consider it as a failure [18:09:10] I got all that by just starting there [18:09:15] or I just stick to IdleConnection [18:10:52] I did the latter for now . https://gerrit.wikimedia.org/r/c/operations/puppet/+/1319533 [18:11:03] I have no idea on how to get Squid to do a 200 + no energy to dig into that :P [18:12:13] worth documenting this somewhere I guess, since in theory nothing prevents http_status set to 400 in the service definition for proxyfetch [18:12:54] anyway, thanks both for confirming!