[07:02:09] <mhbridgebot>	 [discord] <BWM> @Site Reliability Engineering issue on Meta with Varnish?
[07:02:13] <mhbridgebot>	 [discord] <BWM> ```
[07:02:14] <mhbridgebot>	 [discord] <BWM> Error 429 Varnish Rate Limit Exceeded
[07:02:15] <mhbridgebot>	 [discord] <BWM> 
[07:02:16] <mhbridgebot>	 [discord] <BWM> Varnish Rate Limit Exceeded
[07:02:18] <mhbridgebot>	 [discord] <BWM> 
[07:02:19] <mhbridgebot>	 [discord] <BWM> Guru Meditation:
[07:02:20] <mhbridgebot>	 [discord] <BWM> 
[07:02:21] <mhbridgebot>	 [discord] <BWM> XID: 41848841
[07:02:23] <mhbridgebot>	 [discord] <BWM> 
[07:02:24] <mhbridgebot>	 [discord] <BWM> Varnish cache server
[07:02:25] <mhbridgebot>	 [discord] <BWM> ```
[07:02:44] <mhbridgebot>	 [discord] <CosmicAlpha> It means you personally made to many requests.
[07:03:04] <mhbridgebot>	 [discord] <BWM> what do i do
[07:03:19] <mhbridgebot>	 [discord] <CosmicAlpha> You for rate limited. Maybe a lot of different personal JS or CSS files?
[07:04:06] <mhbridgebot>	 [discord] <BWM> I don't have any JS or CSS files
[07:04:07] <mhbridgebot>	 [discord] <BWM> afaik
[07:04:28] <mhbridgebot>	 [discord] <CosmicAlpha> then you reloaded to many tabs at once, or to quickly.
[07:05:08] <mhbridgebot>	 [discord] <CosmicAlpha> It should work if you reload again now.
[07:05:49] <mhbridgebot>	 [discord] <BWM> trying
[07:06:02] <mhbridgebot>	 [discord] <BWM> same issue
[07:06:18] <mhbridgebot>	 [discord] <BWM> oh never mind it's good now
[07:06:45] <mhbridgebot>	 [discord] <CosmicAlpha> are you using a proxy or something?
[07:07:18] <mhbridgebot>	 [discord] <BWM> no
[07:07:25] <mhbridgebot>	 [discord] <BWM> maybe i should..
[07:07:32] <mhbridgebot>	 [discord] <BWM> it works now though
[07:07:37] <mhbridgebot>	 [discord] <CosmicAlpha> no
[07:07:39] <mhbridgebot>	 [discord] <BWM> just had to close the tab and reopen metawiki
[07:07:43] <mhbridgebot>	 [discord] <BWM> oops
[07:08:20] <mhbridgebot>	 [discord] <CosmicAlpha> I am getting SO annoyed at this getting logged out thing!
[07:08:42] <mhbridgebot>	 [discord] <BWM> i'm still logged in
[07:08:57] <mhbridgebot>	 [discord] <CosmicAlpha> I was 5 minutes ago, not anymore...
[07:09:00] <mhbridgebot>	 [discord] <BWM> though it generally happens at like 3am PT for me so i have a bit more time
[07:09:31] <mhbridgebot>	 [discord] <CosmicAlpha> I get logged out what seems like every hour or everytime I access Meta...
[07:10:06] <mhbridgebot>	 [discord] <BWM> interesting, it doesn't occur that much for me
[07:10:10] <mhbridgebot>	 [discord] <BWM> any ideas as to cause?
[07:10:22] <mhbridgebot>	 [discord] <NotAracham> Yeah, that particular post-upgrade quirk is getting very old for me.  It's presently 2-3 times a day
[07:10:44] <mhbridgebot>	 [discord] <NotAracham> (on a good day)
[07:10:49] <mhbridgebot>	 [discord] <CosmicAlpha> I am not sure it had anything to do with MW upgrade in particular.
[07:11:20] <mhbridgebot>	 [discord] <NotAracham> I'm of a similar mind -- I don't think it was the upgrade specifically but something is now exposed/shaken loose that wasn't showing up before
[07:11:50] <mhbridgebot>	 [discord] <BWM> agreew
[07:11:56] <mhbridgebot>	 [discord] <CosmicAlpha> Maybe we should try resetting session cache completely...
[07:12:37] <mhbridgebot>	 [discord] <CosmicAlpha> Oddly it only seems to happen for some of us though....
[07:12:56] <mhbridgebot>	 [discord] <CosmicAlpha> I wonder if certain-attached wikis to CentralAuth or something.
[07:12:57] <mhbridgebot>	 [discord] <NotAracham> Perhaps.  If I was interested in betting zppix I'd bet it's related to a similar request timeout issue like what we were seeing w/ createwiki.
[07:13:31] <mhbridgebot>	 [discord] <BWM> betting Zppix would be interesting indeed...
[07:13:46] <mhbridgebot>	 [discord] <BWM> might work
[07:14:01] <mhbridgebot>	 [discord] <BWM> wouldn't we be seeing that in error logs though?
[07:14:23] <mhbridgebot>	 [discord] <NotAracham> *shrug* not my department
[07:14:31] <mhbridgebot>	 [discord] <BWM> haha true
[07:16:34] <mhbridgebot>	 [discord] <CosmicAlpha> Getting logged out so much is so very annoying, especially when you have 2FA, and takes so long to log in when you have so many attached accounts that you sometimes get a 504 trying to log in...
[07:16:45] <mhbridgebot>	 [discord] <BWM> omg yes
[07:16:53] <mhbridgebot>	 [discord] <BWM> it's so frickin annoying
[07:17:14] <mhbridgebot>	 [discord] <CosmicAlpha> I have kinda avoided going on-wiki for this reason lately.
[07:17:37] <mhbridgebot>	 [discord] <Orduin> Have any of you been keeping track of which wikis you've been visiting before this happens? I've pretty much only been on meta, and I've been fine.
[07:17:52] <mhbridgebot>	 [discord] <BWM> `brandonwmwiki` for me a lot
[07:18:07] <mhbridgebot>	 [discord] <BWM> i visit it, and the next time i visit another wiki besides it i'm logged out
[07:18:17] <mhbridgebot>	 [discord] <BWM> that's the only recurring one
[07:18:23] <mhbridgebot>	 [discord] <CosmicAlpha> Only Meta and commonswiki today I think @Orduin and it still to often happens.
[07:18:29] <mhbridgebot>	 [discord] <BWM> `commonswiki` and `testwiki` are others
[07:18:34] <mhbridgebot>	 [discord] <BWM> but not as usual
[07:18:44] <mhbridgebot>	 [discord] <BWM> and of course `metawiki`
[07:19:41] <mhbridgebot>	 [discord] <NotAracham> It doesn't seem to follow any particular rhyme or reason.
[07:19:46] <mhbridgebot>	 [discord] <CosmicAlpha> Though maybe not.... sometimes I accidentally clicked a link in #cvt-feed-experimental when testing some things also...
[07:20:00] <mhbridgebot>	 [discord] <NotAracham> Mostly metawiki, as I've not been doing so many wiki approvals/interwikis lately
[07:20:19] <mhbridgebot>	 [discord] <NotAracham> One common theme is that I've frequently got custom CSS on most of the wikis I'm visiting
[07:20:34] <mhbridgebot>	 [discord] <NotAracham> But incredibly rudimentary stuff, nothing extra-fancy
[07:20:48] <mhbridgebot>	 [discord] <BWM> not sure if that is the cause as i don't have CSS on my account
[07:21:00] <mhbridgebot>	 [discord] <BWM> could be account-specific, certain triggers for different accounts i suppose
[07:21:10] <mhbridgebot>	 [discord] <CosmicAlpha> CSS wouldn't cause this.
[07:21:36] <mhbridgebot>	 [discord] <BWM> but more likely something global, which if we combine everything together that we've noticed, `commonswiki` and `metawiki` overlap
[07:21:53] <mhbridgebot>	 [discord] <BWM> and i can give you both access to `brandonwmwiki` if you'd like, no issues at all there
[07:22:25] <mhbridgebot>	 [discord] <CosmicAlpha> if it had anything to do with commonswiki or metawiki more people would be affected, especially metawiki, then almost everyone would be.
[07:22:51] <mhbridgebot>	 [discord] <BWM> they might be, and people just haven't reported it
[07:22:53] <mhbridgebot>	 [discord] <CosmicAlpha> But I haven't felt like going on-wiki as much lately because of this issue... it is very quite annoying.
[07:22:57] <mhbridgebot>	 [discord] <BWM> it's not as if an error is appearing
[07:23:02] <mhbridgebot>	 [discord] <BWM> just a log out
[07:23:09] <mhbridgebot>	 [discord] <BWM> they probably think it's normal or something
[07:23:25] <mhbridgebot>	 [discord] <CosmicAlpha> Void would have been affected in that case; just said hadn't been.
[07:23:37] <mhbridgebot>	 [discord] <BWM> true
[07:25:05] <mhbridgebot>	 [discord] <CosmicAlpha> It is definitely widespread, if we poll everyone I've really talked to, 80% were affected in some way though.
[07:25:56] <mhbridgebot>	 [discord] <CosmicAlpha> Talked to 10 people 8/10 had gotten suddenly logged out.
[07:26:26] <mhbridgebot>	 [discord] <BWM> granted rights to @Orduin and @CosmicAlpha in case they need it
[07:26:48] <mhbridgebot>	 [discord] <BWM> though I suppose it doesn't matter either way since stewards and SRE have bypass on private wikis
[07:26:56] <mhbridgebot>	 [discord] <CosmicAlpha> we dont need rights on your wiki.
[07:27:33] <mhbridgebot>	 [discord] <BWM> yep
[07:27:36] <mhbridgebot>	 [discord] <BWM> happy to remove
[07:29:25] <mhbridgebot>	 [discord] <BWM> @CosmicAlpha maybe related to a certain edit count?
[07:29:38] <mhbridgebot>	 [discord] <CosmicAlpha> Unlikely.
[07:29:44] <mhbridgebot>	 [discord] <BWM> like every x edits, you get logged out
[07:30:07] <mhbridgebot>	 [discord] <CosmicAlpha> I think it is related to memcached, potentially not anything with MediaWiki itself.
[07:30:33] <mhbridgebot>	 [discord] <BWM> are you going to reset the session cache rn?
[07:30:37] <mhbridgebot>	 [discord] <CosmicAlpha> no
[07:30:38] <mhbridgebot>	 [discord] <BWM> oh wait that would log everyone out
[07:30:43] <mhbridgebot>	 [discord] <CosmicAlpha> Yes.
[07:31:15] <mhbridgebot>	 [discord] <CosmicAlpha> That's why I won't do it until I discuss with other members of SRE and see if it is a good way forward to try.
[07:31:31] <mhbridgebot>	 [discord] <BWM> ah okay
[07:31:36] <mhbridgebot>	 [discord] <CosmicAlpha> But I am going off for tonight.
[07:31:41] <mhbridgebot>	 [discord] <BWM> hopefully a solution can be found and avoid that
[07:31:45] <mhbridgebot>	 [discord] <BWM> seeya
[07:31:49] <mhbridgebot>	 [discord] <BWM> gnight
[15:55:56] <mhbridgebot>	 [discord] <Colleiflower> ughhghguh
[15:56:00] <mhbridgebot>	 [discord] <Colleiflower> i am also logged out now
[15:56:33] <mhbridgebot>	 [discord] <Legroom> I got logged out couple of hours ago
[15:57:08] <mhbridgebot>	 [discord] <BWM> yepp
[15:57:19] <mhbridgebot>	 [discord] <BWM> current ongoing task at Phabricator