[06:58:48] Check out the thread count on these codfw wdqs hosts https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wdqs&from=1647457172391&to=1647500081971&viewPanel=22 compared to when they dropped offline (as evidenced by null triples reported): https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wdqs&from=1647457172391&to=1647500081971&viewPanel=7 [06:59:15] Very solid example of the "thread leak" problem that blazegraph seems to suffer from under certain circumstances [07:01:21] (At least 2003 and 2004 displayed that behavior, 2001's thread count didn't do anything too crazy while it was offline) [08:09:29] ryankemper: nice find! [08:12:46] could not convince jvmquake to do what I wanted on wdqs1010 ( https://grafana.wikimedia.org/d/000000489/wikidata-query-service?orgId=1&var-cluster_name=wdqs-test&from=1647432470813&to=1647472390330 ) but blazegraph remained alive in the end so I guess I did not find the right stress test scenario... [08:13:48] pondering whether to move forward and see how it behaves on real traffic or to continue to do stress tests on wdqs1010 [08:14:37] tempted to move forward as it did not blow up the jvm so the risk is pretty low if we are just in "warning mode" [09:41:15] dcausse: I would definitely move forward! [11:08:04] lunch [11:45:40] errand [13:10:10] back [13:12:17] greetings [13:13:15] o/ [13:13:20] ryan-kemper that's really interesting re: thread count, do we/can we alert on rate of thread count increase or something? [14:18:39] dcausse: : Use of wfWikiID was deprecated in MediaWiki 1.35. [Called from CirrusSearch\SearchConfig::__construct in /var/www/html/w/extensions/CirrusSearch/includes/SearchConfig.php at line 71] in [14:19:09] I got this [14:19:45] WHen i added cirrussearch to it [14:20:03] ejoseph: looking but looks like you cloned an old version of CirrusSearch? [14:20:22] I have WikiMap::getCurrentWikiId on my repo [14:20:32] not wfWikiID [14:20:55] I think the guide is out of date [14:20:57] can you double check with git log to see at which commit you are? [14:21:05] Cos that was the latest versions [14:21:21] can you link again to the guide? [14:21:51] docs tend to be out of date indeed :/ [14:22:07] but I wonder how you could have fetched such an old version of CirrusSearch [14:22:34] unless the guide explicit tells to fetch a particular branch/tag [14:29:47] my bad it works [14:30:57] I downloaded stable version instead of the master [14:33:56] It works now [14:40:46] nice! [14:41:07] would you have time so we can check it out the [14:46:12] ejoseph: I'm in the openhangout if you want [14:46:58] lrt me get water [14:47:00] 5 moins [14:47:03] mins [15:05:49] \o [15:11:27] o/ [15:36:13] gehel LMK if you have a minute to look over that cookbook stuff ( https://integration.wikimedia.org/ci/job/tox-docker/24455/console ) . Getting a lot of linter errors that look like false positives [15:36:36] inflatador: i suspect the problem there is noqa has been applied for so long that it has accumulated errors [15:37:25] can help work through them if you want, most are relatively minor and either asking for shorter lines or more documentation [15:37:39] ebernhardson Sure! I'm seeing stuff like "missing docstring" where it looks like there actually is a docstring. [15:37:59] meet.google.com/uno-djtc-vgz [17:00:40] dcausse ebernhardson ryankemper ejoseph do any of you have any puppet patches you need help with? We still have ~20m on the calendar mtg or I'm happy to meet sometime this afternoon my time (UTC 1900 or later) [17:01:23] inflatador: I have a couple [17:03:14] OK, come join https://meet.google.com/iqe-wcuz-mpn if you like or just link them if not [17:04:35] i have two as well :) [17:35:25] dinner [17:45:41] workout/lunch , back in ~1h [19:06:34] back [19:19:03] doctor appointment, back in ~1h [20:15:35] lunch [20:37:31] back [20:50:37] back