[10:44:33] running the spark job to export the cirrus indices with 1 core did not fix the mem issue :( [10:45:25] Lunch [10:59:03] wondering if it's not the spark driver actually being killed... [11:05:52] hm.. If I'm reading this right: "22/11/08 09:55:42 ERROR ApplicationMaster: RECEIVED SIGNAL TERM" that seems to be the driver [11:06:57] lunch [13:47:32] ryankemper: I had a quick look at https://grafana.wikimedia.org/d/l-3CMlN4z/wdqs-uptime-slo?orgId=1 after our conversation with Reuven. It looks like the dips are related to a high number of 403 (which indicates a client that isn't following the 429). [13:54:17] also note that the graph don't match what I find in Turnilo: [13:54:17] https://turnilo.wikimedia.org/#webrequest_sampled_128/4/N4IgbglgzgrghgGwgLzgFwgewHYgFwhLYCmAtAMYAWcATmiADTjTxKoY4DKxaG2A5lHyh+NTDAAO3GhGJC8AM0RRiAXyYYAtsWQ5i+EAFE05APQBVACoBhRiAUQEaYjXkBtUGgCeE/QS36TDTECgYA+mEBdgEACi5YACbuoAkwNOhYuAQxAMwADAAidlDOEvikAIwaPn6E6HL0qgC66p41BiUyAnbBoQRpEGGUmCV2cOQcWSAQuExgiDBy+G4gAI6LNF4AdADuEADWEAnocFuYNPwgTUzYmPSKymrXIFASSGjJIN6+HWhdl0wEhBtNgoJkDJReBIwiV0DAhEwoOd7qBehCIB9ou0CCoZEtARBghNwQQEnJyMRsEDukxfDJMAkDCB1IRg [13:54:18] Rj8BU8q0vtivsDAiAgSCwThwpF+XZRHBsDAELQMV4DDEKkVEcjhCA0QQIlFqj9SfzQeCCUTJgY4FAKVSZpcWUhNOy8DKEAhmoj4ks8B5NSEXJSKejMUwFOdNOgNd9amSlLL6HMFrVmXratoLWl9G6QBIZiQEgVDSLQctntnsLnOOqCJQMUJVEA== [13:54:36] or better, a short URL: https://w.wiki/5w2v [14:02:31] ok restarted the import with driver.memory=8g, hopefully it'll pass this time... [14:07:57] dcausse, pfischer_: proposal to tune checkstyle indentation rule: https://gerrit.wikimedia.org/r/c/wikimedia/discovery/discovery-maven-tool-configs/+/854522 [14:17:15] +1 on the case indent, but for arrayInitIndent spotless wants 4 so this might not work [14:17:22] pfischer_: ^ [14:18:12] personally, I don't really care about what rule we use, but I'd like to limit the mismatch between different tools. [14:19:00] We could also update our IntelliJ config https://github.com/wikimedia/wikimedia-discovery-discovery-maven-tool-configs/tree/master/src/editors/intellij [14:29:59] I don't mind either ways as long as the 3 systems we have: checkstyle, spotless and intelliJ are aligned [14:31:46] at the moment it seems that checkstyle is inline with spotless, only intellij disagrees [14:32:21] we could also drop the indentation checks from checkstyle and rely on spotless [14:32:31] https://gerrit.wikimedia.org/r/c/wikimedia/discovery/discovery-maven-tool-configs/+/854522 will align checkstyle with intelliJ but will put spotless in disagreement [14:33:19] if spottless is enforced then some checkstyle rules are useless indeed [14:35:48] I'm not sure that I want to dig into spotless right now. I was planning a quick re-enable some rules in checkstyle and that's it. It's already taking more time than I expected. [14:36:15] so it sounds that updating the IntelliJ config is probably an easy step forward [14:36:25] +1 [14:42:56] well actually caseIndent should still be 4 :) [14:43:51] it's what spotless&intellij wants [14:52:14] And that's the default for checkstyle. Not sure why we diverged from the default previously. [14:52:19] https://gerrit.wikimedia.org/r/c/wikimedia/discovery/discovery-maven-tool-configs/+/854529 [15:05:04] dcausse: another minor review (upgrading the parent pom): https://gerrit.wikimedia.org/r/c/wikimedia/discovery/discovery-parent-pom/+/854530 [16:33:32] dcausse: more pom updates: https://gerrit.wikimedia.org/r/c/search/glent/+/853408 (and previous) [16:57:12] gehel: how do you identify explicit dependencies? [16:57:46] mvn dependencies:analyze [18:57:24] lunch [18:57:33] err, dinner [19:08:57] Is this the right place to ask questions about CirrusSearch? [20:00:00] hexmode: yes :)