[10:31:04] kindrobot zeljkof : Hi there! For the integration issue, may we consider a fallback solution if we have difficulties to run the CI? If kindrobot is working on kubernetes and if that would solve our issue with the CI (because we would run the selenium tests against a temporary WikiFunctions server), we just may create a parallel branch to master and once the CI works, we may merge all the selenium tests we would have written all [10:31:05] at once. [10:31:05] My guess is that the selenium tests code we will write should not conflict with the rest of the code, so my guess is that there should not be much drawbacks of maintaining that branch waiting for kubernetes. What do you think? [12:47:41] That sounds reasonable, an alternative is to take the end to end tests out of the gerrit pre-merge checks for now, and add them to the daily job that can access beta-cluster. Do you have a strong opinion? [12:54:05] I prefer a simpler solution. 😅 Merge the test into master but run them in a daily job that only targets the beta cluster. [12:54:54] Yeah, the kindrobot's solution seems wiser to me! [12:55:02] (as zeljkof, the simpler, the better) [14:55:20] OK. I'll work on that today. Thanks zeljkof and florent :) [14:56:26] Alright! [15:40:05] Wait kindrobot zeljkof, as selenium tests are modifying functions, is it a good idea to run selenium tests against the wikifunctions beta platform? [15:41:14] For the test being reviewed, it should have no effect as we just evaluate an echo function: https://phabricator.wikimedia.org/T321932 [15:43:03] But for the other ones, since they change stuffs, shouldn't we rather wait for kubernetes and block the corresponding issues until then? [15:45:00] That's an excellent point florent. Though, we'll ideally want to run these test against staging "beta cluster" and "production" as well as against the ephemeral test environments for pre-merge, so we'd want to make sure that these tests can run against a longstanding environment without being destructive. [15:46:06] So for example, when we write the "create a function" test, we'll probably want to give it a name like "integration-test-function-add-${timestamp}". [15:46:53] And when we want to modify a function in a test, we'll probably want to create that function with a similar name scheme first. [15:46:56] Aren't they ways to duplicate the beta and / or production wikifunctions instances? Or export data and reimporting them? [15:48:06] There are, but we'll want to run e2e tests against production e.g. after a deploy to make sure the deploy was successful. [15:49:25] If the created test functions have a well defined named prefix, we can even clean them up after the tests have passed, and leave them around if they failed for someone to debug them. [15:54:09] So the purpose of running selenium tests against production is to ensure the deployment was successful and every scenarios work as expected? [15:58:00] If so, maybe then we may run the selenium tests right after the deployment while the server is inaccessible publicly, and somehow snapshot the state of the production environment in order to restore it afterwards [15:58:56] Then we would limit the risk to destroy anything in production [15:59:23] (my fear is that some tests are badly written and destroy some function by accident) [15:59:39] or alter it in a bad way [16:08:45] Yeah, it's a valid fear. [16:09:13] But we' probably getting ahead of ourselves. Given that we don't have a production yet. ;) [16:09:49] Are you comfortable with the idea of using prefixed-named functions on beta-cluster? [16:17:13] Less incomfortable as long as we don't run against production but only beta and we use the prefixed-name functions trick. [16:20:16] 2 questions though: [16:20:16] - are we able to rollback changes on an entity (objects, functions, implementations, tests, ...) currently? or is there anything planned for that? [16:20:16] - is this assumed the beta can somehow be lost or corrupted? I could see that on the notwikilambda first page, but AFAICT nothing about this on the homepage of the beta wikifunctions website [16:22:14] You are able to run it back in the sense, it's just stored as the ZObject on-wiki, and it has the same revisions/versioning as a Wikipedia article. I don't think there's a nice UI for it yet. [16:23:09] (That being said, the corruption can occur because of the contribution rules of the wikis…) [16:23:25] Yeah, so it's fine for me [16:24:56] I also don't think we guarantee the "safety" of data on beta-cluster. We _might_ take database backups, but most likely if it got corrupted, we'd just blow it away and re-seed it with the same stuff we use to seed local development environments. [16:25:40] Fair point