[03:17:44] <강철> Hello, I work WMKR [03:17:44] <강철> Is there a tool that can be bundled into xml and pdf pages and exported from wiki? [03:17:45] <강철> e.g. (1.xml, 2.xml .... 1000.xml) or zip file [03:17:47] <강철> e.g. (1.pdf, 2.pdf .... 1000.pdf) or zip file [09:07:53] 강철: I don't know any, did you have a look in toolhub? https://toolhub.wikimedia.org/ [09:08:21] <강철> I already check it. thank you [09:21:20] With friendly support by Andrew, I was able to set up a Trove Postgres instance for pm20database project and to enable root access in order to create databases and users. However, I cannot figure out how to access as root: Which tool (ssh, psql, ...)? Which host? Which username (root, postgres, my own)? [09:46:39] j. have you followed https://wikitech.wikimedia.org/wiki/Help:Trove_database_user_guide#Enable_root_access ? [09:49:07] I think I have. However, I don't get any respone from ssh or psql. Is access restricted to a certain network? [09:50:07] you can only access from within cloud vps (a VM, or toolforge) [09:50:51] OH - thanks, I'll try [10:19:34] That worked, thanks. I could create a database. But how can I access it with an standard tool (e.g. pgAdmin) to load and edit data? [10:28:05] j.: you can use a SSH tunnel. pgAdmin should support it, see https://www.enterprisedb.com/blog/ssh-tunneling-pgadmin-4 [10:29:56] you can also create a tunnel from the command line and use it with pgAdmin or other tools [10:31:45] OK - this is quite a stack of layers ... Thank you very much for pointing this out. [10:38:55] Could it alternatively work, and not considered misuse, to install phppgadmin for this database as a toolforge project? That could spare non-tech project members a rather challenging setup for contributing to the database. [10:59:56] hmm that's an interesting idea. I don't think it's misuse, but I would like to here the opinion of some other toolforge admins :) [11:00:08] s/here/hear/ [12:43:45] !log taavi@tools-bastion-12 tools.wikibugs toolforge jobs restart irc [12:43:48] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL [12:44:02] bd808: ^ unfortunately it seems like I cursed myself with the barnstar I sent to you earlier today [13:01:35] !log paws use upstream jupyter-rsession-proxy T360800 [13:01:38] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [13:01:38] T360800: Update jupyter-rsession-proxy - https://phabricator.wikimedia.org/T360800 [14:18:00] dhinus: that's a webUI that provides a direct query interface to postgres? That seems like a security disaster (we have always forbidden phpmyadmin which seems equivalent) [14:26:11] andrewbogott: yep that would be it, but with access limited to a single Trove instance I imagine [14:26:42] yeah, not as dangerous but still discouraged. [14:27:17] I don't have personal experience with it but I know that any time we're faced with a public or web-based db query interface we shut it down. [14:27:24] I agree there are some security risks, but I'm not sure how big. also, do we have something related in our policies? [14:39:58] It would not be a public interface, but used only by very selected members of the project group. Probably access to the web project/GUI could require basic authentification (in addtion to what phppgadmin provides). Would that mitigate the security objections? [14:41:59] in this case 'public' doesn't refer to it having a login or not, it refers to it being publicly addressable on the internet [14:43:21] dhinus: I don't know if there's a specific policy, I've looked a bit but not everywhere. [14:52:08] !log anticomposite@tools-sgebastion-10 tools.stewardbots SULWatcher/manage.sh restart # SULWatchers disconnected [14:52:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [14:57:31] !log anticomposite@tools-sgebastion-10 tools.stewardbots ./stewardbots/StewardBot/manage.sh restart # RC reader not reading RC [14:57:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [15:03:04] taavi: well poop. What was the signal that the bot needed a restart? I snuck in a change yesterday that might make the bot seem broken when it isn't. I added a backoff when polling for Phabricator changes that inserts a sleep after an empty response. That backoff varies from 1.5s to 31s depending on how long it has been since a Phabricator event was seen. [15:03:07] https://gitlab.wikimedia.org/toolforge-repos/wikibugs2/-/commit/a2340dda555c9b3946606dff6ba25a78bd9f5b02 [15:04:37] bd808: it stopped relaying any changes, both from gerrit and phab.. [15:05:09] @j.: If you put some authentication requirement on getting access to the phppgadmin UI you would probably be fine. The main thing to worry about is random internet people being able to use the database. [15:09:35] taavi: I started work on T360860 yesterday. I have an idea that would build on the new webservice that task will create to replace Redis in the tool. It is still my hunch that the Redis polling loop is what breaks these days rather than the IRC output. [15:09:35] T360860: Reimagine channel configuration (re)loading to avoid need for git pull - https://phabricator.wikimedia.org/T360860 [15:11:44] Wikibugs only uses Redis as a distributed work queue. The gerrit and phorge jobs stick stuff in Redis that the irc job later pulls out and processes. I'm thinking about replacing that with a couple of webservice URLs and an in-memory queue there. Maybe with websockets in the mix if I decide to be fancy. [15:16:32] bd808: hmm. with the znc bouncer now in front of the IRC connection anyway, why not have the individual gerrit and phab containers connect to the bouncer directly? [15:18:49] taavi: that could be done, but it would add an irc client to each job which is kind of messy from a separation of concerns/DRY point of view. The IRC bot core is not light or easily understood either. [15:20:01] * bd808 is not yet sure he likes the irc3 library generally, but has not been crabby enough to replace it [15:21:33] I've done some projects with https://github.com/jesopo/ircrobots previously and generally like it [15:21:40] Merging the irc client into the webservice would be technically possible too in the future. That would remove a long polling connection to the work queue. [15:26:10] If we decide to always count on having a bouncer in the mix I suppose the irc client could be heavily trimmed. Most of the complexity in a write-only irc bot comes from authn and connection management. [15:35:01] taavi: interestingly the wikibugs-testing deployment has kept running overnight again with no issues. The disconnects seem to be very chaotic. [15:59:03] !log paws upgrade jupyter chart. New cluster T360643 [15:59:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [15:59:07] T360643: update helm chart and jupyterhub - https://phabricator.wikimedia.org/T360643 [18:34:14] !log cloudinfra moved mx records to point to service names (mx-out-a.wmcloud.org and mx-out-b.wmcloud.org) and assoviated those service names with new mx servers mx-out05 and mx-out06 [18:34:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cloudinfra/SAL [19:53:42] cannot SSH to dev.toolforge.org. Has anything chenged recently? [19:54:37] Seems to work fine for me, albeit a bit slow [19:54:48] @Yetkin: yes, https://lists.wikimedia.org/hyperkitty/list/cloud@lists.wikimedia.org/thread/UAMLGQ42CVHLRZ5W2CZBJDJFRNSBT4DC/ The new instance should still allow the same folks to ssh in however. [19:54:51] End up on tools-bastion-12 [19:58:51] /me remembers building https://github.com/yuvipanda/kubessh with the idea of replacing bastions [19:59:54] @yuvipanda: I was talking about the dream of ssh directly to a container as a tool with folks earlier today [20:00:24] Yeah very very doable. [20:01:09] I archived kubessh because i didn’t have time but as it is, i think it would serve most of the use cases i remember bastions serving [20:01:21] Because the primary hard thing was grid engine support [20:01:25] The others on the call wondered if we are close enough to push-to-deploy and remote API access to make it worthwhile at this point to try and build out. All part of the classic problem of which things to focus on first/next. [20:02:08] Thanks. Upgraded PUTTY and tunneled successfully 😃 (re @wmtelegram_bot: @Yetkin: yes, https://lists.wikimedia.org/hyperkitty/list/cloud@lists.wikimedia.org/thread/UAMLGQ42CVHLRZ5W2CZBJDJFRNSBT...) [20:02:14] Getting rid of grid engine does open up new possibilities that were blocked before [20:02:27] Yeah that makes sense. Interactive ssh use is its own helpful paradigm though, that is distinct from just using it as a way to deploy stuff. And the fundamental bastion problem of noisy neighbors can be tackled with something like kubessh [20:03:32] WE mostly squashed the noisy neighbor problem on the bastions with tight per-user quotas and the wheel-of-misfortune process killer script [20:04:32] It is still possible to make things sad by hogging NFS bandwidth, but we have far fewer problems with resource contention than in the 2015-2018 past [20:04:39] Nice!!! [20:05:45] https://github.com/wikimedia/operations-puppet/blob/production/modules/toolforge/files/wmcs_wheel_of_misfortune.py [20:06:39] Hahaha nice [20:06:50] Does sshd have any cgroups support yet? [20:06:57] systemd's support to applying cgroup restrictions on login shells does the quota bits [20:08:27] I think arturo was the one who figured out how to do that once we had systemd everywhere [20:10:03] Nice [20:12:15] https://wikitech.wikimedia.org/wiki/Systemd_resource_control [20:14:26] /me peeks at systemd-cgls and systemd-cgtop just for fun [20:15:11] These should be the current cgroup limits -- https://github.com/wikimedia/operations-puppet/blob/production/modules/profile/files/toolforge/bastion-user-resource-control.conf [20:46:48] tools.superyetkin@tools-bastion-12:~/ybot$ php [20:46:48] -bash: php: command not found [20:46:50] [20:46:51] What does this mean? [20:47:57] You're running on a bastion rather than inside k8s [20:49:17] ? [20:49:33] See https://lists.wikimedia.org/hyperkitty/list/cloud-announce@lists.wikimedia.org/thread/UAMLGQ42CVHLRZ5W2CZBJDJFRNSBT4DC/ [20:50:26] PHP not installed here? [20:50:49] Probably not [20:51:16] why is that? how can I run my scripts there? [20:51:40] For one off, I suspect https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework#Creating_one-off_jobs [20:52:24] this is not a job. I just want to run "any" PHP script interactively [20:54:06] bd808: still around? [20:59:53] RhinosF1: yup. what's up? [21:01:39] @Yetkin: you can either use the older bastion at login.toolforge.org, or more modernly you can become a tool and run your php script on the Kubernetes backend by doing something like `webservice php8.2 shell` and then running things manually there. [21:02:01] bd808: answering that question [21:03:08] We are experimenting with "thin" bastions now that grid engine is gone. Folks were never meant to use the bastions to run much code, and we have a chance to make that more difficult to accidentally or purposefully ignore now. [21:05:55] Chances are good that login.toolforge.org will become "thin" soon as well, but after some discussion earlier today we are likely to keep a "fat" bastion at login-buster.toolforge.org until we can decide on T360818 or another solution that will still let folks use language specific tools as part of managing their Kubernetes jobs. [21:06:02] my script gets "killed" when I try to run it after webservice php7.4 shell" (re @wmtelegram_bot: @Yetkin: you can either use the older bastion at login.toolforge.org, or more modernly you can become a tool and run you...) [21:06:05] T360818: Consider adding `kubectl`, `webservice`, and `toolforge` binaries to shell container images - https://phabricator.wikimedia.org/T360818 [21:07:15] @Yetkin: Try adding more RAM with the `--mem` cli argument. The default is 512M. [21:07:25] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes#Quotas_and_Resources [21:09:45] —mem 1G does not work either [21:10:55] @Yetkin: "Does not work" in what way? I can't see your terminal to debug. [21:11:14] killed (re @wmtelegram_bot: @Yetkin: "Does not work" in what way? I can't see your terminal to debug.) [21:11:37] Then your script is using still more ram than 1GiB [21:12:02] you can turn the number up to 3G I believe... [21:12:53] —mem 4G is also killed [21:13:49] @Yetkin: do you mind sharing what your script is trying to do? That does sound like a lot of RAM for a boring wiki edit. [21:14:34] * bd808 is still waiting for his test shell with `--mem 3G` to start.... [21:14:58] It reads an XML file and parses it using [21:14:59] [21:15:00] $arr = new SimpleXMLElement($str); [21:15:42] it should not take 1 GB of memory, I believe [21:16:36] huh. unless that xml is a wikidump or has a billion laughs attack in it, something "interesting" must be happening [21:17:49] @Yetkin: Your notes here have said `-mem` with only one hyphen. Can you confirm that on the actually command you used 2? `webservice --mem 4G php7.4 shell` [21:20:15] tools.superyetkin@shell-1711400583:~/ybot$ webservice --mem 4G php7.4 shell [21:20:15] bash: webservice: command not found [21:21:00] The"@shell-1711400583" part means you are already inside a Kubernetes container [21:21:34] Currently we don't add the `webservice`, `toolforge`, or `kubectl` commands to the container images. [21:25:16] This command works fine. Thanks so much 😊 (re @wmtelegram_bot: @Yetkin: Your notes here have said `-mem` with only one hyphen. Can you confirm that on the actually command you used 2?...)