[07:58:53] <sebastian-wmse>	 !log wikispeech Deploy latest from Git master: c455310
[07:58:55] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikispeech/SAL
[08:03:03] <sebastian-wmse>	 !log wikispeech Deploy latest from Git master: 2c16169, 84c80a6, 1dea391, 67d3618, 10e4ac5, 043335c (T279620), 1cbfbd8, 29f572a, b7ca5be (T280239, T280258), ac721c8, 8eb2f15 (T133857), d5182d6, 9ddab13 (T279652), 8eebc3d, 36c217f (T280244), dc93055 (T279652), 35abf5c (T277153), f922c2d (T277242), d91b7e8, 898e716 (T279632), 3cbb3b9 (T281262),
[08:03:03] <sebastian-wmse>	 6d90626, 5fd1158, fb1a29b (T280233), f731c76, 9b5c6be, 6e83947, d5bb89d, 26b9ecc (T201491), ef986d4, f94623d, 3831c8f, 0d6ccd8, 03ae1d1, 33f7d49 (T281222), 679f60a (T280258, T281663, T281665), 5d95701 (T280240), 58ac3cc, e9c2396, fa7a71e, 8eddbf9 (T243289), 34f653e, e99a700 (T281667), 807fc3e, d726871 (T282513), 3da7f5d (T281803), d845b51, f29200d
[08:03:04] <sebastian-wmse>	 (T282513), 4f27fc6, 0e6c80d (T163682), 8195214, 84a1d69, 2a1f72d, 384626c, e4a2437, 4d42a8e, 7278639, 891112c (T282151), 056ffda, 0d9e6d1 (T282151), 3aee538, b8692a4, b8aa5f6, 0790eb4, 0dc7294 (T189982), c8ce725, 3eae7d9, fc95d4c, fc20f7e, 05c6748 (T283063), b5dbc8c (T281898), 295eff8, 41d1e42, eab6cb4, e9e3f15, 3d358a8, 16eb450, f9ed69f, 623dedb
[08:03:04] <sebastian-wmse>	 (T284138, T249108), 68a184f, e98d036 (T284765), 5d4c736, 7725a72, e574519, 1a768db (T285083), 40edb58, f3c0d08, 1f69641, 236d5df, 69e7c1c (T275710), 0e6d7c0, 6e631b9, 5f3b260 (T286372), 25e020a (T286198), dd2cb46, 6d616b9, 84652f9, 5a9f362, de8bf62 (T285141), fe6bd14 (T286579), ca15090, 7f49224, 8e3eb9a, 90f5e95, c9fc0b6, 5e57d20, e6a3949, e2c7328,
[08:03:05] <sebastian-wmse>	 ea12b93 (T280592, T280592), c3bd058, facaaa1, 4da6994, a9a1009, 128fcd1, 20ca9b4, 5fc8367, 86d5cf5 (T293043), 798a26f, fa30457, 98288ac, 04408f2, 4be3dfc, 2178cf2 (T286694), e9f10fc, 5b5daa4, 48eb2a4, 4ac2b11 (T293578), 615e70e, e8d7514, 1f95626, 94280ad (T300120), c5191e4, 55a00f8, 99df4c4 (T299994), 8a0d497 (T250045), f8ea4f1 (T201491), 2d6a895,
[08:03:05] <sebastian-wmse>	 ee15d9f, 16869fa, 7f46cdf, 24b89fc, b2b41a0, 3df63f4 (T297688), 44ebacf, 98ed77b (T300104), 7f88874, 22fd3e6, cc651a3, 1d1a30f, df0ce92, 6c761eb, 3ad73cd (T295285), 400ade9, 8de6809 (T300104), 1533f06, 6cda6c4 (T287377), 0ee7dd0, 7896f39 (T285048), bf3f2ec, 221b15d, 6fce00e
[08:03:23] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikispeech/SAL
[08:03:24] <stashbot>	 T280258: Mock mw.Api instead of server - https://phabricator.wikimedia.org/T280258
[08:03:24] <stashbot>	 T281262: Treat Speechoid lexicon id (Pronlex id) as int rather than string - https://phabricator.wikimedia.org/T281262
[08:03:24] <stashbot>	 T277242: Ensure new lexicon edits are CC0 - https://phabricator.wikimedia.org/T277242
[08:03:24] <stashbot>	 T277153: Allow user to decide on updating or adding new entry if entry with same spelling exists - https://phabricator.wikimedia.org/T277153
[08:03:25] <stashbot>	 T280244: Optionally mark utterance files with TTL in Swift - https://phabricator.wikimedia.org/T280244
[08:03:25] <stashbot>	 T279620: i18n for wikispeech-edit-lexicon user group - https://phabricator.wikimedia.org/T279620
[08:03:25] <stashbot>	 T133857: Link to Special:EditLexicon from player - https://phabricator.wikimedia.org/T133857
[08:03:26] <stashbot>	 T280239: Use GET requests for segments and utterances - https://phabricator.wikimedia.org/T280239
[08:03:26] <stashbot>	 T279652: Credit PTS in repo code - https://phabricator.wikimedia.org/T279652
[08:03:27] <stashbot>	 T279632: Implement local lexicon in new wiki namespace - https://phabricator.wikimedia.org/T279632
[08:03:27] <stashbot>	 T281665: Load modules and settings - https://phabricator.wikimedia.org/T281665
[08:03:28] <stashbot>	 T280240: Convert Wikispeech to using abstract schema - https://phabricator.wikimedia.org/T280240
[08:03:28] <stashbot>	 T201491: Fix common typos in code - https://phabricator.wikimedia.org/T201491
[08:03:29] <stashbot>	 T281663: Communication between consumer and producer - https://phabricator.wikimedia.org/T281663
[08:03:29] <stashbot>	 T281222: Many @since tags are in wrong order - https://phabricator.wikimedia.org/T281222
[08:03:30] <stashbot>	 T282513: Config variable for producer mode - https://phabricator.wikimedia.org/T282513
[08:03:30] <stashbot>	 T280233: Control creation of LexiconStorage from config in extension.json - https://phabricator.wikimedia.org/T280233
[08:03:31] <stashbot>	 T281667: Load user options from consumer wiki - https://phabricator.wikimedia.org/T281667
[08:03:31] <stashbot>	 T281803: Disambiguate segments and utterances from different consumers - https://phabricator.wikimedia.org/T281803
[08:03:32] <stashbot>	 T243289: Keyboard shortcut for play/stop doesn't work - https://phabricator.wikimedia.org/T243289
[08:03:32] <stashbot>	 T163682: Feedback when server isn't responding - https://phabricator.wikimedia.org/T163682
[08:03:33] <stashbot>	 T281898: Hook in to status of HAProxy in front of TTS - https://phabricator.wikimedia.org/T281898
[08:03:33] <stashbot>	 T283063: Title highlighting is wrong - https://phabricator.wikimedia.org/T283063
[08:03:34] <stashbot>	 T282151: UI for user preferences - https://phabricator.wikimedia.org/T282151
[08:03:34] <stashbot>	 T189982: Move all apihelp translatable messages to separate files - https://phabricator.wikimedia.org/T189982
[08:03:35] <stashbot>	 T275710: Deprecate IContextSource::getWikiPage and IContextSource::canUseWikiPage - https://phabricator.wikimedia.org/T275710
[08:03:35] <stashbot>	 T286198: Empty node causes Speechoid error - https://phabricator.wikimedia.org/T286198
[08:03:36] <stashbot>	 T286579: Create metrics logging for Wikispeech - https://phabricator.wikimedia.org/T286579
[08:03:36] <stashbot>	 T285083: Introduce class Segments - https://phabricator.wikimedia.org/T285083
[08:03:37] <stashbot>	 T284765: Introduce class Segment - https://phabricator.wikimedia.org/T284765
[08:03:37] <stashbot>	 T284138: Playback stalled if utterance fail to load when prepared by previous utterance starting to play - https://phabricator.wikimedia.org/T284138
[08:03:38] <stashbot>	 T286372: Parse and translate numbers, years and dates to aliases before sending  them to Speechoid - https://phabricator.wikimedia.org/T286372
[08:03:38] <stashbot>	 T285141: Refactor Segmenter - https://phabricator.wikimedia.org/T285141
[08:03:39] <stashbot>	 T249108: Replace trigger() with triggerHandler() in tests - https://phabricator.wikimedia.org/T249108
[08:03:39] <stashbot>	 T293578: Pre render pages - https://phabricator.wikimedia.org/T293578
[08:03:40] <stashbot>	 T280592: Add a `composer phpcs` command to all repos that require mediawiki/mediawiki-codesniffer - https://phabricator.wikimedia.org/T280592
[08:03:40] <stashbot>	 T300120: Failing Jenkins tests for mw.loader - https://phabricator.wikimedia.org/T300120
[08:03:41] <stashbot>	 T293043: Remove MediaWikiTestCase class alias - https://phabricator.wikimedia.org/T293043
[08:03:41] <stashbot>	 T286694: Drop legacy cruft arising from introduction of ResultWrapper - https://phabricator.wikimedia.org/T286694
[08:03:42] <stashbot>	 T299994: Buttons do not show up in the player UI - https://phabricator.wikimedia.org/T299994
[08:03:42] <stashbot>	 T250045: Clean up QUnit test runner for MW (2020) - https://phabricator.wikimedia.org/T250045
[08:03:43] <stashbot>	 T297688: Hard deprecate and remove the deprecated WikiPage instantiation methods - https://phabricator.wikimedia.org/T297688
[08:03:43] <stashbot>	 T285048: Introduce class Utterance - https://phabricator.wikimedia.org/T285048
[08:03:44] <stashbot>	 T295285: Upgrade primary branch of all Wikimedia-deployed repos to a version of mediawiki-tools-phan including T270553 - https://phabricator.wikimedia.org/T295285
[08:03:44] <stashbot>	 T287377: Handle en/ett better in DigitsToSwedishWords  - https://phabricator.wikimedia.org/T287377
[08:03:45] <stashbot>	 T300104: Sort out footer - https://phabricator.wikimedia.org/T300104
[08:04:07] <Lucas_WMDE>	 ._.
[08:18:40] <arturo>	 wow
[08:29:54] <dcaro>	 xd
[08:32:16] <Lucas_WMDE>	 sebastian-wmse: that message got truncated / split btw, most of it didn’t make it into the SAL
[08:33:23] * dcaro Are there any other ways to send messages to SAL? Maybe without the irc limitations? (ex. api)
[08:33:41] <dcaro>	 just wondering though
[08:34:20] <wm-bot>	 I will let you know when I see sebastian-wmse and I will deliver that message to them
[08:34:20] <Lucas_WMDE>	 @notify sebastian-wmse your super-long !log message got split and most of it didn’t end up in the SAL
[08:43:40] <godog>	 dcaro: I'd need to deploy a grafana change to cloudmetrics1003, please LMK when puppet got re-enabled, thanks!
[08:45:41] * dcaro godog: can I just run puppet there? or you need some specific actions taken? (/me just enabled puppet)
[08:47:05] <godog>	 dcaro: no specific actions no, only a puppet run!
[08:47:13] <dcaro>	 👍
[08:48:42] <dcaro>	 done!
[08:50:14] <godog>	 neat, thanks dcaro !
[08:50:52] <godog>	 context is: blocking public access to /metrics on grafana, we get regular security audits mentioning that's a "vulnerability"
[08:51:04] <godog>	 it isn't really, and nothing private, but might as well 
[09:48:44] <arturo>	 !log toolsbeta manually starting gridengine-master.service on toolsbeta-sgegrid-master (T318788)
[09:48:47] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL
[09:48:47] <stashbot>	 T318788: toolsbeta grid is down - https://phabricator.wikimedia.org/T318788
[12:49:44] <wm-bb>	 <bd808> You can edit the appropriate wiki page, but that won’t be reflected in the elasticsearch index that powers sal.wikimedia.org. (re @wmtelegram_bot: <dcaro> Are there any other ways to send messages to SAL? Maybe without the irc limitations? (ex. api))
[12:50:41] <wm-bb>	 <bd808> sal.toolforge.org 🤦
[12:51:28] <dcaro>	 👍
[19:17:02] <wm-bb>	 <harej> Special:MediaStatistics reports that the files for that wiki uses a certain amount of space. Is that just the latest version of each file, or historical versions as well?
[19:29:16] <wm-bb>	 <lucaswerkmeister> is anyone else having emacs issues on toolforge?
[19:29:45] <wm-bb>	 <lucaswerkmeister> if I try to edit a file with emacs, I get an error `Symbol’s value as variable is void: ispell-menu-map-needed`, and it opens the scratch buffer instead of the file
[19:30:09] <wm-bb>	 <lucaswerkmeister> I don’t have a ~/.emacs file in my user, and I think I’ve had it happen in some tool accounts as well, so I don’t think it’s just me…
[19:32:40] <roy649>	 hmmm, I'm not having any problem
[19:33:17] <wm-bb>	 <lucaswerkmeister> weird
[19:33:21] <roy649>	 Could be https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=982756
[19:34:41] <roy649>	 I assume you're running "GNU Emacs 26.1"
[19:34:50] <roy649>	 (run emacs --version)
[19:35:09] <wm-bb>	 <lucaswerkmeister> nope, 27.1
[19:35:21] <wm-bb>	 <lucaswerkmeister> /usr/bin/emacs on tools-sgebastion-10
[19:35:36] <roy649>	 hmmm
[19:35:45] <roy649>	 I'm on tools-sgebastion-11
[19:36:12] <roy649>	 lrwxrwxrwx 1 root root 23 Sep  7  2021 /usr/bin/emacs -> /etc/alternatives/emacs
[19:36:15] <wm-bb>	 <lucaswerkmeister> hm, indeed on toolforge-dev (sgebastion-11) it works, and emacs is 26.1
[19:36:34] <roy649>	 lrwxrwxrwx 1 root root 18 Sep  7  2021 /etc/alternatives/emacs -> /usr/bin/emacs-nox
[19:36:52] <wm-bb>	 <lucaswerkmeister> sgebastion-10 has no emacs-nox at all
[19:36:57] <roy649>	 I thought -10 and -11 were supposed to be identical, but maybe not.
[19:37:00] <wm-bb>	 <lucaswerkmeister> and vice versa -11 has no emacs-gtk
[19:37:06] <wm-bb>	 <lucaswerkmeister> yeah I thought so too, this is quite surprising
[19:37:09] <Platonides>	 harej: it is just current images
[19:37:24] <wm-bb>	 <lucaswerkmeister> they’re both debian 10 (buster)
[19:37:43] <Platonides>	 there is some fake title thing going on, but it seems to be just a SUM( img_size ) of image table
[19:38:44] <wm-bb>	 <lucaswerkmeister> I’ll file a phab task, thanks for pinpointing the issue roy649
[19:38:54] <roy649>	 Well, my suggestion is work on -11 for now and open a phab ticket
[19:41:32] <roy649>	 Always happy to help a fellow emacs user.
[19:41:42] <roy649>	 There's so few of us left, we need to stick together :-)
[19:43:50] <wm-bb>	 <lucaswerkmeister> :D
[19:43:52] <wm-bb>	 <lucaswerkmeister> filed T318858
[19:44:26] <wm-bb>	 <lucaswerkmeister> (or https://phabricator.wikimedia.org/T318858 if we don’t have an autolinking bot here)
[19:48:24] <roy649>	 TIL about namei
[19:51:11] <wm-bb>	 <lucaswerkmeister> it’s neat
[19:51:20] <wm-bb>	 <lucaswerkmeister> though infuriatingly hard to remember the name of
[19:51:34] <wm-bb>	 <lucaswerkmeister> but can be useful to debug permission / path resolution problems
[19:55:36] <roy649>	 Actually, for me it's easy to remember, but I've done some kernel hacking in my day.
[19:55:54] <roy649>	 namei() is the routine in the kernel which maps pathnames into inode numbers
[19:56:07] <roy649>	 I just didn't know there was a command-line version of it.
[19:59:06] <bd808>	 stashbot: hello?
[19:59:06] <stashbot>	 See https://wikitech.wikimedia.org/wiki/Tool:Stashbot for help.
[19:59:40] <bd808>	 oh... task id was from telegram side that's why stashbot didn't respond.
[19:59:41] <Platonides>	 is login-dev.toolforge.org not directly accessible?
[19:59:56] <bd808>	 Platonides: it;s just dev.toolforge.org
[19:59:56] <taavi>	 login-dev?
[20:00:41] <Platonides>	 gah, the task mentions login-dev.toolforge.org, not dev.toolforge.org
[20:00:45] <Platonides>	 and it does have a dns entry
[20:01:15] <Platonides>	 dev.toolforge.org has address 185.15.56.122
[20:01:21] <Platonides>	 login-dev.toolforge.org has address 185.15.56.11
[20:01:29] <taavi>	 .11 is the web proxy wildcard
[20:01:46] <wm-bb>	 <lucaswerkmeister> oops, did I get the host name wrong
[20:02:02] <wm-bb>	 <lucaswerkmeister> yeah dev.toolforge.org is the HostName in my ssh config, sorry bout that
[20:03:19] <taavi>	 for the emacs issue: https://gerrit.wikimedia.org/r/836279
[20:04:21] <wm-bb>	 <lucaswerkmeister> so Debian just ships different Emacs versions for nox and gtk?
[20:04:27] <wm-bb>	 <lucaswerkmeister> I guess I shouldn’t be too surprised
[20:05:56] <taavi>	 it shouldn't, at least it's a single source package
[20:05:56] <wm-bb>	 <lucaswerkmeister> (I remember previously getting annoyed that Buster ships npm and node versions that aren’t compatible)
[20:06:30] <wm-bb>	 <lucaswerkmeister> you’re right, https://packages.debian.org/buster/emacs-gtk and https://packages.debian.org/buster/emacs-nox are both 26.1
[20:06:32] <taavi>	 ah. emacs-gtk is available via buster-backports and -nox isn't
[20:06:38] <taavi>	 why do we install backports by default?
[20:06:43] <wm-bb>	 <lucaswerkmeister> ah
[20:06:46] <bd808>	 #reasons
[20:07:06] <taavi>	 this kind of setup really feels like it's going to break up something important some day
[20:07:29] <bd808>	 it does, and then we fix it
[20:07:38] <bd808>	 everything breaks ;)
[20:07:39] <Platonides>	 buster-backports does have emacs-nox
[20:07:50] <Platonides>	 https://packages.debian.org/buster-backports/emacs-nox
[20:07:56] <wm-bb>	 <lucaswerkmeister> maybe it didn’t whenever one of the servers was provisioned
[20:08:14] <Platonides>	 I guess it's more like backports being preferred when one system was installed but not with the other
[20:09:37] <Platonides>	 just like it choose the gtk version in one and the nox in the other
[20:10:48] <taavi>	 backports being prefered isn't specific to this one host, I've definitely got bit by that before
[20:59:17] <taavi>	 @lucaswerkmeister: the puppet patch was merged, so if you run puppet on the broken host it should fix things
[20:59:47] <wm-bb>	 <lucaswerkmeister> oh right, I should be able to do that shouldn’t I
[21:00:12] <taavi>	 yes!
[21:00:16] <wm-bb>	 <lucaswerkmeister> `sudo run-puppet-agent`?
[21:00:21] <taavi>	 yep
[21:01:10] <wm-bb>	 <lucaswerkmeister> !log tools lucaswerkmeister@tools-sgebastion-10:~$ sudo run-puppet-agent # T318858
[21:01:19] <wm-bb>	 <lucaswerkmeister> oh wait
[21:01:25] <wm-bb>	 <lucaswerkmeister> that won’t work over the bridge
[21:01:26] <taavi>	 although that might be a bit slow on the bastions because that inherits the per-user cpu limiting we have in place, usually I log in directly as root on the bastions because of that
[21:01:46] <wm-bb>	 <lucaswerkmeister> so `sudo -i`? or how does it work
[21:02:02] <wm-bb>	 <lucaswerkmeister> oops, broken packages error
[21:02:31] <wm-bb>	 <lucaswerkmeister> emacs-nox depends on emacs(-bin)-common 1:26.1 but 1:27.1 is to be installed
[21:03:03] <wm-bb>	 <lucaswerkmeister> maybe we need to first absent emacs-gtk and then ensure emacs-nox?
[21:03:33] <taavi>	 no, you can add an ssh key to the root user via horizon hiera
[21:03:37] <lucaswerkmeister>	 ah ok
[21:03:48] <taavi>	 maybe, or you could just remove emacs-gtk manually I guess
[21:03:52] <lucaswerkmeister>	 (puppet is still running)
[21:04:00] <lucaswerkmeister>	 if that’s acceptable then I can also try that, sure
[21:05:52] <lucaswerkmeister>	 do I just edit the YAML in puppet > project puppet > hiera config by hand?
[21:05:57] <lucaswerkmeister>	 (and be careful to keep the syntax intact)
[21:06:00] <lucaswerkmeister>	 or am I missing a better way
[21:07:09] <taavi>	 yes :-/
[21:12:24] <Platonides>	 maybe apt-get install emacs-common=1:26.1
[21:12:27] <Platonides>	 then install emacs-nox
[21:13:54] <Platonides>	 well, =1:26.1+1-3.2+deb10u2
[21:15:20] <lucaswerkmeister>	 !log tools added root SSH key for myself, manually ran puppet on tools-sgebastion-10 to apply it (seemingly successfully)
[21:15:22] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[21:17:33] <lucaswerkmeister>	 I’ll wait for that puppet run to finish and then try manually removing emacs-gtk
[21:17:39] <lucaswerkmeister>	 (with autoremove so it should remove -common too)
[21:21:31] <lucaswerkmeister>	 oh, emacs-gtk is already gone
[21:21:40] <lucaswerkmeister>	 so I guess the puppetized removal of the package didn’t autoremove and that’s the issue
[21:22:50] <lucaswerkmeister>	 !log tools on tools-sgebastion-10: apt remove emacs-common emacs-bin-common # fix package conflict, T318858
[21:22:53] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[21:22:53] <stashbot>	 T318858: login.toolforge.org (tools-sgebastion-10) has broken emacs-gtk installed - https://phabricator.wikimedia.org/T318858
[21:23:32] <lucaswerkmeister>	 !log tools on tools-sgebastion-10: run-puppet-agent # T318858
[21:23:34] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[21:25:01] <lucaswerkmeister>	 taavi: that made the puppet run much faster, thanks
[21:27:53] <lucaswerkmeister>	 and there’s a working emacs on tools-sgebastion-10 now \o/
[21:31:06] <bd808>	 lucaswerkmeister: I generally use either `sudo -i ...` or `sudo su -` to just fork into a root shell without needing to connect with my root key.
[21:31:41] <bd808>	 but whatever works for you works! ;)
[21:31:52] <lucaswerkmeister>	 I’ll probably still do that for one-off things
[21:32:11] <lucaswerkmeister>	 but the puppet run went from ~5 minutes to ~1 minute with the ssh-as-root connection
[21:32:19] <lucaswerkmeister>	 so that definitely seems to have made a difference ^^
[21:33:08] <bd808>	 yeah, it's mostly having /root as your $HOME that makes it so much faster. There are really gross things that puppet does in scanning for ruby classes
[21:35:06] <lucaswerkmeister>	 hm, I thought one of the slow puppet runs was already from sudo -i, so the home/cwd should’ve been /root
[21:35:11] <lucaswerkmeister>	 but maybe not, I’ve closed that terminal already
[23:51:22] <wm-bb>	 <Joko> /metrics@wmtelegram_bot