[00:30:11] !log tools.rustup Moved shared rustup installation to /data/project/rustup/rustup/ [00:30:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.rustup/SAL [02:22:20] bd808: done [05:34:43] !log tools.apt-browser Switch webservice to use Python 3.9 [05:34:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.apt-browser/SAL [09:03:19] !log testlabs test message - cookbook ran by arturo@endurance [09:03:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Testlabs/SAL [11:16:03] !log admin Adding new OSDs ['cloudcephosd1021.eqiad.wmnet'] to the cluster (T295012) - cookbook ran by arturo@endurance [11:16:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:19:47] !log admin Adding new OSDs ['cloudcephosd1021.eqiad.wmnet'] to the cluster (T295012) - cookbook ran by arturo@endurance [11:19:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:19:51] !log admin Adding OSD cloudcephosd1021.eqiad.wmnet... (1/1) (T295012) - cookbook ran by arturo@endurance [11:19:53] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:20:50] !log admin Rebooting node cloudcephosd1021.eqiad.wmnet - cookbook ran by arturo@endurance [11:20:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:23:44] !log admin Finished rebooting node cloudcephosd1021.eqiad.wmnet - cookbook ran by arturo@endurance [11:23:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:26:44] !log admin Added OSD cloudcephosd1021.eqiad.wmnet... (1/1) (T295012) - cookbook ran by arturo@endurance [11:26:47] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:26:49] !log admin Added 1 new OSDs ['cloudcephosd1021.eqiad.wmnet'] (T295012) - cookbook ran by arturo@endurance [11:26:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [11:41:38] stashbot: ? [11:41:38] See https://wikitech.wikimedia.org/wiki/Tool:Stashbot for help. [12:56:33] !log tools.lexeme-forms deployed d649d7a24a (l10n updates) [12:56:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [14:28:56] dcaro: I'm ready for the osd thing [14:29:17] arturo: ack, give me one minute and I'll be there too [14:29:35] however I have a 30m meeting in 30, not sure if we should wait for after the meeting [14:34:11] arturo: /me here [14:37:50] shall we wait for after the meeting? [14:43:57] sure [14:44:17] keep in mind that it might take an hour to complete though [14:47:53] dcaro: I'm OK running it, but mind that I wont be around for 30 minutes [14:48:08] so most likely will miss some of the debugging [14:48:11] your call! [14:48:31] maybe wait then, you have more network foo [14:49:17] ok [14:49:52] Hello. Not urgent, but I've got a new instance that is stuck in a 'Scheduling' state. https://horizon.wikimedia.org/project/instances/bdfdb9ea-0835-4d2e-9327-0a631c0bcb3e/ [14:49:52] I've deleted and tried re-creating with the same name, but it keeps getting stuck at this point. Is there anything I can do? [14:50:37] that sounds like an andrewbogott question ^ [14:53:01] btullis: recreate but not with the same name [14:53:58] Will do, thanks. [14:55:04] just saying that because we sometimes had issues when an instance was deleted and recreated with the same name shortly after, something I would try [14:57:47] btullis: I will check in a minute; let me know if your retry has the same problem. [14:58:35] andrewbogott: Thanks, the retry went through. I changed the name from puppet-01 to puppetmaster-01 [14:59:00] btullis: great. Is the puppet-01 VM still trying to schedule or did you delete it? [14:59:16] I deleted it, sorry. [15:00:23] no problem -- I'll just ignore this for now but please lmk if you see that same behavior again [15:07:02] Will do. Thanks again. [16:00:19] !log admin Adding new OSDs ['cloudcephosd1022.eqiad.wmnet'] to the cluster (T295012) - cookbook ran by arturo@endurance [16:00:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:00:23] !log admin Adding OSD cloudcephosd1022.eqiad.wmnet... (1/1) (T295012) - cookbook ran by arturo@endurance [16:00:25] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:12:28] !log admin Adding new OSDs ['cloudcephosd1022.eqiad.wmnet'] to the cluster (T295012) - cookbook ran by arturo@endurance [16:12:31] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:12:32] !log admin Adding OSD cloudcephosd1022.eqiad.wmnet... (1/1) (T295012) - cookbook ran by arturo@endurance [16:12:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:13:01] !log admin Rebooting node cloudcephosd1022.eqiad.wmnet - cookbook ran by arturo@endurance [16:13:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:16:01] !log admin Finished rebooting node cloudcephosd1022.eqiad.wmnet - cookbook ran by arturo@endurance [16:16:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:17:42] !log admin Added OSD cloudcephosd1022.eqiad.wmnet... (1/1) (T295012) - cookbook ran by arturo@endurance [16:17:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:17:46] !log admin Added 1 new OSDs ['cloudcephosd1022.eqiad.wmnet'] (T295012) - cookbook ran by arturo@endurance [16:17:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:33:47] !log admin Adding new OSDs ['cloudcephosd1023.eqiad.wmnet'] to the cluster (T295012) - cookbook ran by arturo@endurance [16:33:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:33:51] !log admin Adding OSD cloudcephosd1023.eqiad.wmnet... (1/1) (T295012) - cookbook ran by arturo@endurance [16:33:54] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:34:20] !log admin Rebooting node cloudcephosd1023.eqiad.wmnet - cookbook ran by arturo@endurance [16:34:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:37:20] !log admin Finished rebooting node cloudcephosd1023.eqiad.wmnet - cookbook ran by arturo@endurance [16:37:24] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:39:01] !log admin Added OSD cloudcephosd1023.eqiad.wmnet... (1/1) (T295012) - cookbook ran by arturo@endurance [16:39:05] !log admin Added 1 new OSDs ['cloudcephosd1023.eqiad.wmnet'] (T295012) - cookbook ran by arturo@endurance [16:39:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:39:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:42:58] !log cloudinfra provisioning cloud-cumin-03 as bullseye [16:42:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Cloudinfra/SAL [23:53:10] I kind of touched labweb* (wikitech) but made really sure nothing broke. That was to make it possible that we can first remove font packages from "labtestwikitech" before we do actual wikitech. and then we can be really sure and look at https://labtestwikitech.wikimedia.org without any font packages first [23:59:07] naming conventions are fun :)