[00:18:54] bless our tech gurus for all they do [00:24:27] like unplugging live servers [01:56:30] > Please note that WD does not recommend Green drives for use as System drives. They are primarily intended as secondary data drives. If nothing else, a Green would be rather slow compared to a Blue or Black. [01:56:31] heh [01:57:56] šŸ˜‚ [01:58:08] Paladox do we know who suggested WD Green? [01:58:32] Im trying to figure out how we ended up with em [01:59:28] To beat their ass [01:59:33] i've been told we did it for cheapness [01:59:45] Makes sense [01:59:56] Think from now on we should use a grade upwards from green [02:00:03] Y e a h [02:01:12] Im thinking as soon as everything gets settled we see if theres a way can exchange em towards something else [02:01:21] we need to replace cloud12/10 urgently tho [02:01:40] they are using very outdated ilos which makes it very hard for us to connect to the remote console [02:01:49] Oh yay [02:01:57] i'm having to use a insecure java on my mac just to be able to use it [02:02:15] Paladox if you can price me an upgraded SSD setup for bare min, Iā€™ll see what I can pull off [02:02:33] sure, i'll work with void and see what we can come up with [02:03:07] I know rams an issue but I want off those SSDs lol [02:03:15] rams less of an issue [02:03:21] we have 128g on cloud13/14 [02:03:37] i'll like to leave like 15-20g per server for like a db [02:04:04] but other then that we have enough for 10-20g expansion of existing vms or creating another vm [02:04:30] disk is a pressing issue. Cloud14 is slow asf [02:04:39] like it has a disk issue and i don't know why. [02:04:57] any kind of i/o from one of the vms causes issues for other vms. [02:05:07] Have we ran a disk diagnostic on it by chance? [02:05:29] I think we did if my memory is correct, don't think we identified anything. [02:05:36] That worries me [02:06:01] possibly the wiring or what ever connection it uses [02:06:13] Is it safe to assume its another WD green šŸ˜… [02:06:33] i think so but i carn't be 100%. My network rn is ipv4 only so i cannot connect rn to check [02:06:47] We probably should find a way to fix that [02:07:01] Another issue to add to my list to look into [02:09:01] oh i have the asset check list [02:09:04] yeh it's wd green [02:09:18] Oh lord im gonna just go cry [02:09:27] we also have kingston? [02:09:45] Kingston, odd choice but ill take it [02:09:50] i wonder if we used kingston last year and had to use the wd green ones we installed just a bit before the incident [02:10:12] Oh maybe [02:10:49] tbh i love the dell servers [02:11:08] lifetime idrac license (not having to pay yearly to access the most basic features you need) [02:11:19] i also love that they use html5 console [02:11:24] Isnt that because dell makes IDRAC? [02:11:37] yeh but hp makes ilo [02:11:54] they just have different sub models. [02:12:01] Well.. HP is HP [02:14:35] Buy Samsung, their SSD are really cheap now [02:16:23] Mm [02:16:39] I was thinking either Seagate or something [02:17:01] Newer Seagate are really bad [02:17:48] Samsung SSD are top tier these days and their price is plummeted really hard [02:18:48] Idk it just depends on what they price it at, on what if Iā€™ll even be able to even buy it [02:22:51] [1/2] https://www.newegg.com/global/uk-en/samsung-500gb-870-evo-series/p/N82E16820147792 [02:22:52] [2/2] Not sure if this price included shipping [02:23:49] I rather price what we would need, and newegg aint really where Iā€™d prefer to buy server hardware [02:24:39] I thought MH server use same interface as regular computer? [02:24:57] It probably does [02:25:06] But I rather not buy server hardware from newegg [02:25:28] Oh, I have no idea how ppl buy server hardware from the UK [02:25:44] I dont trust newegg to not send out a ā€œrefurbishedā€ piece of junk [02:25:58] Iā€™d likely just buy it in the US and have it shipped to the UK [02:29:34] Also the Kingston SATA SSD are mostly garbage tier [02:30:12] the kingston we have has been running as long as we have had the server im pretty sure [02:30:18] So cant be that terrible [02:30:39] They certainly have a better track record than the WD greens [02:30:56] Kingston used to be my go-to for inexpensive SSDs, Samsungs are now my default buy. [02:31:01] I asked paladox to price me out some SSDā€™s and Iā€™ll pay for what I can lol [02:31:02] But def understand y'all are on a limited budget. [02:31:10] Though, I suspect part of our problem may be controller related (at least with cloud11) [02:31:30] I'm thinking the first big budgeting project for the new board should be disk upgrades :) [02:31:44] Yea, I know that but the Samsung SSD price go down so hard so it's the only chance [02:31:56] Im gonna hopefully be able to afford enough for bare min, so that should help [02:35:36] Those for rarely writes only, not constant 24/7 like servers [02:37:17] If I have a chart of SSDs available to buy, it should be easier to choose [02:37:30] https://www.disctech.com/Samsung-PM983-MZ-QLB9600-960GB-U2-PCIe-NVMe-Solid-State-Drive?gad_source=1&gclid=Cj0KCQiApOyqBhDlARIsAGfnyMqVdAw9GxgETGrNu7f5sNLBbbvqnzrlK6Bzjm9rYu1diHftXINp0yAaAqxnEALw_wcB [02:38:00] https://www.disctech.com/Samsung-MZ-7LH24000-MZ7LH240HAHQ-00005-240GB-SFF-SATA-RI-1-3-DWPD-Solid-State-Drive?gad_source=1&gclid=Cj0KCQiApOyqBhDlARIsAGfnyMp1ZPYEyjGLC1fYNFA8nZBM7ODw7t67HKykzni1tRsAHg54Gja-T4UaAi6sEALw_wcB [02:38:02] Well thats a price [02:38:10] PM drives are OEM drives, should be identical to the retail one but the warranty is depend on seller [02:38:33] the later one is cheaper and would be the one for cloud11. [02:38:43] That's 2TB and I would consider that as really cheap [02:38:56] the 1 and 2TB are the best option if you can buy [02:39:44] 500GB is really expensive per GB comparing to the other 2 [02:39:48] > SSD Endurance DWPD: Enterprise Read Intensive [02:39:51] exactly what we want [02:39:57] enterprise graded ssds [02:40:19] Let me look up those models and do some price shopping [02:40:21] wait, isn't that NVMe drive? [02:41:27] i think the first is [02:41:33] the 2nd doesn't look like it? [02:41:59] 2nd is SATA [02:42:07] NVMe are extremely cheap per GB [02:42:18] yeh, we're running old hardware [02:42:25] that being a old server shell [02:42:33] lol, the image on the first is a 2.5'' 900GB SSD, but the description is definitely not that [02:42:52] cloud13/14 use DELL R620 [02:43:02] So at those prices I would only be able to reliably get 2 of the terabyte ones [02:43:07] cloud12/11 use HP DL360 GEN 7 [02:43:30] yea, unless your server don't have NVMe [02:43:56] What are current ones [02:44:12] Are they NVMe or SATA? [02:45:17] [1/9] > root@cloud14:/home/paladox# pveperf [02:45:17] [2/9] > CPU BOGOMIPS: 224076.40 [02:45:17] [3/9] > REGEX/SECOND: 1706476 [02:45:18] [4/9] > HD SIZE: 29.36 GB (/dev/mapper/pve-root) [02:45:18] [5/9] > BUFFERED READS: 31.10 MB/sec [02:45:18] [6/9] > AVERAGE SEEK TIME: 58.51 ms [02:45:19] [7/9] > FSYNCS/SECOND: 1245.01 [02:45:19] [8/9] > DNS EXT: 4482.75 ms [02:45:19] [9/9] > DNS INT: 8.26 ms (miraheze.org) [02:45:20] [1/9] > root@cloud13:/home/paladox# pveperf [02:45:20] [2/9] > CPU BOGOMIPS: 224080.40 [02:45:20] [3/9] > REGEX/SECOND: 1508235 [02:45:20] [4/9] > HD SIZE: 29.36 GB (/dev/mapper/pve-root) [02:45:21] [5/9] > BUFFERED READS: 188.26 MB/sec [02:45:21] [6/9] > AVERAGE SEEK TIME: 0.69 ms [02:45:22] [7/9] > FSYNCS/SECOND: 3045.88 [02:45:22] [8/9] > DNS EXT: 13.68 ms [02:45:23] [9/9] > DNS INT: 7.86 ms (miraheze.org) [02:46:01] yeh sata [02:46:09] we haven't got any nvmes as far as i understand [02:46:14] Oh ok [02:46:19] Let me look for sata then [02:46:41] you can buy NVMe stick then use PCIe mount to install that stick [02:47:50] Holy hell SATA is expensive [02:48:04] Yea, no one manufacture SATA these days [02:48:17] Besides the cheapest brand (and not very reliable) [02:48:50] https://www.newegg.com/p/N82E16820147852 [02:48:54] wonder if that would do [02:49:20] then we'd need like either 960g/1tb for the other cloud servers [02:49:25] like i guess 3/4 x that [02:49:28] https://www.samsung.com/us/business/computing/memory-storage/enterprise-solid-state-drives/pm893-2-5-sata-1-92tb-mz-7l31t900/ isnt bad [02:49:30] or 2 x 2 tb [02:49:41] holy fuck [02:50:44] Lol [02:51:02] Are you ok? [02:51:31] Sometimes its cheaper to buy direct from manufactor [02:52:28] yeh, high price but it'll be worth it i guess [02:52:42] Idk what it shows for you but for me its 160-something [02:53:02] For 2TB [02:53:11] yeh anything >100 dollars is expensive for me :ultraseeth: [02:53:29] Huh, I got a 4tb samsung drive in my laptop, and I could have sworn it was cheaper than that [02:53:29] My rent is like 3x that šŸ˜‚ [02:53:37] Is it Sata? [02:53:45] yeh [02:53:58] These are also enterprise grade too [02:54:15] They are made to be abused [02:54:22] Not sure you can get 4tb NVMe (certainly not when I got mine) [02:55:05] Iā€™m just curious how much theyā€™d charge me to ship it to the DC [02:55:06] i've got a 1tb one in my mac [02:55:37] i wonder if we can buy it in the UK [02:55:41] Cause I rather pay for it in USD, so my bank dont moan [02:55:46] if we can get it in the UK, would be signficantly cheaper [02:55:56] cuz no postage for international delivery etc [02:56:27] Yeah but the conversion rate [02:56:35] I wonder if its more expensive in the UK to begin with [02:56:52] yeh me too [02:57:28] Beware of "enterprise grade" [02:57:57] Some are literally same as regular retail drives [02:59:56] I dont even see it on their uk site [03:01:38] Sadly these "really cheap parts" only appears in certain countries [03:02:03] US and near China are the 2 biggest market for these [03:03:58] https://www.pcmag.com/picks/the-best-cheap-ssds [03:04:06] ā€¦ [03:04:12] Thats not helpful [03:04:21] Not for enterprise uses [03:04:48] oh [03:08:28] [1/2] I guess this is fine for SATA 2TB? [03:08:28] [2/2] https://www.amazon.com/SAMSUNG-Inch-Internal-MZ-77E1T0B-AM/dp/B08QB93S6R?sbo=RZvfv%2F%2FHxDF%2BO5021pAnSA%3D%3D [03:12:07] seems that's for like a laptop or computer (not server)? [03:12:34] As I said above, Samsung "enterprise grade" are retail one sold for OEMs [03:12:54] PM981 = 970 EVO [03:14:23] PM881 = 870 EVO [03:15:07] [1/2] https://www.amazon.com/Samsung-1-92TB-PM883-SATA6-MZ7LH1T9HMLT-00005/dp/B07GD3GKGV/ref=sr_1_7?crid=3EHNXGELFREI8&keywords=enterprise+ssd&qid=1700536439&sprefix=enterprise+ssd%2Caps%2C166&sr=8-7 [03:15:07] [2/2] https://www.amazon.com/Samsung-250GB-Internal-MZ-76E250B-AM/dp/B07864WMK8/ref=sr_1_5?keywords=server+ssd&qid=1700536346&sr=8-5 [03:16:10] PM871 = 860 EVO I guess? [03:18:59] [1/23] At the Cheapest options on amazon here are the specs: [03:18:59] [2/23] PM883: [03:18:59] [3/23] Digital Storage Capacity 1920 GB [03:19:00] [4/23] Hard Disk Interface Serial ATA [03:19:00] [5/23] Connectivity Technology SATA [03:19:00] [6/23] Brand SAMSUNG [03:19:00] [7/23] Special Feature Portable [03:19:01] [8/23] Hard Disk Form Factor 2.5 Inches [03:19:01] [9/23] Hard Disk Description Solid State Drive [03:19:01] [10/23] Compatible Devices Desktop [03:19:02] [11/23] Color Black [03:19:02] [12/23] Hard Disk Size 1.92 TB [03:19:02] [13/23] 860 EVO: [03:19:03] [14/23] Digital Storage Capacity 250 GB [03:19:03] [15/23] Hard Disk Interface ATA-4 [03:19:04] [16/23] Connectivity Technology SATA [03:19:04] [17/23] Brand SAMSUNG [03:19:05] [18/23] Special Feature Data Recovery Service [03:19:05] [19/23] Hard Disk Form Factor 2.5 Inches [03:19:06] [20/23] Hard Disk Description Solid State Hard Drive [03:19:06] [21/23] Compatible Devices windows, mac [03:19:07] [22/23] Installation Type Internal Hard Drive [03:19:07] [23/23] Color Black [03:19:54] [1/2] That tells nothing, really. [03:19:54] [2/2] You need to look at what controller and what flash chip they use. [03:20:11] Usually Samsung has the biggest database [03:21:32] https://www.techpowerup.com/ssd-specs/samsung-860-evo-500-gb.d7 [03:24:09] I cant find anything like that but I found this comparison table: https://www.amazon.com/Samsung-250GB-Internal-MZ-76E250B-AM/dp/B07864WMK8/ref=sr_1_5?keywords=server+ssd&qid=1700536346&sr=8-5#:~:text=Compare%20with%20similar%20items [03:24:32] Paladox my budget rn is roughly 260, I can go up a few dollars if need be (whatever shipping cost is fine dont include it in budget) [03:25:15] I should be able to go higher around dec 1st [03:25:17] Sadly most sell webpages only focus on selling. They don't tell you the deep specs that makes what SSD worth. [03:26:14] ^Only below Intel 3DXPoint [03:26:33] [1/2] found one for the 860 EVO: [03:26:33] [2/2] https://www.disctech.com/Samsung-860-EVO-MZ-76E250B-AM-250GB-SED-SATA-Solid-State-Drive [03:26:36] Side note if you can find a place that will allow me to do multiple payments instead of all up front the budget may stretch farther depending [03:27:17] oh [03:28:04] Ive spent most of this paycheck already otherwise itā€™d be higher lol [03:29:21] SSDs going crap wasnt in my budget lol [03:29:39] Russian SSD Boss fight: https://3dnews.ru/938764/resursnie-ispitaniya-ssd-obnovlyaemiy-material [03:30:18] also found a not quite as deep specs for the PM883: https://www.disctech.com/Samsung-MZ7LH1T9HMLT-1.92TB-SATA-SSD [03:33:53] Deep specs should include the controller and the flash chip, the "SSD tier list" and Techpowerup page is the only one in English I can find. [03:34:24] You can go as far as the Chinese flash chips in HP SSD [03:34:32] the 860 says: NAND Flash Samsung V-NAND 3bit MLC [03:34:41] 3bit = TLC [03:35:02] MLC is just more than 2, another marketing trick [03:35:18] Cant I just buy a ton of flash drives /s [03:35:30] it also says: Controller Samsung MJX Controller [03:35:48] No, you buy the after-warranty TBW, not the only instantly die when reaches TBW [03:36:04] Usually the cheapest one will die fastest [03:36:51] I wonder if my state has any decent drives from decomā€™d servers I could buy [03:37:15] Yea, you could get some from them but beware [03:37:40] I got a 200TBW Samsung 970 for dirt cheap and it still working these days [03:37:41] My dad runs one of the decomā€™d servers from them, its been running 5 years strong [03:37:48] couldnt you use a RAID instead [03:37:52] No [03:37:53] God no [03:37:55] No [03:38:23] several of the disks are in a RAID configuration [03:38:57] I dislike raid Iā€™ve gotten screwed over once by raid [03:39:16] Same, never use RAID again [03:39:20] we don't use a raid for the hdd on cloud11 [03:39:24] we use a raid for the ssd [03:39:41] on cloud12 i think we use a raid for the hdd? i don't remember [03:39:43] thats probably why it died [03:39:53] RAID is probably the only reason we didn't have total data loss when db141 got corrupted back when [03:39:55] No they are just shit drives [03:40:11] Or there's a controller problem [03:40:29] I doubt that [03:40:38] We have to cold boot the server every time otherwise most drives won't show up and get marked as failed. [03:40:44] tbf there's something funky with it [03:40:58] you have to use cold boot in order for the disks to show or to even boot the os [03:41:10] if you don't it'll just say failed to load and do a loop of reboots [03:41:12] why couln't you use usb flash drives, based on what ive read it is one of the best alternatives. [03:41:16] I dont miss our cloud infra lol [03:41:24] Cause thats terrible r/w [03:41:26] My decommissioned SSD drive that doesn't care about warranty [03:42:29] Hell, the health status is like >5x their warranty, crazy! [03:43:01] I actually have 2 HDDs in my desktop that have survived for 10 years [03:43:11] Granted my desktop doesnt work for unrelated reasons [03:44:34] But yeah depending on shipping I could probably manage to figure out how to get 2 of those samsung drives I linked a while ago [03:45:04] [1/2] You are lucky to have CMR drive. [03:45:05] [2/2] These days HDDs are SMR and can have bad sectors if not careful. [03:45:22] I think one of them may actually be smr [03:50:17] Those are dirt cheap flash with the worst TBW [03:52:25] Mmm, brings back memories of some ancient 80?GB HDDs my college used for sample disk forensics. Fun times [03:52:52] Why cant we just be able to only need 80gbs I could buy like hundreds of those šŸ˜‚ [03:54:27] I don't even know if you can buy those anymore. And if you could, you might encounter the problem one of my classmates did, which was that the disk was corrupted in such a way that it kept reading indefinitely. [03:55:54] You can still buy parts for Apple IIs, im sure i can find a few 80GB hdds [04:03:15] brb, gonna go buy some floppy disks [04:05:37] buy tape drivesšŸ‘Œ [04:05:47] I meanā€¦ if it will work [04:05:57] Just dont ask me to set up the dial up [04:06:36] i remember dsl [04:06:43] i also remember when aol did broadband here [04:07:41] lets buy some Punch Cards while were at itšŸ¤£ [04:09:02] Friend of mine has an original windows 95 laptop [04:10:25] my family still has a floppy disk reader, although a few months ago we decided to break all the floppy disks. [04:38:05] Those are actually making a return as dense cold storage [04:38:42] yea, they are designed as cold storage [04:38:53] but the tape reader is too expensive [15:28:34] @paladox heads up, enwiki beta has parsoid by default available [17:02:14] ok [17:02:41] it looks to be going well, we just need ext testing for 1.41 [20:10:18] @paladox @orduin disks should be at the DC tomorrow, got picked up this morning [20:10:31] Thanks! [20:56:00] @paladox: you are amazing! [20:56:36] well void did a lot of the work (in so far as getting the cloud server up and the lvm done) [20:57:37] @orduin: youre also amazing! [20:58:06] * HolidayRhino is rhinosf1 and will be back on 5th December. No access to discord til then. [20:58:51] šŸ‘‹ [21:00:21] will an announcement be made too? [21:11:25] Huh, didnā€™t know thatā€™s an IRC feature [21:12:17] Wait are you saying no one will be able to access discord, or just the relay bot? [21:12:30] Just him [21:17:30] Just me [21:23:03] Just us holiday rhinoceri in this channel [21:23:40] (just finished my workday and am now on 6 days of holiday) [21:37:53] Ok, I was confused because of the way the message was worded.