PSA: Beware of HDD manufacturers submarining SMR technology in HDD's without any public mention

it’s literally the first thing they adress in the product pamphlet… i suppose it’s not in the datasheet because all the drives have this and just like they don’t tell you they are spinning disks nor that they are using fluid bearings because it’s the same across all the drives

so yeah all the drive models listed in this should be CMR
ofc if the model name is even remotely different then this pamphlet isn’t valid and one has to dig deeper… but it does seem very likely that they aim to keep atleast this product series on only CMR atleast for that generation of their drives.

for most manufacturers they have a sort of syntax used in the model serial so that one can literally read exactly what features it has, ofc that requires one being familiar with all or most of the designations / abbreviation

But this is about 7E8 drives, not X16.

7E8 does have clear spec. that it is CMR.

ah yeah you are right…
checked this - and no mention of CMR, SMR Shingled nor Conventional
ofc it also seems that seagate sells those by the pallet load… xD

it’s possible that the entire EXO series is CMR and thus they think it’s already been covered when they stated in for the baseline model… sometimes understanding these kinds of tech sheets requires a lot of work, even if they are suppose to make life simpler…

maybe they are not use to not selling them by the pallet and thus people/datacenters usually know exactly what they are getting…
and the seagate rep just assume you where lost and guided you to the consumer grade gear… but i duno…

1 Like

I’ll say it again. It’s CMR. Or if you want to be precise PMR+TDMR. Tracks are less wide and the read head uses an array to prevent interference from neighboring tracks. All the trickery is on the read end, NOT the write end. And this technology is said to make performance a bit better even.

I don’t have problems with it, because there are no problems with it. Seagate gave you a weird response. 9 times out of 10 these support people work with the same publicly available fact sheets. They didn’t see the CMR mention and thus they didn’t mention the X16 to you. Doesn’t change the fact that it absolutely is a CMR drive. (Though there is very little conventional about using helium filled drives with an array of read heads… the terminology is weird)

2 Likes

I am indeed pretty surprised with this unit’s performance. Copying my blobs (from an NVMe SSD) was a charm.

Use the term non-SMR instead, since that’s about all that matters :wink:

2 Likes

Could this be fixed by creating a write-through-path on a SSD or some faster HDD?

Hello and welcome @SebastiaanFX! :wave:

Well, I do not know what a “write-through-path” is. If it’s a temporary caching system before data gets written down to disk, I guess it would only delay the moment of failure, unless the cache has the size of the storage disk which wouldn’t make much sense…

I can’t think of any solution, instead of throttling down writes whenever the disk is stalling. Which isn’t what --storage2.max-concurrent-requests does (it is throttling all the time), but at least an SMR disk is usable this way. Or… no to use SMR disks all together.

1 Like

by far the easiest solution is to create multiple nodes on multiple smr or other drives… then the load is shared amongst the drives and like say that you have 3 nodes with 1 hdd each… then the load on each hdd is only 1/3 of what one would be… so much easier to smr to keep up with writing the data… it really becomes and issue about random writes… caching also helps… but the real issue isn’t solved, each smr hdd usually already come with 256mb cache for the same reasons…

but no matter the cache it doesn’t change the smr has poor write speeds… thus multiple drives sharing the load is the safe bet solution…

and then ofc move your database to an ssd or such… SMR hdd’s will be very stressed by the DB files i would assume.

2 Likes

So does nearly every modern non-SMR drive, so no, this is not for that reason.

Also, most SMR drives feature a on-disk CMR cache. For that reason.

actually the drive heads can write both types of tracks and thus as the disk gets filled this feature supposedly diminish in effectiveness.

basically what the drive supposedly does, is to write the tracks as CMR and then rewrite them as SMR later to conserve space…

so this is not really an advantage in a 24/7 operational state but only for burst data transfer and longer idle periods to sort and rewrite the data…

infact this sort of setup will just make the SMR drive even worse performing under continual load, which is why it’s important to get 24/7 rated drives, rather than consumer type drives…

SMR works fine so long as one doesn’t go over the limit in what they can write … if one does go over writes go down to near floppy speeds… xD

1 Like

Nope, it does not. It writes to a dedicated CMR cache, than transfers this cache’s contents over to an SMR section of the drive.

So no, this feature does not diminish once the drive gets filled - the CMR always stays fast - it will just take more time to flush it, once the SMR segments become fuller.

Yeah, as every cache is.

This has nothing to do with each other, there are both consumer and enterprise drives rated for 24/7 use and there are both consumer and enterprise drives that have write load ratings (which might also affect warranty). tl;dr: a drive rated for 24/7 is not automatically rated for permanent write loads.

well there you are pretty in deep to the product sheets because i cannot remember having seen any drives having stated that they was rated for continual write.

its usually just 24/7 or not from what i’ve seen…

well there are like 5-7 different SMR technologies on the market… so if one really wants to dig into the details then one will need to start to sketch out exactly what is being talked about … and it’s also a high focus point for manufactures, so i’m sure with every new product series there will be added new advancements / solutions to their hdd’s

the SMR writing CMR is to my understanding done by the same heads on the same platters…
ofc dedicating segments of the platters to only CMR caching would solve some issue, so i’m sure that if they didn’t do that from the beginning, it would end up like that…

long story short… stay the hell away from SMR if one has the choice… and buy enterprise drives if the option is there for not much added costs :smiley:
i’m sure we all can agree on that…

its all cost vs benefit.

1 Like

Check out this screenshot from the dreaded Exos 5E8 datasheet:


It is only rated for 180TB/a, but is certified to run 24/7 (see the column below what I highlighted)

It is, but IN A DEDICATED AREA that will only be used as as CMR zone for caching. Comparable to SLC caching in modern SSDs.

1 Like

Buy a drive that is rated for use with video surveillance - that is 24/7 and continuous random writes (would be sequential if the system had only one camera, but with multiple cameras writing at the same time the writes are random).

2 Likes

from my understanding the throughput rating has to do with the warranty… it’s not related to SMR and has been present long before SMR was a thing… basically it states that if you push more than this amount of data through a drive… which is today around 210-220TB mark for consumer drives… granted this is a bit lower than what i’ve been seeing when checking product sheets, but i don’t look at the SMR ones so maybe they are a bit lower…

i digress…
each drive has like 2 - 5 years warranty and if one pushes more than the rated throughput then the smart data will record it and the warranty will be void…

180 TB is a lot… i mean at the lowest speed i’ve heard SMR drives reach was something like 800kb/s but lets call it 1 mb/s
so 87600 in a day and lets call that 100gb to be nice… so 10days for 1tb and then at its highest strainwork load it will never reach this point… and then 180x that … so would take 1800 days to reach that… if it was only writes atleast…
so thats like 5mb/s sustained traffic for a year… which is quite a bit… much more than storj has been using … and yet the SMR cannot keep up

it’s a warranty thing… enterprise drives are rated upwards of 500TB a year… so maybe 15mb/s sustained throughput over a year… doesn’t mean the drive can keep up tho but it should :smiley:

io can make even super low traffic nearly impossible.

1 Like

However, if you use RAID6 or RAID10, the load gets distributed and each drive sees less TB/year.

1 Like

Keep in mind that the WDC purple is rated for 360TB/a and the Seagate SkyHawk 8TB for 180TB/a

If you send in a defective drive that has apparently been used above its load rating, the manufacturer might just decline your warranty claim. So the workload is part of the warranty deal.

Yeah, but I never said that :smiley:

sorry maybe i misunderstood you…

i thought you tried to make a pointfor it being related to the disk write ability, and i was trying to make a point that even 180tb a year would be way more than the storj avg utilization…

to be fair, i don’t know much about SMR drives… i know enough to stay far away lol
and i knew from the first time i heard about it, because of the inherent concept of the technology…

fine for storage and retrieval… and like beating a lame horse for high write…

Yes after some long deliberation I am going the way of 1 node on 1 disk too vs setting up a RAID 5 or RAID 6. I hope regular disk health scans will be able to show me in time when a drive might reach it’s eol when it will be time to move the data to a new drive. I bought Toshiba Enterprise Capacity 12TB disks which Toshiba gives 5yrs warranty, so I expect them to perform long and well.

1 Like