CPU and RAM usage through the roof

Did you run top or htop to see what uses up your RAM?

IB-3740-C31

ServerCat :smile: I assume it uses top.

it can be that problem also in drivers, it intended to use on windows and mac

I dont know what ServerCat is.Sry to bring you in wrong direction.

:frowning: quote=ā€œVadim, post:23, topic:21876, full:trueā€]
it can be that problem also in drivers, it intended to use on windows and mac
[/quote]

I know that SATA-USB bridges are finicky. Specifically the JMS578 ( see RPi4 USB3.0 slow speed(quirks not help) - Raspberry Pi Forums )

I also tried Quiks mode
https://forums.raspberrypi.com/viewtopic.php?t=318243

Then i realized i think the drive is SMR :frowning:
Seagate Barracuda 2.5 5400
This is probobly my problem

1 Like

yeah you back to moving the databases out of the drive to reduce the iops load on the disk.
its the only way, if you disk is 5400 SMR ā€¦ but i think barracuda is usually pretty fastā€¦ so not sure that would be SMRā€¦

oh yeah its on seagates SMR for 2.5"

i guess its a long time since barracuda was their faster drivesā€¦
donā€™t you just love when a manufacture, change their branding so their older good model names become their new crappy modelsā€¦ lol

bait and switchā€¦ tsk tsk seagate

old barracuda marketing from when it was cool writing windows xp in their marketing :smiley:

you mean icy box? interesting.

@SGC when you say databases you what are you referring to. How does one move databases? just copy a folder over?

I can throw in a small 250gb ssd i have available now that i have 2 usb3.0 ports :slight_smile:

would be something like thatā€¦ there should be plenty of documentation on it on the forum or in the storj documentation.

pi@raspberrypi:/mnt/WD1001 $ dd if=/dev/zero of=test.bin status=progress
..
292727808 bytes (293 MB, 279 MiB) copied, 12.2682 s, 23.9 MB/s

23.9 MB/s seems not to be much - but much more than storj transfer rates.

Load seems to normalize, no? @YurkoWasHere

At least on my side success rates coming back to 81%, increasing.

my ingress was certain down yesterdayā€¦ by like 25%
my downloaded successrate isnā€™t all that great thoā€¦

========== AUDIT ==============
Critically failed:     0
Critical Fail Rate:    0.000%
Recoverable failed:    0
Recoverable Fail Rate: 0.000%
Successful:            687
Success Rate:          100.000%
========== DOWNLOAD ===========
Failed:                404
Fail Rate:             1.711%
Canceled:              6217
Cancel Rate:           26.331%
Successful:            16990
Success Rate:          71.958%
========== UPLOAD =============
Rejected:              0
Acceptance Rate:       100.000%
---------- accepted -----------
Failed:                30
Fail Rate:             0.073%
Canceled:              47
Cancel Rate:           0.114%
Successful:            41209
Success Rate:          99.814%
========== REPAIR DOWNLOAD ====
Failed:                0
Fail Rate:             0.000%
Canceled:              0
Cancel Rate:           0.000%
Successful:            1186
Success Rate:          100.000%
========== REPAIR UPLOAD ======
Failed:                0
Fail Rate:             0.000%
Canceled:              0
Cancel Rate:           0.000%
Successful:            918
Success Rate:          100.000%
========== DELETE =============
Failed:                0
Fail Rate:             0.000%
Successful:            12535
Success Rate:          100.000%

but got some GEā€™s running which cause a good deal of egress, so that might affect itā€¦
so will have to wait until thats done, for an accurate number on my successrates

1 Like

you canā€™t expect full bandwidth when the HDD is under loadā€¦
HDDā€™s can only do like 100-800 IOPS depending on if its random or sequential read or write iops.

usually like 200 read / write is normal under random loads. but it can get much worse if working with small files.

1 Like

Yep seems to be back to normal.

Been trying to find replacement 2.5" drives to use with my setup but they all seem to be SMR now :sob:

Aside from runs like this, STORJ doesnt seem to be that intense with read/writes. Is there any long term advantages going SMR over CMR or even SSD?

SMR or singled magnetic recording is a technology that allows for more capacity at the cost of writing multiple paths on the platter at the same time and thus its slower and much more cumbersome to write data a diskā€¦

initially when the disk is empty or not fully loaded there isnā€™t much disadvantage to SMR, but during continual workloads the write iops drop by nearly a factor of 10.

any 2.5" HDD with more than 6TB will be SMR

CMR is just better, because it can write fasterā€¦ which is very good for storj.
SMR is good for archival dataā€¦ like storing video and other big files, then you would most likely never notice its SMR

because read on SMR is the same as CMR, its only writes in certain conditions that sees a degradation of performance.

1 Like

Right because it has to re-write all the sectors during writesā€¦

But in the context of Storj, does BETTER write io improve anything or is the network not saturated enough to matter at this point.

Storj write IO is very heavy for HDDā€™s

SMR usually canā€™t keep up without assistance, like moving the databases to an SSD

1 Like

The RPi neither. Donā€™t try. Use CMR. IOWAIT will kill your node after 6 months.

Although my Rpi node (model 4B, 8GB RAM & USB3 ext4 HDD) was fine even during heavy traffic time in these days (65GB/day ingress + 16GB/day egress), another node (VirtualBox, 2 cores, 5GB RAM & USB3 ext4 HDD through USB2 for Storj, SSD for host & guest OS) had a similar issue. IOWAIT and memory usage were increased to approx 4.2GB RAM + 6GB swap; but the node was helped by swapspace (dynamic swap manager)! The node is back to normal operation now, 400MB RAM + zero swap :slight_smile:

Is that recommended, having in mind, that write operations are asked to be minified on the SD card? I would expect one single IOWAITing operation is as often happening as there are io reads/writes on the HDD with storj. The SD might live longer, missing 15-25% of heavy traffic success rates. What do you think?