Updates on Test Data

to much conections spinning up cpu usage in my case.

After almost 1 month of testing, any update in how long will the test phase last?
Hopefully it ends anytime soon and stops being the opium of the people.

Please scroll all the way up and read the top post. You’ll find an answer right there.

2 Likes

so the “new normal” is this massive test data with increased power consumption? :smiley:

@Th3Van, did you notice any increase in total power consumption in your setup? tyia

http://www.th3van.dk/SM01-power-consumption.txt

it says Total watt consumption ~2050W and at the begining of the test campaign it was around 1850W, if i recall correctly.

1 Like

Yes, the tests are emulating expected customer usecase. Best case — the test traffic will be replaced by customer traffic.

8 Likes

I have a 12-bay NAS and a 10-bay dumb USB attached storage box. That 10-bay storage box uses pretty much the exact same power as before. Around 78 watts. The NAS went from about 108 watts up to about 137 watts. Undoubtedly mostly the additional CPU use. This is with 14 nodes + 1 on testnet that is mostly idle.

4 Likes

Around a +25% increase. These 137W were measured today or when the test data was around 120Mbps per node?

Average of yesterday. The way I understand it the current tests are the expected behavior. So I thought that would be fair. Though +25% isn’t really correct. On the CPU side it’s much more as that previous 108 watts included 12 HDD’s. Assuming they use about the same as the HDD’s in the external bay, those HDD’s accounted for around 94 watts. CPU and rest of the system only about 14 watts. That jumped to 45 watts. The CPU has a TDP of 35 watts (I know, that’s not equal to usage). Since those 45 watts include the rest of the system as well as 2 NVMe SSD’s it seems the system is working at close to max wattage most of the time. Anyway, system without disks 14->45 is more like a 200+% increase. If you want to include disks, you should also include the external box too as it runs off the same system. 186->215 so about 15% increase. Nothing too shocking. But this is a very energy efficient system. Heavier systems will almost certainly see even less of an increase as their baseline usage is already higher.

1 Like

Your setup sounds amazing. My rule-of-thumb is 10w/HDD (to cover power, cooling, JBOD/enclosure losses) etc. Like if a 24-bay SAS enclosure uses less than 240w that’s acceptable. But you’re doing 10w/HDD including compute!

2 Likes

Yep, plus this system serves other purposes as well. :slight_smile: I like to keep things lean. Though the CPU is struggling a little to be fair.
I lucked out to be honest. For now it seems to hit just the sweet spot for performance vs energy usage. Speed of the other things it runs is not critical. It would likely be an issue if it was. I forgot to mention I currently use between 3-4 IP’s of traffic as well. (not enough nodes with free space to keep all IP’s I have active and I’m still a little wary to upgrade HDD space too much, though a new 20TB one should be arriving today.) Hardware is Synology DS3617xs + Icybox ib-3810U3. I had the NAS, but added the IB for Storj.

2 Likes

Sorry for OT:
You use the Icybox connected via USB to the Synology?
Is this performant?
Are you seeing the Drives as additional external drives?

Did anyone else see that we had a bit of a breather today? It looks like the ingress rate is only about 75% of what we’ve seen the last four days or so.

Gives those lazy filewalkers a chance to catch up :wink:

Hasn’t been an issue. But you’re right about the OT. So I’ll stick to a short response. :slightly_smiling_face:

I’ve had 2 drives fill with test data. 40TB this month alone so far.
Just added a new one and it’s received 1TB in one day.

If this is going to be a new normal for data ingress, I’m not going to be able add drives fast enough XD

2 Likes

Hold your horses, with a 30 day TTL, deletes will come in just as fast as ingress pretty soon. Don’t go stacking up a year supply of HDD’s expecting used space will just keep growing.

7 Likes

Yeah… we’ll have to be able to maintain these ingress levels… day after day… to keep our hands on all that sweet sweet test data. Bring it on!

1 Like

That’s what I’m expecting, I don’t have anymore drives lying around. Don’t have the room either

I’ve been running a node for less than a week now, just to compensate the costs to upload my own backups to Storj.

So this test data will be removed in the near future? Or will it get replaced by actual consumer data?

Im currently using 4.19TB out of the 7.4TB I reserved for the node which will bring in enough to compensate my own storage costs, but not when all data gets removed again unfortunately.

Scroll up to read the first post: all this capacity-reservation data is for anticipated growth from prospective customers. We’re all hoping they turn into real customers… but if it doesn’t… I wouldn’t expect Storj to keep paying us for all the reserved space (and they would let the network delete it).

Even if it does turn into customer-paid usage… it sounds like there will be significant churn: in that lots of data would get deleted every day and fresh stuff uploaded. So I’m expecting nodes to be relatively busy most of the day.

6 Likes

Traffic of my node for the past week:

Is today and yesterday lower for everyone compared to the 3 days before? It looks like my node is not overloaded, but with the new selection I may not notice that my node is getting reduced traffic. the only way to do it is to compare the traffic to others, so here I am :slight_smile:

2 Likes