Updates on Test Data

Around a +25% increase. These 137W were measured today or when the test data was around 120Mbps per node?

Average of yesterday. The way I understand it the current tests are the expected behavior. So I thought that would be fair. Though +25% isn’t really correct. On the CPU side it’s much more as that previous 108 watts included 12 HDD’s. Assuming they use about the same as the HDD’s in the external bay, those HDD’s accounted for around 94 watts. CPU and rest of the system only about 14 watts. That jumped to 45 watts. The CPU has a TDP of 35 watts (I know, that’s not equal to usage). Since those 45 watts include the rest of the system as well as 2 NVMe SSD’s it seems the system is working at close to max wattage most of the time. Anyway, system without disks 14->45 is more like a 200+% increase. If you want to include disks, you should also include the external box too as it runs off the same system. 186->215 so about 15% increase. Nothing too shocking. But this is a very energy efficient system. Heavier systems will almost certainly see even less of an increase as their baseline usage is already higher.

1 Like

Your setup sounds amazing. My rule-of-thumb is 10w/HDD (to cover power, cooling, JBOD/enclosure losses) etc. Like if a 24-bay SAS enclosure uses less than 240w that’s acceptable. But you’re doing 10w/HDD including compute!

2 Likes

Yep, plus this system serves other purposes as well. :slight_smile: I like to keep things lean. Though the CPU is struggling a little to be fair.
I lucked out to be honest. For now it seems to hit just the sweet spot for performance vs energy usage. Speed of the other things it runs is not critical. It would likely be an issue if it was. I forgot to mention I currently use between 3-4 IP’s of traffic as well. (not enough nodes with free space to keep all IP’s I have active and I’m still a little wary to upgrade HDD space too much, though a new 20TB one should be arriving today.) Hardware is Synology DS3617xs + Icybox ib-3810U3. I had the NAS, but added the IB for Storj.

2 Likes

Sorry for OT:
You use the Icybox connected via USB to the Synology?
Is this performant?
Are you seeing the Drives as additional external drives?

Did anyone else see that we had a bit of a breather today? It looks like the ingress rate is only about 75% of what we’ve seen the last four days or so.

Gives those lazy filewalkers a chance to catch up :wink:

Hasn’t been an issue. But you’re right about the OT. So I’ll stick to a short response. :slightly_smiling_face:

I’ve had 2 drives fill with test data. 40TB this month alone so far.
Just added a new one and it’s received 1TB in one day.

If this is going to be a new normal for data ingress, I’m not going to be able add drives fast enough XD

2 Likes

Hold your horses, with a 30 day TTL, deletes will come in just as fast as ingress pretty soon. Don’t go stacking up a year supply of HDD’s expecting used space will just keep growing.

7 Likes

Yeah… we’ll have to be able to maintain these ingress levels… day after day… to keep our hands on all that sweet sweet test data. Bring it on!

1 Like

That’s what I’m expecting, I don’t have anymore drives lying around. Don’t have the room either

I’ve been running a node for less than a week now, just to compensate the costs to upload my own backups to Storj.

So this test data will be removed in the near future? Or will it get replaced by actual consumer data?

Im currently using 4.19TB out of the 7.4TB I reserved for the node which will bring in enough to compensate my own storage costs, but not when all data gets removed again unfortunately.

Scroll up to read the first post: all this capacity-reservation data is for anticipated growth from prospective customers. We’re all hoping they turn into real customers… but if it doesn’t… I wouldn’t expect Storj to keep paying us for all the reserved space (and they would let the network delete it).

Even if it does turn into customer-paid usage… it sounds like there will be significant churn: in that lots of data would get deleted every day and fresh stuff uploaded. So I’m expecting nodes to be relatively busy most of the day.

6 Likes

Traffic of my node for the past week:

Is today and yesterday lower for everyone compared to the 3 days before? It looks like my node is not overloaded, but with the new selection I may not notice that my node is getting reduced traffic. the only way to do it is to compare the traffic to others, so here I am :slight_smile:

2 Likes

Throughput seems a bit lower today and we didn’t have the “morning dip”, so the pattern has changed a little bit

1 Like

Bit offtopic.

I see these graphs more often. How do you create them or can anyone else tell me? :slight_smile:

I see the same thing: for a few days they seemed to still be in go-hard-or-go-home mode and were pushing for max throughput. But today (so far) and yesterday they seemed to be more chill: pushing 75% of max just to keep up with capacity-reservation.

Maybe they backed off a bit because 1.105.4 is also slowly getting rolled out: and they know there will be lots of node restarts and used-space-filewalkers running?

You do see those green RRDtool graphs all over the place. However it’s not something you tend to use directly: it’s more just the graphing part of some other monitoring tool. I’d check the web UI for your router first to see if it offers any graphs… or if you’re using something like PFsense there are some built-in (under Status → Monitoring). Or tell us if you’re using Windows or Linux and then someone can let you know the tool they use.

I use two monitoring systems - cacti (graphs with white background) and zabbix (graphs with black background).
Traffic graphs are easy to create, you just need to install SNMP service to the VM or wherever your node runs, cacti already has a graph template for traffic.
Other graphs that I sometimes post are custom. I have uploaded some scripts I use to my github, but there is no documentation :slight_smile:

Thanks. I was thinking that maybe my node is too slow. I even disabled the filewalker, but the traffic did not go up.

1 Like

Yep.

I’ve started a node on the 15th and I saw it using about 170 to 180Mbps with morning dips on the 15th throughout the 18th. Yesterday’s maximum was around 150Mbps. Today has been around 110 to 120Mbps without the morning dip, just a constant flow of traffic since about 3PM CET on the 18th.

1 Like