# Updates on Test Data **URL:** \ **Category:** Announcements\ **Created:** [April 30, 2024, 3:36pm UTC](https://forum.storj.io/t/updates-on-test-data/26034 "2024-04-30T15:36:45Z")\ **Posts on this page:** 20\ **Page:** 43
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [June 7, 2024, 11:24am UTC](https://forum.storj.io/t/updates-on-test-data/26034/947 "2024-06-07T11:24:06Z")
> [@agente](#): > > Hardware that can sustain it untill now is different of what we need today with this “new pattern”. Again, we are not talking about 1/10 nodes! We are talking about PB+ of data. > Scaling job is not easy… if you have infinite money you are going to buy something that can work on 1gbit per node but if you have to manage 100 nodes or more you need to find a compromise. Today pattern broke that compromise 🙂 I get that, which is why we are discussing this: what has been working so far is having issues keeping up, true in some cases. Be realistic: with 1PB of data ($1500/month just in storage) there is literally nothing you can afford to switch from “broken” to “works perfectly”? ---
**Author:** ![ACarneiro](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/acarneiro/32/1129_2.png) [@ACarneiro](https://forum.storj.io/u/ACarneiro)\ **Post date:** [June 7, 2024, 11:24am UTC](https://forum.storj.io/t/updates-on-test-data/26034/948 "2024-06-07T11:24:49Z")
Seems like the testing is over for the time being. Making adjustments or has “mystery contract” fallen by the wayside? 🙂 ---
**Author:** ![Pentium100](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pentium100/32/6082_2.png) [@Pentium100](https://forum.storj.io/u/Pentium100)\ **Post date:** [June 7, 2024, 11:25am UTC](https://forum.storj.io/t/updates-on-test-data/26034/949 "2024-06-07T11:25:48Z")
Switch should see the same MAC on port 2/VLAN10 and on port 3/VLAN20. If a packet with that destination MAC comes on VLAN 20 it should go to port 2 and if it comes on VLAN10 it should go to port 2. I’m pretty sure switches should not be confused with this. Unless the duplicate MACs are on the same VLAN, that would be a problem. As for ARP - switch would only need that for the management interface, which can be assigned on yet another vlan to be in the internal network. ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [June 7, 2024, 11:28am UTC](https://forum.storj.io/t/updates-on-test-data/26034/950 "2024-06-07T11:28:47Z")
If a packet comes in tagged for vlan 1 for MAC abc on your upstream port, and needs to travel untagged to MAC abc which is on your downstream port, how are you going to move this packet? You can’t. It’s like looping an ethernet cable from port 1 to port 2. Why would the switch pick to send this packet to your downstream port? the mac is already on the upstream port. ---
**Author:** ![BrightSilence](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/brightsilence/32/6048_2.png) [@BrightSilence](https://forum.storj.io/u/BrightSilence)\ **Post date:** [June 7, 2024, 11:31am UTC](https://forum.storj.io/t/updates-on-test-data/26034/951 "2024-06-07T11:31:13Z")
I thought tapping out on the topic would do the trick since it was about my setup. Let’s keep it on topic people. We might need this thread for more testing. ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [June 7, 2024, 11:36am UTC](https://forum.storj.io/t/updates-on-test-data/26034/952 "2024-06-07T11:36:04Z")
In [related news](http://www.th3van.dk/): `Wed Jun 05 14:43:22 CEST 2024 : NIC migrated from 10Gbit/s --> 25Gbit/s` Somebody is getting ready for the “new normal” 😜 ---
**Author:** ![Pentium100](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pentium100/32/6082_2.png) [@Pentium100](https://forum.storj.io/u/Pentium100)\ **Post date:** [June 7, 2024, 11:36am UTC](https://forum.storj.io/t/updates-on-test-data/26034/953 "2024-06-07T11:36:36Z")
The mods are going to split it anyway. We’ll probably continue this in DMs. ---
**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\ **Post date:** [June 7, 2024, 11:39am UTC](https://forum.storj.io/t/updates-on-test-data/26034/954 "2024-06-07T11:39:33Z")
Yes but… if you followed the rule of 1 node per hdd, with basic ext4, without caching, you’ll realize that even if you max out ram on your server you broke somethings or maybe not performing great (long gc, long fw all together). Than you need to understand if you must change everything adding caching support (and maybe changing fs) or just wait to see if this new pattern will be real or not. ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [June 7, 2024, 11:45am UTC](https://forum.storj.io/t/updates-on-test-data/26034/955 "2024-06-07T11:45:45Z")
That’s the situation I’m currently in. So far load on one of the node rigs was just fine. With the recent change in traffic pattern, incoming data takes precedence over running my lazy GCs and FWs. So I’m left with two choices: wait it out, or add more RAM to it (to speed up lazy). One isn’t really an option since that mobo doesn’t support more sticks. The only logical path forward for me is to just budget in a new mobo+cpu+ram upgrade. I was planning to do that anyways, so this just inconveniences me only to the “how fast should I do it” part. I’m still using what I have, but I’ve already hit the limit on that, so I need to upgrade. Is that against any of the guidelines? I don’t think so, I don’t expect storj to spend any R&D time on making my rig cope with the traffic. If they tweak things a bit and it stays good, then everything is fine. If they don’t and I still need to update, still everything is fine. Either way I’ve outgrown that rig. If by adding another rig I move on one step forward, that only frees up this rig for a couple of more nodes. ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [June 7, 2024, 11:53am UTC](https://forum.storj.io/t/updates-on-test-data/26034/956 "2024-06-07T11:53:54Z")
If filewalkers/gc run in the background… is it important that they be fast? Like if their runtime doesn’t affect your payouts, there’s no reason to upgrade? Recent trash handling seems way better: so it shouldn’t be causing nodes to appear artificially full (and rejecting uploads) anymore. ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [June 7, 2024, 11:58am UTC](https://forum.storj.io/t/updates-on-test-data/26034/957 "2024-06-07T11:58:09Z")
The thing is that lazy (=running in the background) is exactly that: lower IO priority. As long as something else needs that priority (ie incoming data), lazy can’t run. You can only run it between tests, or when the satellite scales back the traffic. I can speed up lazy by adding RAM (so more metadata is cached) or plan the whole “mortgage the house and move to ZFS” scenario. To me, RAM is easier and more cost effective. No I will not consider LVM cache 🤣 ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [June 7, 2024, 12:10pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/958 "2024-06-07T12:10:49Z")
> [@Mitsos](#): > > The thing is that lazy (=running in the background) is exactly that: lower IO priority. I get it… but is that important? Does it affect payouts? It sounds like a node would have to be _so busy, 24x7,_ that it completely fills its space because it a) didn’t get an accurate report of the space it’s already using - so didn’t know to reject uploads… or b) didn’t keep up with deleting trash. So… SNOs would want to improve fw/gc performance to avoid nodes falling over due to filled disks? That’s the problem you’re having? Storj is throwing data your way… money your way… faster than you can catch it? I don’t have that problem 🙂 - fw/gc can take days… my disks aren’t full… I’m not paying anything to speed up background housecleaning. ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [June 7, 2024, 12:14pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/959 "2024-06-07T12:14:57Z")
Literally my problem: can’t catch enough data because I can’t keep up with the deletes. Shouting “but they told us to use what we have!!!”, isn’t going to fix this. Spending $$$, will. Off topic (again) the new zen 5 looks perfect on paper. ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/5_5575768a8748004e209b776fc1b2916d.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [June 7, 2024, 12:21pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/960 "2024-06-07T12:21:10Z")
> [@Mitsos](#): > > Shouting “but they told us to use what we have!!!”, isn’t going to fix this. > Spending $$$, will. I understand your issue now: sorry to push. If I could start clean… maybe with a 4u 24-bay case… then yeah stuff in 128GB (or more) of RAM for ZFS ARC, and some mirrored SSDs for a special metadata device: and fw/gc would always run in seconds… and the HDDs would only receive the lightest occasional kiss 😘 But… for now I tend to the 🥔 … and spend my nights dreaming of the day I get to add a 20TB drive just for Storj… 💤 ---
**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\ **Post date:** [June 7, 2024, 1:16pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/961 "2024-06-07T13:16:22Z")
> [@Mitsos](#): > > RAM is easier and more cost effective. If you cap your system ram you need to move to the second step… and beleive me… you’ll cap in a basic system! 🙂 how much you can scale with a simple config? (double cpu 1.5tb ram basic ext4 setup basic jbod 6g/12g basic HBA controller) 100? 150? 1000 nodes? Few SNOs today have the answer. New pattern cut maximum nodes number that you can manage a lot. Again… I’m not talking about a couple of nodes. ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [June 7, 2024, 1:20pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/962 "2024-06-07T13:20:03Z")
No, I’ll add a second rig at the 24 nodes (=24 drives) mark. Why do I need to run 1000 nodes off a single host? So I can justify me daisychaining whole racks of JBODs? I’ll be capped at the controller level anyway. ---
**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\ **Post date:** [June 7, 2024, 1:27pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/963 "2024-06-07T13:27:07Z")
cost efficiency… we are talking about KWs of hardware. ---
**Author:** ![littleskunk](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/littleskunk/32/63_2.png) [@littleskunk](https://forum.storj.io/u/littleskunk)\ **Post date:** [June 7, 2024, 1:28pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/964 "2024-06-07T13:28:33Z")
> [@Roxor](#): > > maybe with a 4u 24-bay case… then yeah stuff in 128GB (or more) of RAM for ZFS ARC I think with ZFS you need about 1GB of RAM per TB disk space. So you would have to put in a lot more RAM if you want to run so many disks. ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [June 7, 2024, 1:33pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/965 "2024-06-07T13:33:14Z")
No, we are only talking about 200W of extra power. The drives would be powered anyway through a JBOD. ---
**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\ **Post date:** [June 7, 2024, 1:35pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/966 "2024-06-07T13:35:19Z")
200 more watt every 24 bays with your future setup. scale up! we can split in another thread. “What should I do to manage 1000 nodes when BIG customers comes?” I want to be positive. I want to think that one day storj need a lot of more stored data and everyone have to think how to scale up. [Previous page](https://forum.storj.io/t/updates-on-test-data/26034.md?page=42) [Next page](https://forum.storj.io/t/updates-on-test-data/26034.md?page=44)