# Updates on Test Data **URL:** \ **Category:** Announcements\ **Created:** [April 30, 2024, 3:36pm UTC](https://forum.storj.io/t/updates-on-test-data/26034 "2024-04-30T15:36:45Z")\ **Posts on this page:** 20\ **Page:** 23
**Author:** ![littleskunk](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/littleskunk/32/63_2.png) [@littleskunk](https://forum.storj.io/u/littleskunk)\ **Post date:** [May 31, 2024, 3:50pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/519 "2024-05-31T15:50:28Z")
> [@littleskunk](#): > > We are working on a different success tracker. Deployment on a Friday is a bad idea so it will have to wait for Monday. There is a good description in the PR what the differences are: [https://review.dev.storj.io/c/storj/storj/+/13308](https://review.dev.storj.io/c/storj/storj/+/13308) Ok turns out we are doing this today. It will get deployed shortly. ---
**Author:** ![arrogantrabbit](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/arrogantrabbit/32/11200_2.png) [@arrogantrabbit](https://forum.storj.io/u/arrogantrabbit)\ **Post date:** [May 31, 2024, 4:02pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/520 "2024-05-31T16:02:23Z")
> [@zip](#): > > Really hard to believe, as I was used to a norm the overhead is a few percent at most when running workloads on VMs. This is not a normal workload though. Modern CPUs have built in instructions specifically designed to accelerate encryption and decryption. These, and other extensions might not be available in VMs/not supported by the hypervisor/emulated in software/etc ---
**Author:** ![ACarneiro](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/acarneiro/32/1129_2.png) [@ACarneiro](https://forum.storj.io/u/ACarneiro)\ **Post date:** [May 31, 2024, 4:30pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/521 "2024-05-31T16:30:25Z")
I hope you’re not on call over the weekend! 😄 > [@littleskunk](#): > > Ok turns out we are doing this today. ---
**Author:** ![littleskunk](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/littleskunk/32/63_2.png) [@littleskunk](https://forum.storj.io/u/littleskunk)\ **Post date:** [May 31, 2024, 4:43pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/522 "2024-05-31T16:43:17Z")
It is a feature flag so we can go back to the behavior that was running stable for hours. ---
**Author:** ![KernelPanick](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/kernelpanick/32/62_2.png) [@KernelPanick](https://forum.storj.io/u/KernelPanick)\ **Post date:** [May 31, 2024, 5:03pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/523 "2024-05-31T17:03:01Z")
I have 3 nodes on 1 IP and I’ve been suspicious if that was a good decision I made now at this point. Contemplating preventing 2 of them from filling anymore if there is risk of overload from having more than one. Please let us know if there are adverse findings for this. So far I’m seeing similar data per day combined (350-500GB) and the resource consumption doesn’t even seem too close to the limit. I like this strategy to make the node limit size no larger than my largest single drive size for a DR scenario. ---
**Author:** ![Pentium100](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pentium100/32/6082_2.png) [@Pentium100](https://forum.storj.io/u/Pentium100)\ **Post date:** [May 31, 2024, 5:29pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/524 "2024-05-31T17:29:33Z")
30% on Linux means 30% of one core. If the node process used all 4 cores it would show 400% utilization. ---
**Author:** ![Ruskiem](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/ruskiem/32/64_2.png) [@Ruskiem](https://forum.storj.io/u/Ruskiem)\ **Post date:** [May 31, 2024, 5:32pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/525 "2024-05-31T17:32:49Z")
And what are You doing right now? because i noticed that all nodes started to gain more and more RAM for storagenode.exe(Windows GUI) and disk seems to be keeping up like usually, the network load is ~30%, it was like that and more all day before and the RAM was under control, untill to 19:19 UTC+2 when i noticed that change. Yea im observing RAM for the process is going up by the minute. 5 minutes ago it was 1300MB , now its 1700MB. Similar acros all nodes that gets Ingress. Not occuring in nodes that are full. With more CPU % load in accompaniment ofcourse. I see the HDDs are not working any harder than usually, i see them chilin’, with writes, (if they can afford to show 3-4-5MB/s and then some 1,5MB/s or 748KB/s in Writes, means not so much load, that they could not keep up with or anything) Edit: now 1946MB of ram for storagenode.exe, other nodes similarly climbing, to the machine available limit, will observe and watch what happens Edit2: Yup. The memory for the process started to gain rapidly like 4-5MB per sec. up to 2282MB i saw, then screen flashed black, and the storagenode.exe was terminated unexpectedly, and freed all occupied ram. And in Even Viewer i see prior to that, a warning that machine is running low on virtual RAM. Storagenode.exe was restarted after 2 minutes (because that what i set in services on crash) in logs theres nothing, just abruptly cut, after a line of some normal, file upload work. Edit3: Oh i see the storagenode.exe process back to 2000MB of RAM again. So fast. I though after restart it will climb slowly. But here it is at \>2000MB again. And its gaining like 0,3MB per sec. its at 2200MB again hmmm. Looks like all nodes going in that direction, oh shoe, and i was hoping they will finish the filewalkers this week so i can turn it off oh shoe ☹ the crashes will cancel all the progress Edit4: 20:19 UTC+2 time, i see the process is losing MB finally, and i see the overall network utilization just droped by half ~120 sec ago, thank You. Edit5: yea its very much loosing MB, 5 minutes ago it was ~1500MB, now its ~1000MB 🥳 Edit6: oh no, 20:36 UTC+2 time, its back to, i will call it “bad ingress”, only 30% of the network, but with too much CPU and too much growing RAM. I was enyoing such load today earlier, and days before and the problem wasn’t occurring hmmm the nodes are on 1.104.5 as it’s current minimum. Maebe You Guys enabled something at satelites and forgot nodes cant handle it yet? idk, i see theres ver 1.105.4 in testing available Edit7: i can confirm that at 00:58 (UTC+2) the traffic is back to 30% of my network, same network load as with the problems, but observed node are back to stable RAM and CPU usage, i didn’t change anything, all other nodes also are behaving, hope thats some usefull feedback. ---
**Author:** ![agente](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/agente/32/1137_2.png) [@agente](https://forum.storj.io/u/agente)\ **Post date:** [May 31, 2024, 5:37pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/526 "2024-05-31T17:37:59Z")
my router collapsed… ---
**Author:** ![MarviBiene](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/marvibiene/32/1124_2.png) [@MarviBiene](https://forum.storj.io/u/MarviBiene)\ **Post date:** [May 31, 2024, 6:18pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/527 "2024-05-31T18:18:17Z")
Love that proxmox thinks I am writing with like 200 GB/s 😂 ![1000051973](https://storj-s3.bcdn.literatehosting.net/original/3X/5/3/53a0fa764fb222cbe5c986e5804fdf8b63563f74.png) ---
**Author:** ![Mitsos](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/mitsos/32/1125_2.png) [@Mitsos](https://forum.storj.io/u/Mitsos)\ **Post date:** [May 31, 2024, 6:26pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/528 "2024-05-31T18:26:27Z")
Can confirm, baremetal RAM usage is 5x of what it used to be with minimal network and IO. ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/6_22734026db12803ebb3e059622ac3534.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [May 31, 2024, 7:23pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/529 "2024-05-31T19:23:48Z")
> [@Ruskiem](#): > > Yea im observing RAM for the process is going up by the minute. I wonder if this is just the change from sync to async writes? With the old sync behavior a node was limited by what disks could actually handle (and it would lose races asking for more). But with the new async writes everything is going to get funneled straight into RAM (for the OS to write when-it-can)… so the node is always telling the network “I got this - send me more”. Get some data… stuff it in RAM… get some data… stuff it in RAM… completed writes are a problem for later 😉 ---
**Author:** ![Ruskiem](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/ruskiem/32/64_2.png) [@Ruskiem](https://forum.storj.io/u/Ruskiem)\ **Post date:** [May 31, 2024, 7:28pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/530 "2024-05-31T19:28:52Z")
i personally doubt, it looked like HDD was rather bored and we had some cases of memory leak back in the days 2021 or so, but the problem was solved. i would be fine probably even with 3-4GB sticked to the process, but i can’t put more RAM dices to that motherboard to allocate more ☹ ---
**Author:** ![ACarneiro](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/acarneiro/32/1129_2.png) [@ACarneiro](https://forum.storj.io/u/ACarneiro)\ **Post date:** [May 31, 2024, 8:05pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/531 "2024-05-31T20:05:21Z")
I’ve seen some more RAM used on all my nodes but still nothing especially worrying. With the exception of one of the very first tests where CPU usage was very high, they’re all coping very well with the current test loads… ---
**Author:** ![snorkel](https://storj.bcdn.literatehosting.com/letter_avatar/snorkel/32/6_22734026db12803ebb3e059622ac3534.png) [@snorkel](https://forum.storj.io/u/snorkel)\ **Post date:** [May 31, 2024, 8:31pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/532 "2024-05-31T20:31:30Z")
I don’t get the “huge number of connections to a node” test. If we prepare for a big client, or a few clients, I imagine they have a few servers that will upload to nodes. Why do we simulate a huge number of connections, if we prepare for a few servers? Can’t they just upload with fewer connections instead of more? It’s the same bandwith. 10x1Gbps=1x10Gbps. Why do we need to suffocate the CPU, router, ISP etc with more connections? ---
**Author:** ![pangolin](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/pangolin/32/15685_2.png) [@pangolin](https://forum.storj.io/u/pangolin)\ **Post date:** [May 31, 2024, 8:37pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/533 "2024-05-31T20:37:37Z")
Isn’t this by design (1 piece = 1 connection)? ---
**Author:** ![Roxor](https://storj.bcdn.literatehosting.com/letter_avatar/roxor/32/6_22734026db12803ebb3e059622ac3534.png) [@Roxor](https://forum.storj.io/u/Roxor)\ **Post date:** [May 31, 2024, 10:28pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/534 "2024-05-31T22:28:04Z")
> [@snorkel](#): > > I don’t get the “huge number of connections to a node” test. Are you asking why they’re testing the performance of different configurations… during their performance testing? 😉 ---
**Author:** ![Knowledge](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/knowledge/32/10681_2.png) [@Knowledge](https://forum.storj.io/u/Knowledge)\ **Post date:** [May 31, 2024, 10:42pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/535 "2024-05-31T22:42:59Z")
Because it is a lot of data, from many sources, all hitting at once. The purpose of testing is to know what our best results are under varying high traffic conditions. ---
**Author:** ![Andrew](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/andrew/32/1139_2.png) [@Andrew](https://forum.storj.io/u/Andrew)\ **Post date:** [May 31, 2024, 10:43pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/536 "2024-05-31T22:43:48Z")
Mine too. It would be good to know when the developers plan to finally finish these tests. It’s very annoying. Online and audits are failing, all thanks to the tests. I don’t understand the purpose of these tests. Storj, in its current state, is nowhere near this load of traffic or number of customers. ---
**Author:** ![Aitor](https://storj.bcdn.literatehosting.com/letter_avatar/aitor/32/6_22734026db12803ebb3e059622ac3534.png) [@Aitor](https://forum.storj.io/u/Aitor)\ **Post date:** [May 31, 2024, 10:45pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/537 "2024-05-31T22:45:47Z")
I also think the test it’s too much. ---
**Author:** ![Knowledge](https://storj.bcdn.literatehosting.com/user_avatar/forum.storj.io/knowledge/32/10681_2.png) [@Knowledge](https://forum.storj.io/u/Knowledge)\ **Post date:** [May 31, 2024, 10:48pm UTC](https://forum.storj.io/t/updates-on-test-data/26034/538 "2024-05-31T22:48:38Z")
Also, consider the potential customer may be asking for evidence to show that the Storj network can handle the traffic based on the conditions they specify. They may be actively testing themselves and looking at the results. [Previous page](https://forum.storj.io/t/updates-on-test-data/26034.md?page=22) [Next page](https://forum.storj.io/t/updates-on-test-data/26034.md?page=24)