Bandwidth utilization comparison thread

my egress is a bit lower but I have repair egress of 1-2Mbps

yeah… my stupidity was the fault xD Initially I installed everything correctly on a new SSD in a VM so I could just reboot and switch to the new system, which worked great, until I decided to change some things and realized that I got some automatic standby into the system too… lol… hope it is fixed now so we can all get more ingress now :stuck_out_tongue_winking_eye:

3 Likes

i really should get around to installing my storagenode into a vm, so it can get it’s own graph in proxmox, but that means i need to figure out how to access the storagepool directly…

so yeah thats most likely not going to happen… sadly… maybe when i upgrade… then i might just let the whole thing exist in a vm, seems like it wouldn’t affect the performance much anyways… and the mixing off all vm’s into the proxmox total graph really bugs me… ofc it wasn’t really made with the purpose of having stuff outside the vm’s so…

You can attach any block device to a vm, see: https://pve.proxmox.com/wiki/Physical_disk_to_kvm

yes, but i got a big zfs pool that i already have the storagenode data on… and i just want it to access that folder, so far as i have been able to tell one can only mount a full drive or partition or such, and then copy all the data to the vm… which currently is impossible because the storagenode takes over half my current storage capacity.

was rebooting a vm that has a network drive at the pool, and because it’s doing some torrent related stuff it ofc reads all the files to verify how complete they are or whatever…
and instantly… one is blind on the proxmox primary network graph…

and next to no idea where it’s going or coming from because this doesn’t even leave the machine it’s only on the vmbr…

image

so annoying, i cannot even separate it out into more graphs… this 1 graph represents my 4 nics in the machine + all the vm’s connected to the vmbr… i mean Whiskey Tango Foxtrot
and i might one day add a 4 port 10gbit nic or so making it even worse… one graph for 8 real nics and 8 virtual :smiley:

i even spent time trying to figure out if i could custom it in some way to split out that graph but it seems not… maybe i’m not suppose to use NAS / network shares for the VM’s
but it’s just nice if i don’t need to setup 10 different systems to access the same damn point…

but yeah proxmox rookie… i’m sure ill find a solution to this at one point, because i hate it…

21Jul2020
Node 1:

Node 2:

Did you try Virtual Box? i thas a feature called “shared folders?” it needs “guest additions” to be installed but you just select a folder and it appears as a drive in your client system.
Don’t know how it works under linux though (if you have linux-like OS as a HOST)

1 Like

you can install network monitoring tool in hosted system - that could do the trick
for linux - darkstat, or cactai looks pretty useful (i use darkstat) though cactai has much more feaures and both have http interfaces for accessing stats. Darkstat can monitor specified ports and ignore rest traffic.
for windows - i used networx - simple and free, but has data gathering feature that can show you current flow, and hourly, daily, weekly, monthly traffic, can ignore LAN traffic, can do simple summary for per application traffic.

1 Like

netdata (docker container stats) + prometheus + grafana.

1 Like

yeah netdata is alrite, but kinda rough around the edges… it will show my KSM but it won’t account for it when displaying the memory usage, and the network graphs are highly accurate, but not great for averages… ofc i’m still new to all these tools and every week i have good process on getting more familiar with the setup…

heard a lot about grafana will check that out… haven’t heard of prometheus tho.
server going on 17 days of uptime now… :smiley: so thats a nice change to my continually rebooting gpu pass through fail… haven’t even gotten around to putting the gpu back in… but might do a shutdown for the reupgrade when the new update for storj is out…

the proxmox graphs are just a nice place to have it, because it’s all in one place… also might spin up a Pfsense vm for the firewall stuff… and now the management network is basically all working, i don’t have to worry to much about firewalling myself out of the system lol

pfsense is suppose to be like state of the art in terms of network monitoring and firewall type deal…

@shoofar i’m using proxmox and it has ,… something maybe kvm and support for various stuff… but i prefer not to install to many other virtualization software on it for compatibility issues, and also this might just be me that is at a loss of how to set it up correctly… or that lack some sort of hardware to make it as neat and simple as it could be…

i can see the sense in each server having a single graph to define its ingoing and outgoing network traffic, that makes good sense… but having the vm bridge (virtual switch) /vmbr traffic shown on it also makes it a bit of a mess… but maybe it’s because i am suppose to route my drives not as network drives but as one of those SAS harddisk uplinks instead… from what i understand those can actually suppose multiple server accessing the same raids…

so many things to learn

2 Likes

Take a look here: Prometheus Storj-Exporter
That will give you storj stats on grafana. It’s all fairly easy to set up and they run as docker containers too.
Grafana takes care of calculating averages.

2 Likes

slams head into table
yeah right now i’m putting the final touches on the management network… and ofc i cannot apply my network configurations in proxmox… which is fine… i made it work through the debian terminal… but then now i made another vmbr1 for my management network so i could quickly and easily … ROFL quickly and EASILY … never seems to be the case when i want to do even minor advanced stuff in this OS…

anyways… for some reason i couldn’t just add the nic to the vm, so ill simply add a vmbridge i thought… which i did… but now the proxmox gui doesn’t want to see my new vmbr1 so i cannot select it in the vm… and if i want to apply network changes i will need to install ifupdown2

suppose i can do that also in the terminal… BUT WHATS THE POINT IN A GUI IF IT NEVER CAN DO ANYTHING… its like 40% of the time i need to do something in proxmox and i’m in terminal… tho getting lower every week and getting more and more happy with my setup…

just pisses me off… ofc it makes it easy to suppose if they expect people to only use the gui…

but yeah thanks… ill give it a look when i get to it…sounds pretty good…

might give pfsense a whirl first tho

[a short while later]
… hmmm okay atleast adding the bridge manually to the vm was a breeze… finally
time to test this puppy

1 Like

I don’t know if we’re still sharing stats by the day, but here’s mine for yesterday:
2020-07-21
3mo age
50x50 fiber
98.73TB*h | 9TB (half full)
Ingress - 147.02GB | 6.77GB
Egress - 22.62GB | 37.51MB | 122.88KB

node2:
2mo age
100x10 DOCSIS
48.43TB*h | 2.1TB (full)
Ingress - 0B | 0B
Egress - 7.85GB | 470.53KB | 62.46KB

3 Likes

We use the “total disk space” to make our calculations as it’s more reliable than the “disk space used”.
And also welcome to the forum !

1 Like

Updated it to include the total storage, and my second node.

3 Likes

1 Like

22Jul2020:
Node 1:

Node 2:

1 Like

Wow that graph looks sad haha

23Jul2020:
Node 1:

Node 2:

Have some exciting news, looks like I’ll switching ISPs next week and upgrading to fiber 1Gbit up&down.

2 Likes