Dashboard with very litle information

Hello,
Dashboard shows very little information about the Node, and it does not give the feeling of transparency. More info can be found on the log file, but it is not fast to see relevant information.

Of course there are free Open Source statistics and metrics software like Prometheus, but if you have a dashboard, would be great to have all the info in one place.

For example,
1. for Disk space and usage:

  • total disk space used
  • info about new uploaded/downloaded files in the day (nr of files, space used, download time, average download time, failed, reason)
  • hits on files, and most accessed files in the day/week/month (this is incredibly relevant info since files with many hits can be moved to a ssd, while low rate files can simply stay on big hdd)
  • info about uploads (finished, unfinished, reason)
  • opened files at the moment and over the day, disk usage, transfer rate, activity, etc.
  • info about cpu/ram usage
    2. for Bandwidth usage:
  • a graphic in minutes/hours/days/months specifying bandwidth, latency, nr of uploads/downloads. Minutes and hours usage is important to see congestion and give more bandwidth in that time space. This info could be combined with disk/cpu usage in that moment, so one can know if it is a disk or connection saturation.
  • nr of downloads / nr of uploads (bandwidth used, bandwidth trashed)
  • nr of open threads, total nr of connections opened in a day. Many times ISP limit the number of simultaneous connections to less than 30, which should be taken in consideration besides the bandwidth cap.

Maybe you can make a plugin for Prometheus to read your log file, and use its dashboard instead.

All the info is available, but Dashboard is very basic and its simply not useful. All the extra info and the job to make it possible, of course, if you want quality nodes. Would be interesting in the future to allow multiple folders designation in each node, so files could be easily be moved between hard drives. And even more interesting to accept/reject “clients” and/or files. We are seeing in other network that you can see the number of downloads each file has and the node operator can delete the files it is not interested in hosting.

There’s only one mount point… it’s not really possible for an SNO to move individual files/data pieces from one drive to another.

I think the development is busy ironing out architectural problems first… But since the information is available, I would imagine that at some point in the future the SNO graphical web dashboard will contain more of the available SN information.

  1. At this point you can create a virtual drive with several real hard disks, and depending on the software you use, you can move each file to your desired hard drive. But its pointless since there is no easy info about usage of the files we are storing.

  2. Totally agree with you, things will come in time, and definitely most people will want a easy and fast solution for earning some extra money, so most will never need the extra info. But if Storj wants some long lasting and quality Nodes, the extra info is good to have.

you can use zfs with arc/l2arc caching then the most used files will automatically get cached.

13 posts were split to a new topic: Using ZFS Arc cache on non-ECC RAM

Interesting suggestion, clearly you can do things to improve speed and availability. But as a Node operator if you can only:

  1. be there,
  2. store files,
  3. and hope the files you store are ever used again.
    means that is just about luck. A node operator should be able to decide over the files that he stores, and not just store useless data, that nobody will want it again.

These will lead on some Nodes having lot of traffic and others not so much.

At this point this is pointed as a extra income and like nothing serious, but i am convinced that this is how many storage company’s will work in the near future, and with that in mind I would totally appreciate having control over my Node/s.

No he should definitely not.

4 Likes

The Storj network works like a giant worldwide RAID array. In such architecture, it doesn’t make sense to give individual nodes decision power over which files to store where… the “controller” isn’t the SNO.

2 Likes

it like if gas station not sell you gas, because you not buy 50L and need only 5

I have brought this to the attention of our SNO growth team, they are already working on an improved version of the SNO dashboard, but they may also incorporate some of the suggestions for improvements from this thread.

4 Likes

A post was merged into an existing topic: Using ZFS Arc cache on non-ECC RAM

Absolutely not, the files are not yours and they make zero sense to you as they are just very small chunks of real files, and encrypted. I think you misunderstand what the network is all about.
If a file is popular the network will ensure that more storage operators have a copy, there is no need for SSD or any other improvements as the size of the files are just MBs in size.

Moving files around just means that there is a greater chance that a file will be lost and you will fail in audit, and will be kicked out from the network.

2 Likes

You cannot move files around. They need to be exactly at the same place where the storagenode puts them.

Yep, that’s pretty much it.

What are you talking about? How do yo want to force customers to download their files more frequently?

That’s not what he meant. He wants to be able to “throw out” that data that doesn’t get downloaded and only keep data that gets downloaded frequently enough.

1 Like

Wow, even worse. …

1 Like

And that was not what op means either

It was heavily implied here.

This obviously wouldn’t work as no one would want the data that is never downloaded and so no one would be able to store relatively static long term data on tardigrade. So no, node operators should definitely not get to choose and break the entire system.

1 Like

Yes you are Right, my previous comment seems to be valid, op should not be participating in the network

1 Like

I truly agree with everything said here, no option is best, but as a Node operator information is important, especially see all errors together, counted, organized and with statistics. There could be many failed uploads/downloads for which many ask in this forum (and i see in my logs this, but there is no clear reason of why, and i cannot spend time in organizing the info in the log, since it can be done automatically, just to reveal is a network problem, a software problem, or simply the client canceled the transfer)

Money is a important factor too, since even if one person simply installs the software and lets it run, there are costs. But there are bugs, glitches and things to solve, so you finally end up spending some time. So there should be a balance between spending’s and income, mainly if the Nodes should be there for a long time. It is only my opinion and it doesn’t have to be right, but I truly believe that this project wants and will go a from the “extra income” idea , to a “stable income”, and wants long lasting, stable Nodes.

It would be great to see audit counts by satellite so when I start a new node I have an idea how the vetting progress is going. This could be actual counts, or percent of 100 audits required.

1 Like