Network statistics

The Storj team is actively trying to undo external tracking of the network. There are benefits to running an opaque network. It would be nice to have a Storj Node “Oracle” that accepts hashes of Node IDs.

The proposed “Oracle” would need to be run by Storj and would just give a boolean response to a submitted node ID hash. In this way, validated analytics could be collected anonymously. However, it would require Storj to provide the validation service.

2 Likes

If Storj agrees to it, that might work. That’s a really good idea.

That would be great and easy to implement, sure. I just don’t expect them doing it tbh…

Even with encryption, signing with a key and registration etc you still can’t prevent some trolls from submitting wrong data. The data might not be obviously wrong but could just a couple TB more than they have or a lot less… The only way to verify that would be to compare it to the payment but then you’d need data about all their nodes. And some troll could still just falsify his data to be equivalent to the payment but e.g. immense storage without download. How would you verify data and find out wrong data? Therefore such a project is likely to either fail or not be accurate or and immense workload. The only reasonable and reliable way to do it is from the satellite side but as we know, storjlbas won’t do that.

2 Likes

I think we’ve been here before…

Different threads and subject, but same issue.

“Help! My node DQ-ed. Why can’t I verify my data and do GE?”

Of course, verifying the truthfulness of a node’s claims would require satisfying this common question.

A private blockchain recording some important events in a limited time frame would be able to provide locally verifiable node history.

you can put whatever you want onto the blockchain. how would you know if the data that ends up there is true?

The same holds true for Storj provided analytics or node information.

The only method that I know of would be signed transactions on a shared public ledger… which is used as the basis for payment of services as well as node analytics. If the information is a lie, at least one can show that the lie was accepted and paid…

It’s very unlikely that the information collected via an automated private blockchain would be false. It’s possible to design a system that operates more securely… with an application key signing process from Storj… but now we’re talking massive PKI deployment.

So… to OP… how many millions do you want to invest on the analytics app ?

storj has all the correct data on the satellites as they are paying all nodes. So they either provide correct data or they lie about their network (which they don’t do but there are only those 2 options)

Apart from that I think we are in agreement that as user/client based analytics solution that is “uncheatable” is basically impossible and would have higher requirements than running a node itself :smiley:

lol it costs whatever it costs, right? :joy:

Sure it isn’t. It wouldn’t be impossible with Storj’s help as well, just harder.

The key is to make it hard enough that only a few people will do it (especially as there is no real benefit for them anyway) and the damage one person can do as little as possible is.

aka. one can easily create 3 gmail accounts, but can they create 3000?
Also, outliers can be ignored in the statistics, so people who send ridiculous data won’t be included.

yes statistically speaking it wouldn’t matter if only a few are trolls as long as you have 2000 who aren’t. But my guess is that such a project would get the interest of maybe 100-200 people which is not enough to get any meaningful numbers.

http://storjnet.info/

Personally I am against anyone gathering info about my node even if its anonymous. I would rather get that info from Townhall Meetings.

2 Likes

So just to be clear, your opposed to another SNO having that data or to Storj having it and publishing it in an aggregated and annonomized way? Because, frankly, the Sat’s already have much of the needed data to get a “10,000 foot view” of the network(amount of data stored, avg piece health of 80 chunks, # of nodes connected, ingress for the day, egress for the day).

This kind of stat is published every TownHall

The same doesn’t hold true for Storj. They have signed transfers for all incoming data and do regular audits to verify. Storj can at least be reasonably sure about how much data is stored.
As for free space, that’s never going to be reliable. One of my nodes runs on a 16TB volume with thin provisioning, for which at most 3.6TB of phyisical space is available. No matter how solid you try to make that, the file system will report a total volume of 16TB. So yes, Storj doesn’t know how much space is available. Nobody does.

As for your suggestion to use blockchain, everything you suggest can be done by using public key crypto and doesn’t require a blockchain. Blockchain is not a holy grail and most trust exchanges have already been solved before it ever existed. Neither however prevents faking the actual data. Only Storj or the customer could verify a node’s claims about data stored or transferred and I don’t think we have an option to involve them.

Never underestimate the desire for people to be named at the top of a list…

1 Like

The point about blockchain is not to verify the truth of the input data of a particular event… but to ensure that the historical record hasn’t been modified… All lies are recorded forever, which means people are less likely to lie. Programming errors are also recorded forever, so miscalculation or misreporting of an event can never be purged or memory-holed.

In order to have something similar with stand alone keys, one would need to sign each event and cross-sign each new key and event… which is a loose definition of a blockchain.

Blockchains are a really wonderful tool for ensuring immutable recording of events, which is why I put it in the discussion when someone refers to lying about past events. Private blockchains do not require any monetary support system… which is why they are wonderful tools for recording non-monetary events. All participants “sign-in” with their immutable keys… which is why they are also wonderful tools for authentication to services.

However, blockchains can’t determine the truthfulness of the external inputs… which is why I indicated that an embedded public key on a Storj issued cert would be required as a trust anchor to ensure that the Storj software has not been modified and was faithfully doing whatever it was programmed to do by the authorized entity.

Of course, most of this thread is wishful thinking and arm-chair development of hypothetical features…

I don’t think immutability solves anything in this scenario. You’re storing immutable data without verification. So my lies are immutable? Who cares, I’m still at the top with my 16 yottabytes of stored data!

Immutable records of events are important if those events actually happen to have dependencies. A transaction of tokens leads to different amounts of tokens being available to different token holders. Which in turn allows them to spend them.
My activity report has nu such dependency. I could claim to store 16 yottabytes one day an 5 bytes the next and then report 326 exabytes the day after. Whether this is stored on an immutable blockchain or a database connected to a stats collection service really doesn’t matter. It’s still nonsense.

If Storj makes the cert they use to sign node identities availably publicly, the exact same can be done with a simple signature check on a signed stat report sent by the node. Definitely no need for a blockchain.

And either way, all that does is verify that it’s the owner of the node who lied about his stats.

Known and acknowledged- This dashboard is the “more live” view though of what is going on. While things like QtoQ gives you the big points, but there’s also the day to day stat trackers (think stock watchers) that would want finer granularity than 4 times a year. Also, given there is multiple Sat’s, the case could also be made for the fact that you may want to investigate stats for a specific Sat’s traffic vs another’s.

I would like to see this information too. Are there any information sites dedicated to storj for this info?

@madbitz I like @BrightSilence’s answer to this:

This info is regularly shared with SNOs during town halls, but it’s just what nodes advertise…

2 Likes