Audit Verify Process

Hello there,

I have one doubt want to clarify.

As audit process will be in infinity loop and it verifies data(segments) if that are reliable then it gives audit success count otherwise not.

So my question is which data(segments) will be compared in this process?

Afaik one will be taken from satellitedb (table = segments) another will taken from where?

Thanks in advance

@Alexey

The small part of the random piece from each segment will be requested from nodes for audit. It uses math to determine is it valid or not, see high level description here: Decentralized Auditing and Repair! The Low-key Life of Data Resurrection
and more detailed with math here: Introduction to Reed-Solomon. We’ve open-sourced a high-performance… | by Jeff Wendling | Vivint Innovation Center

1 Like

@Alexey Thanks for your reply,

But I want to know from nodes which data it will take for validation?

Because I have checked node db (sqlitedb) there is no information regarding such thing(data to get for comparision).

Am I missing something?

Please read articles above, there is no comparison, there is a math calculation. The node doesn’t contain required hashes, because otherwise it can try to foolish the satellite. The satellite calculate is the returned data valid or not.

1 Like

Returned from where?
Who returns data to satellite?

@Alexey

It just uses any random piece of data on your node and retrieves all pieces for that same segment from other nodes as well. The Reed Solomon erasure encoding then allows the audit worker to see which pieces from which nodes are correct and corrupted. Your node doesn’t have to do anything special as literally any stripe of any piece can be used for auditing and they are selected by the satellite at random.

3 Likes

To confirm what @Alexey already said…

Your questions are answered there.


Your node returns the audit data to the satellite when requested.

2 Likes

So these info might be stored somewhere…!

I didn’t find it in nodedb(sqlitedb)

Where it actually stores on node side…?

@BrightSilence

Yes exactly I want to know that from where node return data to satellite ?

Where it could be stored on node?

It can be any of the ‘blobs’ data which that satellite thinks your node is storing. So there is no ‘shortcut’ if you see in the log file you can see the specific piece being requested.

1 Like

What info? I’m not sure what you’re looking for.

Also, I don’t mean to presume, but it sounds awefully like you’re trying to find out if you can mess with the audit system to fool it. Don’t bother, you can’t. If that’s not your intention, I apologize.

3 Likes

It’s the data itself, not some sort of metadata of it.

Like, you can compare hash of a file you downloaded to confirm it is a legit one. You check on the main website, or any other trust source, what is the expected hash value. You calculate the hash of the file you downloaded, compare the hashes and that’s it. All you need to get the hash is data itself and the hashing algorithm.

So satellite gets the data and that’s all it needs from you. It then proceeds to calculate stuff and cross-compare all the pieces. More or less - don’t quote me if it happens to do it in a slightly different manner. The core point to take from this is that it needs just the data from you.

Hello @BrightSilence

No that was never my intention…

I just want to know the audit system get audit success in which case theoratically I get understood from article shared by @Alexey .

But I still confused that which data node will return?
In nodedb afaik there is nothing present, that could be returned by node.

Yes if you see in satellite db there is present(segments table in SatelliteDB) but what about node?

Don’t get me wrong I just wanted to understand this process practicallly…

If you could help me with that…

Thanks in advance.

The requested part of the piece from the blobs folder, no databases are involved.
So the node will physically read the requested sequence of bytes from the file from the blobs folder from the filesystem, and return it to the auditor.
You may consider blobs as a filesystem-based database. So selection would be more complicated, then just select * from db_something where id="jtntngng";, but the result would be the same.

1 Like

Okay @Alexey

And the same process will be done at a time of downloading object?

Am I going right?

Yes, and the client can do it like an auditor - request not the whole piece, but only any part of it. However, the default for the client (libuplink) is to request the whole piece.
When it combines pieces back to the file the erasure codes math allows it to combine only valid pieces.

1 Like

A post was split to a new topic: Orders and OrderLimit