Replace DB by metadata (object storage)

Ideas & Suggestions
1

Hello, I would like to suggest this to the community developers.

############## REPLACE DB BY METAS ##########

  1. Architecture with SQL Lite DB
    I guess that DB can be corrupt in a software SPOF (Single point of failure) model (on 1 storage node)
    DB corruption will not ensure a SN to cummunicate with the network.
    Exemple : pieceinfo.db, this file can be corrupt because of a system crash and sync problem

824×720

  1. Architecture without DB
    Database is underlied with metadata resulting to a no-SPOF architecture
    Descriptors (metas) file XML/Json is uploaded with piece and contain link to the piece

ENCAPSULED USER FILE :

  • CONTENT (unmodified)
  • META#1 = PIECEINFO
  • META#2 = PIECEEXPIRATION
  • META#3 = PECESPACEDDUSED
  • META#4 = PIECEMETA

ENCAPSULED SATELLITES INTERNAL DB FILE

  • CONTENT = TXT
  • META#1 = INTERNALDBFILE
    → Parse and update TXT element with regular expressions and separator

In case of issue with DB :
DB is corrupted and there is many elements to correct in the DB because of relations in the database ( SQLite is a relational database)

In case of issue with metadata :
File is corrupted and can be updated again by another storage node in case of crash, maintening files consistences over the network.

Thanks.

I don’t know if research are performed for this subject actually.

Best regards.

2 Likes