Short update from a meeting with the developers today about the db locking:
- The piece size is way too small and causing an IOPS bottleneck that we need to fix. I am not sure about the details but the long term goal is to write bigger pieces to disk. That should improve performance.
- Why do we store serial numbers for 48h in a DB? We should keep that in memory and don’t write it to disk at all. That will solve that problem entirely.
- We still want to write orders into a DB. If the node is restarting we can drop the serial list but we can’t drop unsubmitted orders.
- Something is wrong with our sqlite implementation. Why do we have to call vacuum? That shouldn’t be needed. Autovacuum should also not be needed. It could be a problem with the go sqlite driver. At that point, I faded out. Too many details for me. I was sold on the in memory solution already.