Hi guys! It is interesting to read design draft, nevertheless I have very few knowledge about what is going on under the cover (in current code base), so please sorry in advance.
As we all know - it is not a good practice to organize queues on databases - messing with reads, writes and locks is dramatically reduces performance, so it is a best practice to make queues on something like RabbitMQ or similar staff. Why don’t you use queues for such data-flow?
Also re cockroach db - it has high latencies (for ex. in comparison with proven Postgres) which seems pretty critical for read-write networking tasks… BTW Postgres supports sharding, HA clustering etc.
Sorry for the later than usual response on my end. I’ve caught a bad case of the flu over the past week and still recovering and catching up. I’m happy to see this one as it intents to fix an issue of confusion I have “complained” about in the past. From what I can tell it’s a fitting solution, though I agree that if performance becomes an issue, queuing is better done outside the database.
More surprisingly the benefits do not list perhaps the most important benefit, no more 7-8 days delays payout and bandwidth usage display. Either way, looks good and looking forward to seeing it implemented.