Hashstore rollout commencing!

[jammerdan]

Question on hashstore reads and data location. As I understand it, with hashstore pieces are no longer file pieces but data within “log” files?

I read that this is supposed to help writes.

@jtolio

Hashstore also has the goal of improving alignment overhead and minimizing random writes.

But what about reads? As the pieces are no longer file system objects do frequently accessed pieces get cached? Because there are frequently accessed so called lucky pieces, causing massive amount of downloads:

@jammerdan

I recently had a lucky piece that was being downloaded at a rate of like 100 times per minute and more.

@Krawi

This is a single piece, logs from today, time: 8 hours 45 minutes.
That’s about one download request every 2 seconds, it started on June 12. and continuing since then.

@snorkel

… and the bit goes on…
(if we were back 4 years ago when egress was 20$… )

@EasyRhino

oh I got my first magic data fragment yesterday on one node. normal egress is around 8GB. yesterday it was 146GB

@snorkel

The lucky pieces still on… same 2 nodes last month:

So 2 questions regarding pieces in log files:

1. How (good) are reads cached with hashstore with the big logfiles in place now (because in my opinion such massive downloads should come entirely from RAM)?
2. Is there still a way to tell where a specific piece is stored which was easy with piecestore of course?