Is it possible to move the "storage\temp\blob-*.partial" to a different folder?

SGC · September 7, 2021, 6:57pm

is this suppose to be a docker flag?
because i can’t find it, i would much rather have it in the run command rather than inside the container.

Alexey · September 7, 2021, 7:16pm

github.com

storj/storj/blob/1c3688f1d62b9273cf149d32b2294a4ce3d6fee8/storage/filestore/store.go#L40

    
      
          
          
	_ storage.Blobs = (*blobStore)(nil)
          )
          
          
func monFileInTrash(namespace []byte) *monkit.Meter {
          	return mon.Meter("open_file_in_trash", monkit.NewSeriesTag("namespace", hex.EncodeToString(namespace))) //mon:locked
          }
          
          
// Config is configuration for the blob store.
          type Config struct {
          	WriteBufferSize memory.Size `help:"in-memory buffer for uploads" default:"128KiB"`
          }
          
          
// DefaultConfig is the default value for Config.
          var DefaultConfig = Config{
          	WriteBufferSize: 128 * memory.KiB,
          }
          
          
// blobStore implements a blob store.
          type blobStore struct {
          	log    *zap.Logger

0r

storagenode setup --help | grep filestore
      --filestore.write-buffer-size memory.Size                  in-memory buffer for uploads (default 128.0 KiB)

Toyoo · September 7, 2021, 7:24pm

It’s a storage node option. When using docker images you can pass it by:

docker run … -e RUN_PARAMS="--filestore.write-buffer-size 4096kiB" …

This is the entrypoint script used by Docker, you can see it uses RUN_PARAMS too and deliberately preserves whatever contents you pass from the command line: storj/entrypoint at e5977ec8499c16500e294708a6824a681429adf0 · storj/storj · GitHub

SGC · September 7, 2021, 7:43pm

thanks much appreciated, not really that well versed in docker nor storagenode parameters.

tried it out seems to work like a charm, added it before the docker -d flag + ARG’s…
obviously… but still thought i would mention it for anyone else that might be even worse at this than i am lol

JDA · September 7, 2021, 8:47pm

Ok for the sake of science I had to test and moving the temp folder with a junction link doesn’t work:
2021-09-07T22:41:26.006+0200 ERROR piecestore upload failed {"Piece ID": "HSQG6W32TBHIWNNQ76DMRJGW3D5EBL37F3WJK3LR77THX2EUJBBQ", "Satellite ID": "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S", "Action": "PUT", "error": "pieces error: filestore error: replace D:\\Storj\\Data\\storage\\temp\\blob-878937550.partial D:\\Storj\\Data\\storage\\blobs\\ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa\\hs\\qg6w32tbhiwnnq76dmrjgw3d5ebl37f3wjk3lr77thx2eujbbq.sj1: The system cannot move the file to a different disk drive.", "errorVerbose": "pieces error: filestore error: replace D:\\Storj\\Data\\storage\\temp\\blob-878937550.partial D:\\Storj\\Data\\storage\\blobs\\ukfu6bhbboxilvt7jrwlqk7y2tapb5d2r2tsmj2sjxvw5qaaaaaa\\hs\\qg6w32tbhiwnnq76dmrjgw3d5ebl37f3wjk3lr77thx2eujbbq.sj1: The system cannot move the file to a different disk drive.\n\tstorj.io/storj/storage/filestore.(*blobWriter).Commit:120\n\tstorj.io/storj/storagenode/pieces.(*Writer).Commit.func1:130\n\tstorj.io/storj/storagenode/pieces.(*Writer).Commit:198\n\tstorj.io/storj/storagenode/piecestore.(*Endpoint).Upload:401\n\tstorj.io/common/pb.DRPCPiecestoreDescription.Method.func1:209\n\tstorj.io/drpc/drpcmux.(*Mux).HandleRPC:33\n\tstorj.io/common/rpc/rpctracing.(*Handler).HandleRPC:58\n\tstorj.io/drpc/drpcserver.(*Server).handleRPC:102\n\tstorj.io/drpc/drpcserver.(*Server).ServeOne:60\n\tstorj.io/drpc/drpcserver.(*Server).Serve.func2:95\n\tstorj.io/drpc/drpcctx.(*Tracker).track:52", "Size": 8704}

flo · September 8, 2021, 5:43am

Would be really nice if somebody could write a patch that would allow us to move the temp folder to an SSD… I’m not that good in go unfortunately :-/

SGC · September 8, 2021, 7:11am

didn’t the filebuffer fix it… i mean that should make stuff stay in memory until its ready to write out?

JDA · September 8, 2021, 7:30am

Again, buffering the writes affect IOPs, not fragmentation. Creating the files with a size > or < of what the final size should be WILL generate fragmentation

SGC · September 8, 2021, 7:45am


 ls -lt --block-size=M | more
total 645M
-rw------- 1 root root 4M Sep  8 07:41 blob-113776038.partial
-rw------- 1 root root 4M Sep  8 07:41 blob-734808695.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-099298630.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-104431985.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-105337495.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-136319171.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-205912330.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-213586925.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-262762482.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-286865948.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-363794910.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-433614779.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-463376609.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-484036383.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-628020317.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-685286900.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-686582988.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-782702249.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-797460910.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-815919128.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-831509159.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-909285338.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-940900043.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-951380072.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-959962304.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-966410149.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-095836595.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-156129956.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-415889782.partial
-rw------- 1 root root 0M Sep  8 03:00 blob-483025945.partial
--More--

so i did the filepiece parameter for the storagenode and it clearly fixes the size to whatever i set it to…

now the only question really is if all .partials becomes a certain size.

i just changed it, node was crashed which is why it hasn’t made any since 03:00 until 7:41

now the question is if it will also be 4mb when its moved… and i bet you my zfs will not let that be a 4mb file anyways because it’s compressed…

SGC · September 8, 2021, 8:04am

it does resize for long term storage, so thats good… i’m guessing thats the best we can do for now…

storj/storage/blobs/pmw6tvzmf2jv6giyybmmvl4o2ahqlaldsaeha4yx74n5aaaaaaaa# ls -lt --block-size=M | more
total 77M
drwx------ 2 root root 1M Sep  8 07:59 uq
drwx------ 2 root root 1M Sep  8 07:59 m3
drwx------ 2 root root 1M Sep  8 07:59 k3
drwx------ 2 root root 1M Sep  8 07:59 wi
drwx------ 2 root root 1M Sep  8 07:59 z4
drwx------ 2 root root 1M Sep  8 07:58 at
drwx------ 2 root root 1M Sep  8 07:58 4a
drwx------ 2 root root 1M Sep  8 07:57 zr
drwx------ 2 root root 1M Sep  8 07:56 uo
drwx------ 2 root root 1M Sep  8 07:56 us
drwx------ 2 root root 1M Sep  8 07:55 ks
drwx------ 2 root root 1M Sep  8 07:54 aq
drwx------ 2 root root 1M Sep  8 07:54 eu
drwx------ 2 root root 1M Sep  8 07:52 dx
drwx------ 2 root root 1M Sep  8 07:51 ml
drwx------ 2 root root 1M Sep  8 07:51 ia
drwx------ 2 root root 1M Sep  8 07:50 pn
drwx------ 2 root root 1M Sep  8 07:50 kh
drwx------ 2 root root 1M Sep  8 07:49 mo
drwx------ 2 root root 1M Sep  8 07:48 rr
drwx------ 2 root root 1M Sep  8 07:48 ru
drwx------ 2 root root 1M Sep  8 07:46 46
drwx------ 2 root root 1M Sep  8 07:46 xx
drwx------ 2 root root 1M Sep  8 07:45 gw
drwx------ 2 root root 1M Sep  8 07:44 w5
drwx------ 2 root root 1M Sep  8 07:43 av
drwx------ 2 root root 1M Sep  8 07:43 27
drwx------ 2 root root 1M Sep  8 07:43 x7
drwx------ 2 root root 1M Sep  8 07:42 ky
drwx------ 2 root root 1M Sep  8 07:42 ae
--More--

the docker run command ended up something like this.


docker run -d -e RUN_PARAMS="--filestore.write-buffer-size 4096kiB \
--pieces.write-prealloc-size 4096kiB" \
-d --restart unless-stopped --stop-timeout 300 \
-p 192.168.1.100:28967:28967/tcp -p 192.168.1.100:28967:28967/udp \
#--log-opt max-size=10m \
-p 192.168.1.100:14002:14002 -e WALLET="0x111111111111111111111" \
-e EMAIL="your@email.com" -e ADDRESS="global.ip.inet:28967" \
-e STORAGE="4TB" --mount type=bind,source="/sn3/id-sn3",destination=/app/identity \
--mount type=bind,source="/sn3/storj",destination=/app/config --name sn3 storjlabs/storagenode:latest

best to keep changes in the run command, so that config.yaml is default to avoid confusing in later troubleshooting.
also in case of reinstalls and what not, its nice when its just copy paste…

Adavan · September 9, 2021, 4:04pm

I started, I take care of the node when I have time and otherwise I have no problems . Yes, I do not use an operating system from the Redmon Moloch .