Having difficulties using the download functions for nodejs implementation of uplink

Hello!
Fairly new to these forums so excuse me if this particular post is or is not necessarily meant for this thread.
I’m having difficulties implementing the download functionality of the nodejs implementation for uplink.

the await download.read(buffer,buffer.length) doesn’t seem to be working I do have access to the object (through the download.info() it works!)so there are no errors there, however the read function seems to return an error, any help would be appreciated, thanks in advance guys.

3 Likes

hi @Agoni sorry youre having that issue – one of the mods here pinged engineering about it after seeing your post. the company is closed for christmas so the response is slower, but we will let you know as soon as we hear anything. thank you.!

2 Likes

Hey @Agoni, could you provide your script so I can test it on my end?

1 Like

Are you using this library or a different one? https://www.npmjs.com/package/storj-nodejs

@Agoni - welcome!!

Might want to check out https://www.npmjs.com/package/uplink-nodejs also - storj-nodejs was deprecated. Can you confirm which lib you are using?

Hi, the currently am working with https://github.com/storj-thirdparty/uplink-nodejs
and I’m mainly following through this documentation -> https://storj-thirdparty.github.io/uplink-nodejs/#/tutorial ,
to give more clarification about the code and error

The object that I’m trying to download is being successfully accessed, but the error is being caught at
{await download.read(buffer, buffer.length).then(async (bytesread) => {

            console.log("Inside download.read ->",bytesread);
            fs.writeSync(fileHandle,bytesread);

            fs.closeSync(fileHandle);
          }).catch((err) => {
              console.log("Error has occured", err)
              });

}
I even tried to checkout the error handling files regarding the uplink library, but they don’t really seem to give much information about what’s wrong, my guess is there must be an error with how the bytes are read through the function?

1 Like

thanks for the screenshots, we’re forwarding this to engineering

I’m facing the same problem, when calling download.read(…) I get

{
  err: InternalError: internal error,
   ...
    code: 2,
    details: ''
  }

Any updates on this matter or temporary solution sugestions?

hi @brtteu, I was able to reproduce your issue with such code snippet:

const f> s = require(“fs”);
const storj = require(“uplink-nodejs”);
const libUplink = new storj.Uplink();

libUplink.parseAccess(‘my_access_grant’)
.then((access) => {
access.openProject().then((project) => {

        var downloadOptions = new storj.DownloadOptions();
        downloadOptions.offset = 0;
        downloadOptions.length = -1;

        project.downloadObject('michal', 'README.md', downloadOptions).then((download) => {
            var buffer = new Buffer.alloc(1792);

            download.read(buffer, buffer.length).then(async (bytesread) => {
                fs.writeFileSync("/home/wywrzal/test_README.md", buffer);
                console.log(bytesread)
            }).catch((err) => {
                console.log(err)
            })

            download.close().then(() => {
            }).catch((err) => {
                console.log(err)
            });
        })
    })
})

It looks that error is with closing download and without this part data is downloaded correctly.

InternalError: internal error
at Object.storjException (/home/wywrzal/git/nodejs-test/node_modules/uplink-nodejs/dist/error.js:109:23)
at /home/wywrzal/git/nodejs-test/node_modules/uplink-nodejs/dist/download.js:19:25
at async DownloadResultStruct.read (/home/wywrzal/git/nodejs-test/node_modules/uplink-nodejs/dist/download.js:18:27) {
code: 2,
details: ‘metainfo error: manager closed\n’ +
‘\tstorj.io/drpc/drpcmanager.init:23\n’ +
‘\truntime.doInit:5646\n’ +
‘\truntime.doInit:5641\n’ +
‘\truntime.doInit:5641\n’ +
‘\truntime.doInit:5641\n’ +
‘\truntime.doInit:5641\n’ +
‘\truntime.main:191’
}
I will contact developers and ask for solution.

1 Like

@Agoni and @brtteu - Can you try with the following bit of code:

async function downloadfile(project){
    var downloadOptions = new storj.DownloadOptions();
    downloadOptions.offset = 0;
    downloadOptions.length = -1;

  //Downloading file

    await project.downloadObject(storjConfig.bucketName,storjConfig.uploadPath,downloadOptions).then(async (download) => {
    var objectsize =0;
    
    console.log("Fetching download object info");
    await download.info().then((objectInfo) => {
        objectsize = objectInfo.system.content_length;
    }).catch((err) => {
        console.log("Failed to get downloading object info");
        console.log(err);
    });
    
    var size = { download : 0,
        actuallyWritten : 0,
        totalWritten    : 0};
    var buffer = new Buffer.alloc(BUFFER_SIZE);
    var fileHandle = await fs.openSync(localFullFileName.dest, "w");
    var loop = true;

    while(loop) {
        if((objectsize-size.totalWritten>0)&&(objectsize-size.totalWritten)<BUFFER_SIZE){
            buffer = null;
            buffer = new Buffer.alloc(objectsize-size.totalWritten);
        }

        //Reading data from storj V3 network
        await download.read(buffer,buffer.length).then(async (bytesread) => {
            size.download = bytesread.bytes_read;
            size.actuallyWritten = await fs.writeSync(fileHandle, buffer, 0, size.downloaded, size.totalWritten);
            size.totalWritten = size.totalWritten + size.actuallyWritten;
            if(size.actuallyWritten>=objectsize){
                loop = false;
            }

            if((size.totalWritten>0)&&(objectsize>0)){
                console.log("File Dowloaded : ",((Number(size.totalWritten)/Number(objectsize))*100).toFixed(4)," %");
            }
        }).catch((err) => {
            console.log("Failed to read data from storj V3 network ");
            console.log(err);
            loop=false;
        });
        
        if (size.totalWritten >= objectsize) {
            break;
        }
    }
    fs.closeSync(fileHandle);

    //Closing download
    await download.close().then(() => {
        console.log("Object Downloaded Successfully");
    }).catch((err) => {
        console.log("Failed to download object");
        console.log(err);
    });
}).catch((err) => {
    console.log("Failed to download file");
    console.log(err);
});

}

One thing I did notice is your filesystem i/o is not async’d awaited so it’s possible your code is finishing before those operations complete and are managed properly.

Please do let us know the result. Thanks!