Hello!
Fairly new to these forums so excuse me if this particular post is or is not necessarily meant for this thread.
I’m having difficulties implementing the download functionality of the nodejs implementation for uplink.
the await download.read(buffer,buffer.length) doesn’t seem to be working I do have access to the object (through the download.info() it works!)so there are no errors there, however the read function seems to return an error, any help would be appreciated, thanks in advance guys.
hi @Agoni sorry youre having that issue – one of the mods here pinged engineering about it after seeing your post. the company is closed for christmas so the response is slower, but we will let you know as soon as we hear anything. thank you.!
The object that I’m trying to download is being successfully accessed, but the error is being caught at
{await download.read(buffer, buffer.length).then(async (bytesread) => {
}
I even tried to checkout the error handling files regarding the uplink library, but they don’t really seem to give much information about what’s wrong, my guess is there must be an error with how the bytes are read through the function?
@Agoni and @brtteu - Can you try with the following bit of code:
async function downloadfile(project){
var downloadOptions = new storj.DownloadOptions();
downloadOptions.offset = 0;
downloadOptions.length = -1;
//Downloading file
await project.downloadObject(storjConfig.bucketName,storjConfig.uploadPath,downloadOptions).then(async (download) => {
var objectsize =0;
console.log("Fetching download object info");
await download.info().then((objectInfo) => {
objectsize = objectInfo.system.content_length;
}).catch((err) => {
console.log("Failed to get downloading object info");
console.log(err);
});
var size = { download : 0,
actuallyWritten : 0,
totalWritten : 0};
var buffer = new Buffer.alloc(BUFFER_SIZE);
var fileHandle = await fs.openSync(localFullFileName.dest, "w");
var loop = true;
while(loop) {
if((objectsize-size.totalWritten>0)&&(objectsize-size.totalWritten)<BUFFER_SIZE){
buffer = null;
buffer = new Buffer.alloc(objectsize-size.totalWritten);
}
//Reading data from storj V3 network
await download.read(buffer,buffer.length).then(async (bytesread) => {
size.download = bytesread.bytes_read;
size.actuallyWritten = await fs.writeSync(fileHandle, buffer, 0, size.downloaded, size.totalWritten);
size.totalWritten = size.totalWritten + size.actuallyWritten;
if(size.actuallyWritten>=objectsize){
loop = false;
}
if((size.totalWritten>0)&&(objectsize>0)){
console.log("File Dowloaded : ",((Number(size.totalWritten)/Number(objectsize))*100).toFixed(4)," %");
}
}).catch((err) => {
console.log("Failed to read data from storj V3 network ");
console.log(err);
loop=false;
});
if (size.totalWritten >= objectsize) {
break;
}
}
fs.closeSync(fileHandle);
//Closing download
await download.close().then(() => {
console.log("Object Downloaded Successfully");
}).catch((err) => {
console.log("Failed to download object");
console.log(err);
});
}).catch((err) => {
console.log("Failed to download file");
console.log(err);
});
}
One thing I did notice is your filesystem i/o is not async’d awaited so it’s possible your code is finishing before those operations complete and are managed properly.
I’m unable to download an object using this code snippet. It seems that the text is garbled at the end of each chunk, despite the code being properly awaited.
Hi guys, fixed the problem a while ago. Just completely forgot about the thread, sorry for not replying sooner.
Essentially the problem was defining the downloadOptions.length = -1;
where the downloadOptions.length represents the size of the file that you are attempting to download, since it’s -1 it constantly failed. Once I properly analyzed the size of the files im attempting to download the whole download process was successful.
