Problem accessing Storj using REST API and Python

Hello,

I would like to upload files to my Storj account using the API with Python code, but I’m encountering access problems. Here is a portion of the code and the error message:

storage_options = {
‘key’: ACCESS_KEY_ID,
‘secret’: SECRET_ACCESS_KEY,
‘client_kwargs’: {
‘endpoint_url’: ENDPOINT_URL
}
}

import numpy as np
import pandas as pd

bucket = “mybucket”
key = “random.csv”

Creating a random dataframe.

df = pd.DataFrame(np.random.uniform(0,1,[10**3,3]), columns=list(‘ABC’))

Saving as CSV

df.to_csv(
f"s3://{bucket}/{key}",
index=False,
storage_options=storage_options)

new_df = pd.read_csv(
f"s3://{bucket}/{key}",
storage_options=storage_options)

ERROR


ClientError Traceback (most recent call last)

/usr/local/lib/python3.10/dist-packages/s3fs/core.py in _error_wrapper(func, args, kwargs, retries)
112 try:
→ 113 return await func(*args, **kwargs)
114 except S3_RETRYABLE_ERRORS as e:

18 frames

ClientError: An error occurred (AccessDenied) when calling the CreateBucket operation: Access Denied

The above exception was the direct cause of the following exception:

PermissionError Traceback (most recent call last)

/usr/local/lib/python3.10/dist-packages/s3fs/core.py in _error_wrapper(func, args, kwargs, retries)
138 err = e
139 err = translate_boto_error(err)
→ 140 raise err
141
142

PermissionError: Access Denied

I have configured the accesses with all the necessary permissions (in my Storj Account). Any idea what could be going wrong?

Best Regards!

Hello @jrperassoli,
Welcome to the forum!

You need to provide an endpoint URL https://gateway.storjshare.io, otherwise it will try to connect to the AWS instead.
Since you want to create a bucket, you need to generate s3 credentials without limiting to any bucket, it also should have all permissions, include List. You may disable Delete though.

See also example Using presigned URLs - Storj DCS Docs

For a native integration please take a look on GitHub - storj-thirdparty/uplink-python: Python bindings for libuplink

4 Likes

Perfect. It works now.

1 Like