Subject: S3 MissingContentLength Error with Storj Gateway Using Boto3 and AWS CLI
Hi everyone,
I’m encountering a persistent “MissingContentLength” error when attempting to upload files to the Storj S3 gateway (gateway.storjshare.io ). This issue occurs across multiple standard S3 clients, including a custom Python script and the official AWS CLI, which leads me to believe the root cause is environmental rather than an issue with my code.
1. Python Boto3 Attempt
My initial approach was to use a Python script with the Boto3 library. I was aware that some S3-compatible services require the Content-Length header to be set explicitly. As you can see in the code below, I calculate the file size and pass it directly as the ContentLength parameter in the put_object call, but the error persists.
Upload Function Code:
import os
import boto3
from botocore.exceptions import ClientError
def upload_to_storage(zip_path):
"""
Uploads a file to an S3-compatible bucket, explicitly setting ContentLength.
"""
try:
s3 = boto3.client(
"s3",
aws_access_key_id=os.getenv("aws_access_key_id"),
aws_secret_access_key=os.getenv("aws_secret_access_key"),
endpoint_url=os.getenv("endpoint_url"),
)
bucket_name = os.getenv("aws_bucket_name")
s3_key = os.path.basename(zip_path)
file_size = os.path.getsize(zip_path)
print(f"Uploading {s3_key} ({file_size} bytes) to bucket {bucket_name}...")
with open(zip_path, "rb") as f:
s3.put_object(
Bucket=bucket_name,
Key=s3_key,
Body=f,
ContentLength=file_size
)
print(f"Successfully uploaded {s3_key}")
return True
except ClientError as e:
print(f"Upload failed: {e}")
return False
except FileNotFoundError:
print(f"Upload failed: The file {zip_path} was not found.")
return False
2. AWS CLI Test
To verify that this wasn’t a Boto3-specific problem, I attempted a simple file upload using the AWS CLI. This command also failed with the exact same MissingContentLength error.
Command:
aws s3 cp README.md s3://frappe-database-backup/README.md --endpoint-url [https://gateway.storjshare.io](https://gateway.storjshare.io)
Output:
upload failed: ./README.md to s3://frappe-database-backup/README.md An error occurred (MissingContentLength) when calling the PutObject operation: You must provide the Content-Length HTTP header.
Hypothesis: Intermediate Proxy or Firewall
Since my code explicitly provides the Content-Length header and the standard AWS CLI fails as well, the problem likely lies within my network environment. I suspect that a reverse proxy (e.g., Nginx) or a firewall between my client and the Storj gateway is stripping the Content-Length header from the request.
Has anyone encountered a similar issue? I would greatly appreciate any guidance on the correct server or proxy configurations required to ensure this header is preserved and passed along to the S3 gateway.
Thank you!