Gateway Issue with latest version

Im getting the below error when trying to use the latest gateway. Ive used it before so pretty sure I know what im doing (famous last words!) Any ideas?

EDIT: I should probably add, i get this error in the gateway when i try to upload a file. I can open files fine.

API: ListObjectsV1(bucket=perftesting)
Time: 20:20:37 GMT 03/19/2020
DeploymentID: ca62a51b-6d5e-41e2-bafa-16a549859501
RequestID: 15FDCE15EB452C60
RemoteHost: 127.0.0.1
Host: 127.0.0.1:7777
UserAgent: CloudBerryLab.Base.HttpUtil.Client 5.9.1 (http://www.cloudberrylab.com/)
Error: uplink: SetupProject error: prefix should end with slash
4: \go\pkg\mod\github.com\minio\minio@v0.0.0-20200306214424-88ae0f119610\cmd\api-errors.go:1786:cmd.toAPIErrorCode()
3: \go\pkg\mod\github.com\minio\minio@v0.0.0-20200306214424-88ae0f119610\cmd\api-errors.go:1811:cmd.toAPIError()
2: \go\pkg\mod\github.com\minio\minio@v0.0.0-20200306214424-88ae0f119610\cmd\bucket-handlers-listobjects.go:342:cmd.objectAPIHandlers.ListObjectsV1Handler()
1: net\http\server.go:2007:http.HandlerFunc.ServeHTTP()

Hey, I just tried the gateway from https://github.com/storj/gateway/releases/latest/download/gateway_linux_amd64.zip and I was able to use it to upload to us-central with no issues. What OS are you on? And could you provide more info so we can try to replicate?

Hey,

Im running this on win server, it says… running on version v1.0.0 - this the correct one?

Yeah, I am testing on v1.0.0-rc.6 from here https://github.com/storj/gateway/releases/tag/v1.0.0-rc.6

I will see if anyone can try to replicate this on Windows.

I’m tried to upload via Gateway on Windows

PS C:\Users\USER> aws s3 --endpoint http://localhost:7777 cp .\gateway_windows_amd64.exe.zip s3://test/gateway/gateway_windows_amd64.exe.zip
upload: .\gateway_windows_amd64.exe.zip to s3://test/gateway/gateway_windows_amd64.exe.zip

PS C:\Users\USER> aws s3 --endpoint http://localhost:7777 rm s3://test/gateway/gateway_windows_amd64.exe.zip
delete: s3://test/gateway/gateway_windows_amd64.exe.zip

Please, try to use a different bucket

Ive deleted the gateway.exe, rebooted the machine and tried again. Still get the same issue as per screenshot below. Im using the CloudBerry Explorer software as a GUI which has worked in previous versions.

I’m able to reproduce it without this Explorer.

aws s3 --endpoint http://localhost:7777 ls s3://test/gateway

An error occurred (InternalError) when calling the ListObjectsV2 operation (reached max retries: 4): We encountered an internal error, please try again.

In the gateway console:

API: ListObjectsV2(bucket=test)
Time: 23:41:37 MSK 03/20/2020
DeploymentID: 0993c209-d13c-45bb-8dcd-64a9343d105c
RequestID: 15FE1DCFC668BFEC
RemoteHost: 127.0.0.1
Host: localhost:7777
UserAgent: aws-cli/1.16.210 Python/3.6.0 Windows/10 botocore/1.12.200
Error: uplink: SetupProject error: prefix should end with slash
       4: \go\pkg\mod\github.com\minio\minio@v0.0.0-20200306214424-88ae0f119610\cmd\api-errors.go:1786:cmd.toAPIErrorCode()
       3: \go\pkg\mod\github.com\minio\minio@v0.0.0-20200306214424-88ae0f119610\cmd\api-errors.go:1811:cmd.toAPIError()
       2: \go\pkg\mod\github.com\minio\minio@v0.0.0-20200306214424-88ae0f119610\cmd\bucket-handlers-listobjects.go:263:cmd.objectAPIHandlers.ListObjectsV2Handler()
       1: net\http\server.go:2007:http.HandlerFunc.ServeHTTP()
2020-03-20T23:41:45.023+0300    ERROR   gateway error:  {"error": "uplink: SetupProject error: prefix should end with slash", "errorVerbose": "uplink: SetupProject error: prefix should end with slash\n\tstorj.io/uplink/private/metainfo/kvmetainfo.(*DB).ListObjectsExtended:290\n\tstorj.io/uplink.(*ObjectIterator).loadNext:97\n\tstorj.io/uplink.(*ObjectIterator).Next:76\n\tstorj.io/gateway/miniogw.(*gatewayLayer).ListObjectsV2:345\n\tstorj.io/gateway/miniogw.(*layerLogging).ListObjectsV2:100\n\tgithub.com/minio/minio/cmd.objectAPIHandlers.ListObjectsV2Handler:261\n\tnet/http.HandlerFunc.ServeHTTP:2007\n\tgithub.com/minio/minio/cmd.httpTraceAll.func1:341\n\tnet/http.HandlerFunc.ServeHTTP:2007\n\tgithub.com/minio/minio/cmd.collectAPIStats.func1:376\n\tnet/http.HandlerFunc.ServeHTTP:2007\n\tgithub.com/gorilla/mux.(*Router).ServeHTTP:212\n\tgithub.com/minio/minio/cmd.customHeaderHandler.ServeHTTP:748\n\tgithub.com/minio/minio/cmd.securityHeaderHandler.ServeHTTP:763\n\tgithub.com/minio/minio/cmd.bucketForwardingHandler.ServeHTTP:624\n\tgithub.com/minio/minio/cmd.requestValidityHandler.ServeHTTP:608\n\tgithub.com/minio/minio/cmd.httpStatsHandler.ServeHTTP:544\n\tgithub.com/minio/minio/cmd.requestSizeLimitHandler.ServeHTTP:68\n\tgithub.com/minio/minio/cmd.requestHeaderSizeLimitHandler.ServeHTTP:93\n\tgithub.com/minio/minio/cmd.crossDomainPolicy.ServeHTTP:51\n\tgithub.com/minio/minio/cmd.redirectHandler.ServeHTTP:247\n\tgithub.com/minio/minio/cmd.minioReservedBucketHandler.ServeHTTP:315\n\tgithub.com/minio/minio/cmd.cacheControlHandler.ServeHTTP:274\n\tgithub.com/minio/minio/cmd.timeValidityHandler.ServeHTTP:385\n\tgithub.com/rs/cors.(*Cors).Handler.func1:207\n\tnet/http.HandlerFunc.ServeHTTP:2007\n\tgithub.com/minio/minio/cmd.resourceHandler.ServeHTTP:525\n\tgithub.com/minio/minio/cmd.authHandler.ServeHTTP:449\n\tgithub.com/minio/minio/cmd.sseTLSHandler.ServeHTTP:798\n\tgithub.com/minio/minio/cmd.reservedMetadataHandler.ServeHTTP:136\n\tgithub.com/minio/minio/cmd.criticalErrorHandler.ServeHTTP:780\n\tgithub.com/minio/minio/cmd/http.(*Server).Start.func1:104\n\tnet/http.HandlerFunc.ServeHTTP:2007\n\tnet/http.serverHandler.ServeHTTP:2802\n\tnet/http.(*conn).serve:1890"}

BTW You can use a minio via browser if you want to have a GUI. http://localhost:7777

thanks @Alexey at least its not just me! Thanks also for the heads up on the browser thing, didn’t know about that!

I created an issue for the team.

2 Likes

This has been added to the next sprint for the user-growth team

1 Like

any update on this guys? Just tested it with the latest gateway release and issue still appears. I think its a pretty critical issue since it results in data loss/appearance of data uploaded correctly?

Hey @will.topping! Quick update- It’s in the next sprint. It was not in last sprint because the focus was on getting the QNAP app done. Also, what do you mean it results in data loss?

Great, thanks for the update. Re the data loss…from the GUI perspective the file seems to have uploaded fine but im assuming from the error in the Gateway that the file didn’t actually upload correctly and therefore won’t be retrievable.

The devs already implemented an error message in the gateway console indicating that you needed to add a / after the prefix. So the user should not assume that the file was successfully uploaded when they get this error. Therefore, there is no danger of data loss unless the user chooses to ignore the error.

Please, try in the new version

docker run -it --rm -p 7777:7777 -v /path/to/gateway/config/folder:/root/.local/share/storj/gateway/ storjlabs/gateway:3dcf2e4-v1.1.4-go1.13.8 run

You run the new version in docker?

Yes. The binary should be available soon too.

1 Like