If you’re developing a client-side web application, you can make your bucket public and embed content directly. Everyone on the internet will have read-only access to the files.
There is no way to lock the bucket down if you ONLY have a client side web application, you would need your own server to achieve that. You can either proxy traffic through your server, or use presigned urls. The server would need some sort of user authentication to ensure no one else can view the urls. If you go the server route, yes, use the AWS SDK for javascript for the best experience. For an example, see Sample NodeJS and React code using presigned urls and AWS SDK V3