Check if S3 bucket name is already taken when generating bucket name

[larissa-n]

Is your feature request related to a problem? Please describe.

Deployment of new install (following tutorial) kept failing at bucket creation with error 403.

Trying to create bucket of the same name in AWS directly, the response is: cannot create bucket with same name as one that already exists.

AWS policy is that S3 buckets must have unique names across all of AWS, so if you pick a bad name and don't check, bucket creation will fail. It seems that that's what happened with the webiny script.

Describe the solution you'd like.

Automatically generate bucket name that does not already exist on AWS.

AWS recommends checking if a bucket name already exists before generating a new one, and has a command for it: https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/head-bucket.html

Describe alternatives you've considered.

It might be worthwhile to mention the issue in the docs, and as a temporary workaround recommend to delete the project and start from scratch.

[Pavel910]

@larissa-n that's weird, because Pulumi generates unique resource names, by suffixing our names:

How did you come to a conclusion that it's due to a bucket name collision, are there any other error logs you could share?
Thanks!

[larissa-n]

Deployment failed every time with 403 error (i.e. bucket permission issue), see screenshot above. Trying to manually create bucket of the name that webiny had picked in AWS directly was refused with: can't create bucket of name that already exists.

How does pulumi check if a bucket name is unique? Bucket names in AWS have to be unique across all of AWS, not just unique within the user's account.

[Pavel910]

@larissa-n Webiny doesn't pick a full name, only the base name. Pulumi appends a random suffix to make it unique. We never had any issue with it, and we run our CI dozens of times each day. Can you please send the bucket name you tried to create manually?

[larissa-n]

In this case, the bucket name pulumi came up with was fm-bucket-394a717.

"Random" of course does not guarantee "unique". If you guys think it's fine and the cases where the randomly generated bucket names are already taken will be rare, sure, don't bother to add a check. If you wanted, you might add a note to the docs. Certainly not the highest priority issue, but it might become more relevant as you guys scale. In either case, AWS has a standard function for checking if a bucket name is taken, and recommends developers make use of it.

[Pavel910]

Need to see if Pulumi itself provides any mechanisms for that. You're the first one who reported this issue, so it's really weird. Webiny is not an IaC tool, and we want to minimize the amount of custom stuff on top of the existing IaC tools we use, otherwise we'll quickly find ourselves in maintaining stuff which is really not ours to handle in the first place.

First, we need to check what checks Pulumi does itself, and if it doesn't check, see if we can make it check this for us.
It's really surprising to me that we never ran into resource naming issues over the years 🤔 If it was that easy to run into, we'd be seeing this every other day.

[webiny-bot]

This issue is stale because it was opened 120 days with no activity. Remove the "stale-issue" label or leave a comment to revive the issue. Otherwise, it will be closed in 7 days.