If you are making a CloudFormation template and including an S3 bucket for logs you are probably setting a bucket policy. Here’s a hard to find problem. Let’s say you are configuring VPC flog logs and CloudTrail logs to go to the same logs bucket. Makes sense right? So to make that happen you make your bucket and you give it a bucket policy. In the bucket policy you add the necessary policies to allow both VPC flow logs and CloudTrail log to be written to the bucket. You kick back and admire your work.
But when you create a stack from the template you get an error that says the CloudTrail log bucket has an invalid bucket policy. Wait, what? You double-checked in your CloudFormation template and all is as it should be. So then you go to S3 to look at the bucket policy for yourself. The bucket policy set on the logs bucket doesn’t look anything like what you have set in your template. In fact, the statement IDs don’t even match what’s in your template! They aren’t in there anywhere! What is happening?
Well, it turns out there is a gotcha. I’ll let the AWS documentation try to explain it:
If the user creating the flow log owns the bucket, has PutBucketPolicy permissions for the bucket, and the bucket does not have a policy with sufficient log delivery permissions, we automatically attach the preceding policy to the bucket. This policy overwrites any existing policy attached to the bucket.
Catch that? A policy automatically gets set for the bucket and it overwrites any existing policy! (Read it for yourself at https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-s3.html.)
That means the beautiful policy you spent valuable time on is just getting wiped out by an automatically set policy. Well, now that we know what the problem is we can fix it. How you fix it (ahem workaround) it is up to you, e.g. split the buckets into two, etc.