I can't seem to resolve this one issue. Without www it doesn't work. For background it was originally pointing to a www. bucket but I've since made a new one and it's pointing to that one. I have a namecheap CNAME records pointing to cloudfront (added the * host CNAME today):
Cloudfront:
S3:
For the CNAME record portion of the set up I just added a @ as the host, this link helped