Deploy gzip content on Amazon S3

Question

I use gzip compression for Amazon S3. I gzip HTML, JS, CSS files and keep images unchanged.

I sync everything using s3cmd:

s3cmd sync --cf-invalidate ./deploy s3://n12v.com

Unfortunately, this doesn’t set Content-Encoding: gzip to all necessary files.

I need to find all updated gzip files and set Content-Encoding: gzip for each one of them. The best solution I could came up with:

• find all gzipped files by running gzip --test filepath on every one
• s3cmd put --add-header='Content-Encoding: gzip' filepath s3://n12v.com/filepath, e.g. upload files again just to add a header.

This is very ad-hoc solution, is there any better ways of doing this?

Answer 1

I was having the same type of issue so I wrote s3tup. Unfortunately it doesn't offer anything like s3cmd's cf-invalidate flag (yet), but it makes configuring while rsyncing a lot easier. You'd write a config file that looks something like this.

# config.yml
---
bucket: n12v.com
rsync: ./deploy
key_config:
  - patterns: ['*.html', '*.css', '*.js']
    content_encoding: gzip

And then rsync from the command line like this.

$ s3tup config.yml --rsync_only

It's still early in development but maybe you'll find it useful.