Amazon S3 Bucket Policy Permissions Don't Allow Dropbox API to Pull Files from Bucket

Question

This has been making me crazy all night.

I wrote a DropBox app in PHP/MYSQL that worked perfectly, it pulls files from an Amazon S3 Bucket and sends them to users Dropbox folders.

Then I changed the bucket policy on the Amazon S3 bucket to allow files to be pulled from only a handful of referrers, and signed URLS (example: /musicfile.mp3?AWSAccessKeyId=[accesskeyid]&Expires=[expires]&Signature=[signature]).

This works great for all purposes, except I learned my Dropbox functionality no longer works, it's because you pass the Dropbox API the URL of the mp3 on Amazon S3, and on Dropbox's side they pull the file in, so now that I have the bucket policy allowing only certain referrers, dropbox gets a permission denied and the API tells me it failed.

So I thought easy fix, I would simply add the ?AWSAccessKeyId= blah blah to the end of the file being passed to dropbox and all would work instantly, but, it doesn't because the file then doesn't end in an extension Dropbox recognizes so it again fails to work.

Then I thought I'd simply add the referrer from Dropbox to my bucket policy, I still have no idea what it is however and have added every variation of dropbox.com and api.dropbox with and without https, all with no luck.

If anyone has any idea or solution you will seriously make my week.

The absolute last thing I want to do is be forced to download the file first to my server, then send to dropbox, I really don't want to do that and I know I had this working perfectly already as it was, and it works instantly when I remove my bucket policy entirely, I just want it to work with it.

Answer 1

I assume, because you mention passing a URL to Dropbox, that you're using the Saver? If so, you can tell the Saver what file name to use, so give it the authorized URL and specify a filename so there's a file extension. E.g.:

<a href="https://...?AWSAccessKeyId=..." class="dropbox-saver" data-filename="myfile.txt"></a>

or, in JavaScript:

Dropbox.save('https://...?AWSACcessKeyId=...', 'myfile.txt');

When you say that "because the file then doesn't end in an extension Dropbox recognizes so it again fails to work," what do you mean, exactly? What goes wrong when the file doesn't have an extension?

Answer 2

When all else fails... check the logs.

Turn on logging for your bucket, run some tests, wait a few minutes for a log to appear, and then examine the logs to see what the referer is. It seems a safe bet that there won't be a referer because a user agent that isn't a web browser (such as Dropbox's back-end processes) would typically not have a reason to send a referer.

If it's any consolation, "securing" a bucket by constraining the referer is pretty much like not securing the bucket at all. It's extremely simple to defeat, and so it's only really effective protection against two classes of people:

• honest people
• lazy people

http://en.wikipedia.org/wiki/Referer_spoofing