Saving JSON Client-side to an S3 Bucket

| Comments

A co-worker came up with an interesting problem today. What’s the cheapest and easiest way to save relatively low traffic text content without having to create a server side component for it.

After thinking about it for a bit, I thought about using an public-writable S3 bucket and letting the client-side JavaScript PUT the JSON to the bucket. With a little bit of research and playing around, I was able to make it work.

This assumes you have an EC2 account with Amazon. If you do, you can log in to the console. Then, go to the S3 section of the console and create a new bucket that will hold the uploaded JSON files.

With that bucket selected, go to “Properties”, open up the “Permissions” tab and click on the “Add CORS Configuration” button. Put something like this in there:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
    <CORSRule>
        <AllowedOrigin>http://fiddle.jshell.net</AllowedOrigin>
        <AllowedMethod>PUT</AllowedMethod>
        <AllowedHeader>*</AllowedHeader>
    </CORSRule>
</CORSConfiguration>

But be sure to change the AllowedOrigin to whatever host name you’ll be doing your uploading from (I use jsfiddle for testing javascript so it’s a good one to test with).

Then, you’ll want to “Add a bucket policy” to the bucket to allow the world to upload to the bucket. Here is the one I came up with (replace the bucket name with yours):

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "AllowPuts",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME/*"
        },
                {
            "Sid": "DenyGetsToAllButMyIP",
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::YOUR-BUCKET-NAME/*",
            "Condition": {
                "NotIpAddress": {
                    "aws:SourceIp": "YOUR.IP.ADD.RESS/32"
                }
            }
        }
    ]
}

This policy has the effect of removing anyone’s ability to download or view the file who isn’t at whatever IP you put in the NotIpAddress CIDR address. This is an alternative to setting up an IAM policy that lets a specific user or set of users view the file, which I’ve found to be quite fiddly to get working.

If you’re having trouble with this policy, you can use the AWS Policy Generator to generate your own.

Then, just write your javascript to upload the JSON (this requires jQuery, replace YOUR-BUCKET-NAME in the upload path and set the data to whatever JSON you want it to be, you can tweak this fiddle):

var value = "foobar";
// generate random guid for filename
var guid = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
    var r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8);
    return v.toString(16);
});

// CHANGE THIS TO YOUR BUCKET NAME
var uploadPath = 'http://YOUR-BUCKET-NAME.s3.amazonaws.com/' + guid + ".json";

console.log(uploadPath);

$.ajax({
    type: "PUT",
    url: uploadPath,
    dataType: 'json',
    async: false,
    data: JSON.stringify({ "value": value })
});

Because this relies on CORS, it only works in IE8+, but for most applications, that should be OK.

This solution isn’t appropriate for all uses, but for low-traffic, relatively insecure stuff (like website “feedback” forms) it should be fine. There is also the possibility that someone malicious could upload whatever they want to your bucket (and make you pay the charges for it). Hopefully, setting the policy so that it’s only world-writable mitigates the potential for misuse.

Comments