Sync old sonata media files to S3 after shift filesystem from local to S3

Question

We were storing all sonata media files on local directory earlier but now we have moved to AWS S3. After shifting to S3 now sonatamedia is unable to access old local files. Sonatamedia is trying to find old files on S3 as well. New files are uploading on S3 and accessible.

Now please advice how to sync our old data to S3 or SonataMedia bundle can look for old files on local instead of S3.

Our current SonataMedia configuration is as mentioned below

sonata_media:
filesystem:
    local:
        directory:  %kernel.root_dir%/../web/uploads/media
        create:     true
    s3:
        bucket: %sonata_media_s3_bucket%
        accessKey: %sonata_media_s3_accessKey%
        secretKey: %sonata_media_s3_secretKey%
        region: %sonata_media_s3_region%
        create: true
        .....

Answer 1

I was in the same dilemma, and I could successfully solve it. SonataMediaBundle has a cli sync command, basically it regenerates the routes for the media contexts based on the cdn config, so if you perform:

app/console sonata:media:sync

You'll get something like:

Please select the provider
    [0] sonata.media.provider.image
    [1] sonata.media.provider.file
    [2] sonata.media.provider.youtube
    [3] sonata.media.provider.dailymotion
    [4] sonata.media.provider.vimeo

These contexts belongs to my project, you may have a similar structure. I my case I just had images, it means just the first one: sonata.media.provider.image. Then after setting your option, e.g: 0 you'll be asked to select the context, e.g:

Please select the context
    [0] default
    [1] news
    [2] collection
    [3] category
    [4] profile

Just select all the contexts you currently use (of course one by one, step by step).

For each step you'll get something like:

Loaded 52 medias (batch #1, offset 0) for generating thumbs (provider: sonata.media.provider.image, context: default)
Generating thumbs for Scenario - 1
...
...
...
Done (total medias processed: 52).

Once all the processes has finished, if you list all the images within your admin dashboard, you will see all of them have the new url, that belongs to AWS S3.

As first step, assure you have not the local storage set, so your settings should look like:

sonata_media:
    filesystem:
        s3:
            bucket: %sonata_media_s3_bucket%
            accessKey: %sonata_media_s3_accessKey%
            secretKey: %sonata_media_s3_secretKey%
            region: %sonata_media_s3_region%
            create: true

instead of:

sonata_media:
    filesystem:
        local:
            directory:  %kernel.root_dir%/../web/uploads/media
            create:     true
        s3:
            bucket: %sonata_media_s3_bucket%
            accessKey: %sonata_media_s3_accessKey%
            secretKey: %sonata_media_s3_secretKey%
            region: %sonata_media_s3_region%
            create: true

There's no need for you to set the local storage to sync the new content with AWS S3 in spite of you've been storing your images locally.

The sync process just rebuilds the path for the stored media. Right now it does not push the content to AWS S3, that why you must upload your uploads directory by hand, straight to the root of your bucket where you want to store the media from now on.

The media's documentation suggests a cdn path based on the static storage in S3, you if you are no using an static storage, I recommend you to use the default URL, e.g:

...
cdn:
    # define the public base url for the uploaded media
    server:
        path: "https://s3.amazonaws.com/%sonata_media.s3.bucket_name%/%sonata_media.cdn.host%"
...

Let's assume you've already finished running the cli sync command and you've already uploaded your media into AWS S3.

The last one step is to re-save each content which contains images or medias (e.g: all the posts of your blog which contain images), it means you should open them one by one from your admin dashboard and then must click on the update and close button in order to update the media sources (images / videos / files) because these weren't automatically updated.

I recommend you to perform all these steps in your development / staging environment before proceeding to perform it in production.

Once you've successfully performed the previous steps, you can remove the old uploads directory (old local storage).

Done!