Setting up Django with GeoDjango Support in AWS Beanstalk or EC2 Instance

Question

So I have at one point had this going via Beanstalk, using Amazon Instance (2013.09) ami-35792c5c . At the time this ebextension scripts worked great when placed in the root of your repo in .ebextensions/

00_repo.config

packages:
    rpm:
        pgdg-redhat93-9.3-1: 'http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-redhat93-9.3-1.noarch.rpm'
        remi: 'http://rpms.famillecollet.com/enterprise/remi-release-6.rpm'

files:
    "/etc/yum.repos.d/pgdg-93-redhat.repo":
        mode: "000644"
        owner: root
        group: root
        content: |
            [pgdg93]
            name=PostgreSQL 9.3 $releasever - $basearch
            baseurl=http://yum.postgresql.org/9.3/redhat/rhel-6-$basearch
            enabled=1
            gpgcheck=1
            gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-93

            [pgdg93-source]
            name=PostgreSQL 9.3 $releasever - $basearch - Source
            failovermethod=priority
            baseurl=http://yum.postgresql.org/srpms/9.3/redhat/rhel-6-$basearch
            enabled=0
            gpgcheck=1
            gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-93


commands:
    epel_repo:
        command: yum-config-manager -y --enable epel

    remi_repo:
        command: yum-config-manager -y --enable remi

01_app.config

packages:
    yum:
        libtiff-devel: ''
        libjpeg-devel: ''
        libzip-devel: ''
        freetype-devel: ''
        postgresql-devel: ''
        gdal: ''
        gdal-python: ''
        geos: ''
        proj: ''
        libmemcached: ''
        libmemcached-devel: ''
        cyrus-sasl-devel: ''
        zlib-devel: ''

container_commands:
    01_collectstatic:
        command: 'PYTHONPATH=.:..:../lib cd site/kpmkhv && ./manage.py collectstatic -c --noinput && cd ../..'
        leader_only: true
    02_syncdb:
        command: 'PYTHONPATH=.:..:../lib cd site/kpmkhv && ./manage.py syncdb --noinput && cd ../..'
        leader_only: true
    03_migrate:
        command: 'PYTHONPATH=.:..:../lib cd site/kpmkhv && ./manage.py migrate --noinput && cd ../..'
        leader_only: true

option_settings:
    - namespace: aws:elasticbeanstalk:container:python
      option_name: WSGIPath
      value: site/kpmkhv/wsgi.py
    - namespace: aws:elasticbeanstalk:container:python:staticfiles
      option_name: /static/
      value: site/kpmkhv/static/
    - option_name: DJANGO_SETTINGS_MODULE
      value: settings_prod

So now when I use the same instance and launch my environment, I get this error regarding a dependency.

Error: Package: gdal-libs-1.9.2-5.rhel6.x86_64 (pgdg93)
            Requires: libpoppler.so.5()(64bit)
  You could try using --skip-broken to work around the problem
  You could try running: rpm -Va --nofiles --nodigest

Looks like the same repo is now returning a newer version of poppler, it was 12.x and now its 22.x and gdal needs the old version.

I also tested this out on an EC2 Instance and got the same error. But then I ran into this document from amazon on locking an AMI to its original repository version.

So adding this to User Options via the EC2 Console when you launch fixes the problem on EC2:

#cloud-config
repo_releasever: 2014.03

What is the best way to always have this option when your Beanstalk launches the EC2 Instance on your behalf? I read about cloud-init and perhaps a script deployed via ebextensions would be the best bet?

Any insight on this is appreciated, thanks.

Answer 1

So I now have a working ebextensions workflow on 2013.09 stack ami ami-35792c5c. For the 2014.09 stack see the other solution. The solution below works with postgis by installing the needed gdal component, 00_repo.config needs to look like this:

files:
  "/etc/yum.repos.d/pgdg-93-redhat.repo":
    mode: "000644"
    owner: root
    group: root
    content: |
      [pgdg93]
      name=PostgreSQL 9.3 $releasever - $basearch
      baseurl=http://yum.postgresql.org/9.3/redhat/rhel-6-$basearch
      enabled=1
      gpgcheck=1
      gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-93

      [pgdg93-source]
      name=PostgreSQL 9.3 $releasever - $basearch - Source
      failovermethod=priority
      baseurl=http://yum.postgresql.org/srpms/9.3/redhat/rhel-6-$basearch
      enabled=0
      gpgcheck=1
      gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-PGDG-93

commands:
  epel_repo:
    command: yum-config-manager -y --enable epel
  remi_repo:
    command: yum-config-manager -y --enable remi

packages:
  rpm:
    pgdg-redhat93-9.3-1: 'http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-redhat93-9.3-1.noarch.rpm'
    remi: 'http://rpms.famillecollet.com/enterprise/remi-release-6.rpm'
    qt4-devel: 'http://mirror.centos.org/centos/6/os/x86_64/Packages/qt-4.6.2-28.el6_5.x86_64.rpm'

The 2nd extension stays in tact. This works on Amazon Instance (2013.09) ami-35792c5c, I haven't tried the newer containers yet with it.

Alternative 1:

If you want less dependencies on repos / rpms from the ebextensions file, you could upload all required rpms to S3 and update the ebextensions 'packages' to point to your s3 rpms. Setup your S3 bucket for public get access using CORS headers.

Alternative 2:

Create a custom AMI where you compile all dependencies from source. This way there will be no rpm conflicts and you don't have to temper with the default repos supplied by the AMI you are customizing. See this answer: Configuring Amazon Elastic Beanstalk with PostGIS

Also check out the tool I made:

https://github.com/radlws/django-awseb-tasks

Answer 2

My solution to a 2017.03 image was to:

commands:
  01_yum_update:
    command: sudo yum -y update
  02_epel_repo:
    command: sudo yum-config-manager -y --enable epel
  03_install_gdal_packages:
    command: sudo yum -y install gdal gdal-devel

files:
  "/etc/httpd/conf.d/wsgihacks.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      WSGIPassAuthorization On

packages:
  yum:
    git: []
    postgresql95-devel: []
    gettext: []
    libjpeg-turbo-devel: []
    libffi-devel: []

Answer 3

I was finally able to get this to work on the 2015.03 image and PostgreSQL 9.4 with a suggestion I found on the PostgreSQL yum mailing list (http://www.postgresql.org/message-id/[email protected])

Specifically, I installed libpoppler straight from a CentOS 6 mirror. I also used Mike Edward's suggestion of the Amazon Linux specific pgdg rpm. In whole, it ended up being a rather simple solution, which looked like:

yum -y install http://yum.postgresql.org/9.4/redhat/rhel-6-x86_64/pgdg-ami201503-94-9.4-1.noarch.rpm
yum --enablerepo=epel -y install postgis2_94 http://centos-distro.cavecreek.net/centos/6.6/os/x86_64/Packages/poppler-0.12.4-3.el6_0.1.x86_64.rpm

Answer 4

I had a similar issue on the 2014.09 stack, ami id ami-246ed34c and got around the problem like this:

1. I found and downloaded the required dependencies online here and here. These are the versions that work:

   • lib64jpeg8-8b-5.1.mga1.x86_64.rpm
   • lib64poppler5-0.12.4-2.1mdv2010.1.x86_64.rpm

2. Zipped both .rpm files

3. Uploaded the .zip files to S3 and make sure the server can access them.
4. Added to ebextensions/00_sources.config the following code: /etc/mylibs/jpeg8/ : https://s3.amazonaws.com/path-to-first-rpm-file.rpm.zip /etc/mylibs/poppler/ : https://s3.amazonaws.com/path-to-second-rpm-file.rpm.zip

5. Installed via yum in ebextensions/01_packages.config. I had to use these commands to be able to install them.

   commands: 
     jpeg8_repo:
       command: yum -y install /etc/mylibs/jpeg8/lib64jpeg8-8b-5.1.mga1.x86_64.rpm
       ignoreErrors: true
     poppler_repo:
       command: yum -y install /etc/mylibs/poppler/lib64poppler5-0.12.4-2.1mdv2010.1.x86_64.rpm 
       ignoreErrors: true
   

Answer 5

For the 2015.03 image, I had good luck using this value for the pgdg repo:

pgdg-redhat93-9.3-1: 'http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/pgdg-ami201503-93-9.3-1.noarch.rpm'

With that, I was able to install gdal without needing to bring in libpoppler, etc., as separate rpms from S3.