Cannot create docker container in OpenStack

Question

I have problem about setup nova-docker (docker hypervisor) on multi-node OpenStack (version Juno), my OS is Ubuntu 14.04 I followed: https://wiki.openstack.org/wiki/Docker, when I create a container vm by dashboard or nova boot, I received error in nova-compute.log

2015-06-22 14:41:40.059 2550 **ERROR nova.compute.manager [-] [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105] Instance failed to spawn**
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105] Traceback (most recent call last):
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2267, in _build_resources
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105]     yield resources
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2137, in _build_and_run_instance
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105]     block_device_info=block_device_info)
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105]   File "/usr/local/lib/python2.7/dist-packages/novadocker/virt/docker/driver.py", line 301, in spawn
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105]     **instance_id=instance['name'])**
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105] **InstanceDeployFailure: Cannot create container**
2015-06-22 14:41:40.059 2550 TRACE nova.compute.manager [instance: 62d7a3d3-7008-44d7-b17f-129b4c10c105]

****UPDATE****** Error in docker debug: docker -d -D

INFO[0000] Docker daemon                                 commit=0baf609 execdriver=native-0.2 graphdriver=aufs version=1.7.0    
DEBU[0018] Calling GET /images/{name:.*}/json           
INFO[0018] GET /v1.13/images/tutum/mysql/json           
DEBU[0018] Calling POST /containers/create              
INFO[0018] POST /v1.13/containers/create?name=nova-0e24a2ef-57f3-421e-b357-6cbc71dbe15b 
ERRO[0018] **Handler for POST /containers/create returned error: json: cannot unmarshal string into Go value of type []string** 
ERRO[0018] **HTTP Error                                    err=json: cannot unmarshal string into Go value of type []string statusCode=500**
DEBU[0018] Calling GET /containers/ps                   
INFO[0018] GET /v1.13/containers/ps?all=1               
DEBU[0018] Calling GET /containers/{name:.*}/json       
INFO[0018] GET /v1.13/containers/86bbca94cb7cf4a8dc81c4eaba55a46ca92c9e6bb8b79b63dfdb1466bee15233/json 
DEBU[0018] Calling GET /containers/ps                   
INFO[0018] GET /v1.13/containers/ps?all=1               
DEBU[0018] Calling GET /containers/{name:.*}/json       
INFO[0018] GET /v1.13/containers/86bbca94cb7cf4a8dc81c4eaba55a46ca92c9e6bb8b79b63dfdb1466bee15233/json 

• My configuration:

/etc/nova/nova.conf

  [DEFAULT]
    compute_driver = novadocker.virt.docker.DockerDriver
    network_api_class = nova.network.neutronv2.api.API
    security_group_api = neutron
    linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver
    firewall_driver = nova.virt.firewall.NoopFirewallDriver
    dhcpbridge_flagfile=/etc/nova/nova.conf
    dhcpbridge=/usr/bin/nova-dhcpbridge
    logdir=/var/log/nova
    state_path=/var/lib/nova
    lock_path=/var/lock/nova
    force_dhcp_release=True
    libvirt_use_virtio_for_bridges=True
    verbose=True
    ec2_private_dns_show_ip=True
    api_paste_config=/etc/nova/api-paste.ini
    enabled_apis=ec2,osapi_compute,metadata

    rpc_backend = rabbit
    rabbit_host = controller
    rabbit_password = openstack 


    rbd_user=volumes
    rbd_secret_uuid=f514c965-78b3-46ed-b986-838b3cb91655


    instance_usage_audit = True
    instance_usage_audit_period = hour
    notify_on_state_change = vm_and_task_state
    notification_driver = messagingv2

    my_ip = 10.0.0.33
    vnc_enabled = True
    vncserver_listen = 0.0.0.0
    vncserver_proxyclient_address = 10.0.0.33
    novncproxy_base_url = http://controller:6080/vnc_auto.html
    auth_strategy = keystone
    [keystone_authtoken]
    auth_uri = http://controller:5000/v2.0
    identity_uri = http://controller:35357
    admin_tenant_name = service
    admin_user = nova
    admin_password = openstack 

    [glance]
    host = controller


    [neutron]
    url = http://controller:9696
    auth_strategy = keystone
    admin_auth_url = http://controller:35357/v2.0
    admin_tenant_name = service
    admin_username = neutron
    admin_password = openstack
    service_metadata_proxy = True
    metadata_proxy_shared_secret = af5a6fbca1ef8da0d018


    #[libvirt]
    #live_migration_flag=VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_TUNNELLED

/etc/nova/nova-compute.conf

[DEFAULT]
compute_driver = novadocker.virt.docker.DockerDriver
#[libvirt]
#virt_type=kvm

/etc/nova/rootwrap.d/docker.filters

# nova-rootwrap command filters for setting up network in the docker driver
# This file should be owned by (and only-writeable by) the root user

[Filters]
# nova/virt/docker/driver.py: 'ln', '-sf', '/var/run/netns/.*'
ln: CommandFilter, /bin/ln, root

/etc/glance/glance-api.conf in controller node

.....
container_formats = ami,ari,aki,bare,ovf,docker
....

I uploaded one image docker:

root@compute3:~# glance image-list
+--------------------------------------+-------------------+-------------+------------------+------------+--------+
| ID                                   | Name              | Disk Format | Container Format | Size       | Status |
+--------------------------------------+-------------------+-------------+------------------+------------+--------+
| 0b20f57c-0862-42dc-a705-eb415832fff0 | cirros32-ceph     | qcow2       | bare             | 12336128   | active |
| 4b591eba-8fc1-409a-8d39-35ab792811c6 | tutum/wordpress   | raw         | docker           | 507855360  | active |
+--------------------------------------+-------------------+-------------+------------------+------------+--------+

root@compute3:~# docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
tutum/wordpress          latest              a540e82bd70a        5 weeks ago         492.3 MB
hello-world              latest              91c95931e552        9 weeks ago         910 B
bacongobbler/memcached   latest              81ed21c66f64        15 months ago       263.7 MB

Can anyone help me?, thanks all!

Answer 1

This week there was an update to docker from 1.6.2 to 1.7.0. This updated version apparently does not support any version of the api below 1.14. The nova-docker driver uses the 1.13 api. For right now the solution is to downgrade to docker 1.6.2 (apt-get purge lxc-docker-1.7.0 && apt-get install lxc-docker-1.6.2). Hopefully the nova docker driver will be updated to support a more recent version of the API in the near future.

UPDATE: the problem seems to be that the novadocker driver passes a default VolumesFrom value of "" (empty string), when the updated version of docker wants this to be a null value. I'm currently trying to figure out if this should be considered a bug in docker or the novadocker driver.