Configure Plone to use Relstorage as blobstorage

Question

I have a installation of Plone 4.3.3 with one site. First the buildout was configured to use the Data.fs file in var/filestorage and a shared blob storage in var/blobstorage. Then I added a Relstorage to the buildout and converted the content of the Data.fs file to the underlying MySQL database. Now Plone is using Relstorage instead of Data.fs.

But now I want also to use the Relstorage instead of the blobstorage. Because I am relatively new to Plone and especially the Relstorage thing, my idea was to first setup a new empty Plone. Then I copied the buildout.cfg and base.cfg from the first one to the new one. Then I created a new database userZodb and changed the base.cfg for using the new database and I also changed the ports for zeoserver and clients. The next step was to reconfigure the relstorage for not using the file based blobstorage.

rel-storage =
    type mysql
    db userZodb
    user zodbuser
    passwd innzop
    blob-dir ${buildout:var-dir}/blobstorage
    shared-blob-dir false

# shared blobs are much faster if we're on the same server.
# if not, turn it off.
shared-blob = off

Then I ran the buildout. All was built successfully. After starting the zeoserver, I got this error from the client:

user@server:~/Plone433-dev/zeocluster3$ ./bin/zeoserver start
. 
daemon process started, pid=35136
user@server:~/Plone433-dev/zeocluster3$ ./bin/client1 fg
2014-12-17 14:50:31 INFO ZServer HTTP server started at Wed Dec 17 14:50:31 2014
    Hostname: 0.0.0.0
    Port: 9180
2014-12-17 14:50:32 INFO Products.PloneFormGen gpg_subprocess initialized, using /usr/bin/gpg
Traceback (most recent call last):
  File "/home/user/Plone433-dev/zeocluster3/parts/client1/bin/interpreter", line 289, in <module>
    exec(compile(__file__f.read(), __file__, "exec"))
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/run.py", line 76, in <module>
    run()
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/run.py", line 22, in run
    starter.prepare()
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/__init__.py", line 86, in prepare
    self.startZope()
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/__init__.py", line 262, in startZope
    Zope2.startup()
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/__init__.py", line 47, in startup
    _startup()
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/App/startup.py", line 81, in startup
    DB = dbtab.getDatabase('/', is_root=1)
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py", line 287, in getDatabase
    db = factory.open(name, self.databases)
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py", line 185, in open
    DB = self.createDB(database_name, databases)
  File "/home/user/Plone433-dev/buildout-cache/eggs/Zope2-2.13.22-py2.7.egg/Zope2/Startup/datatypes.py", line 182, in createDB
    return ZODBDatabase.open(self, databases)
  File "/home/user/Plone433-dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/config.py", line 101, in open
    storage = section.storage.open()
  File "/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/config.py", line 33, in open
    return RelStorage(adapter, name=config.name, options=options)
  File "/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/storage.py", line 212, in __init__
    self.blobhelper = BlobHelper(options=options, adapter=adapter)
  File "/home/user/Plone433-dev/buildout-cache/eggs/RelStorage-1.6.0b2-py2.7.egg/relstorage/blobhelper.py", line 118, in __init__
    fshelper.create()
  File "/home/user/Plone433-dev/buildout-cache/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/blob.py", line 359, in create
    (self.layout_name, self.base_dir, layout))
ValueError: Directory layout `zeocache` selected for blob directory /home/user/Plone433-dev/zeocluster3/var/blobstorage/, but marker found for layout `bushy`

Unfortunality I have no idea where the problem could be. Anyone a suggestion? Thank you!

Answer 1

The solution was to use zodbconvert again. Correctly configured it can convert from one storage to another, for example from Blobstorage to Relstorage. In my case the configuration looks like that:

<filestorage source>
  path     /home/user/Plone433-dev/zeocluster/var/filestorage/Data20141230.fs
  blob-dir /home/user/Plone433-dev/zeocluster/var/blobstorage
</filestorage>

<relstorage destination>
  shared-blob-dir false
  # ZODB Cache Dir
  blob-dir ./var/cacheblob
  blob-cache-size 10mb
  <mysql>
    host localhost
    db Zodb
    user zodbuser
    passwd XXXXXXXXX
  </mysql>
</relstorage>

After that you have to change your base.cfg and buildout.cfg for using only relstorage. You can find more information about how exactly it works here: https://www.techidiots.net/notes/plone-1/plone-4-3-3-relstorage

Answer 2

If you're using relstorage the blob directory is still used for caching (Check for more infos). In your case there's a problem with your directory layout.

You can remove the hole var directory in ${buildout:directory} and rerun buildout. This will create you a new var directory. After starting the instance you should have a new blob directory with the correct layout.

OR

You can modify the .layout file in ${buildout:directory}/var/blobstorage and change the value from bushy to zeocache

Explanation:

The first time you started the Plone instance, it creates the blobstorage directory with the given layout - in your case it was the default bushy. Since you changed the storage it expects zeocache. But the marker file .layout is not changed automatically.

If this doesn't help, please post your full buildout.cfg.