Loading different data types from XML into a dictionary in python

Question

I'm using cElementTree to extract xml tags and values in a loop and then storing them into a dictionary.

XML file contains:

<root>
    <tag1>['item1', 'item2']</tag1>
    <tag2>a normal string</tag2>
</root>

Python code (roughly):

import xml.etree.cElementTree as xml

xmldata = {}
xmlfile = xml.parse(XMLFile.xml)
for xmltag in xmlfile.iter():
    xmldata[xmltag.tag] = xmltag.text

The problem I have encountered is that the xml file contains different data types, which include string and list. Unfortunately Element.text saves all the xml values as string (including the lists).

So when I load from the XML file I have:

{'tag1':"['item1', 'item2']", 'tag2':'a normal string'}

When I'd prefer to have:

{'tag1':['item1', 'item2'], 'tag2':'a normal string'}

Is there an easy way to do this?
e.g a command that saves to the dictionary in the original format

Or do I need to set up if statements to determine the value type and save it seperately using an alternative to Element.text?

Answer 1

You can use literal_eval to try to parse complex python literals. Since your strigns are unquoted, they will raise a SyntaxError in lteral eval, but that is simle to work around:

import xml.etree.cElementTree as xml
from ast import literal_eval

xmldata = {}
xmlfile = xml.parse(XMLFile.xml)
for xmltag in xmlfile.iter():
    try:
        xmldata[xmltag.tag] = literal_eval(xmltag.text)
    except SyntaxError:
        xmldata[xmltag.tag] = xmltag.text

Unlike Python's builtin "eval", ast.literal_eval does not allow the execution of expressions, and thus is safe, even if the XML data come from an untrusted source.

Answer 2

I think that you are not using xml in all its mighty power!

Why don't you organize your .xml like:

<root>
    <tag1>
        <item>item1</item>
        <item>item2</item>
    </tag1>
    <tag2>a normal string<tag2>
</root>

This way your python code will be handling every <tag1> as a container of <item>, and I think that's better.

Note: You may also want to take a look here. (I agree with the "Favorite Way" of the author)

Answer 3

Here is a proposed solution: check for the existence of [, then parse the list. It's not failsafe (it won't work if the separator is not exactly , with a space) but I think that it'll be easy for you to improve it.

import xml.etree.cElementTree as xml

xmldata = {}
xmlfile = xml.parse("data.xml")
for xmltag in xmlfile.iter():
    # it's a list
    if "[" in xmltag.text:
        d = xmltag.text.lstrip("[").rstrip("]")
        l = [item.lstrip("'").rstrip("'") for item in d.split(", ")]
        xmldata[xmltag.tag] = l
    else:
        xmldata[xmltag.tag] = xmltag.text

print xmldata

Prints: {'root': '\n', 'tag1': ['item1', 'item2'], 'tag2': 'a normal string'}