Parse XML (musicbrainz) In Python

Question

I am trying to import urls like this one (http://musicbrainz.org/ws/2/artist/72c536dc-7137-4477-a521-567eeb840fa8) into python and extract the value of the "gender".

import urllib2
import codecs
import sys
import os
from xml.dom import minidom
import xml.etree.cElementTree as ET

#urlbob = urllib2.urlopen('http://musicbrainz.org/ws/2/artist/72c536dc-7137-4477-a521-567eeb840fa8')
url = 'dylan.xml'

#attempt 1 - using minidom
xmldoc = minidom.parse(url)
itemlist = xmldoc.getElementsByTagName('artist') 

#attempt 2 - using ET
tree = ET.parse('dylan.xml')
root = tree.getroot()

for child in root:
    print child.tag, child.attrib 

I can't seem to get at gender either via the mini-dom stuff or the etree stuff. In its current form, the script returns

{http://musicbrainz.org/ns/mmd-2.0#}artist {'type': 'Person', 'id': '72c536dc-7137-4477-a521-567eeb840fa8'}

Answer 1

That is because you're looping root which is only the root of the tree, does that make sense? When you loop the root, it will only return the next child and stop there.

You need to loop the iterable so it will get returning next node and get the result, see this:

tree = ET.parse('dylan.xml')
root = tree.getroot()

# loop the root iterable which will keep returning next node
for node in root.iter(): # or root.getiterator() if < Python 2.7
    print node.tag, node.attrib, node.text

Results:

{http://musicbrainz.org/ns/mmd-2.0#}metadata {} None
{http://musicbrainz.org/ns/mmd-2.0#}artist {'type': 'Person', 'id': '72c536dc-7137-4477-a521-567eeb840fa8'} None
{http://musicbrainz.org/ns/mmd-2.0#}name {} Bob Dylan
{http://musicbrainz.org/ns/mmd-2.0#}sort-name {} Dylan, Bob
{http://musicbrainz.org/ns/mmd-2.0#}ipi {} 00008955074
{http://musicbrainz.org/ns/mmd-2.0#}ipi-list {} None
{http://musicbrainz.org/ns/mmd-2.0#}ipi {} 00008955074
{http://musicbrainz.org/ns/mmd-2.0#}ipi {} 00008955172
{http://musicbrainz.org/ns/mmd-2.0#}isni-list {} None
{http://musicbrainz.org/ns/mmd-2.0#}isni {} 0000000121479733
{http://musicbrainz.org/ns/mmd-2.0#}gender {} Male
{http://musicbrainz.org/ns/mmd-2.0#}country {} US
{http://musicbrainz.org/ns/mmd-2.0#}area {'id': '489ce91b-6658-3307-9877-795b68554c98'} None
{http://musicbrainz.org/ns/mmd-2.0#}name {} United States
{http://musicbrainz.org/ns/mmd-2.0#}sort-name {} United States
{http://musicbrainz.org/ns/mmd-2.0#}iso-3166-1-code-list {} None
{http://musicbrainz.org/ns/mmd-2.0#}iso-3166-1-code {} US
{http://musicbrainz.org/ns/mmd-2.0#}begin-area {'id': '04e60741-b1ae-4078-80bb-ffe8ae643ea7'} None
{http://musicbrainz.org/ns/mmd-2.0#}name {} Duluth
{http://musicbrainz.org/ns/mmd-2.0#}sort-name {} Duluth
{http://musicbrainz.org/ns/mmd-2.0#}life-span {} None
{http://musicbrainz.org/ns/mmd-2.0#}begin {} 1941-05-24

Answer 2

## This prints out the tree as the xml lib sees it 
## (I found it made debugging a little easier)
#def print_xml(node, depth = 0):
#    for child in node:
#        print "\t"*depth + str(child)
#        print_xml(child, depth = depth + 1)
#print_xml(root)

# attempt 1
xmldoc = minidom.parse(url)
genders = xmldoc.getElementsByTagName('gender') # <== you want gender not artist
for gender in genders:
    print gender.firstChild.nodeValue

# attempt 2
ns = "{http://musicbrainz.org/ns/mmd-2.0#}"
xlpath = "./" + ns + "artist/" + ns + "gender"
genders = root.findall(xlpath) # <== xpath was made for this..
for gender in genders:
    print gender.text

So.. the problem with your first attempt is that you're looking at a list of all the artist elements not the gender elements (the child of the only artist element in the list).

The problem with your second attempt is that you are looking at a list of the children of the root element (which is a list containing a single metadata element).

The underlying structure is:

<artist>
    <name>
    <sort-name>
    <ipi>
    <ipi-list>
        <ipi>
        <ipi>
    <isni-list>
        <isni>
    <gender>
    <country>
    <area>
        <name>
        <sort-name>
        <iso-3166-1-code-list>
            <iso-3166-1-code>
    <begin-area>
        <name>
        <sort-name>
    <life-span>
        <begin>

so you need to get root -> artist -> gender, or just search for the node you actually want (gender in this case).