Parsing xml from a NCBI entrez in R

Question

I want to extract some informations from the features section of a NCBI entry and I'm using this code. To download the data

fetch2 <- entrez_fetch(db = "nucleotide", id = 1028916732, 
                       rettype = "gbc", retmode="xml", parsed = TRUE)

To analyse the data

xmltop = xmlRoot(fetch2) #gives content of root
class(xmltop)#"XMLInternalElementNode" "XMLInternalNode" "XMLAbstractNode"
xmlName(xmltop) 
xmlSize(xmltop)
xmlName(xmltop[[1]]) 
features <- xmltop[[1]][[20]][[1]][[4]]

I'm only interested in the features

  <INSDFeature_quals>
      <INSDQualifier>
        <INSDQualifier_name>organism</INSDQualifier_name>
        <INSDQualifier_value>Alanphillipsia aloeigena</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>mol_type</INSDQualifier_name>
        <INSDQualifier_value>genomic DNA</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>strain</INSDQualifier_name>
        <INSDQualifier_value>CPC 21286</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>isolation_source</INSDQualifier_name>
        <INSDQualifier_value>leaves</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>host</INSDQualifier_name>
        <INSDQualifier_value>Aloe melanacantha</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>culture_collection</INSDQualifier_name>
        <INSDQualifier_value>CBS:136408</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>culture_collection</INSDQualifier_name>
        <INSDQualifier_value>CPC:21286</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>type_material</INSDQualifier_name>
        <INSDQualifier_value>culture from holotype of Alanphillipsia aloeigena</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>db_xref</INSDQualifier_name>
        <INSDQualifier_value>taxon:1414674</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>country</INSDQualifier_name>
        <INSDQualifier_value>South Africa: Namakwaland, Koegap Nature Reserve</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>collected_by</INSDQualifier_name>
        <INSDQualifier_value>M.J. Wingfield</INSDQualifier_value>
      </INSDQualifier>
      <INSDQualifier>
        <INSDQualifier_name>note</INSDQualifier_name>
        <INSDQualifier_value>ex-holotype culture of Alanphillipsia aloeigena</INSDQualifier_value>
      </INSDQualifier>
    </INSDFeature_quals> 

I would like to create a table like

Organism | culture_collection | host  
Alanphillipsia aloeigena | CBS:136408 | Aloe melanacantha

However I don't understand how to retrieve data with

<INSDQualifier_name>
<INSDQualifier_value>

I have seen some tutorials from Pubmed and it works well but the outputs has a different structure. In end I want to make a loop to extract data from a list of ID and since not all the entries have the same structure I would like to use the tags like host organism to retrieve that info.

Answer 1

Since your XML is fairly flat, consider XML's convenient handler, xmlToDataFrame:

library(XML)

fetch2 <- ...
doc <- xmlParse(fetch2)
df <- xmlToDataFrame(doc, nodes=getNodeSet(doc, "//INSDQualifier"))

df
#    INSDQualifier_name                               INSDQualifier_value
# 1            organism                          Alanphillipsia aloeigena
# 2            mol_type                                       genomic DNA
# 3              strain                                         CPC 21286
# 4    isolation_source                                            leaves
# 5                host                                 Aloe melanacantha
# 6  culture_collection                                        CBS:136408
# 7  culture_collection                                         CPC:21286
# 8       type_material culture from holotype of Alanphillipsia aloeigena
# 9             db_xref                                     taxon:1414674
# 10            country  South Africa: Namakwaland, Koegap Nature Reserve
# 11       collected_by                                    M.J. Wingfield
# 12               note   ex-holotype culture of Alanphillipsia aloeigena

And then run a transpose with column and row names cleanup if each of the above rows should be columns with corresponding values

final_df <- data.frame(t(df), stringsAsFactors = FALSE)

colnames(final_df) <- as.character(final_df[1,])
final_df <- final_df[-1,]
rownames(final_df) <- NULL

final_df 
#                   organism    mol_type    strain isolation_source              host culture_collection culture_collection                                     type_material
# 1 Alanphillipsia aloeigena genomic DNA CPC 21286           leaves Aloe melanacantha         CBS:136408          CPC:21286 culture from holotype of Alanphillipsia aloeigena
#          db_xref                                          country   collected_by                                            note
#  1 taxon:1414674 South Africa: Namakwaland, Koegap Nature Reserve M.J. Wingfield ex-holotype culture of Alanphillipsia aloeigena