VTD-XML Exception: Name space qualification Exception: prefixed attribute not qualified

Question

I receive an XML via a web service and I am using legacy code (which uses dom4j) to perform some xml transformation. Loading/parsing the original XML into VTD-XML (VTDGen) works fine, no exceptions thrown. However, after loading the xml into dom4j, I noticed some of the element namespace declarations and attributes are re-arranged. Apparently, this re-arrangement causes VTD-XML to throw the following exception:

Exception: Name space qualification Exception: prefixed attribute not qualified

Line Number: 101 Offset: 1827

Here is the element at this line number in the original XML:

<RR_PerformanceSite:PerformanceSite_1_4 RR_PerformanceSite:FormVersion="1.4" xmlns:NSF_ApplicationChecklist="http://apply.grants.gov/forms/NSF_ApplicationChecklist-V1.1" xmlns:NSF_CoverPage="http://apply.grants.gov/forms/NSF_CoverPage-V1.1" xmlns:NSF_DeviationAuthorization="http://apply.grants.gov/forms/NSF_DeviationAuthorization-V1.1" xmlns:NSF_Registration="http://apply.grants.gov/forms/NSF_Registration-V1.1" xmlns:NSF_SuggestedReviewers="http://apply.grants.gov/forms/NSF_SuggestedReviewers-V1.1" xmlns:PHS398_CareerDevelopmentAwardSup="http://apply.grants.gov/forms/PHS398_CareerDevelopmentAwardSup_1_1-V1.1" xmlns:PHS398_Checklist="http://apply.grants.gov/forms/PHS398_Checklist_1_3-V1.3" xmlns:PHS398_CoverPageSupplement="http://apply.grants.gov/forms/PHS398_CoverPageSupplement_1_4-V1.4" xmlns:PHS398_ModularBudget="http://apply.grants.gov/forms/PHS398_ModularBudget-V1.1" xmlns:PHS398_ResearchPlan="http://apply.grants.gov/forms/PHS398_ResearchPlan_1_3-V1.3" xmlns:PHS_CoverLetter="http://apply.grants.gov/forms/PHS_CoverLetter_1_2-V1.2" xmlns:RR_Budget="http://apply.grants.gov/forms/RR_Budget-V1.1" xmlns:RR_KeyPersonExpanded="http://apply.grants.gov/forms/RR_KeyPersonExpanded_1_2-V1.2" xmlns:RR_OtherProjectInfo="http://apply.grants.gov/forms/RR_OtherProjectInfo_1_2-V1.2" xmlns:RR_PerformanceSite="http://apply.grants.gov/forms/PerformanceSite_1_4-V1.4" xmlns:RR_PersonalData="http://apply.grants.gov/forms/RR_PersonalData-V1.1" xmlns:RR_SF424="http://apply.grants.gov/forms/RR_SF424_1_2-V1.2" xmlns:RR_SubawardBudget="http://apply.grants.gov/forms/RR_SubawardBudget-V1.2" xmlns:SF424C="http://apply.grants.gov/forms/SF424C-V1.0" xmlns:att="http://apply.grants.gov/system/Attachments-V1.0" xmlns:codes="http://apply.grants.gov/system/UniversalCodes-V2.0" xmlns:globlib="http://apply.grants.gov/system/GlobalLibrary-V2.0">

Here is the same element after loaded into dom4j:

<RR_PerformanceSite:PerformanceSite_1_4 xmlns:RR_PerformanceSite="http://apply.grants.gov/forms/PerformanceSite_1_4-V1.4" xmlns:NSF_ApplicationChecklist="http://apply.grants.gov/forms/NSF_ApplicationChecklist-V1.1" xmlns:NSF_CoverPage="http://apply.grants.gov/forms/NSF_CoverPage-V1.1" xmlns:NSF_DeviationAuthorization="http://apply.grants.gov/forms/NSF_DeviationAuthorization-V1.1" xmlns:NSF_Registration="http://apply.grants.gov/forms/NSF_Registration-V1.1" xmlns:NSF_SuggestedReviewers="http://apply.grants.gov/forms/NSF_SuggestedReviewers-V1.1" xmlns:PHS398_CareerDevelopmentAwardSup="http://apply.grants.gov/forms/PHS398_CareerDevelopmentAwardSup_1_1-V1.1" xmlns:PHS398_Checklist="http://apply.grants.gov/forms/PHS398_Checklist_1_3-V1.3" xmlns:PHS398_CoverPageSupplement="http://apply.grants.gov/forms/PHS398_CoverPageSupplement_1_4-V1.4" xmlns:PHS398_ModularBudget="http://apply.grants.gov/forms/PHS398_ModularBudget-V1.1" xmlns:PHS398_ResearchPlan="http://apply.grants.gov/forms/PHS398_ResearchPlan_1_3-V1.3" xmlns:PHS_CoverLetter="http://apply.grants.gov/forms/PHS_CoverLetter_1_2-V1.2" xmlns:RR_Budget="http://apply.grants.gov/forms/RR_Budget-V1.1" xmlns:RR_KeyPersonExpanded="http://apply.grants.gov/forms/RR_KeyPersonExpanded_1_2-V1.2" xmlns:RR_OtherProjectInfo="http://apply.grants.gov/forms/RR_OtherProjectInfo_1_2-V1.2" xmlns:RR_PersonalData="http://apply.grants.gov/forms/RR_PersonalData-V1.1" xmlns:RR_SF424="http://apply.grants.gov/forms/RR_SF424_1_2-V1.2" xmlns:RR_SubawardBudget="http://apply.grants.gov/forms/RR_SubawardBudget-V1.2" xmlns:SF424C="http://apply.grants.gov/forms/SF424C-V1.0" xmlns:att="http://apply.grants.gov/system/Attachments-V1.0" xmlns:codes="http://apply.grants.gov/system/UniversalCodes-V2.0" xmlns:globlib="http://apply.grants.gov/system/GlobalLibrary-V2.0" RR_PerformanceSite:FormVersion="1.4">

The problem is regarding the attribute (at offset 1827, at the end of the element) in the new XML element: RR_PerformanceSite:FormVersion="1.4"

Here is what removes the exception: 1. Adding the RR_PerformanceSite xmlns declaration for this element to the root element of the XML doc. 2. Replacing new element with original element. This SEEMS to lead me to believe that the order of the attributes/ns declarations affects VTD when parsing.

NOTE: I parse the xml doc setting ns aware to 'true' with both xml docs (original and post-dom4j xml). Also, new VTD objects are created for each xml, original and post-dom4j.

I tried to put 'RR_PerformanceSite:FormVersion="1.4"' at the beginning of the element like the original but that does not remove the exception. The offset in the error message is different due to the change of location of the attribute. Does the order of the xmlns declarations affect VTD?

I have looked at the VTDGen source code and cannot figure out why this exception is being thrown.

Why would dom4j parse the new doc and vtd is unable to? Can anyone can shed some light on this?

Answer 1

It appears to be a bug on VTD-XML, related with namespace declaration order.

Always reproducible using the following Java code

public class SchemaTester {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {

        String bad = "C:/Temp/VTD_bad.xml"; // XML files to test
        String good = "C:/Temp/VTD_good.xml";

        StringBuilder sb = new StringBuilder();

        char[] buf = new char[4*1024];
        FileReader fr = new FileReader(bad);
        int readed = 0;

        while ((readed = fr.read(buf, 0, buf.length)) != -1) {
            sb.append(buf, 0, readed);
        }

        fr.close();

        String x = sb.toString();

        //instantiate VTDGen
        //and call parse 
        VTDGen vg = new VTDGen();
        vg.setDoc(x.getBytes("UTF-8"));
        vg.parse(true);  // set namespace awareness to true
        VTDNav vn = vg.getNav();



        AutoPilot ap = new AutoPilot (vn);
        ap.selectXPath("//*/@*");

        int i= -1;
        while((i=ap.evalXPath()) != -1) {
            // i will be attr name, i+1 will be attribute value
            System.out.println("\t\tAttribute ==> " + vn.toNormalizedString(i));
            System.out.println("\t\tValue ==> " + vn.toNormalizedString(i+1));
        } 

    }
}

The OP has uploaded the XML to https://gist.github.com/2696220