Error "Duplicate namespace in XML input" when retrieving a list over NETCONF using Opendaylight

Question

I am using Opendaylight (release 0.10.1) as NETCONF client to send requests to a Netopeer2 server.
My YANG model is a third-party model consisting of several modules which contain, among other attributes, several list elements.
This translates into an XML request and response pair that looks something like this (simplified):

Request:

<rpc message-id="m-3" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
    <get-config>
        <source>
            <running/>
        </source>
        <filter xmlns:ns0="urn:ietf:params:xml:ns:netconf:base:1.0" ns0:type="subtree">
            <TopElement xmlns="top-element">
                <id>1</id>
            </TopElement>
        </filter>
    </get-config>
</rpc>

Response:

<rpc-reply xmlns="urn:ietf:params:xml:ns:netconf:base:1.0" message-id="m-3">
    <data>
        <TopElement xmlns="top-element">
            <id>1</id>
            <ParentElement xmlns="parent-element">
                <id>1</id>
                <ChildElement1 xmlns="child-element1">
                    <id>1</id>
                    <attributes>
                        ...
                    </attributes>
                </ChildElement1>
                <ChildElement1 xmlns="child-element1">
                    <id>2</id>
                    <attributes>
                        ...
                    </attributes>
                </ChildElement1>
                <ChildElement2 xmlns="child-element2">
                    <id>1</id>
                    <attributes>
                        ...
                    </attributes>
                </ChildElement2>
                <ChildElement1 xmlns="child-element1">
                    <id>3</id>
                    <attributes>
                        ...
                    </attributes>
                </ChildElement1>
            </ParentElement>
        </TopElement>
    </data>
</rpc-reply>

However, such a request causes a "Master is down" error, which I have tried investigating using the Karaf log file. I tracked the problem down to a akka.pattern.AskTimeoutException timeout which seems to be caused by a parsing error of the XML response:

2020-04-08T09:09:11,733 | ERROR | nioEventLoopGroupCloseable-3-1 | AbstractFuture                   | 42 - com.google.guava - 25.1.0.jre |  -  | RuntimeException while executing runnable CallbackListener{org.opendaylight.netconf.sal.connect.netconf.sal.NetconfDeviceRpc$1@165744d4} with executor MoreExecutors.directExecutor()
java.lang.IllegalArgumentException: Failed to parse data response [data: null]
    at org.opendaylight.netconf.sal.connect.netconf.schema.mapping.NetconfMessageTransformer.toRpcResult(NetconfMessageTransformer.java:285) ~[386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at org.opendaylight.netconf.sal.connect.netconf.schema.mapping.NetconfMessageTransformer.toRpcResult(NetconfMessageTransformer.java:75) ~[386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at org.opendaylight.netconf.sal.connect.netconf.sal.NetconfDeviceRpc$1.onSuccess(NetconfDeviceRpc.java:60) ~[386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at org.opendaylight.netconf.sal.connect.netconf.sal.NetconfDeviceRpc$1.onSuccess(NetconfDeviceRpc.java:57) ~[386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1355) ~[42:com.google.guava:25.1.0.jre]
    at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398) ~[42:com.google.guava:25.1.0.jre]
    at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1024) [42:com.google.guava:25.1.0.jre]
    at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:866) [42:com.google.guava:25.1.0.jre]
    at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:689) [42:com.google.guava:25.1.0.jre]
    at org.opendaylight.netconf.sal.connect.netconf.listener.UncancellableFuture.set(UncancellableFuture.java:45) [386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at org.opendaylight.netconf.sal.connect.netconf.listener.NetconfDeviceCommunicator.processMessage(NetconfDeviceCommunicator.java:338) [386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at org.opendaylight.netconf.sal.connect.netconf.listener.NetconfDeviceCommunicator.onMessage(NetconfDeviceCommunicator.java:270) [386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at org.opendaylight.netconf.sal.connect.netconf.listener.NetconfDeviceCommunicator.onMessage(NetconfDeviceCommunicator.java:49) [386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    at org.opendaylight.netconf.nettyutil.AbstractNetconfSession.handleMessage(AbstractNetconfSession.java:64) [379:org.opendaylight.netconf.netty-util:1.6.1]
    at org.opendaylight.netconf.nettyutil.AbstractNetconfSession.channelRead0(AbstractNetconfSession.java:187) [379:org.opendaylight.netconf.netty-util:1.6.1]
    at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [71:io.netty.transport:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [71:io.netty.transport:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) [71:io.netty.transport:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) [71:io.netty.transport:4.1.34.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) [66:io.netty.codec:4.1.34.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) [66:io.netty.codec:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [71:io.netty.transport:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) [71:io.netty.transport:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) [71:io.netty.transport:4.1.34.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) [66:io.netty.codec:4.1.34.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) [66:io.netty.codec:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) [71:io.netty.transport:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:38) [71:io.netty.transport:4.1.34.Final]
    at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:350) [71:io.netty.transport:4.1.34.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [68:io.netty.common:4.1.34.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [68:io.netty.common:4.1.34.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:495) [71:io.netty.transport:4.1.34.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) [68:io.netty.common:4.1.34.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [68:io.netty.common:4.1.34.Final]
    at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[-1,-1]
Message: Duplicate namespace "child-element1" element "ChildElement1" in XML input
    at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.read(XmlParserStream.java:383) ~[410:org.opendaylight.yangtools.yang-data-codec-xml:2.1.10]
    at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.read(XmlParserStream.java:321) ~[410:org.opendaylight.yangtools.yang-data-codec-xml:2.1.10]
    at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.read(XmlParserStream.java:403) ~[410:org.opendaylight.yangtools.yang-data-codec-xml:2.1.10]
    at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.read(XmlParserStream.java:321) ~[410:org.opendaylight.yangtools.yang-data-codec-xml:2.1.10]
    at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.read(XmlParserStream.java:403) ~[410:org.opendaylight.yangtools.yang-data-codec-xml:2.1.10]
    at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.parse(XmlParserStream.java:214) ~[410:org.opendaylight.yangtools.yang-data-codec-xml:2.1.10]
    at org.opendaylight.yangtools.yang.data.codec.xml.XmlParserStream.traverse(XmlParserStream.java:244) ~[410:org.opendaylight.yangtools.yang-data-codec-xml:2.1.10]
    at org.opendaylight.netconf.sal.connect.netconf.schema.mapping.NetconfMessageTransformer.toRpcResult(NetconfMessageTransformer.java:281) ~[386:org.opendaylight.netconf.sal-netconf-connector:1.9.1]
    ... 34 more

Can it really be because of the list? The XML is compliant to RFC 7950:

7.8.5. XML Encoding Rules

A list is encoded as a series of XML elements, one for each entry in the list. Each element's local name is the list's identifier, and its namespace is the module's XML namespace (see Section 7.1.3). There is no XML element surrounding the list as a whole.

Any suggestion would be greatly appreciated. Thanks!

Answer 1

I'm not sure I know the answer here, but could the problem just be that some of the list items are named the same? as in:

<ChildElement1 xmlns="child-element1">

I know the id's are unique, but that's what stood out to me.

Answer 2

I guess the problem here is that the response from the server contains, at the /TopElement/ParentElement , the list entries (in this order):

• ChildElement1[id=1]
• ChildElement1[id=2]
• ChildElement2[id=1]
• ChildElement1[id=3]

So I assume both ChildElement1 and ChildElement2 are both lists. The problem here is that the 3 entries of ChildElement1 are not sequential, you have an entry of ChildElement2 in the middle.

I don't see anything in the NETCONF spec saying that that is an issue, but clearly OpenDaylight has a problem with it. I assume OpenDaylight consider the handling of the ChildElement2 entry as meaning that the ChildElement1 list has ended being delcared, so after it finds another entry later, it considers it as a separate list, and generates the exception Duplicate namespace "child-element1" element "ChildElement1" in XML input that you had.

If you want to avoid that problem, I'd recommend to sort the entries in the response from the server, so that entries of the same list stand together.

So to reiterate, the problem is not the order of individual list entries, but the fact that you are interleaving entries belonging to two different lists.

Answer 3

I might have stumbled upon something.
I tried the same thing with a different NETCONF server (ConfD) and the same models, and it seems to work fine.
One difference I noticed is that when doing a get-request on TopElement or ParentElement, ConfD returns the child lists ordered, i.e. first all ChildElement1 elements and then all ChildElement2 elements, even though they where created in a random order. Sysrepo-netopeer2, on the other hand, returns the XML tree with the elements in the same order as they were created, so ChildElement1 elements and ChildElement2 elements are mixed (see my example).
I'm not sure if they are both good approaches, or if there is a right one and a wrong one.

EDIT: In my original post I quoted RFC 7950 and somehow missed the relevant part:

The XML elements representing list entries MUST appear in the order specified by the user if the list is "ordered-by user"; otherwise, the order is implementation dependent. The XML elements representing list entries MAY be interleaved with elements for siblings of the list, unless the list defines RPC or action input or output parameters.

So, assuming I am correct in my assumption, it looks like ODL is actually being too restrictive in this case.