Remote registry fails to read on deploy

Question

I have a 3 node ESB cluster. Each node has its own mssql registry db. Additionally, there is a shared db for config and governance mount points.

I have a CAR which contains a proxy with a wspolicy. This car fails to deploy with a m says saying that there is a problem with the registry stream. Looking at synaps sources show that the resource stream is unlikely to be open.

I made a CAR with only the policy artefact. It deploys to the remote Registry fine. I can see it via the Mgt GUI. I can see it via SQL query.

I set TRACE logging on the entire esb. When i deploy the original CAR I can see the policy committed to the registry, later when synapse tries to Read the policy on the proxy deploy the error is raised.

I have a second cluster setup the same way. Just different host names and sql. Server. This AR file deploys fine on that.

I have checked my axis2.xml and registry.xml and see no problems.

Any clue what can cause synapse to have a problem reading on deploy but not on simpl install of policy? I noticed that my mgt host name was not right in the DNS and will get network Guys to fix but I am not sure this will be any impact here. That is used in the URL for the remote registry elment. I am using jdbc registry not web service so is that even used?

[2017-01-27 13:29:44,823]  INFO -  Building Axis service for Proxy service : Authenticate_XXXXXXXX {org.apache.synapse.core.axis2.ProxyService}
[2017-01-27 13:29:44,823] DEBUG -  Entry: AxisOperation::getInputAction {org.apache.axis2.description.AxisOperation}
[2017-01-27 13:29:44,823] DEBUG -  Exit: AxisOperation::getInputAction null {org.apache.axis2.description.AxisOperation}
[2017-01-27 13:29:44,823] DEBUG -  Entry: AxisOperation::setSoapAction, previous soapAction: null updated soapAction: urn:mediate {org.apache.axis2.description.AxisOperation}
[2017-01-27 13:29:44,823] DEBUG -  Exit: AxisOperation::setSoapAction {org.apache.axis2.description.AxisOperation}
[2017-01-27 13:29:44,823] DEBUG -  mapActionToOperation: Mapping Action to Operation: action: out; operation: org.apache.axis2.description.InOutAxisOperation@15f408b5named: mediate {$
[2017-01-27 13:29:44,823] DEBUG -  DEBUG_FRAME = org.apache.axis2.util.JavaUtils.callStackToString(JavaUtils.java:564)
[2017-01-27 13:29:44,824] DEBUG -  mapActionToOperation: Mapping Action to Operation: action: in; operation: org.apache.axis2.description.InOutAxisOperation@15f408b5named: mediate {o$
[2017-01-27 13:29:44,824] DEBUG -  DEBUG_FRAME = org.apache.axis2.util.JavaUtils.callStackToString(JavaUtils.java:564)
[2017-01-27 13:29:44,824] DEBUG -  mapActionToOperation: Mapping Action to Operation: action: mediate; operation: org.apache.axis2.description.InOutAxisOperation@15f408b5named: media$
[2017-01-27 13:29:44,824] DEBUG -  DEBUG_FRAME = org.apache.axis2.util.JavaUtils.callStackToString(JavaUtils.java:564)
 [2017-01-27 13:29:44,824] DEBUG -  Entry: AxisOperation::getInputAction {org.apache.axis2.description.AxisOperation}
[2017-01-27 13:29:44,824] DEBUG -  Debug: AxisOperation::getInputAction - using soapAction {org.apache.axis2.description.AxisOperation}
[2017-01-27 13:29:44,824] DEBUG -  Exit: AxisOperation::getInputAction urn:mediate {org.apache.axis2.description.AxisOperation}
[2017-01-27 13:29:44,824] DEBUG -  mapActionToOperation: Mapping Action to Operation: action: urn:mediate; operation: org.apache.axis2.description.InOutAxisOperation@15f408b5named: m$
[2017-01-27 13:29:44,824] DEBUG -  DEBUG_FRAME = org.apache.axis2.util.JavaUtils.callStackToString(JavaUtils.java:564)
[2017-01-27 13:29:44,825] DEBUG -  Will not  evaluate the value of the remote entry with a key conf:/ws-policy/Secure_XXXXXXXX.xml,  because the registry is not availabl$
[2017-01-27 13:29:44,825]  WARN -  Cannot convert null to a StreamSource {org.apache.synapse.config.SynapseConfigUtils}
[2017-01-27 13:29:44,825] ERROR -  ProxyService Deployment from the file : /usr/local/wso2/wso2esb-4.9.0/tmp/carbonapps/-1234/1485485984115XXXXXXX_Deploy_.car/Authenticate_XXXXXXXX
org.apache.synapse.SynapseException: Cannot convert null to a StreamSource
        at org.apache.synapse.config.SynapseConfigUtils.handleException(SynapseConfigUtils.java:578)
        at org.apache.synapse.config.SynapseConfigUtils.getStreamSource(SynapseConfigUtils.java:79)
        at org.apache.synapse.core.axis2.ProxyService.getPolicyFromKey(ProxyService.java:822)
        at org.apache.synapse.core.axis2.ProxyService.buildAxisService(ProxyService.java:608)
        at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:80)
        at org.wso2.carbon.proxyadmin.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:46)

Answer 1

Turns out that the {$CARBON}/repository/deployment/server/synapse-configs/default/registry.xml file was missing on my install. I have no idea how that happened. Replacing from original install archive solved the issue.