disable-output-escaping broken with libxslt library on MacOS Catalina

Question

A curious problem after upgrading to MacOS Catalina and linking against the libxslt and libxml2 included in the 10.15 SDK, is that all uses of disable-output-escaping="yes" are broken and escape the output anyway. A couple examples from various XSLT sheets:

<xsl:text disable-output-escaping="yes">&lt;--test comment--&gt;</xsl:text>
<xsl:value-of select="/*[1]/LBL_CM_3" disable-output-escaping="yes" />

Where the LBL_CM_3 node has some HTML in it, e.g. cm<sup>3</sup>. Both of these cases are escaped (unexpected result) when processed by my code, using very straight-forward API use. Here is a stripped down version with error checking and cleanup omitted:

    xmlDocPtr tmpXmlDoc = xmlParseMemory(inputString.c_str(), (int)inputString.size());
    xmlDocPtr tmpXslDoc = xmlParseMemory(xslInputStr, xslInputStrSize);
    xsltStylesheetPtr tmpXslStyleSheet = xsltParseStylesheetDoc(tmpXslDoc);
    xmlDocPtr tmpHtmlDoc = xsltApplyStylesheet(tmpXslStyleSheet, tmpXmlDoc, params);

However, if I process the same input and stylesheet using /usr/bin/xsltproc, the output is not escaped (expected result). We both link the same library on the system:

$ otool -L /usr/bin/xsltproc | grep -E 'xml|xslt'
/usr/bin/xsltproc:
    /usr/lib/libexslt.0.dylib (compatibility version 9.0.0, current version 9.15.0)
    /usr/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)
    /usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
$ otool -L TestXslt.app/Contents/MacOS/TestXslt | grep -E 'xml|xslt'
    /usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
    /usr/lib/libexslt.0.dylib (compatibility version 9.0.0, current version 9.15.0)
    /usr/lib/libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)

The system xsltproc version info:

$ /usr/bin/xsltproc --version
Using libxml 20904, libxslt 10129 and libexslt 817
xsltproc was compiled against libxml 20904, libxslt 10129 and libexslt 817
libxslt 10129 was compiled against libxml 20904
libexslt 817 was compiled against libxml 20904

The 10.15 SDK usr/include/libxml/xmlversion.h shows the same version:

/**
 * LIBXML_DOTTED_VERSION:
 *
 * the version string like "1.2.3"
 */
#define LIBXML_DOTTED_VERSION "2.9.4"

/**
 * LIBXML_VERSION:
 *
 * the version number: 1.2.3 value is 10203
 */
#define LIBXML_VERSION 20904

This worked as expected previously with the MacOS Sierra SDK, with the same calling code and same XSLT sheets. All the rest of the XSLT processing appears to be working as expected, only disable-output-escaping="yes" is being ignored.

Answer 1

This turned out to be caused by mixed linking of conflicting versions of libxml2 and libxslt from another dependency. The system versions:

libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
libxslt.1.dylib (compatibility version 3.0.0, current version 3.26.0)

The bundled versions with pre-built Postgres 12:

libxml2.2.dylib (compatibility version 12.0.0, current version 12.4.0)
libxslt.1.dylib (compatibility version 3.0.0, current version 3.29.0)

If you get the e.g. the libxml2 from Postgres but the libxslt from the system, then it mostly works but disable-output-escaping mysteriously breaks. If you use either pair (both system, or both bundled with Postgres), then disable-output-escaping works. You have to take into account the linker ordering (e.g. as reported by otool -L if more than one version is present.