modcluster (Wildfly) is not detecting Advertize from Apache

Question

I am trying to configure WildFly 8.1.0 with mod_cluster. Both WildFly and Apache are running on the same machine. The machine is Ubuntu 12.04 with Apache 2.2.x

Apache is set up correctly (I believe). I have tested that the advertise module is working correctly by running the test class Advertise found in the mod_proxy source code (github). There are no errors in the apache logs.

I am starting the server as follows: ./standalone.sh -c standalone-ha.xml

If any one can see something wrong with the configuration below and help put me out of days of misery, I would be really grateful....

Apache configuration

CreateBalancers 1

<IfModule manager_module>

    #Listen 127.0.1.1:6666
    Listen *:6666
    ManagerBalancerName mycluster

    <VirtualHost *:6666> 

        KeepAliveTimeout 300
        MaxKeepAliveRequests 0
        AdvertiseFrequency 5
        ServerAdvertise On
        AllowDisplay On

        <Location />
            Order deny,allow
            Allow from 127.0.1
        </Location>

       <Location /mod_cluster_manager>
            SetHandler mod_cluster-manager
            Order deny,allow
            #Deny from all
            #Allow from 127.0.1
            Allow from all
        </Location>
     </VirtualHost>
</IfModule>

VirtualHost configuration

<VirtualHost *:80>

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        ProxyPass / balancer://mycluster stickysession=JSESSIONID|jsessionid nofailover=On
        ProxyPassReverse / balancer://mycluster
        ProxyPreserveHost On

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

The results of the Advertize test class

received from /178.62.50.xxx:23364
received: HTTP/1.0 200 OK
Date: Sat, 26 Jul 2014 20:03:12 GMT
Sequence: 121
Digest: 4dedd3761d451227f36534b63ca2a8a1
Server: b23584e2-314f-404d-8fde-05069bfe5dc7
X-Manager-Address: 127.0.1.1:6666
X-Manager-Url: /b23584e2-314f-404d-8fde-05069bfe5dc7
X-Manager-Protocol: http
X-Manager-Host: 127.0.1.1

The print out from mod_cluster_manager (178.62.50.xxx:6666/mod_cluster_manager)

mod_cluster/1.2.6.Final
start of "httpd.conf" configuration
mod_proxy_cluster.c: OK
mod_sharedmem.c: OK
Protocol supported: http AJP 
mod_advertise.c: OK
Server: 127.0.1.1 
Server: 127.0.1.1 VirtualHost: *:80
Server: 127.0.1.1 VirtualHost: *:6666 Advertising on Group 224.0.1.105 Port 23364 for http://127.0.1.1:6666 every 5 seconds
end of "httpd.conf" configuration

Lastly, here are the relevant bits taken from standalone-ha.xml

<subsystem xmlns="urn:jboss:domain:modcluster:1.2">
    <mod-cluster-config advertise-socket="modcluster" connector="ajp">
        <dynamic-load-provider>
            <load-metric type="cpu"/>
        </dynamic-load-provider>
    </mod-cluster-config>
</subsystem>

<socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>

<interfaces>
    <interface name="management">
        <inet-address value="178.62.50.xxx"/>
    </interface>
    <interface name="public">
        <inet-address value="127.0.1.1"/>
    </interface>
    <interface name="unsecure">
        <inet-address value="${jboss.bind.address.unsecure:127.0.1.1}"/>
    </interface>
</interfaces>

The only part of the server log that relates to modcluster (output during server startup)

15:53:29,805 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) JBAS017519: Undertow HTTP listener default listening on /127.0.1.1:8080
15:53:29,811 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) JBAS017519: Undertow AJP listener ajp listening on /127.0.1.1:8009
15:53:29,905 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000001: Initializing mod_cluster version 1.3.0.Final
15:53:29,967 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364

Answer 1

I managed to figure out the problem here, finally. The public interface also needs to reference the server IP, not 127.0.1.1.

The updated interface configuration is:

<interfaces>
    <interface name="management">
        <inet-address value="178.62.50.xxx"/>
    </interface>
    <interface name="public">
        <inet-address value="178.62.50.xxx"/>
    </interface>
    <interface name="unsecure">
        <inet-address value="${jboss.bind.address.unsecure:127.0.1.1}"/>
    </interface>
</interfaces>

Answer 2

You need to:

• use EnableMCPMReceive in that <VirtualHost *:6666> so as to allow it to consume MCMP messages from WildFly

• <Location /> in that EnableMCPMReceive enabled VirtualHost must allow IP addresses of WildFly servers

• with WildFly, never bind to localhost -- it doesn't make any sense in the mod_cluster environment except when the boxes (Apache and WildFly instances) are located all in a single box; which is likely a local development situation only

• the whole chain of events is as follows:

  1. Apache sends a UDP multicast message stating its location (by default the VirtualHost with which is the advertising enabled)
  2. WildFly catches the message and mod_cluster subsystem sends CONFIG and STATUS messages containing its JvmRoute (instance-id), Host, Port, protocol..., load... The Host and Port must be reachable from the Apache HTTP Server.
  3. Apache HTTP Server talks to the WildFly server on its Host:Port with STATUS-RSP messages etc. The communication is bidirectional.

Please, don't confuse this with request routing, by default, client requests are delivered from Apache HTTP Server to WildFly and back via AJP whereas the MCMP messages use HTTP.

HTH K