Can Spring Integration be used in Mutli Server Environment

Question

Can we use Spring Integration to configure directory polling for files such that -

With 2 servers configured, polling occurs on 1 server and corresponding processing get distributed b/w both the servers.

Also, can we switch the polling on either of the servers on runtime ?

Edit - Tried configuring JBDC MetaStore and run the two instances separately, able to poll and process but getting intermittently DeadLockLoserDataAccessException

Configuration below

@Bean
public MessageChannel fileInputChannel(){
return new DirectChannel();
}

@Bean(PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller(){
PollerMetadata pollermetadata = new PollerMetadata();
pollermetadata.setMaxMessagesPerPoll(-1);
pollermetadata.setTrigger(new PeriodicTrigger(1000));
return pollermetadata;
}

@Bean
@InBoundChannelAdapter(value = "fileInputChannel"){
FileReadingMessageSource source = new FileReadingMessageSource();
source.setDirectory("Mylocalpath");
FileSystemPersistentAcceptOnceFileListFilter acceptOnce = new FileSystemPersistentAcceptOnceFileListFilter();
ChainFileListFilter<File> chainFilter = ChainFileListFilter(".*\\.txt"));
chainFilter.addFilter(acceptOnce);
source.setFilter(chainFilter);
source.setUseWatchService(true);
source.setWatchEvents(FileReadingMessageSource.WatchEventType.CREATE,FileReadingMessageSource.WatchEventType.MODIFY);
return source;
}

@Bean
public IntegrationFlow processFileFlow(){
return IntegrationFlows.from("fileInputChannel")
.handle(service).get();
}

Answer 1

It is really one of the features of Spring Integration to easy implement a distributed solution. You just need add a messaging middle ware into your cluster infrastructure and have all the nodes connected to some destination for sending and receiving. A good example could be a SubscribableJmsChannel which you just simple can declare in your application context and all the nodes of your cluster are going to subscribe to this channel for round-robin consumption from JMS queue. It already wouldn't matter which node produces to this channel.

See more in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/jms.html#jms-channel.

Another sample of similar distributed channels are: AMQP, Kafka, Redis, ZeroMQ.

You also can have a shared message store and use it in the QueueChannel definition: https://docs.spring.io/spring-integration/docs/current/reference/html/system-management.html#message-store