I am able to import data from my MS sql to HDFS using JDBCHDFC Spring Batch jobs.But if that containers fails , the job does not shift to other container. How do I proceed to make the job fault tolerant. I am using spring xd 1.0.1 release
jdbc to HDFS import using spring batch job
158 Views Asked by Neeraj Kumar At
1
There are 1 best solutions below
Related Questions in SPRING
- HTTPS configuration in Spring Boot, server returning timeout
- Multi Tenancy in Spring - Partitioned Data Approach
- How to create beans of the same class for multiple template parameters in Spring
- org.telegram.telegrambots.meta.exceptions.TelegramApiException: Bot token and username can't be empty
- Springboot: How to get an entity optional property and check null?
- How do I propagate the current SecurityContext to my @RabbitListener in Spring Boot?
- Spring's XML based bean configuration for Object Mapper's Case Insensitive property
- Failed to configure a DataSource: 'url' attribute is not specified and no embedded datasource could be configured. I'm using Postgresql
- springboot class org.hibernate.mapping.Bag cannot be cast to class org.hibernate.mapping.SimpleValue
- Issue while deploying JDK 17 and Spring 6 application in Tomcat 10.1.20
- Spring JPA Data Auditing - How to design it?
- Springframework test: Async not started
- Error: Cannot invoke "jakarta.servlet.http.HttpSession.getAttribute(String)" because "session" is null
- How does spring-retry determine which methods to retry when @Retryable is placed at the class level?
- problem with edge server registration in Eureka
Related Questions in HDFS
- Can anyoone help me with this problem while trying to install hadoop on ubuntu?
- ERROR: org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "maprfs"
- How to optimize writing to a large table in Hive/HDFS using Spark
- Update hadoop hadoop-2.6.5 to haddop 3.x. Operation category WRITE is not supported in state standby
- Copy/Merge multiple HDFS files using Nifi Processor
- HDFS too many bad blocks due to "Operation category WRITE is not supported in state standby" - Understanding why datanode can't find Active NameNode
- distcp throws java.io.IOException when copying files
- ERROR flume.SinkRunner: Unable to deliver event
- Apache flume does not run hadoop 3.1.0 Flume 1.11
- Livy session to submit pyspark from HDFS
- ClickHouse Server Exception: Code: 210.DB::Exception: Fail to read from HDFS:
- Confluent HDFS Sink connector error while connecting HDFS to Hive
- Node Transitioned from NEW to UNHEALTHY and Attempting to remove non-existent node
- Error associated with Azure Datalake Gen2 and Hadoop connection
- How do I directly read files from HDFS using dask?
Related Questions in SPRING-BATCH
- How to customize the Spring Batch default configuration without breaking autoconfiguration
- Is there any catch in Reading and Updating to the same Oracle table in Single Threaded Spring batch job?
- Starting first job causes - Dispatcher has no subscribers for channel exception when declared multiple DirectChannels
- Looping Over a Group of Steps in Spring Batch
- Already value [datasource.ConnectionHolder@6b144bd8] for key [datasource.DriverManagerDataSource@56739ee9] bound to thread [main]
- How to Use ControllerAdvice for Exception Handling in Spring Batch Application?
- Writing in multiple related tables in spring batch
- Restarting a FAILED job is not processing the failed chunk data again : Continuation
- Spring Batch Not persisting Data Properly in MYSQL Database
- Strange Spring Batch exception, something I did wrong?
- Parameter 0 of method jobBean in com.nextuple.BatchProcessing.config.BatchConfig required a bean named 'dataSource' that could not be found
- is spring batch supports parquet file reading and writing
- Spring Batch 5 - Don't want it to automatically create tables in database
- Transaction flow for distributed transaction in Spring Batch
- How to solve JobBuilder deprecated in SpringBatch 5.1.1
Related Questions in SPRING-BATCH-ADMIN
- Running multiple instances of spring batch job in parallel
- Creating a Generic Utility for JDBCItemReader in a Multi-Step Spring Batch Application with @StepScope
- How to make the JobRegistry contain the job information on server restart in spring batch
- Spring batch 5: how to invoke a particular job from commandline?
- Spring Batch Can we have in memory for batch job and application have other data source
- Spring Batch - How to dynamically invoke a Step Configuration to reinitialise partitioner instance
- How to integrate Spring Cloud Data Flow with Spring Boot Admin for batch applications monitoring?
- In Spring boot batch, should the itemWriter, itemProcessor, itemWriter be step-scoped?
- DataSourceTransactionManager bean not created after Introducing Spring Batch @EnableBatchProcessing Annotation
- Spring Batch - Need help and suggestion to decide on the best approach to compare two different Databases and create an excel report with differences
- Set job name in Spring Batch Test
- In Spring Batch, linked with a ItemReader call I want to call a static util method to populate a string
- Spring Batch- how to pass list of multiple items from input to ItemReader, ItemProcessor and ItemWriter
- How to integrate Spring Batch ( without Spring Boot ) with Spring Cloud Data Flow Server
- FlatFileItemWriter - Writer must be open before it can be written to
Related Questions in SPRING-XD
- Can't install Spring-XD properly on Windows
- Not able to convert Message from Json String to Object. ClassCastException
- SpringXD Kryo-shaded-3.0.3.jar and NoClassDefFoundError and ClassNotFoundException And Enum
- kafka consumer get a message which was consumed before
- Highest RabbitMq compatibility as Message Bus for Spring XD 1.3.0.RELEASE
- In a Spring-XD job module, how to write data to two files respectively at the same path in two servers where XD containers are running?
- An illegal reflective access operation has occurred while setting up spring xd
- reading CSV file from s3 using spark
- FTP source - obtaining file reference each time a file is transferred
- Class org.apache.kafka.common.serialization.StringSerializer not found
- I am trying to run a Groovy script from Spring XD shell. But script does not trigger
- springxd JMS sink acknowledge propery
- How to make the job restartable in Spring xD
- Spring-xd Redis sink with Collection type MAP
- Headers in http source Spring cloud flow
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular # Hahtags
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
You don't mention which version of Spring XD you're currently using so I can't verify the exact behavior. However, on a container failure with a batch job running in the current version, the job should be re-deployed to a new eligible container. That being said, it will not restart the job automatically. We are currently looking at options for how to allow a user to specify if they want it restarted (there are scenarios that fall into both camps so we need to allow a user to configure that).