I need to consume a stream from Jetstream by batch and store bulk in database with acknowledgment after database bulk insert. At this time I use pull consumer, store all messages and commit 1 by 1 on successful bulk insert. Do I need to ack all messages? I saw that I can commit the latest message but for this I need to be in consumer push mode.
whats the best way to consume and commit by batch with NATS Jetstream
1.5k Views Asked by erich At
1
There are 1 best solutions below
Related Questions in NATS.IO
- communication between processes: tcp vs unix sockets, ipc vs nats
- Golang Nats subscribe issue
- NATS async reply to request is not asynchronnous
- How to connect STAN to NATS using NKeys?
- What is the best way to deploy NATS using Azure
- How did consumer handle the messages without start a new Thread in NATS-JetStream?
- Timeout error in nats client connectivity for react native | Android ONLY
- NATS jetstream timeout inside anyio taskgroup
- jsAccountLimits::MaxBytesRequired is a bool ean, should it be int64_t?
- Spark Streaming output mode to process only new messages
- nats-spark-connector with Java giving an error
- Modeling multiple, replicated consumers with Jetstream
- NATS-streaming - Where i can find clusterID? I watched tons of websites, but there's nowhere i could find how to know it
- Spark Streaming - After application restart, pulled message count is not correct
- Is it possible for NATS JetStream Publisher to get notification about discarded message?
Related Questions in NATS-JETSTREAM
- How did consumer handle the messages without start a new Thread in NATS-JetStream?
- Modeling multiple, replicated consumers with Jetstream
- Nats jetstream load balancing
- Is it possible for NATS JetStream Publisher to get notification about discarded message?
- KEDA NATS scaler not activating deployment
- nats update stream replicas - no suitable peers for placement
- JetStream: How to make Push Subscription with Durable consumer? (i get "No interest")
- whats the best way to consume and commit by batch with NATS Jetstream
- Is it possible to redirect NATs Jetstream message to Elastic search
- Can NATS JetStream partially replicate streams on different NATS Servers?
- Nats/Jetstream: naming conventions for commands' subjects
- How to list the messages of a nats jetstream and know if they were acknowledged?
- Where does "has" come from in "this.has" expression?
- Can NATS keep messages in memory if no workers are online at the moment (producer-consumers pattern)?
- Does NATS Jetstream provide message ordering by a key?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
ConsumerConfiguration.AckWaitis probably what you need.For pull mode, it is required that you ack every message. You could set the
AckWaitto a longer duration, which should give you enough time to make sure your bulk store works and then go back and individually ack the messages.If you use push mode, you could use
AckPolicy.Alland set theAckWaitlonger also. You can then read messages in push mode, (which looks like one message at a time, but the server has actually pushed many messages to the client and the client is buffering them.) Once you've collected the number of messages you want to write to the database, you can write them in bulk, then you can ack the last message when you know the database write has succeeded.