I need to consume a stream from Jetstream by batch and store bulk in database with acknowledgment after database bulk insert. At this time I use pull consumer, store all messages and commit 1 by 1 on successful bulk insert. Do I need to ack all messages? I saw that I can commit the latest message but for this I need to be in consumer push mode.
whats the best way to consume and commit by batch with NATS Jetstream
1.6k Views Asked by erich At
1
There are 1 best solutions below
Related Questions in NATS.IO
- How are Consumers implemented in NATS
- User authentication via JWT token in NATS WebSocket
- How to configure Nats supercluster to publish messages across two super-clusters running with different operators via gateways
- TLS handshake error: read tcp in NATS server K8 pod
- Why NATS doesn't map subjects?
- GCP NATS Subscriber/Workers
- Out of the box, does NATS / JetStreams guarantee message delivery?
- docker nuts connection refused
- NestJs NATS listener
- How to build Kafka-like queue with NATS
- Executor instance is not getting created while running Spark app on Kubernetes ( Minikube)
- NATS-Jetstream two consumer run issue
- How can my client from another device access my docker nats-server?
- Is it possible to get the current number of attempts to process the message msg jetstream.Msg?
- NATS local server that provides remote jetstream
Related Questions in NATS-JETSTREAM
- How did consumer handle the messages without start a new Thread in NATS-JetStream?
- Modeling multiple, replicated consumers with Jetstream
- Nats jetstream load balancing
- Is it possible for NATS JetStream Publisher to get notification about discarded message?
- KEDA NATS scaler not activating deployment
- nats update stream replicas - no suitable peers for placement
- JetStream: How to make Push Subscription with Durable consumer? (i get "No interest")
- whats the best way to consume and commit by batch with NATS Jetstream
- Is it possible to redirect NATs Jetstream message to Elastic search
- Can NATS JetStream partially replicate streams on different NATS Servers?
- Nats/Jetstream: naming conventions for commands' subjects
- How to list the messages of a nats jetstream and know if they were acknowledged?
- Where does "has" come from in "this.has" expression?
- Can NATS keep messages in memory if no workers are online at the moment (producer-consumers pattern)?
- Does NATS Jetstream provide message ordering by a key?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
ConsumerConfiguration.AckWaitis probably what you need.For pull mode, it is required that you ack every message. You could set the
AckWaitto a longer duration, which should give you enough time to make sure your bulk store works and then go back and individually ack the messages.If you use push mode, you could use
AckPolicy.Alland set theAckWaitlonger also. You can then read messages in push mode, (which looks like one message at a time, but the server has actually pushed many messages to the client and the client is buffering them.) Once you've collected the number of messages you want to write to the database, you can write them in bulk, then you can ack the last message when you know the database write has succeeded.