I am new to GCP and while reading the documentation about Auto-tuning by Dataflow service they are talking about backlog and auto-scaling that depends on it. In this particular case what is backlog? If my pipeline is reading from a pub/sub, is it the age of oldest message or the number of unacknowledged messages?
Backlog in Google Cloud Pub/Sub
1k Views Asked by lookout At
1
There are 1 best solutions below
Related Questions in GOOGLE-CLOUD-PLATFORM
- Google Logging API - What service name to use when writing entries from non-Google application?
- Custom exception message from google endpoints exception
- Unable to connect database of lamp instance from servlet running on tomcat instance of google cloud
- How to launch a Jar file using Spark on hadoop
- Google Cloud Bigtable Durability/Availability Guarantees
- How do I add a startup script to an existing VM from the developer console?
- What is the difference between an Instance and an Instance group
- How do i change files using ftp in google cloud?
- How to update all machines in an instance group on Google Cloud Platform?
- Setting up freeswitch server on Google cloud compute
- Google Cloud Endpoints: verifyToken: Signature length not correct
- Google Cloud BigTable connection setup time
- How GCE HTTP Cross-Region Load Balancing implemented
- Google Cloud Bigtable compression
- Google cloud SDK code to execute via cron
Related Questions in PUBLISH-SUBSCRIBE
- using Message oriented middleware for communications within single web application realm
- Faye Ruby Server Side Publish on Heroku - EventMachine buffer overflow detected
- Pub/Sub Vs Observer Vs Reactive
- Adding pubsubhubbub link tag to Atom feed
- $rootScope.$broadcast in factory cannot change a value in other $scope
- Knockout component updating observable not being subscribed to by parent view model?
- Meteor show spinner on method call
- Meteor Counts in Router Level Subscription
- Meteor JS - Exception from sub id (id that does not exist)
- Is it posible to hook redis before key expired
- Redis hooking (publish-subscribe) under stress tests - performance under load
- How i can unsubscribe to a 'changefeed' in RethinkDB?
- Share profile picture change to all users in my roster
- ZeroMQ: How to send messages to a specific subscriber
- Node.js redis pubsub
Related Questions in BACKLOG
- VSTS Feature/PBI record the release version
- Team Foundation server 2013 backlog order API
- How to hide not done PBI from backlog in TFS 2013
- Socket Backlog behaviour
- What are sprints children nodes of backlog iteration in TFS 2012?
- How to install plugins on Redmine hosted on Heroku.com
- Measuring scale for story points in Scrum framework
- Cassandra: MigrationStage cannot keep up
- listen() ignoring backlog value
- What is the best way to gauge how much request Heroku is rejecting due to Backlog too deep?
- Free online application for doing scrum on rails
- How to set a custom work item to a back log level
- Priorize a single workitem in TFS
- Java ServerSocket prevent clients queueing
- listen() backlog upper limits
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Backlogs in Dataflow aren't related to PubSub. Dataflow always get a message from PubSub when it is here. But the processing queue can increase internally in Dataflow: that is the backlogs. If it's too big, and the CPU consumption too high a new worker is added to the pipeline.
In streaming mode, you still have backlog, but you also have a predictive backlog. In fact, it compare the number of message in each time windows and if the number of message increase that can be the beginning of a spike and dataflow can scale up proactively.