I am new to AWS and I don't understand the use of AWS Kinesis quite well. I know it is used for processing streaming data, but why not just use AWS Lambda to process the incoming data and possibly store them to DB eventually? As a real case scenario, imagine there is a web crawler that constantly crawls a website for specific activities and sends them in JSON format. Why should I use AWS Kinesis in this case when I can do all my data processing through a lambda function? Is it because Kinesis can handle large amounts of data through auto calling its shards? Also, after using Kinesis to store my data into S3 I realized that my data got stored into S3 bucket with almost 3-4 minutes of delay which is not acceptable. Can anyone help me, please?
Why use AWS Kinesis, when AWS Lambda can handle the task?
195 Views Asked by Michael At
0
There are 0 best solutions below
Related Questions in AMAZON-WEB-SERVICES
- "Access Denied" - User's Permissions to S3 Bucket
- Cohort analysis with Amazon Redshift / PostgreSQL
- Using Amazon KMS service on Heroku
- can't ssh in after cloning an EC2 instance on Amazon AWS
- Using HDFS with Apache Spark on Amazon EC2
- How can I access Mule ESB Community edition via browser?
- AWS EC2: Migrating from Windows to Linux Server
- AWS ELB Load Balancer: is it possible to set multiple session cookies?
- AWS Flow Framework: Can we run activity worker and activity task on different EC2 instances
- Unable to access files from public s3 bucket with boto
- Cloudfront stream only part of the video
- s3cmd not working as cron-task when echos/dates are added
- How to deploy django 1.8 on Elastic Beanstalk using Docker
- InstanceProfile is required for creating cluster - create python function to install module
- How to fix WordPress HTTPS issues when behind an Amazon Load Balancer?
Related Questions in AMAZON-S3
- Convert JSON.gz to JSON in node js
- Downloading objects from S3 with presigned URL
- "Access Denied" - User's Permissions to S3 Bucket
- jQuery file upload to S3 (and rails) with CORS headers
- copying file from local machine to Ubuntu 12.04 returning permission denied
- AWS Flow Framework: Can we run activity worker and activity task on different EC2 instances
- Unable to access files from public s3 bucket with boto
- s3cmd not working as cron-task when echos/dates are added
- AWS S3 object listing
- React-native upload image to amazons s3
- S3 restrictions on quantity of object downloads
- How to upload a photo in Meteor to S3 and have it sync to database item?
- Limit upload size to S3 with presigned URL
- dragonfly-s3 with S3 IAM user causing a forbidden 403 response from Amazon
- Split S3 files into multiple output files
Related Questions in AWS-LAMBDA
- How to get rows count from Amazon DynamoDB using Lambda AWS
- Querying DynamoDB with Lambda does nothing
- undefined is not a function after refactor
- Async AWS Lambda not executed if caller returns too early
- In amazon lambda, resizing multiple thumbnail sizes in parallel async throws Error: Stream yields empty buffer
- How to upload an object into S3 in Lambda?
- How to do image overlay and watermark using node.js in amazon lambda function
- Base64 encode UserData parameter for EC2 RunInstances using AWS Lambda
- AWS Lambda PHP Create Function with Zip
- Triggering a AWS Lambda from a form post
- Zip Files & Folders With No Base Directory
- Dynamically loading jar from arbitrary url
- AWSTask is not instantiable
- AWS Custom Authorizer with request parameters
- Parse OSM PBF in AWS Lambda and S3
Related Questions in APACHE-CAMEL-AWS-KINESIS
- Returning from http lambda endpoint for longer than 30s
- aws lambda throws internal lambda error during streaming from kinesis
- Kinesis Agent Input - Log file created for each day
- Trying to understand WebRTC Signaling Channels
- Lambda function triggering more than once
- AWS kinesis getRecords returns empty Records[]
- How to decode AWS Kinesis Video Stream GetMedia API output to mp3/wav?
- Trigger AWS lambda for specific Kinesis event push
- AWS Kinesis, Is S3 mandatory to be used for kinesis?
- AWS kinesis video stream in raspberrypi. Unable to create Kinesis Video stream
- Why use AWS Kinesis, when AWS Lambda can handle the task?
- Camel Kinesis ignoring maxResultsPerRequest parameter
- Using Apache Camel AWS-KINESIS endpoint, how do I checkpoint the message in the Kinesis stream?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?