Is there a way to preserve the order on a query from Athena? Assume the data in the s3 bucket or data lake are partitioned and are in parquet files. Every time I query something, the order is different each time. I am not sure how Athena works, but it makes sense to have multiple workers performing the query for performance and just combining the results together, which will make sense why the order is different each time. But is it possible to preserve the order of the results if all data is just coming from a single parquet file?
Athena preserve order
948 Views Asked by user1179317 At
1
There are 1 best solutions below
Related Questions in AMAZON-WEB-SERVICES
- "Access Denied" - User's Permissions to S3 Bucket
- Cohort analysis with Amazon Redshift / PostgreSQL
- Using Amazon KMS service on Heroku
- can't ssh in after cloning an EC2 instance on Amazon AWS
- Using HDFS with Apache Spark on Amazon EC2
- How can I access Mule ESB Community edition via browser?
- AWS EC2: Migrating from Windows to Linux Server
- AWS ELB Load Balancer: is it possible to set multiple session cookies?
- AWS Flow Framework: Can we run activity worker and activity task on different EC2 instances
- Unable to access files from public s3 bucket with boto
- Cloudfront stream only part of the video
- s3cmd not working as cron-task when echos/dates are added
- How to deploy django 1.8 on Elastic Beanstalk using Docker
- InstanceProfile is required for creating cluster - create python function to install module
- How to fix WordPress HTTPS issues when behind an Amazon Load Balancer?
Related Questions in AMAZON-S3
- Convert JSON.gz to JSON in node js
- Downloading objects from S3 with presigned URL
- "Access Denied" - User's Permissions to S3 Bucket
- jQuery file upload to S3 (and rails) with CORS headers
- copying file from local machine to Ubuntu 12.04 returning permission denied
- AWS Flow Framework: Can we run activity worker and activity task on different EC2 instances
- Unable to access files from public s3 bucket with boto
- s3cmd not working as cron-task when echos/dates are added
- AWS S3 object listing
- React-native upload image to amazons s3
- S3 restrictions on quantity of object downloads
- How to upload a photo in Meteor to S3 and have it sync to database item?
- Limit upload size to S3 with presigned URL
- dragonfly-s3 with S3 IAM user causing a forbidden 403 response from Amazon
- Split S3 files into multiple output files
Related Questions in AMAZON-ATHENA
- Cannot load simple Json file with AWS Athena
- Amazon Athena and compressed S3 files
- Amazon AWS Athena -- Removing Temp Files
- How to use aws athena using nodejs?
- AWS Athena JSON format failure
- Athena not adding partitions after msck repair table
- Location of S3 data for Amazon Athena
- read csv file where value contain comma in AWS athena
- AWS Glue unable to access input data set
- determine address of underlying s3 bucket in Athena
- running aws athena query via pyathena
- how to read escape character in Athena
- How to query Athena string to date in hadoop.hive.serde2.OpenCSVSerde format
- Athena Write Performance to AWS S3
- Athena date format unable to convert string to date formate
Related Questions in PYATHENA
- running aws athena query via pyathena
- Pyathena is super slow compared to querying from Athena
- Pyathena in Jupyter Notebook Can't find aws config file
- Which one is faster for querying Athena: pyathena or boto3?
- raise NoCredentialsError() botocore.exceptions.NoCredentialsError: Unable to locate credentials while connecting S3 using Athena
- PyAthena - Invalid length for parameter RoleArn
- AWS Athena is giving multiple "Connection pool is full, discarding connection" warnings while firing queries from a pyathena package
- Catch AWS's TooManyRequestsException when using Pyathena
- pyathena returns columns name and value separately
- How to Make Sure SQL Alchemy ORM Translates Each Numerical Object to Raw Query Correctly?
- Athena preserve order
- Why pyathena doesn't work on longer running queries while Athena runs them?
- Pyathena "s3_staging_dir" file - how can I get this filename to use it?
- Choosing data catalog in pyathena?
- Fixing error in a SHOW TABLES IN DATABASE name query
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
If the data in your original files is already time sorted, adding an
order by time_columnwon't add complexity to the query. Conceptually, each worker will sort a small fraction of the data, then merge-sort results from workers. For data that's already sorted these are inexpensive operations.