Have large fastq.gz files in an s3 bucket and want to look at a few lines and check that their format is correct. Working locally I use zcat <filepath> | head -n 8 which works fine, but on AWS to get the relevant filename I use aws s3 ls s3://bucket_name/filepath/ and get something like 2022-04-01 3244653538 filename.fastq.gz. Attempting to feed this filepath to zcat - with or without the datestamp and keycode - returns No such file or directory. I am an AWS noob and would greatly appreciate your input.
How to use zcat | head on large file in s3 bucket
398 Views Asked by jklaus At
0
There are 0 best solutions below
Related Questions in AMAZON-S3
- Convert JSON.gz to JSON in node js
- Downloading objects from S3 with presigned URL
- "Access Denied" - User's Permissions to S3 Bucket
- jQuery file upload to S3 (and rails) with CORS headers
- copying file from local machine to Ubuntu 12.04 returning permission denied
- AWS Flow Framework: Can we run activity worker and activity task on different EC2 instances
- Unable to access files from public s3 bucket with boto
- s3cmd not working as cron-task when echos/dates are added
- AWS S3 object listing
- React-native upload image to amazons s3
- S3 restrictions on quantity of object downloads
- How to upload a photo in Meteor to S3 and have it sync to database item?
- Limit upload size to S3 with presigned URL
- dragonfly-s3 with S3 IAM user causing a forbidden 403 response from Amazon
- Split S3 files into multiple output files
Related Questions in FASTQ
- Using python to compare differences in strings from an input fastq file
- Bash: Converting 4 columns of text interleaved lines (tab-delimited columns to FASTQ file)
- Using Biopython SeqIO.convert over an entire directory
- applying the same command to multiple files in multiple subdirectories
- Parse file and use some of the fields as variables using the header as name in bash
- FASTQ sequence record does not begin with @NAME line error
- Delete an item from a dictionary generated by SeqIO.index
- Convert FASTQ ASCII to decimal and hexadecimal in R
- assign file names dynamically to match directory name
- Read same extension multiple files in one directory in Perl
- QFileDialog "destroys" the name of the files
- Read FASTQ file into a AWS Glue Job Script
- In a fastq file, how do I change the sequence headers to the file name and a unique identifier?
- python dictionary, make every odd number line to key and even number line to value from a file
- How can I make my Python script faster?
Related Questions in ZCAT
- Using zcat in windows
- zcat a file, output its contents to another file based on original filename
- Reconstructing an archived SQLite3 database file
- Trying to combine awk and zcat with multiple filtering criteria
- Http get ungzip response in bash
- Segmentation fault 11, zcat pipe into postgresql 9.0.5 (Lion)
- How to use zcat | head on large file in s3 bucket
- MySQL import using zcat log time script is complete
- invalid predicate zcat
- Cron job for generate Go Access report not working
- Zcat multiple compressed files in a for loop
- zcat can assign as a variable in bash
- zcat filename expansion in for loop
- How do I zcat two files inline to perform diff?
- Combine log files (some gzipped) in Java
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?