I have an ES cluster for which we started to stream data using an Azure Function (on python) into Azure Blobs. Basically, I am getting the last 5 min data in JSON format, compressing the list of JSON's and storing them on the Blob for live data. We have an S3 bucket where snapshots have gone into. Is there a way to read the JSON's in the files with python without having to restore the indexes on a separate ES to get historical data? It's a lot of data and having to run a separate instance just to restore it so then I can extract the JSON's is just not cost-effective.
How to read Elasticsearch snapshot files in python?
570 Views Asked by Andrei Budaes At
1
There are 1 best solutions below
Related Questions in PYTHON
- new thread blocks main thread
- Extracting viewCount & SubscriberCount from YouTube API V3 for a given channel, where channelID does not equal userID
- Display images on Django Template Site
- Difference between list() and dict() with generators
- How can I serialize a numpy array while preserving matrix dimensions?
- Protractor did not run properly when using browser.wait, msg: "Wait timed out after XXXms"
- Why is my program adding int as string (4+7 = 47)?
- store numpy array in mysql
- how to omit the less frequent words from a dictionary in python?
- Update a text file with ( new words+ \n ) after the words is appended into a list
- python how to write list of lists to file
- Removing URL features from tokens in NLTK
- Optimizing for Social Leaderboards
- Python : Get size of string in bytes
- What is the code of the sorted function?
Related Questions in JSON
- getting undefined while iterating json
- How can I serialize a numpy array while preserving matrix dimensions?
- What is best way to check if any of the property of object is null or empty?
- How to query JSON data according to JSON array's size with Spark SQL?
- Extracting data from json_decode with lat and lng geolocation
- Convert JSON.gz to JSON in node js
- How do I get the type to convert to when deserializing from Jackson
- Escape dot in jquery validate plugin
- Are allOf and properties keywords interchangeable?
- Sort continents by amount of countries
- Is there a data format lighter than json?
- Object of class CS_REST_Wrapper_Result could not be converted to string in CAMPAIGN MONITOR
- How to read JSON data from a web server running PHP and MySQL?
- Parse Nsmutabledictionary and extract value
- Handle empty JSON values in Java
Related Questions in ELASTICSEARCH
- Elasticsearch schema for multiple versions of the same text
- Elasticsearch nested filter query
- Elasticsearch data model
- search with filter by token count
- Usage of - operator in elasticsearch
- Running multiprocessing on two different functions in Python 2.7
- How to get an Elasticsearch aggregation with multiple fields
- How to implement custom sort in elasticsearch?
- Custom Analyzer not working Elasticsearch
- How to implement full text search using Elasticsearch in Rails?
- UnresolvedAddressException in Logstash+elasticsearch
- Elasticsearch Fiddler No DNS
- Monolithic ETL to distributed/scalable solution and OLAP cube to Elasticsearch/Solr
- how to disable page query in Spring-data-elasticsearch
- Create Custom Analyzer after index has been created
Related Questions in AZURE-BLOB-STORAGE
- Cordova with Windows Azure Storage, if the access to my container is public, do I still need authentification?
- Is it possible to copy from a RA-GRS secondary using Start-AzureStorageBlobCopy?
- Azure Blob video is not streaming
- 409 Conflict when attempting to revert a Page Blob to a previous snapshot
- How big a factor is blob size on retrieval time in Azure Storage?
- Upload file to Azure Blob from HTML webpage (C#)?
- Excel file (azure blob) does not download in chrome
- Getting error 400 bad request when accessing Azure blob storage - all latest components
- Azure storage 'put block list' returns 400
- Windows Azure Website and static html in Blob storage
- how to check if there is data in an azure block blob
- Azure - is one 'block blob' seen as one file?
- How to call Azure Blob Storage using REST API with SAS
- Automatically delete Blob files in Azure based on a future date
- Browser crashes while downloading large size files
Related Questions in ELASTICSEARCH-SNAPSHOT
- AWS ElasticsearchService: Automated snapshot is running for more than 20days
- Elasticsearch Restore from S3 Snapshot
- How does Elasticsearch incremental snapshots deal with the deleted docs?
- Elasticsearch Restoring large snapshot incomplete due to size
- Elasticsearch - repository_exception
- "Read-only file system" error while creating snapshot in elasticsearch with multiple-nodes
- How to delete ElasticSearch snapshots by prefix
- ElasticSearch Snapshot: reasons for partial status
- How to take snapshots of specific indices in Elastic Cloud Enterprise?
- elasticsearch on docker - snapshot and restore - access_denied_exception
- Unable to register snapshot repository for AWS elasticsearch in fs
- How to manually invoke snapshot endpoint for ElasticSearch using JestClient
- Unable to restore elasticsearch snapshots from S3 using the repository-s3 plugin after nodes data directory got deleted from all nodes
- How to read Elasticsearch snapshot files in python?
- Elasticsearch snapshot for a cluster
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
No, it's not possible, to access the data in the snapshots you will need to restore them.
But you do not need a separate instance/cluster, you can restore using the same instance/cluster you have and renaming the restored index as the example in the documentation.