Atlas is product of choice for Hadoop data lineage question. Is there any clear product for data lineage tracking on aws Athena or Glue.
How is data lineage tracked in aws athena and glue?
2k Views Asked by Shishir Choudhary At
1
There are 1 best solutions below
Related Questions in AWS-GLUE
- AWS GLUE child node execution order of same level
- Is there a way to import Redshift Connection in PySpark AWS Glue Job?
- Retrieving a list of all failed Glue jobs via CLI
- How do I change the data type in a Glue Crawler?
- Loading around 50gb of parquet data to Redshift taking indefinite time to load
- Glue Notebook not starting: Failed to start notebook
- old aws-glue libraries in the Glue streaming ETL job 4.0?
- Add File name column to Dynamic Frame
- How to test Glue jobs and Athena queries locally on dummy data?
- AWS Glue throws AWSBadRequestException when loading DynamicFrame from s3 with local Glue docker
- AWS Glue Insert and update into oracle table
- SQL query to extract incremental data from a table in SQL Server
- redshift spectrum type conversion from String to Varchar
- Apply transformation on nested json column in dataframe
- Access Denied while creating crawler
Related Questions in AMAZON-ATHENA
- How can I join data to my table that isn't available for everyone without losing results?
- How do I change the data type in a Glue Crawler?
- How can I determine the column level differences between 2 tables?
- How to test Glue jobs and Athena queries locally on dummy data?
- AWS Athena: how to use LIKE in the query
- Using map_agg with order in Athena Presto
- Using Lambda function to create prefixes/folder like Athena when doing query
- How to register a UDF in AWS Athena?
- View uses get_json_object fails in Athena, but works in Databricks
- AWS Glue table schema comments vs no comments
- Trino/Presto SQ: Replace NULL with a value only if the NULL comes after the first non-NULL value in the group
- Programatically querying Delta Table via Athena is failing
- Submit multiple SQL queries in a dynamic way
- How do I make the rounding function of athena match redshift?
- Aws Athena SQL Query is not working in Apache spark
Related Questions in DATA-LINEAGE
- ODI 12c Data Lineage Query with Source, Staging, Target table column details
- Data Lineage in Unity Catalog is not shown in lineage tab in databricks
- How is marquez aware of the structure that airflow sets up?
- BigQueryInsertJobOperator data_lineage doesn't work on Google Cloud Composer with tableDefinitions
- Salesforce API, extract lineage
- data lineage and provenance of airflow pipeline
- How to login to Collibra from AWS EC2 instance?
- PySpark OpenLineage configuration
- Is it possible to find the queries in BigQuery triggered by "looker studio"/ "data studio" using INFORMATION_SCHEMA.JOBS_BY_PROJECT?
- How to convert an arbitrary SQL statement to column level lineage information via an open source solution?
- How can you create lineage between Power BI datasets and Databricks sql warehouse
- How to inject inlets and outlets parameters in Airflow PythonOperator executable function
- BigQuery Data Lineage using AuditLogs, PubSub, Dataflow, ZetaSQL and Data Catalog
- How to generate DBT data lineage graphs in client's production environment?
- How to get metadata from Talend Data Management Platform?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular # Hahtags
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
No, there isn't.
Athena is managed Presto. Glue is a mix of a managed Hive Metastore and a Serverless Spark Cluster.
You can use Atlas on Elastic Map Reduce (EMR), there is a blog post about that:
Metadata classification, lineage, and discovery using Apache Atlas on Amazon EMR