Fastest way to connect Data Lake Gen2 to PowerBi?

Question

What is the fastest way to get data from an Azure Data Lake Gen2 to PowerBI? Data is Updated every 5 minutes and it must ingested trough DirectQuery

Currently:

• I have large tables that get updated every 5 minutes
• I have Data in Delta tables in a Azure Data Lake Gen2
• I have External Tables in a Azure Synapse Serveless Pool
• I am ingesting the data trough Direct Query from Azure Synapse

The visualizations are taking too long to appear and I cannot use a dedicated pool in synapse

Answer 1

I am not sure I understood your question correctly. You mentioned you are using Direct Query to access data from delta lake on ADLS Gen2 using Serverless SQL Pool. Direct query pulls data from Delta lake using Serverless SQL when you are trying to access the report. If you are referring to a solution that updates the dataset that you have created on Power BI, then you need to schedule an auto refresh and set the frequency as per your requirement. Refer to Power Query, I think this fits your needs, but does not involve Direct Query, rather you pull data from Delta lake and host it on the Power BI workspace.

Answer 2

Suggestion:

use Microsoft Fabric

1. go to Lakhouse section of fabric and create a new lakehouse
2. click "get data" and create shortcuts to your Delta tables
3. create a dataset from the shortcuts
4. create a Power BI report from the dataset in fabric, it will be created in Direct Lake mode

So no data is shifted or copied anywhere. In Power BI (and also in shortcuts and dataset) the data is always updated automatically without the need to refresh.

directlake-overview