how can we create alerts for data drift and data quality for some dataset by giving threshold using python. Which is better package to capture data quality and data drift? evidently, deepchecks or Y-data profiling? How can we convert RDD for large dataset into pandas dataframe? for this it is throwing the error requirejs error. What is the best way for converting large RDD into a Dataframe?
expecting code in python by giving threshold and raising alert for the issue.