This is a difficult one: I want to know where the buttons / textviews etc are located in the current focused activity which might not be mine. The app shall only run on my device so I can guarantee root / su and system-signatures + Android 4.2 and up (API 17).
I thought of getting a Screenshot via screencap or fb2png, passing it to an edge detection algorithm and look for squares, this would cover some views, better then nothing but this is not really satisfying me.
An other way would be to unpack the app, look for the layout files and parse them in what way ever (but if it has multiple layout files it will be very hard / impossible). Thus Android must now which views are in foreground to make them touchable.
Where is this info stored? How can I access it? Or maybe other ideas?