Is there a way to determined what element is at a point in a WKWebView or SFSafariViewController view?

Question

I would like to know what element is "under" a particular point in a web view. For example, in the screenshot below, I would like to know what element is at point 550x, 275y. I'm hoping to get the result:

<img alt=​"Typhoon Mangkhut approaching the Philippines on September 14" src=​"/​/​upload.wikimedia.org/​wikipedia/​commons/​thumb/​3/​30/​Mangkhut_2018-09-14_0750Z.jpg/​140px-Mangkhut_2018-09-14_0750Z.jpg" width=​"140" height=​"111" srcset=​"/​/​upload.wikimedia.org/​wikipedia/​commons/​thumb/​3/​30/​Mangkhut_2018-09-14_0750Z.jpg/​210px-Mangkhut_2018-09-14_0750Z.jpg 1.5x, /​/​upload.wikimedia.org/​wikipedia/​commons/​thumb/​3/​30/​Mangkhut_2018-09-14_0750Z.jpg/​280px-Mangkhut_2018-09-14_0750Z.jpg 2x" data-file-width=​"1219" data-file-height=​"963">​

Answer 1

If the webview supports it, you could have javascript determine what's there, and report it to the webview. I apologize but I haven't run this code, I don't have the means to test it at the moment.

If you don't have the ability to add javascript to the page then you'll have to do it all in WKWebView .evaluateJavaScript

find the element value with:

var elem = document.elementFromPoint(550, 275);

Then tell your webview about it by creating a custom scheme request. you can do it this way:

var iframe = document.createElement("IFRAME");
iframe.setAttribute("src", "customScheme://" + JSON.stringify(elem);
document.documentElement.appendChild(iframe);
iframe.parentNode.removeChild(iframe);
iframe = null;

Intercept it in the webview delegate.

public func webView(_ webView: WKWebView,
                    decidePolicyFor navigationAction: WKNavigationAction,
                    decisionHandler: @escaping (WKNavigationActionPolicy) -> Void) {
    let url = navigationAction.request.url
    if(url.range(of:"customScheme://") != nil){
         // this is your reported info
    }
    return true
}

This approach will only tell you the element as it relates to the web browser coordinates, not it's position on the device's screen. If you want to do it via device position you'll have to translate that and pass the coordinates over to the webview by invoking some javascript on the device.