Is there a best practice for fetching link previews?

Question

Essentially given any url, I could fetch the webpage in Ruby using

doc = Nokogiri::HTML(open(my_url))
title = doc.at('meta[property="og:title"]')['content']
...

and extract the elements I need

Is there a best practice before fetching any links? It seems like a potential security risk as well.

I'm assuminig large compaines like facebook might run an image through some model to determine if it should be censored?

Answer 1

Essentially given any url, I could fetch the webpage in Ruby using

I am using metainspector to get OG data from various media URLs. It works very well and it might save you some headaches.

Is there a best practice before fetching any links? It seems like a potential security risk as well.

It depends on your application, what info you scrape and what you show to the user. If you are concerned about obscene words, you can filter them out (there might be some gems), but usually in the OG meta I didn't see any of them. You could blacklist adult website domains, or allow just some domains..

I'm assuminig large compaines like facebook might run an image through some model to determine if it should be censored?

Image recognition is a way to do it but it requires a lot of work. A lot.