data harvesting from Wall Street Journal articles With "rvest" package

20 Views Asked by At

I intend to collect " all news articles from Wall Street Journal which mentions Microsoft." This will help with my study on the popularity and media presence of Microsoft. Is there a possible R code to help with this? Thanks in advance.

I tried using the rest package but keep getting the error below

url <- "https://www.wsj.com/search/term.html?KEYWORDS=microsoft"
page <- read_html(url)
headlines <- page %>%
html_nodes(".headline") %>%
html_text()
print(headlines)


page <- read_html(url)
Error in open.connection(x, "rb") : HTTP error 401.
0

There are 0 best solutions below