Twitter API access using R's http

49 Views Asked by At

I am tearing my hair out over this issue! I am pretty confident I am using the correct bearer token (I have reset this to check), I also have paid API access for twitter. I also don't believe I am over any rate limits, I am just confused.

The code started to produce blank tables of data so I set up a little helper function that tells me which bits fail.

It's basically producing the below error

Hopefully, someone can help :)

Failed to fetch tweets for nespresso
Failed to fetch tweets for keurig
Failed to fetch tweets for tassimo
Failed to fetch tweets for sage
Failed to fetch tweets for breville
Failed to fetch tweets for sense

Here is the code~

# Load necessary libraries
library(httr)
library(tidyverse)

bearer_token <- Sys.getenv("TWITTER_BEARER_TOKEN")

# Twitter API v2 endpoint for searching tweets
endpoint_url <- "https://api.twitter.com/2/tweets/search/recent"

# List of competitors
competitors <- c("nespresso", "keurig", "tassimo", "sage", "breville", "senseo")

# Initialize a list to hold data for all competitors
all_tweets <- list()

# Function to compile tweets data
compile_tweets_data <- function(content_list) {
    map_df(content_list$data, ~tibble(
        Text = .x$text,
        AuthorID = .x$author_id,
        Likes = .x$public_metrics$like_count,
        Retweets = .x$public_metrics$retweet_count,
        Replies = .x$public_metrics$reply_count,
        Username = content_list$includes$users[[which(sapply(content_list$includes$users, function(u) u$id == .x$author_id))]]$username,
        Verified = content_list$includes$users[[which(sapply(content_list$includes$users, function(u) u$id == .x$author_id))]]$verified
    ))
}

# Functions to process API responses
process_response <- function(response, competitor) {
    if (http_status(response)$category == "success") {
        process_success(response, competitor)
    } else {
        message("Failed to fetch tweets for ", competitor)
        all_tweets[[competitor]] <- tibble()
    }
}

process_success <- function(response, competitor) {
    content_list <- content(response, as = "parsed", type = "application/json")
    
    if (!is.null(content_list$data)) {
        tweets_data <- compile_tweets_data(content_list)
        all_tweets[[competitor]] <- tweets_data
    } else {
        all_tweets[[competitor]] <- tibble()
    }
}

for (competitor in competitors) {
    # Construct the query parameters with additional fields
    params <- list(
        'query' = paste(competitor, "lang:en"),
        'max_results' = '10',
        'tweet.fields' = 'text,author_id,public_metrics',
        'expansions' = 'author_id',
        'user.fields' = 'username,verified'
    )
    
    # Make the GET request to the Twitter API
    response <- GET(url = endpoint_url, add_headers(Authorization = paste("Bearer", bearer_token)), query = params)
    
    # Process the response
    process_response(response, competitor)
}

# Combine all tweets into a single tibble
all_tweets_combined <- bind_rows(all_tweets, .id = "Competitor")
0

There are 0 best solutions below