How can you tell if a Twitter query is returning the correct data?

60 Views Asked by At

I am just starting to use the Twitter API (v2) in Java and it doesn't seem like I'm getting the correct response to a query on the recent search endpoint. In the code (modified from example code) below, I am using a query of

"#" has:hashtags lang:en

to look for English language tweets with a hashtag (using a backslash to escape Java processing so I'm really searching on "#", which is needed as there has to be one standalone operator) that have occurred in the last second, and returning tweet.fields entities so I can look for the hashtags.

private static String search(String bearerToken) throws IOException, URISyntaxException {
    String searchResponse = null;

    HttpClient httpClient = HttpClients.custom().setDefaultRequestConfig(RequestConfig.custom()
                    .setCookieSpec(CookieSpecs.STANDARD).build()).build();

    URIBuilder uriBuilder = new URIBuilder("https://api.twitter.com/2/tweets/search/recent");
    ArrayList<NameValuePair> queryParameters;
    queryParameters = new ArrayList<>();
    queryParameters.add(new BasicNameValuePair("query", "\"#\" has:hashtags lang:en"));
    DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss'Z'");
    LocalDateTime datetime = LocalDateTime.now();
    String currentTime = datetime.format(formatter);
    String currentTimeMinusOneSec = currentTime.substring(0,17)+String.valueOf(Integer.parseInt(currentTime.substring(17,19))-1)+currentTime.substring(19);
    queryParameters.add(new BasicNameValuePair("start_time", currentTimeMinusOneSec));
    queryParameters.add(new BasicNameValuePair("tweet.fields", "entities"));
    uriBuilder.addParameters(queryParameters);

    HttpGet httpGet = new HttpGet(uriBuilder.build());
    httpGet.setHeader("Authorization", String.format("Bearer %s", bearerToken));
    httpGet.setHeader("Content-Type", "application/json");

    HttpResponse response = httpClient.execute(httpGet);
    HttpEntity entity = response.getEntity();
    if (null != entity) {
        searchResponse = EntityUtils.toString(entity, "UTF-8");
    }
    return searchResponse;
}

In the response, I get 10 entities sections, but only seven of those have hashtags. I would have thought the three entities sections w/o hashtags would have been dropped by has:hashtags. In addition, I have one tweet in Mandarin characters, so clearly not language en (English). If anyone can see what I'm doing wrong or offer advice, that would be great!

0

There are 0 best solutions below