Why is the same http post request failing with jsoup but working with AsyncHttpClient?

26 Views Asked by At

want to get the data from reddit profile pages. Some of these pages are tagged as nsfw, so users have to click accept in gui. (e.g. https://old.reddit.com/user/3DigitIQ) After clicking the page send a post request with "over18=yes" body and gets redirected to get request.

Now I want to rebuild this request in jsoup. the first request is allways getting redirected to reddit startpage. The second AsyncHttpClient request works fine and redirects correctly to https://old.reddit.com/user/3DigitIQ . I think its the same request. So one of these libaries is doing something in the background I dont understand. Someone knows whats the difference?

        String url2 = " https://old.reddit.com/over18?dest=https://old.reddit.com/user/3DigitIQ";
        String res = Jsoup.connect(url2)
            .method(Connection.Method.POST)
            .header("Content-Type","application/x-www-form-urlencoded;charset=UTF-8")
            .requestBody("over18=yes")
            .followRedirects(true)
            .execute()
            .body();

    System.out.println(Jsoup.parse(res).select("title").text());
    // REDIRECT TO REDDIT MAIN PAGE

    AsyncHttpClient client = new DefaultAsyncHttpClient();
    String newPageBody = client.prepare("POST", url2)
            .setBody("over18=yes")
            .setFollowRedirect(true)
            .execute()
            .get()
            .getResponseBody();

    System.out.println(Jsoup.parse(newPageBody).select("title").text());
    // REDIRECT TO EXPECTED REDDIT USER PAGE
0

There are 0 best solutions below