want to get the data from reddit profile pages. Some of these pages are tagged as nsfw, so users have to click accept in gui. (e.g. https://old.reddit.com/user/3DigitIQ) After clicking the page send a post request with "over18=yes" body and gets redirected to get request.
Now I want to rebuild this request in jsoup. the first request is allways getting redirected to reddit startpage. The second AsyncHttpClient request works fine and redirects correctly to https://old.reddit.com/user/3DigitIQ . I think its the same request. So one of these libaries is doing something in the background I dont understand. Someone knows whats the difference?
String url2 = " https://old.reddit.com/over18?dest=https://old.reddit.com/user/3DigitIQ";
String res = Jsoup.connect(url2)
.method(Connection.Method.POST)
.header("Content-Type","application/x-www-form-urlencoded;charset=UTF-8")
.requestBody("over18=yes")
.followRedirects(true)
.execute()
.body();
System.out.println(Jsoup.parse(res).select("title").text());
// REDIRECT TO REDDIT MAIN PAGE
AsyncHttpClient client = new DefaultAsyncHttpClient();
String newPageBody = client.prepare("POST", url2)
.setBody("over18=yes")
.setFollowRedirect(true)
.execute()
.get()
.getResponseBody();
System.out.println(Jsoup.parse(newPageBody).select("title").text());
// REDIRECT TO EXPECTED REDDIT USER PAGE