NPM Request not get html from thai url. (Web Scraping)

150 Views Asked by At

This code will not show html in console it show error

var request = require('request');
var thai_url = "http://pantip.com/tag/Honda_(มอเตอร์ไซค์)";

request(thai_url, function (error, response, html) {
        if (!error && response.statusCode === 200) {
            console.log(html);
        } else {
            console.log("Error");
        }
});

But I change thai_url from มอเตอร์ไซค์ to %E0%B8%A1%E0%B8%AD%E0%B9%80%E0%B8%95%E0%B8%AD%E0%B8%A3%E0%B9%8C%E0%B9%84%E0%B8%8B%E0%B8%84%E0%B9%8C

Like This

var request = require('request');

var thai_url = "http://pantip.com/tag/Honda_(%E0%B8%A1%E0%B8%AD%E0%B9%80%E0%B8%95%E0%B8%AD%E0%B8%A3%E0%B9%8C%E0%B9%84%E0%B8%8B%E0%B8%84%E0%B9%8C)";

request(thai_url, function (error, response, html) {
        if (!error && response.statusCode === 200) {
            console.log(html);
        } else {
            console.log("Error");
        }
});

The second code working right. Can I change มอเตอไซค์ to %E0%B8%A1%E0%B8...(more)? Or If it's possible, How am I doing for http://pantip.com/tag/Honda_(มอเตอร์ไซค์) working with npm request ? Thank you for any help.

2

There are 2 best solutions below

0
On BEST ANSWER

Update. Now I can solve this problem by this

var thai_url = encodeURI("http://pantip.com/tag/Honda_(มอเตอร์ไซค์)");

Use encodeURI. http://www.javascripter.net/faq/escape.htm

2
On

First one is working fine. Just log the error instead of "error" string.

Make sure your IDE supports utf character. otherwise on saving it might be changing it. That why you might get error.