iconv(1) inconv(3) giving different conversions - Ubuntu 16.04 64 bit

Question

The following

u8"abc ß α € àḃç Málaga uña"

Is being converted on the command line by

echo abc ß α € àḃç Málaga uña| iconv --verbose -f UTF-8 -t ASCII//TRANSLIT

to

"abc ss ? EUR abc Malaga una"

However, in my 64 bit C++ programme (gcc 6.3.0), using iconv_open(u8"US-ASCII//TRANSLIT", u8"UTF-8"), it is being converted to

"abc ss ? EUR ??? M?laga u?a"

iconv(3) doesn't return -1.

Any help much appreciated.

Answer 1

After some experimentation (your code is easy enough to reproduce, but you should have posted it), the solution is to call, at the beginning of your main():

setlocale(LC_ALL, "");  //#include <locale.h> of course

Why is this needed? Well, arguably transliteration rules may depend on the language used (that will be your LANG environment variables), so you have to apply that by calling setlocale().

Knowing that, it is easy to reproduce your issue in the command line:

echo abc ß α € àḃç Málaga uña| LANG=C iconv --verbose -f UTF-8 -t ASCII//TRANSLIT

Output is:

abc ss ? EUR ??? M?laga u?a

Answer 2

Thanks for the replies. I stumbled upon the solution here

http://php.net/manual/en/function.iconv.php

As rodrigo said, I'd missed out

std::setlocale(LC_ALL, "en_GB.UTF-8");