How to forward a text with special characters in REST responses, in Delphi?

Question

Please help. I'm trying to learn REST concepts and make the first programs using Delphi and REST objects. I came across a problem that I don’t know how to solve. In the database I have a test with special characters from my mother tongue (Bosnian): č, ć, đ, š, ž. When I pass this text via the GET method, objects that parse and display this text in objects on the form display these characters as "?", "æ", "æè" etc. I researched and tried unsuccessfully to solve the problem. I tried to use the Utf8ToAnsi function and put iso-8859-2 instead of UTF-8 in the RESTClient, RESTRequest and RESTResponce object parameters. Please for help, instruction or advice on how to solve or investigate this problem.

Source code on server side:

procedure TWebModule1.UsersGet(Request: TWebRequest; Response: TWebResponse);
var
  a: TJSONArray;
  o: TJSONObject;
  i: Integer;
  Q1: TADOQuery;
begin
  Q1:=TADOQuery.Create(nil);
  Q1.Connection:= BasicDBConn.Konekcija;
  with Q1 do
  begin
    Active:=False;
    SQL.Clear;
    SQL.Add('Select USER_ID, USER_NAME, ROLE_NAME  from USERS, ROLES  ' +
      ' where USERS.ROLE_ID=ROLES.ROLE_ID ');
    Active:= True;
  end;
  a := TJSONArray.Create;

  if Q1.RecordCount > 0 then
  begin
    for i:=1 to Q1.RecordCount do
    begin
      o := TJSONObject.Create;
      o.AddPair('USER_ID', Q1.Fields.Fields[0].Value);
      o.AddPair('USER_NAME', Q1.Fields.Fields[1].Value);
      o.AddPair('ROLE_NAME', Q1.Fields.Fields[2].Value);
      Q1.Next;
      a.AddElement(o);
    end;
  end;

  Response.ContentType := 'application/json';
  Response.Content := a.ToString;
  a.DisposeOf;
  Q1.Free;
end;

on client side:

procedure TGlavna.Button1Click(Sender: TObject);
begin
  RESTRequest.Resource := 'Users';
  RESTRequest.Method   := TRESTRequestMethod.rmGet;
  RESTRequest.Response := RESTResponse;

  RESTRequest.Execute;

  if assigned(fJSONArray) then fJSONArray.DisposeOf;
  fJSONArray := TJSONObject.ParseJSONValue(RESTResponse.Content) as TJSONArray;

  if RESTResponse.Content.IsEmpty then Application.MessageBox('Empty', 'Information', MB_OK)
    else Memo1.Lines.Add(RESTResponse.Content);
end;

This is what a data in DB looks like

USER_ID USER_NAME ROLE_ID u1 Test User 2
u2 T_č_ć_š_đ_ž 1
u3 t_Č_Ć_Ž_Đ_Š 1
u4 Bradić Kenan 1

In web browser, responce data is:

[{"USER_ID":"u1","USER_NAME":"Test User","ROLE_ID":"2"},{"USER_ID":"u3","USER_NAME":"t_È_Æ_Ž_Ð_Š","ROLE_ID":"1"},{"USER_ID":"u4","USER_NAME":"Bradiæ Kenan","ROLE_ID":"1"},{"USER_ID":"u2","USER_NAME":"T_è_æ_š_ð_ž","ROLE_ID":"1"}]

Headers in web browser:
General:
    Request URL: http://localhost:8080/Users
    Request Method: GET
    Status Code: 200 OK
    Remote Address: [::1]:8080
    Referrer Policy: strict-origin-when-cross-origin
Response Headers:
    Connection: close
    Content-Length: 228
    Content-Type: application/json; charset=ISO-8859-1
    Date: Mon, 27 Dec 2021 17:07:54 GMT
Request Headers:
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
    Accept-Encoding: gzip, deflate, br
    Accept-Language: en-US,en;q=0.9
    Cache-Control: max-age=0
    Connection: keep-alive
    Host: localhost:8080
    sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
    sec-ch-ua-mobile: ?0
    sec-ch-ua-platform: "Windows"
    Sec-Fetch-Dest: document
    Sec-Fetch-Mode: navigate
    Sec-Fetch-Site: none
    Sec-Fetch-User: ?1
    Upgrade-Insecure-Requests: 1
    User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36

Thank you...

Answer 1

The HTTP response contains this line:

Content-Type: application/json; charset=ISO-8859-1

So the client handles interprets the Response content as ISO-85591 encoded instead of UTF-8. Your server side codes contains this line:

Response.ContentType := 'application/json';

Your server side code should clear the charset=ISO-85591 part.

Answer 2

You can encode your value with base64, and create a JSON payload to translate the JSON payload with REST API.

Moreover, if the value is sensitive, you can encrypt it with AES, and then translate it with REST API.

What technology will you use in server side? PHP? or .NET or JAVA? All 3 technologies used AES in native code, but there are little differences between the technologies.

If you used PHP or PHP + Laravel, you might reference my sample project in my GitHub Repo, the description you can read from my article.