How to add UTF-8 support to SmartGWT application

Question

I am implemtnting a smartgwt application for a client that need support to utf-8 language(RTL). Now I have a listgrid which takes in persian text and when i try to save it to my mssql 2008 database, it saves it like this ????? instead of the perisan text itself. In my database table I have defined the col type as nvarchar which is the requirement for saving utf-8. but the problem is smartgwt is not sending the text in proper format. I have put this in my html file

<meta http-equiv="content-type" content="text/html; charset=UTF-8">

In the developer Console when I look into the RPC tab. I see the DSREquest is being sent in UTF-8 format but in the DSResponse and in my db I get this ??????. In my database table I have defined the col type as nvarchar which is the requirement for saving utf-8. but the problem is smartgwt is not sending the text in proper format into the db. can somebody help me to get this correct.

In the console I see everything correct

=== 2015-04-22 12:45:25,929 [9-27] INFO PoolManager - [builtinApplication.importExcelDS_add] SmartClient pooling started for 'SQLServer' objects

=== 2015-04-22 12:45:25,929 [9-27] DEBUG PoolableSQLConnectionFactory - [builtinApplication.importExcelDS_add] Initializing SQL config for 'SQLServer' from system config - using DriverManager: net.sourceforge.jtds.jdbc.Driver

=== 2015-04-22 12:45:25,929 [9-27] DEBUG PoolableSQLConnectionFactory - [builtinApplication.importExcelDS_add] net.sourceforge.jtds.jdbc.Driver lookup successful

=== 2015-04-22 12:45:25,929 [9-27] DEBUG PoolableSQLConnectionFactory - [builtinApplication.importExcelDS_add] DriverManager fetching connection for SQLServer via jdbc url jdbc:jtds:sqlserver://localhost:1433;DatabaseName=dbilling;useUnicode=true;characterEncoding=UTF-8;sendStringAsUnicode=true;User=teoit;Password=seop

=== 2015-04-22 12:45:25,929 [9-27] DEBUG PoolableSQLConnectionFactory - [builtinApplication.importExcelDS_add] Passing JDBC URL only to getConnection

=== 2015-04-22 12:45:25,999 [9-27] INFO SQLDriver - [builtinApplication.importExcelDS_add] Executing SQL update on 'SQLServer': INSERT INTO sales (available, productMth, productName, productYr) VALUES (NULL, NULL, 'به آسا اماکن ۸ ٪ - ۴ لیتر - محلول ضد عفونی کننده', NULL)

The only thing that is missing is the N which should have been there before the utf-8 string. like so

INSERT INTO sales (available, productMth, productName, productYr) VALUES               
(NULL, NULL, N'به آسا اماکن ۸ ٪ - ۴ لیتر - محلول ضد عفونی کننده', NULL)
can somebody help me to get this correct.

Answer 1

When I had problem with Russian characters I added this code in constructor of my data source:

DSRequest dsRequest = new DSRequest();
dsRequest.setContentType("application/json; charset=utf-8");
setRequestProperties(dsRequest);

I'm using REST DataSource with JSON format.

Answer 2

OK, got it to work and for peopel who want to get the answer to this problem. I got this working by assigning the type of the field as ntext. so here is datasource file and the field called productName where I have my UTF-8 text

<DataSource ID="importExcelDS" serverType="sql" tableName="sales">
<fields>
    <field name="importID" type="sequence" hidden="true" primaryKey="true" />
    <field name="productName" title="Product Name" type="ntext" />
    <field name="productMth" title="product Mth" type="int" />
    <field name="productYr" title="product Yr" type="int" />
    <field name="available" title="Quantity" type="int" />
</fields>