How do I get Spanish characters to display properly using a Velocity template?

Question

I'm using Velocity and a message resource bundle to generate html pages. When I specify Mexico as my locale, my messages-es_MX.properties gets processed as the source for the message resources. This is as I expect it to be. But the characters (áéíóúüñ¿¡) aren't displayed properly.

My message property:

customer.greeting=áéíóúüñ¿¡

For my first attempt, I've got the following:

• html header in the generated page: <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
• velocity properties contains:
  • input.encoding=utf-8
  • output.encoding=utf-8
• html file encoding: UTF-8
• messages_es_MX.properties encoding: ISO-8859-1

Output to html for ${customer.greeting}:

�������

I then realized that the encoding of the properties file isn't correct; it should also be UTF-8.

Second attempt:

• html header in the generated page: <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  • velocity properties contains:
    • input.encoding=utf-8
    • output.encoding=utf-8
• html file encoding: UTF-8
• messages_es_MX.properties encoding: UTF-8

Output to html:

áéíóúüñ¿¡

Any suggestions on how to get this to work?

Answer 1

Well, this turned out to be trickier than I thought it would be, but here's the solution. Before beginning, ensure that all html and properties files are encoded in UTF-8. And that all Velocity configurations reference UTF-8 as well. There should be no references to ISO-8859-1.

The underlying problem is that by default, ResourceBundle assumes property files are in ISO-8859-1 encoding. It's possible to override this, but takes a piece of custom code.

Instead of calling

ResourceBundle bundle = ResourceBundle.getBundle("messages", MEXICO);

I needed to add an argument for a custom Control implementation:

ResourceBundle bundle = ResourceBundle.getBundle("messages", MEXICO, new UTF8Control());

The UTF8Control class looks like this:

package test;

import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.util.Locale;
import java.util.PropertyResourceBundle;
import java.util.ResourceBundle;
import java.util.ResourceBundle.Control;

public class UTF8Control extends Control {

    public ResourceBundle newBundle(String baseName, Locale locale,
            String format, ClassLoader loader, boolean reload)
            throws IllegalAccessException, InstantiationException, IOException {

        // The below is a copy of the default implementation.
        String bundleName = toBundleName(baseName, locale);
        String resourceName = toResourceName(bundleName, "properties");
        ResourceBundle bundle = null;
        InputStream stream = null;
        if (reload) {
            URL url = loader.getResource(resourceName);
            if (url != null) {
                URLConnection connection = url.openConnection();
                if (connection != null) {
                    connection.setUseCaches(false);
                    stream = connection.getInputStream();
                }
            }
        } else {
            stream = loader.getResourceAsStream(resourceName);
        }
        if (stream != null) {
            try {
                // Only this line is changed to make it to read properties files
                // as UTF-8.
                bundle = new PropertyResourceBundle(new InputStreamReader(
                        stream, "UTF-8"));
            } finally {
                stream.close();
            }
        }
        return bundle;
    }

}

Credit to @BalusC who originally posted this related answer.

So, part 1 solved. But there's still a problem, because I'm using VelocityTools and I can see in the source code for ResourceTool that it calls ResourceBundle.getBundle(...) without passing in any Control implementation. This means that it's going to use ISO-8859-1, and I have no way to pass in my UTF8Control...

Unless I override the ResourceTool class's method to get the bundle:

package test;

import java.util.Locale;
import java.util.ResourceBundle;

import org.apache.velocity.tools.ConversionUtils;
import org.apache.velocity.tools.generic.ResourceTool;

public class UTF8ResourceTool extends ResourceTool {

    /**
     * Retrieves the {@link ResourceBundle} for the specified baseName and
     * locale, if such exists, using UTF-8 encoding. If the baseName or locale is null or if the
     * locale argument cannot be converted to a {@link Locale}, then this will
     * return null.
     */
    protected ResourceBundle getBundle(String baseName, Object loc) {
        Locale locale = (loc == null) ? getLocale() : toLocale(loc);
        if (baseName == null || locale == null) {
            return null;
        }
        return ResourceBundle.getBundle(baseName, locale, new UTF8Control());
    }

    /* Copied here from parent class because it's private there */
    private Locale toLocale(Object obj) {
        if (obj == null) {
            return null;
        }
        if (obj instanceof Locale) {
            return (Locale) obj;
        }
        String s = String.valueOf(obj);
        return ConversionUtils.toLocale(s);
    }

}

In my UTF8ResourceTool class, I override just one method getBundle (and I had to copy a private method in verbatim from the parent class). The overridden method passes the UTFControl as the third argument when getting the resource bundle, so that we can pull in the resources using UTF-8 encoding.

The only thing left now is to update my Velocity configuration so that I'm referencing my custom UTF8ResourceTool instead of Velocity Tools' ResourceTool:

        EasyFactoryConfiguration config = new EasyFactoryConfiguration();
    config.toolbox(Scope.APPLICATION).tool("msg", UTF8ResourceTool.class)
            .property("bundles", "messages/messages").property("locale", locale)
            .tool("date", DateTool.class).tool("number", NumberTool.class)
            .tool("string", StringTool.class).tool("esc", EscapeTool.class)
            .tool("base64", Base64Tool.class);

Put it all together, and my html output for ${customer.greeting} is:

áéíóúüñ¿¡