Kivy isn't showing (Bengali) joining character properly?

Question

Kivy Python does not support Bengali joining characters,Is there any other way to solve this problem?

Can any one please describe what is the problem? What can i do to solve it?

Expected Output:

Program Output:

Answer 1

Appropriate text renderer is needed to handle bangla fonts (and other fonts with complex glyphs) properly. The text renderer pango has features like fallback, complex glyph processing among many other features. Kivy supports pango text renderer with limited features as of now. Currently this has been tested in MacOS and Linux. Below is the procedure to demonstrate installation of pango text renderer with Kivy and an example app showing correct rendering of bangla text. The procedure assumed that Kivy is already installed in Linux (Ubuntu 18.04).

1. Check with the following command if pango is already installed correctly:

   pkg-config --libs pangoft2

   if pango is installed correctly, then it would produce following output:

   -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lfontconfig -lfreetype

   Otherwise, if it produces no output, then it is not installed correctly. In this case proceed to next step.

2. Issue the following command and wait for it to finish the execution:

   sudo apt-get update

3. Next issue the following command to install pango:

   sudo apt install libfreetype6-dev libpango1.0-dev libpangoft2-1.0-0

   Again check with command in step 1 if installation is done properly.

4. After pango installation, Kivy needs to be re-compiled. For this first issue the following command:

   sudo apt-get install -y \
       python-pip \
       build-essential \
       git \
       python \
       python3-dev \
       ffmpeg \
       libsdl2-dev \
       libsdl2-image-dev \
       libsdl2-mixer-dev \
       libsdl2-ttf-dev \
       libportmidi-dev \
       libswscale-dev \
       libavformat-dev \
       libavcodec-dev \
       zlib1g-dev
   

5. Then issue the following three commands sequentially:

   sudo pip3 install cython
   export USE_PANGOFT2=1
   export KIVY_TEXT=pango
   

   You may receive Requirement already satisfied message for Cython installation, which is ok.

6. Now uninstall Kivy with following command:

   sudo python3 -m pip uninstall kivy

7. And install Kivy with following command:

   sudo pip3 install kivy

8. Once Kivy is re-installed successfully, you can run the following example:

import os
os.environ['KIVY_TEXT'] = 'pango'
from kivy.app import App
from kivy.lang import Builder


APP_KV = """
BoxLayout:
    Label:
        text: "সকালে"
        font_size: '48sp'
        font_name: 'font/kalpurush.ttf'
"""

class MainApp(App):
    def build(self):
        return Builder.load_string(APP_KV)

if __name__ == '__main__':
    MainApp().run()

The KIVY_TEXT environment setting may not be needed as already exported earlier. But this is to show how you can select the text renderer. You need to have the font file kalpurush.ttf (or any other bangla unicode font file) under sub directory named font.

The Kivy log when you run the program is as follows:

[INFO   ] [Logger      ] Record log in /home/kivy/.kivy/logs/kivy_20-08-31_9.txt
[INFO   ] [Kivy        ] v1.11.1
[INFO   ] [Kivy        ] Installed at "/usr/local/lib/python3.6/dist-packages/kivy/__init__.py"
[INFO   ] [Python      ] v3.6.8 (default, Oct  7 2019, 12:59:55) 
[GCC 8.3.0]
[INFO   ] [Python      ] Interpreter at "/usr/bin/python3"
[INFO   ] [Factory     ] 184 symbols loaded
[INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_pil, img_gif (img_ffpyplayer ignored)
[INFO   ] [Window      ] Provider: sdl2(['window_egl_rpi'] ignored)
[INFO   ] [GL          ] Using the "OpenGL" graphics system
[INFO   ] [GL          ] Backend used <sdl2>
[INFO   ] [GL          ] OpenGL version <b'3.1 Mesa 19.0.8'>
[INFO   ] [GL          ] OpenGL vendor <b'VMware, Inc.'>
[INFO   ] [GL          ] OpenGL renderer <b'llvmpipe (LLVM 8.0, 256 bits)'>
[INFO   ] [GL          ] OpenGL parsed version: 3, 1
[INFO   ] [GL          ] Shading version <b'1.40'>
[INFO   ] [GL          ] Texture max size <8192>
[INFO   ] [GL          ] Texture max units <32>
[INFO   ] [Window      ] auto add sdl2 input provider
[INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
[INFO   ] [Text        ] Provider: pango
[INFO   ] [Base        ] Start application main loop
[INFO   ] [GL          ] NPOT texture support is available
[INFO   ] [WindowSDL   ] exiting mainloop and closing.
[INFO   ] [Base        ] Leaving application in progress...

Please note that the Text provider is pango.

The output of the program:

So, the text is rendered correctly. You can test with any other compound bangla letters, it will render correctly.

For other platforms like Windows, the challenge is to correctly install pango. It may not be trivial to do so as there is no simple way I have found so far. May be pango needs to be compiled from source with it's dependencies.

Answer 2

In my previous answer I have outlined procedure to install and activate pango text rendered back-end in Linux. Here I will explain a working procedure to install and activate pango for Kivy in Windows environment. Following is the environment where the procedure is tested:

• Windows 10
• Python 3.6.8 (64-bit)
• Kivy 2.0.0rc3

Following are the steps to be followed:

1. Create a directory named src in the root of C: drive and go to that directory:

   C:\>mkdir src 
   C:\>cd src 
   

2. Run the following command to clone the vcpkg to the src directory:

   git clone https://github.com/microsoft/vcpkg
   

   You need to have Git installed to run the above command.

3. Then bootstrap vcpkg and move to vcpkg directory:

   C:\src>.\vcpkg\bootstrap-vcpkg.bat
   C:\src>cd vcpkg
   

4. Now run the following command to install pango (with it's dependencies):

   vcpkg install pango --triplet x64-windows
   

5. After successful installation of pango, download the source tar ball of Kivy 2.0.0rc3 release from https://pypi.org/project/Kivy/2.0.0rc3/#files and extract in any suitable folder.

6. Now go to the folder where Kivy 2.0.0rc3 source files are extracted and open the setup.py file in a text editor (preferably Notepad++). Search for the line containing if c_options['use_pangoft2'] in (None, True) and add the following block of code before that line and save the file:

   if c_options['use_pangoft2'] in (None, True) and platform == 'win32':
       pango_flags = {}
       pango_flags['include_dirs'] = (['C:/src/vcpkg/installed/x64-windows/include', 'C:/src/vcpkg/installed/x64-windows/include/harfbuzz'])
       pango_flags['library_dirs'] = (['C:/src/vcpkg/installed/x64-windows/lib'])
       pango_flags['libraries'] = (['pangoft2-1.0', 'pango-1.0', 'gobject-2.0', 'glib-2.0', 'harfbuzz', 'freetype', 'fontconfig'])
       c_options['use_pangoft2'] = True
       pango_depends = {'depends': ['lib/pango/pangoft2.pxi', 'lib/pango/pangoft2.h']}
       sources['core/text/_text_pango.pyx'] = merge(base_flags, pango_flags, pango_depends)
       import subprocess
       subprocess.call(["setx", "FONTCONFIG_FILE", "C:\\src\\vcpkg\\installed\\x64-windows\\tools\\fontconfig\\fonts.conf"]) 
   

   This manual configuration is required as vcpkg currently doesn't produce .pc files for automatic population of compiler linking variables through pkg-conifg. Also as of now there is no pkg-config tool integration with vcpkg.

7. Add C:/src/vcpkg/installed/x64-windows/lib and C:/src/vcpkg/installed/x64-windows/bin to the PATH system environment variable.

8. Now Kivy needs to be compiled from it's source. If Kivy is already installed please uninstall it first. And then the procedure mentioned in the official kivy site can be followed - though I am listing here the required steps to be followed. Upgrade pip and wheel with:

   python -m pip install --upgrade pip wheel setuptools
   

9. Get Visual C++ Build Tools and install the following components from the "visual c++ build tools" section: "Windows 10 SDK (version)", "C++ CMake tools for Windows", "C++/CLI support", "MSVC v142 - VS 2019 C++ x64/x86 build tools".

10. Set the required environment variables:

    set USE_SDL2=1
    set USE_PANGOFT2=1
    set USE_GSTREAMER=1
    

11. Install the latest versions of kivy dependencies:

    python -m pip install Cython==0.29.10 docutils pygments pypiwin32 kivy_deps.sdl2 kivy_deps.glew kivy_deps.gstreamer kivy_deps.glew_dev kivy_deps.sdl2_dev kivy_deps.gstreamer_dev
    

12. Now go to the root folder where the Kivy 2.0.0rc3 source files are extracted (where the ```setup.py`` file is located) and issue the following command to compile and install Kivy:

    python -m pip install .
    

13. After successful installation of Kivy, you can run the following example:

    import os
    os.environ['KIVY_TEXT'] = 'pango'
    from kivy.app import App
    from kivy.lang import Builder
    
    
    APP_KV = """
    BoxLayout:
        Label:
            text: "অল্প বিস্তর"
            font_size: '48sp'
            font_name: 'font/kalpurush.ttf'
    """
    
    class MainApp(App):
        def build(self):
            return Builder.load_string(APP_KV)
    
    if __name__ == '__main__':
        MainApp().run()
    

    You need to have the font file kalpurush.ttf (or any other bangla unicode font file) under sub directory named font.

    The Kivy log when you run the program is as follows:

    [INFO   ] [Logger      ] Record log in C:\Users\user\.kivy\logs\kivy_20-09-06_12.txt
    [INFO   ] [deps        ] Successfully imported "kivy_deps.gstreamer_dev" 0.2.0
    [INFO   ] [deps        ] Successfully imported "kivy_deps.gstreamer" 0.2.0
    [INFO   ] [deps        ] Successfully imported "kivy_deps.glew" 0.2.0
    [INFO   ] [deps        ] Successfully imported "kivy_deps.glew_dev" 0.2.0
    [INFO   ] [deps        ] Successfully imported "kivy_deps.sdl2" 0.2.0
    [INFO   ] [deps        ] Successfully imported "kivy_deps.sdl2_dev" 0.2.0
    [INFO   ] [Kivy        ] v2.0.0rc3, git-Unknown, 20200906
    [INFO   ] [Kivy        ] Installed at "C:\Python\Python36\lib\site-packages\kivy\__init__.py"
    [INFO   ] [Python      ] v3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)]
    [INFO   ] [Python      ] Interpreter at "C:\Python\Python36\python.exe"
    [INFO   ] [Factory     ] 185 symbols loaded
    [INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2 (img_pil, img_ffpyplayer, img_gif ignored)
    [INFO   ] [Window      ] Provider: sdl2
    [INFO   ] [GL          ] Using the "OpenGL" graphics system
    [INFO   ] [GL          ] GLEW initialization succeeded
    [INFO   ] [GL          ] Backend used <glew>
    [INFO   ] [GL          ] OpenGL version <b'4.6.0 - Build 26.20.100.7812'>
    [INFO   ] [GL          ] OpenGL vendor <b'Intel'>
    [INFO   ] [GL          ] OpenGL renderer <b'Intel(R) HD Graphics 520'>
    [INFO   ] [GL          ] OpenGL parsed version: 4, 6
    [INFO   ] [GL          ] Shading version <b'4.60 - Build 26.20.100.7812'>
    [INFO   ] [GL          ] Texture max size <16384>
    [INFO   ] [GL          ] Texture max units <32>
    [INFO   ] [Window      ] auto add sdl2 input provider
    [INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
    [INFO   ] [Text        ] Provider: pango
    [INFO   ] [Base        ] Start application main loop
    [INFO   ] [GL          ] NPOT texture support is available
    

    Please note that the Text provider is pango.

    The output of the program:

    The text is rendered correctly. With pango other non-english language fonts having requirements of font fallback, complex glyph processing would render correctly.

    There are other procedures for installing pango, like with gvsbuild scripts as mentioned in the official site of GTK. It has the pkg-config integration as well. But it doesn't integrate successfully with Kivy. The installation through vcpkg, as outlined in this procedure, is the only successful procedure I have found.