How can I install and use Pynini on Windows 10?

Question

OpenGrm Pynini is a open-source finite-state transducer python library developed by K.Gorman and available here for download: http://www.openfst.org/twiki/bin/view/GRM/Pynini

I would like to be able to develop on my Windows 10 laptop with PyCharm/IntelliJ using the latest Pynini version 2.0.8. The problem is that Pynini depends on C++ OpenFST 1.7.3 which apparently has some issues to build on Windows.

The solutions or workarounds I have found so far are:

• Use the 'Port of the OpenFST library to Windows' I have found on github.com:kkm000/openfst but not yet at version 1.7.3 and could be an issue for some people.
• Use Cygwin or mingw-w64 as suggested here on stackoverflow.com: Issues installing pynini

But is there any other simple way to make it work like using Debian for Windows Subsystem for Linux (WSL)?

Answer 1

So, I spent some time trying to achieve this and I have found that this setup works so well that I have decided to post it here in case it can help someone. The integration with PyCharm (pro) works so well that you don't need any linux box or shell or ssh tunnel. PyCharm can see your WSL instance, it will automatically start it, and call your python interpreter when you will run your script.

Here is all the steps I have executed to complete my setup:

Install Debian WSL

Install and Setup a Debian instance from Microsoft Store

Then based on Pynini readme, here is what we need:

• GCC > 4.8
• Built OpenFST 1.7.3 built with ./configure --enable-grm and headers
• A Python version: 2.7 or 3.6+ and headers

Install GCC

sudo apt update && sudo apt -y upgrade
sudo apt install build-essential       # to install GCC and others build libs and tools

Install OpenFST

We need to install wget to be able to download openfst and pynini.

sudo apt install wget                        
cd /usr/local/src 
sudo wget http://www.openfst.org/twiki/pub/FST/FstDownload/openfst-1.7.3.tar.gz
sudo tar -xvf openfst-1.7.3.tar.gz && sudo chown -R root:root openfst-1.7.3
cd openfst-1.7.3 && sudo ./configure --enable-grm
sudo make && sudo make install

Install Pynini

First we need to install Python

sudo apt install python3 python3-dev python3-pip python3-venv

Then download and build Pynini; sorry but I am addicted to virtual environments:

• python3 -m venv ~/venv373; . ~/venv373/bin/activate;
• cd /usr/local/src
• sudo wget http://www.opengrm.org/twiki/pub/GRM/PyniniDownload/pynini-2.0.8.tar.gz
• sudo tar -xvf pynini-2.0.8.tar.gz && sudo chown -R 1000:1000 pynini-2.0.8
• cd /usr/local/src/pynini-2.0.8
• sudo env PATH='$PATH'; python setup.py install;

And that's it, Pynini should be installed.

PyCharm integration

Please note that this integration with WSL is only available on PyCharm/IntelliJ Professional edition.

Here is the link on how to add you WSL python interpreter in PyCharm: https://www.jetbrains.com/help/pycharm/using-wsl-as-a-remote-interpreter.html

One screenshot from my IntelliJ where you can see the import pynini statement is recognized and auto-completion works as well.