How compile training neural network as stand-alone app in MATLAB?

Question

I want to compile my MATLAB application that uses neural networks to a stand-alone application, but as you know MATLAB can't compile training neural network as stand-alone and can only compile already trained neural networks.

The core of my application consists of training a neural network on an imported data. How can I do that? Is there an alternative way to do this? My MATLAB version is R2014a.

I tried using deploytool for compiling, but according to the MATLAB Compiler documentation:

THIS CAN BE COMPILED
  * Pre-trained network
  * command line functions

THIS CANNOT BE COMPILED
  * All other command line functionality
  * All GUIs provided with toolbox
  * Simulink blocks
  * gensim

So we get error after compiling the app if we have functions like newff or patternnet or other training functions in our code.

I know this is a limitation of the MATLAB Compiler and I searched for solutions for months but I didn't find any workarounds or alternative ways.

Apparently there is a function added to newer versions of MATLAB for using trained neural networks in MATLAB compiler: Deploy Neural Network Functions.

Answer 1

The bottom line is that MATLAB Compiler only supports deploying pre-trained neural networks.

Neural Network Toolbox

Can be compiled:

• Pre-trained network command line functions

Cannot be compiled:

• All other command line functionality
• Apps and UIs
• Simulink blocks
• gensim

This means you cannot mcc-compile functions with training functionality (anything containing TRAIN, ADAPT, etc...), you can only deploy functions that evaluate/simulate an already trained network object (SIM function and the like).

---

For the supported scenario (deploying a pre-trained network), there are a couple of ways to go about it:

1) save/load pre-trained network object to a MAT-file

In a normal MATLAB session, load the training data you have, then create and train a neural network using desired settings (keep tuning the network parameters until you are satisfied with the result). Finally save the network object to disk (export as variable in a MAT-file).

% sample regression dataset
[x,y] = simplefit_dataset();

% feed-forward neural network (one hidden layer with 4 neurons)
net = fitnet(4);
net = configure(net, x, y);            % configure net to match data
net.trainParam.showWindow = false;     % dont show training GUI
net.trainParam.showCommandLine = true; % display output in command line
net.trainParam.show = 1;               % display output every iteration

% train networks (data is divided into train/validation/test sets)
net = init(net);           % initialize network weights
[net,tr] = train(net, x, y);

% save pre-trained network to MAT-file
save('pretrained_network.mat', 'net')

Next create a deployable function that loads the saved network, and uses it to predict the output given some test data (note the use of %#function pragma line):

simulateSavedNet.m

function y_hat = simulateSavedNet(x)
    % this is a special pragma for MATLAB Compiler
    % used to declare "network" class as dependency in deployed mode
    %#function network

    % load pre-trained network
    S = load('pretrained_network.mat', 'net');
    net = S.net;

    % predict outcome given input data
    %y_hat = net(x);
    y_hat = sim(net, x);
end

2) Generate a standalone M-function from pre-trained network

You can generate a standalone MATLAB function from a pre-trained network object using genFunction, which can then be used to simulate network output. This functionality was introduced in MATLAB R2013b.

It will basically hard-code network settings, structure, and weights all in one M-function. The generated function is fully compatible with MATLAB Compiler mcc (compiled into one of the supported targets) as well as MATLAB Coder codegen (converted to standalone C/C++ code).

% generate standalone M-function from the trained net
genFunction(net, 'simulateStandaloneNet.m', 'MatrixOnly','yes', 'ShowLinks','no')

Here is the code for the generated function:

simulateStandaloneNet.m

3) Manually simulate the pre-trained network

For simple static neural networks (feed-forward and the like), it is relatively easy to evaluate a pre-trained network and simulate its output (the hard part is training them!).

I've showed how to do this in previous answers. You basically extract the learned weights from the network, then plug those numbers into the transfer functions, feed it the input, and compute the propagated outputs (one layer at-a-time). You will have to take care of applying any pre-/post- processing on the data and use the same transfer functions in each layer.

In fact, this is basically what genFunction does in the previous approach, only it is automated and handles all cases (works for all kinds of neural networks, not just feed-forward ANNs).

Here is an example from the network trained above:

simulateManualNet.m

function y_hat = simulateManualNet(x)
    % pre-trained feed-forward neural network
    % contains one hidden layer with 4 neurons, 1D input, 1D output
    % We assume the default transfer functions, preprocessing, etc..

    % The following hardcoded values were obtained
    % from net.IW, net.LW, net.b properties using MAT2STR

    % hidden layer weights/biases
    b1 = [6.0358701949521; 2.72569392497815; 0.584267717191459; -5.1615078566383];
    W1 = [-14.0019194910639; 4.90641117353245; -15.2282807645331; -5.26420794868803];
    % output layer weights/biases
    b2 = -0.756207251486408;
    W2 = [0.548462643231606 -0.435802343861239 -0.085111261420613 -1.13679228253379];

    % scale input
    in = mapFcn(x);

    % hidden layer
    hid = hiddenLayerTransferFcn(bsxfun(@plus, W1*in, b1));

    % output layer
    out = outputLayerTransferFcn(W2*hid + b2);

    % inverse scale output
    y_hat = mapInverseFcn(out);
end

function xx = mapFcn(x)
    % linear mapping from [mn,mx] to [-1,1]
    mn = 0; mx = 9.97628374728129;
    xx = (x - mn)*2 / (mx - mn) - 1;
end
function x = mapInverseFcn(xx)
    % inverse linear mapping from [-1,1] to [mn,mx]
    mn = 0; mx = 10;
    x = (xx + 1) * (mx - mn)/2 + mn;
end
function out = hiddenLayerTransferFcn(in)
    % Hyperbolic tangent sigmoid transfer function
    out = tanh(in);
end
function out = outputLayerTransferFcn(in)
    % Linear transfer function
    out = in;
end

4) Generate Simulink block from pre-trained network, and convert it using Simulink Coder

The idea here is to generate a Simulink block from the pre-trained network using gensim, then convert the generated block into a standalone C/C++ application using Simulink Coder (formerly known as Real-Time Workshop). Compiling neural network to Simulink blocks was introduced in R2010b.

I'm not a Simulink expert, so I'll leave it to you to explore this approach:

gensim(net)

---

In each approach above (first three anyway), the idea is to compile the simulate functions into one of the supported targets by MATLAB Compiler (standalone executable, shared library, Java package, .NET assembly) then deploy the generated component.

(Actually approach #2 and #3 can also be converted into C/C++ source code using MATLAB Coder codegen).

Here is how to compile each into a shared library using mcc command (you could use deploytool if you like):

% 1) saved network
mcc -v -W cpplib:libANN -T link:lib -N -p nnet simulateSavedNet.m -a pretrained_network.mat

% 2) standalone simulation function (genFunction)
mcc -v -W cpplib:libANN -T link:lib -N simulateStandaloneNet

% 3) standalone simulation function (manual)
mcc -v -W cpplib:libANN -T link:lib -N simulateManualNet

To check the resulting DLLs, below is a C++ test program that links against the produced shared library:

% 1)
mbuild -output test_savedNet -DSIMFCN=simulateSavedNet -I. test_net.cpp libANN.lib

% 2)
mbuild -output test_standaloneNet -DSIMFCN=simulateStandaloneNet -I. test_net.cpp libANN.lib

% 3)
mbuild -output test_manualNet -DSIMFCN=simulateManualNet -I. test_net.cpp libANN.lib

The code for the test program:

test_net.cpp

#include <cstdlib>
#include <iostream>
#include "libANN.h"

// choose one!
//#define SIMFCN simulateSavedeNet
//#define SIMFCN simulateStandaloneNet
//#define SIMFCN simulateManualNet

int main()
{
    // initialize MCR and lib
    if (!mclInitializeApplication(NULL,0))  {
        std::cerr << "could not initialize the application" << std::endl;
        return EXIT_FAILURE;
    }
    if(!libANNInitialize()) {
        std::cerr << "Could not initialize the library" << std::endl;
        return EXIT_FAILURE;
    }

    try {
        // create input data (1x5 vector)
        double x[] = {1.0, 3.0, 5.0, 7.0, 9.0};
        mwArray in(1, 5, mxDOUBLE_CLASS, mxREAL);
        in.SetData(x, 5);

        // predict network output by simulating network
        mwArray out;
        SIMFCN(1, out, in);
        double y[5];
        out.GetData(y, 5);

        // show result
        std::cout << "y = net(x)" << std::endl;
        std::cout << "y = \n" << out << std::endl;

    } catch (const mwException& e) {
        std::cerr << e.what() << std::endl;
        return EXIT_FAILURE;
    } catch (...) {
        std::cerr << "Unexpected error thrown" << std::endl;
        return EXIT_FAILURE;
    } 

    // cleanup
    libANNTerminate();   
    mclTerminateApplication();

    return EXIT_SUCCESS;
}

Here is the output of the resulting programs, compared against the original network object and the source M-functions:

>> net([1 3 5 7 9])
ans =
    9.5620    7.7851    7.2716    6.1647    2.4073
>> simulateSavedNet([1 3 5 7 9])
ans =
    9.5620    7.7851    7.2716    6.1647    2.4073
>> simulateStandaloneNet([1 3 5 7 9])
ans =
    9.5620    7.7851    7.2716    6.1647    2.4073
>> simulateManualNet([1 3 5 7 9])
ans =
    9.5620    7.7851    7.2716    6.1647    2.4073

>> !test_savedNet.exe
y = net(x) 
y =  
9.5620    7.7851    7.2716    6.1647    2.4073 

>> !test_standaloneNet.exe
y = net(x) 
y =  
9.5620    7.7851    7.2716    6.1647    2.4073 

>> !test_manualNet.exe
y = net(x) 
y =  
9.5620    7.7851    7.2716    6.1647    2.4073 

This ended up being a long post, but I wanted to cover all possible cases for this question and future ones :) HTH

Answer 2

It is unfortunately indeed correct that you cannot create a standalone neural network program via the deploytool (so with matlab compiler).

Alternatives:

1. You can first train a network, and then build the standalone program, but it seems like you want to do the training after its creation.
2. You can look into matlab coder; this is basically 'the other way' to create programs from matlab. I have not been able to find whether it supports neural networks, but you could contact the mathworks if you are considering this option.
3. Consider not making a standalone program. Depending on your needs you could for example call matlab from a different program or from the command line to do its job.

Answer 3

It's not possible to deploy the network training functionality of Neural Network Toolbox using either the deployment products (MATLAB Compiler, MATLAB Builder products) or the code generation products (MATLAB Coder etc).

You might look into using a third-party toolbox for neural networks such as Netlab. Netlab doesn't include all the neural network functionality of Neural Network Toolbox, but it includes most typically-used functionality, and quite a few extra methods that are covered better by Statistics Toolbox, such as K-means clustering.

I'm not aware of any technical problems in deploying Netlab functionality, and I believe it's licensed under a BSD open-source license, so you should be able to include and redistribute it with your project with no problems.

---

Edit: As of R2016b, it is now possible to compile network training functionality from Neural Network Toolbox (or Deep Learning Toolbox as it is now known).