Encog 3.3 library for c#: I get a 0.79 error on my network but does not improve

Question

Im new at programming, and I am trying to learn Encog 3.3 Library. I worked on making my first network. I was able to write and understand the Code; However, My error rate does not go below 0.79, I used TANH activation function. My network is suppose to return 1 of three values -1,0,1 based on a set of variables I input it. Has anyone Have this same Problem?

this is the Code:

        static void Main(string[] args)
    { 

        // creating the neural net : network
        var network = new BasicNetwork();
        network.AddLayer(new BasicLayer(null, true,21));
        network.AddLayer(new BasicLayer( new ActivationTANH(), true,15));
        network.AddLayer(new BasicLayer(new ActivationTANH(), true, 15));
        network.AddLayer(new BasicLayer(new ActivationTANH(), true,1));
        network.Structure.FinalizeStructure();
        network.Reset();


     // creating the training Data
        string Path = "";
        var listArray = GetFile(Path); 
        int amountNumbersY = GetYSize(listArray);
        int amountNumbers = GetXSize(listArray[1]);
        string[,] matrixString = new string[listArray.Length, amountNumbers];                    matrixString = splitter(listArray, amountNumbers); 
        double[][] allData = new double[amountNumbers][]; 
        for (int i = 0; i < allData.Length; i++) 
            allData[i] = new double[amountNumbersY]; 

        allData = ConvertToDouble(matrixString, amountNumbers);
        // creating the inpuit and output
        double[][] XOR_INPUT = new double[amountNumbersY][];  
        for (int i = 0; i < amountNumbersY; i++)
        {
            XOR_INPUT[i] = new double[amountNumbers - 1];
        }
        double[][] XOR_IDEAL = new double[amountNumbersY][];
        for (int i = 0; i < amountNumbersY; i++)
        {
            XOR_IDEAL[i] = new double[1];
        }


        XOR_INPUT = GetInput(allData, amountNumbers, amountNumbersY, 1);
        XOR_IDEAL = GetIdealOutPut(allData, amountNumbers, amountNumbersY, 1);

         // normalizing the Arrays
        double[][] temp_Input = new double[amountNumbersY-1][];
         for (int i = 0; i < amountNumbersY-1; i++) // initializing the x axis
        {
            temp_Input[i] = new double[amountNumbers - 1];
        }
         double[][] temp_Ideal = new double[amountNumbersY-1][]; // same as above for output matrix
        for (int i = 0; i < amountNumbersY-1; i++)
        {
            temp_Ideal[i] = new double[1];
        }
        double[][] closedLoop_temp_Input = new double[amountNumbersY-1][];
        for (int i = 0; i < amountNumbersY-1; i++) // initializing the x axis
        {
            closedLoop_temp_Input[i] = new double[amountNumbers - 1];
        }
        double[][] closedLoop_temp_Ideal = new double[amountNumbersY-1][]; 
        for (int i = 0; i < amountNumbersY-1; i++)
        {
            closedLoop_temp_Ideal[i] = new double[1];
        }
        var hi = 1;
        var lo = -1;
        var norm = new NormalizeArray { NormalizedHigh = hi, NormalizedLow = lo };
        for (int i = 0; i < amountNumbersY-1; i++)
        { 
            temp_Input[i] = norm.Process( XOR_INPUT[i]);

        }
        closedLoop_temp_Input = EngineArray.ArrayCopy(temp_Input);
        var Ideal_Stats = new NormalizedField(NormalizationAction.Normalize,"Temp_Ideal",1,-1,-1,1);
        for (int i = 0; i < amountNumbersY - 1; i++)
        {
            temp_Ideal[i][0] = Ideal_Stats.Normalize(XOR_IDEAL[i][0]);

        }
        closedLoop_temp_Ideal = EngineArray.ArrayCopy(temp_Ideal);
        IMLDataSet trainingSet = new BasicMLDataSet(closedLoop_temp_Input, closedLoop_temp_Ideal);


         // training the network
        IMLTrain train = new ResilientPropagation( network, trainingSet);
        ICalculateScore score = new TrainingSetScore(trainingSet);
        IMLTrain annealing = new NeuralSimulatedAnnealing(network,score,10,2,10);
        int epoch = 1;
         do
         {
             if (epoch == 50)
             {
                 int i = 0;
                 do
                 {
                     annealing.Iteration();
                     Console.WriteLine("Annealing: " + i +", Error: " + annealing.Error);
                     i++;
                 } while (i < 5);
             }
             train.Iteration();
               Console.WriteLine(@" Epoch: "+epoch+ @", Error: "+train.Error+"...");
             epoch ++;
         } while ( train.Error<0.01 || epoch < 1000);
     // testing the network


    }
}

}

Answer 1

The training rate not falling is one of the most common problems in machine learning. Often it is because the data given to the model simply does not support a prediction. You are training a neural network to predict an output given the input. Consider if you were to train on the following inputs and expected outputs. The data might be noisy or it might be contradictory.

A few suggestions. First, dump your training data to a file and have a look at it. Is it what you expect? Are all of the values between -1 and 1. You have quite a bit of code happening before the training. There could be something going wrong there.

You also have a somewhat hybrid training approach going on, with RPROP and annealing. Perhaps just stick to RPROP and see what happens.