Neural Networks
There are neural networks in all sorts and sizes. I attempted to create a framework to
facilitate the creation of a wide variety of concrete neural networks. The framework that
is presented here is capable of creating most of the neural networks as described by
McClelland and Rumelhart in their "Explorations in parallel distributed processing".
Even though the framework is generic, it is still capable of creating only a small subset
of the existing neural networks. For now I will leave it at that, with the intention
to expand its capability in the future. What was implemented:
What was not implemented:
It's obvious that it is a major undertaking, if at all possible, to design a framework that is capable of creating all possible neural networks. Here are some things I didn't implement from the McClelland and Rumelhart workbook:
Remarks
Sample use
This code implements a simple XOR model, a three layer model that is trained by backpropagation. #include <conio.h> #include "aiNNModel.h" #include "genericArray.h" /// The XOR model, which is a 3 layer network, trained by backpropagation /// to act as an xor logical gate. /// /// from "Explorations in Parallel Distributed Processing" (chapter 5) /// by McClelland & Rumelhart #define EPOCHCOUNT 300 #define PATTERNCOUNT 4 // error criterion (4 times 0.1 squared) #define ECRIT 0.04f void Xor(void) { CNNModel XorModel; CArray<float> SourcePatterns[PATTERNCOUNT]; CArray<float> TargetPatterns[PATTERNCOUNT]; float TotalSumOfSquares = 1.0f; int EpochIndex; SourcePatterns[0].Add(0); SourcePatterns[0].Add(0); TargetPatterns[0].Add(0); SourcePatterns[1].Add(0); SourcePatterns[1].Add(1); TargetPatterns[1].Add(1); SourcePatterns[2].Add(1); SourcePatterns[2].Add(0); TargetPatterns[2].Add(1); SourcePatterns[3].Add(1); SourcePatterns[3].Add(1); TargetPatterns[3].Add(0); // create pools CNNPool *Inputs = XorModel.AddPool("Input", 2); CNNPool *Hiddens = XorModel.AddPool("Hidden", 2); CNNPool *Outputs = XorModel.AddPool("Output", 1); Inputs>SetPoolType(POOLTYPE_INPUT); Hiddens>SetPoolType(POOLTYPE_HIDDEN); Outputs>SetPoolType(POOLTYPE_OUTPUT); // create connections between pools, interconnect all units CNNPoolConnection *I2HConnection = XorModel.AddPoolConnection(Inputs, Hiddens); CNNPoolConnection *H2OConnection = XorModel.AddPoolConnection(Hiddens, Outputs); // pool parameters Hiddens>SetUpdateMethod(UPDATEMETHOD_CONTINUOUS_SIGMOID); Outputs>SetUpdateMethod(UPDATEMETHOD_CONTINUOUS_SIGMOID); // randomize weights and biases (to break symmetry) I2HConnection>RandomizeWeights(); H2OConnection>RandomizeWeights(); Hiddens>RandomizeBiases(); Outputs>RandomizeBiases(); // train the model XorModel.SetTrainingMethod(TRAININGMETHOD_GENERALIZED_DELTA); XorModel.SetLearningRate(0.5f); for (EpochIndex=0; EpochIndex<EPOCHCOUNT; EpochIndex++) { TotalSumOfSquares = 0.0f; for (int PatternIndex=0; PatternIndex<PATTERNCOUNT; PatternIndex++) { Inputs>SetOutputPattern(SourcePatterns[PatternIndex]); Outputs>SetTargetPattern(TargetPatterns[PatternIndex]); XorModel.TrainPattern(); TotalSumOfSquares += Outputs>GetPatternSumOfSquares(); } // total error below error criterion? if (TotalSumOfSquares < ECRIT) break; } printf("Epochs used (needed): %d\n\n", EpochIndex+1); // check the results XorModel.SetUpdateOrder(UPDATEORDER_HIDDEN_OUTPUT); for (int PatternIndex=0; PatternIndex<PATTERNCOUNT; PatternIndex++) { // set input pattern Inputs>SetOutputPattern(SourcePatterns[PatternIndex]); // update all units XorModel.Update(); // print results printf("%d xor %d = %f\n", (int)SourcePatterns[PatternIndex].Get(0), (int)SourcePatterns[PatternIndex].Get(1), (Outputs>GetUnit(0)>GetOutput()) ); } } Links
