Previous  Up  Next

Module Ann_config


module Ann_config: sig .. end
Parsing the program parameters from the command line. The program may be runned in several modes (Train, Test, Run, or Predict) with different options and arguments.

This module details the global variables used as program parameters, and the functions used to parse the command line and to set the program parameters to the chosen values.


Main types for anniml configuration



type mode =
| Train (*Train the network on a patterns file.*)
| Test (*Test the trained network on a patterns file.*)
| Run (*Run the trained network on a new input vector passed as argument.*)
| Predict (*Make predictions on a new input data file.*)
The program can be runned in several modes

type opti =
| Gradient_basic of float (*Basic steepest descent with step eta (see [Bishop96], section 7.5, p.263).*)
| Gradient_momentum of float * float (*Gradient descent with step eta and momentum mu (see [Bishop96], section 7.5.2, p.267).*)
| Bfgs (*Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton local optimization method (see documentation of the MATH/BFGS module). Section 7.10 of [Bishop96] (p.287) also provides an overview of quasi-Newton methods and BFGS, although it is not sufficient for an efficient implementation of BFGS.*)
Type of optimization method used to minimize the error.

type gradient_learning =
| Batch (*All patterns are used to compute the gradient.*)
| On_line (*Only one pattern is used at a time to compute the gradient, and make a descending step.*)
| Chunk of int (*Patterns are taken by blocks (chunks) of chosen size.*)
Type allowing to choose the number of patterns used to compute the gradient of the error, before making a step in the descent direction (in the weights space). This choice is specific to gradient descent methods.

Global variables of the anniml program


val raw_layers : int array ref
Number of units in each layer, without counting the bias units which are added when the fully connected topology is created.
val dir : string ref
Working directory.
val fpatterns : string ref
Patterns file. Each line of the patterns file must contain first the input vector x, and second the target vector t. Be cautious to use a network's topology that is consistent with your patterns file (dimension of x must be equal to the number of input units, and dimension of target vector t must be equal to the number of output units).

There is an option allowing to select the columns that you really want to use in your patterns file. A few examples of pattern files can be found in the examples/ directory.

val columns : int list option ref
Columns of the patterns file that will be used to build the input vectors of the neural network. This variable is updated with the -c option which takes a file as argument. The column file may contain a list of integer, or expressions like 2 6:35, which mean that column 2 and columns 6 to 35 will be used to build the input vectors.
val finputs : string ref
File containing input vectors only (whereas patterns files contain also target vectors). When the -predict option is selected by the user, the program will make predictions on the input vectors of this file. Option -p with a filename as argument allows to choose the file where the results are saved. If -p option is not used, results are saved by default in a new file with the same root as finputs but with a .pred extension.
val fpredict : string ref
File where the network's ouptuts are saved of the -predict option is used. fpredict is updated with the -p option when parsing the command line. If -p option is not used, fpredict will reference a filename with the same root as finputs but with a .pred extension.
val fwts : string ref
Weights file. The default filename is the patterns file basename with a '.wts' extension. You can choose your own weights file with the -w option.
val fnet : string ref
Network file, containing the network's topology: units per layer, connections.
val max_iter : int ref
Maximum number of iterations. See abstol for details on the stop criterion
val abstol : float ref
Absolute tolerance for the stop criterion. The training stops when the maximum number of iterations is reached or when two successive values e1 and e2 of the error are closer than the absolute tolerance abstol, or when the relative difference between the two values falls under the relative tolerance reltol.

The exact formulation of the two last conditions is: |e2-e1|< abstol or |e2-e1|< reltol*(|e1|+reltol). We add reltol to |e1| to take account of the case when e1 is zero).

val reltol : float ref
Relative tolerance for the stop criterion. See abstol for details on the stop criterion.
val eta : float ref
Step of the gradient descent. The quality of the training's result with gradient descent may vary significantly with the chosen value of eta. The optimal value of the parameter depends on the problem. The default value for etat is 0.2.
val mu : float ref
mu is the momentum parameter of the gradient descent with momentum method. It must be chosen between 0 and 1.
val opti : opti ref
Optimization method used to minimize the error during the training.
val learning : gradient_learning ref
learning allows to choose how many patterns are used to compute the gradient of the error, before making a step in the descent direction (in the weights space). This choice is specific to the gradient descent methods. The options are: Batch when all patterns are used, On_line when only one pattern is used, or Chunk n when a block of n patterns is used to compute the gradient.
val rand : int ref
Root for the random generator (default 0).
val norm : bool ref
Boolean flag deciding if the patterns should be normalized before being taken as inputs. This flag is set by the -norm option. Normalization is made by removing the average value and dividing by the standard deviation. However, this normalization operates only on the pattern file given as argument of the command line. So you may not normalize the same way on your training set and your test set.

You should better pre-process your initial data set, and normalize the inputs before splitting it into a training set and a test set. Whatever method is used, please do not forget to normalize the inputs of the neural network before training it.

val mode : mode ref
Program's mode, which may be either Train, Test, Run, or Predict. Default is Train. The choice can be changed with adequate options in the command line.
val freq_verbose : int ref
Frequency of the verbose output on stdout (default 10). It can be modified with the -fv option.
val print_class : bool ref
Boolean flag selecting the type of results printed on the standard output: classification results when true, or regression results otherwise. The default option is to print regression results. Use -prc option to print classification results.

Parse command line, and print program parameters


val int_args : int list ref
List of integer arguments found when parsing the command line. We should find integer arguments only when training the network (-train option). These arguments should then be the number of units in each layer of a fully connected network.
val float_args : float list ref
List of float arguments found when parsing the command line. We should find float arguments only when running the neural network with a new input vector (-run option).
val parse_arguments : unit -> unit
parse_arguments () parses the arguments and the options of the command line.
val fprint_parameters : out_channel -> unit
fprint_parameters ch prints the global variables used as program parameters on channel ch.

Select the network's functions


val get_functions : unit -> Ann_func.nn_func
get_functions () returns the neural network's functions chosen by the user (command line options).