Re: Overflow

Heavens, 

This problem has a much simpler solution than that.  Just pick some
boundary conditions above which you use an asymptotic value.

For example, in the case of the logistic function (and similar for
tanh), we have
                        1
           f(x) = -------------
                   1 + exp(-x)

For |x| > 50, we see that since exp(x) ~ 5 x 10^21, double precision
arithmetic will cause f(x) to be either 0 or 1.  This means that you
can test for values outside this range and just return 0 or 1 instead.
 Moreover since the derivative is given by:

          f'(x) = f(x) f(-x)

you can pull the same trick.

One important thought, though, is that if you are encountering values
large enoug to cause numerical overflow with double precision, then
you may be having much more substantial problems with lack of
convergence in your weight computation.  Have you tried a simpler
approach such as logistic regression or a NN with no hidden layers
first?

[EMAIL PROTECTED] (Greg Heath) wrote in message news:<[EMAIL PROTECTED]>...
> "Greg Heath" <[EMAIL PROTECTED]> wrote
> 
> > "Krzysztof Kolago" <[EMAIL PROTECTED]> wrote in message 
> > news:<[EMAIL PROTECTED]>...
> > 
> > > I prepare some neural networks using our program, but in my last
> > > projects I've to use very long list of training data. When I've add 504
> > > example for training there was an error: "exp: OVERFLOW error". 
>
> > I assume this is occuring in the activation calculation step. One
> > cure is to replace the theoretical tanh or logsig containing 
> > exponentials with a piecewise differentiable low order polynomial.
> > Other models are given in Tveter's home page.