Re: Simple ANN question

Peder Ydalus <[EMAIL PROTECTED]> wrote in message 
\news:<[EMAIL PROTECTED]>...
> If a node (hidden or output) is supposed to calculate the sum of the
> products of its input weights times its input values, and then apply
> some squashing function, how can ever the ANN produce other values than
> <0, 1>?

If the theoretical outputs don't have natural fixed limits,
shift to zero mean and scale to unit variance (See the FAQ).
Then use an identity output activation:

yk  = b2k + SUM(j=1,H){ w2kj*zj },          k = 1,2,...O,

zj  = tanh{ b1j + SUM(i=1,I){ w1ji*xi }},   j = 1,2,...H

If the theoretical outputs do have natural *fixed* limits,
you can still do as above. However, in some cases it is
useful to shift and scale to the compact interval (-1,1) 
or (0,1) and use tanh or logsig as the ouput activation.
This is equivalent to using two hidden layers with each
output connected to one neuron in the second hidden layer.


yk   =  b3k + w3k*z2k,                        k = 1,2,...O,

z2k  = tanh{ b2k + SUM(j=1,H){ w2kj*z1j }},  k = 1,2,...O,

z1j  = tanh{ b1j + SUM(i=1,I){ w1ji*xi },    j = 1,2,...H


> Am I missing something very fundamental?
> 
> From what I understand, every layer in the backpropagation algorithm
> will tend to return an error term = 0 when neuron output 'ok' climbs
> towards 1. dk = ok * (1 - ok) * (tk - ok). This will happen when vextors
> w*x returns something big because of the squashing function.
> 
> My ANN has numerical inputs where some are always low and others are
> thousands of times higher. I thought the weights would eventually even
> themselves out to create that equilibrium. Target output is in the range
> 100-400 and thus not <0, 1>

Are 100 and 400 theoretical limits or just bounds for your data set?

Hope this helps.

Greg
 
> Or am I just totally misunderstanding the whole concept?