Re: Simple ANN question

I'm doing a University project on COM interaction in the win32
environment using Python, and I figured making an ANN and reading the
values from an excel sheet would be good project.

My ANN har 12 imputs. These are all stock data, so there are no bounds
and the values vary a lot. For instance, a beta value is usually in the
range <0, 1>, whereas trading volumes are at the extreme more than a
billion.

I made a plain ANN with 12 input neurons, 3 hidden neurons and one
output neuron. The input neurons are a stock ticker id (numbered myself
from 0 to about 250), a stock industry sector id (numbered myself from 0
to about 100), the beta value (normally never outside <-1, 2>), date
(counting days since 01.01.1995), trading price (highly varying from
stock to stock), trading volume (varying even more from stock to stock),
pluss date, price and volume for the day before and the day before that. 

The output neuron is supposed to propose a price for the next day.

However, as I train, the big input values quickly sets the values in the
hidden neurons to either 1 or -1. This in turn leads to weight updates
of 0. My ANN doesn't get passed2 or 3 iterations before I can turn off
the training.

Finally, the sigmoid function in the last neuron will, of course, lead
to an output in the range <0, 1>. Compared to the supposed value, e.g.
250, leads to weight updates following an error of about 249.

What am I not getting?

Yes, I have read the FAQ, but it didn't clear up my misconceptions.

Finally why do we need biases and bias weights? And when we do, if they
are all set to 1 does it really make a difference? 

Btw, anyone know any good ANN litterature, taking things from the
basics? I'm using a book called Machine Learning by Mitchell, but want
something better.

Thanks!



- Peder -




Greg Heath wrote:
> 
> Peder Ydalus <[EMAIL PROTECTED]> wrote in message
> \news:<[EMAIL PROTECTED]>...
> > If a node (hidden or output) is supposed to calculate the sum of the
> > products of its input weights times its input values, and then apply
> > some squashing function, how can ever the ANN produce other values than
> > <0, 1>?
> 
> If the theoretical outputs don't have natural fixed limits,
> shift to zero mean and scale to unit variance (See the FAQ).
> Then use an identity output activation:
> 
> yk  = b2k + SUM(j=1,H){ w2kj*zj },          k = 1,2,...O,
> 
> zj  = tanh{ b1j + SUM(i=1,I){ w1ji*xi }},   j = 1,2,...H
> 
> If the theoretical outputs do have natural *fixed* limits,
> you can still do as above. However, in some cases it is
> useful to shift and scale to the compact interval (-1,1)
> or (0,1) and use tanh or logsig as the ouput activation.
> This is equivalent to using two hidden layers with each
> output connected to one neuron in the second hidden layer.
> 
> yk   =  b3k + w3k*z2k,                        k = 1,2,...O,
> 
> z2k  = tanh{ b2k + SUM(j=1,H){ w2kj*z1j }},  k = 1,2,...O,
> 
> z1j  = tanh{ b1j + SUM(i=1,I){ w1ji*xi },    j = 1,2,...H
> 
> > Am I missing something very fundamental?
> >
> > From what I understand, every layer in the backpropagation algorithm
> > will tend to return an error term = 0 when neuron output 'ok' climbs
> > towards 1. dk = ok * (1 - ok) * (tk - ok). This will happen when vextors
> > w*x returns something big because of the squashing function.
> >
> > My ANN has numerical inputs where some are always low and others are
> > thousands of times higher. I thought the weights would eventually even
> > themselves out to create that equilibrium. Target output is in the range
> > 100-400 and thus not <0, 1>
> 
> Are 100 and 400 theoretical limits or just bounds for your data set?
> 
> Hope this helps.
> 
> Greg
> 
> > Or am I just totally misunderstanding the whole concept?

-- 
Det er bedre å putte nesen sin i andres ting
enn å putte andres ting i nesen

        http://www.stud.ifi.uio.no/~pedery