
www.Usenet.com
Group Index
Comp Thread Archive from Usenet.com
nn in information retrieval
- __From__: Thomas Krämer
- __Subject__: nn in information retrieval
- __Date__: Thu, 04 Dec 2003 09:58:11 -0600
Hello fuzzy / neural network implementers
i am building an extension for lucene, the search engine project within
apache , that integrates the user´s
query history into the retrieval process.
one idea is to implement a neural network, that adapts to the users
information need by reweighting search terms.
furthermore there are 2 different levels of text representation, one by
a large collection of metadata and pure fulltext search, and a second
based on the same metadata AND the entire document.
the latter is permanently changed in order to obtain a representation
closer to what a single user considers relevant.
first, the application performs a full-text search on a metadata corpus
using lucene.
the user then selects the apparently most interesting documents (by its
dc metadata). this second user-input should be used to adapt the neural
network weights.
until now, i am not entirely decided which neural network api i should
use. joone, at a first glance, seems to be quite user friendly.
question 1 :
does an easy to use "term-document-matrix builder" exist, that
constructs t-dimensional matrix containing all the terms in the corpus
mapped to the documents where they occured?
it would be helpful if it were easily possible to compute the
tf-idf-measure instead of the absolute term-occurence values (integers).
i thought of a simple 2-layered nn, in wich one of the layers represents
documents and the second one all the terms in the corpus (cf. google:
cosimir)
so, once the users decides to see the full text of several docs
refenrenced in metadata retrieved by a full text search according to teh
users initial query terms, it must be possible to increase the
activation of those neurons that represent the documents selected for
full retrieval.
quetion 2:
do you know whether there is a more appropriate (open source) software
to build such a nn?
i am a real newbie to ai and appreciate your opinion.
thanks for any hint!
thomas krämer
- nn in information retrieval,
Thomas Krämer
Please check out one of the premium Usenet Newsgroup Service Providers below for access to Usenet.