Re: what is SVM? what are the good features of SVM?

"Aleks Jakulin" <jakulin@@ieee.org> wrote in message news:<[EMAIL PROTECTED]>...
> "walala" <[EMAIL PROTECTED]> wrote:
> > I saw a lot buzzword about SVM? What is it? And why is it hot? Can
> > it be used for curve fitting/extrapolate/prediction problem?
> 
> SVM is hot because you can use relatively robust quadratic programming
> optimization apparatus instead of the incremental back-propagation
> most people use for neural networks: there are very few parameters
> that you need to set, and even if you don't set them, it's likely to
> work. Furthermore, the whole methodology is a bit more rigorous
> mathematically. Finally, the tools are simple and accessible.

Interestingly, regularized logistic regression is very nearly
equivalent to SVM techniques and can be trained more efficiently than
the normal QP algorithms used for training SVM's.  Adding the
regularization also greatly improves the convergence properties of the
logistic regression itself.

This fact doesn't seem widely known but given the impact, it should
be.

Jason Rennie has written a lucid description of the underlying
mathematics for one form of regularized logistic regression -
http://www.ai.mit.edu/people/jrennie/writing/lr.pdf

Researchers at CMU have used this for text categorization with good
results:

http://portal.acm.org/citation.cfm?id=860471&jmp=references&dl=portal&dl=ACM

This group has also showed that SVM results can be exactly replicated
using logistic regression techniques.  They use a sequence of
functions so that the logistic regression penalty function uniformly
converges to that used by SVM.

I haven't seen anybody present the relatively simple proof that
demonstrates the near equivalence of regularized logistic regression
and structural risk minimization.  It is simple enough that somebody
must have published it already.