Why is NLP such a hard topic?

I have long been wandering about that question, and did not really find any
good answers:
There are many (profitalbe) uses for natural language processing (automated
translation, search engines, spam filters, database querys...), and lexicons
(even including kind of semantic information) are available, and yet (as far
as I know) there is no really good implementation for natural language
processing (Think of something like English in BNF).

Why is this the case?

It's surely not because it would be too much work to write down a few
thousand rules or more (theres money in it).

Can anyone explain me (or point me to information that does) why this is
such a hard topic?
Is the number of rules simply too big???
Can syntax not be analyzed without semantics?
Is some mystical "soul" required to process natural language????

Thanks a lot

Niki