Natural Language Search - A New Breakthrough?
Published 4 months ago by Andreas Blumauer
While I am still waiting for an invitation from Twine (probably you too?) I have received one from Powerset - natural language search. Powerset obviously is a promising company (and is promising a lot), so I was excited when I was starting to play around with this new tool which still isn't available for the public.
The very first impression was good. The interface is well done and there are a couple of new ideas how Wikipedia (and similar knowledge bases) can be navigated in the future. But unfortunately after a while it was clear, that search results must be improved. However Powerset might be implemented, the only benchmark which counts at the end is, which improvement the new application (semantic web or not) delivers compared to existing ones. Some examples:
- The question: "who is the president of the united states?" delivers some similar questions or related articles of Wikipedia but NOT the right answer.
In comparison: ask.com delivers this perfect result.
- My next question "where was mozart born?" delivers "Getreidegasse" which is correct but actually too much detailed. Again, ask.com delivers the perfect answer.
A third try which was an even more difficult question: "how far is london from paris?" was again correctly answered by ask.com, powerlabs wasn't even close...
When I was asking START, the world's first Web-based question answering system, which has been on-line and continuously operating since December, 1993 (!) those three questions - all of them - were answered correctly.
So finally I was asking something really tricky: "What is the relationship between RDF and XML?": Only START gave the right answer which was: "Unfortunately, I don't have that information."
Trackback URL for this entry:
http://www.semanticfocus.com/blog/tr/id/119111/
Spam protection by Akismet
Post a comment
Posted by Elder on February 13, 2008 at 6:46am
There is a very definite line that must be drawn between question answering, on the one hand, and search, on the other. Question-answering systems must deliver "answers" and correct answers at that. The task of a Search engine is to deliver a set of resources that can plausibly contain the information the user is looking for. One may fairly criticize a search engine for retrieving irrelevant documents in response to a query, but not for failing to give a "perfect answer", That is like criticizing a French-Englsh translator for being unable to speak fluent Urdu. Powerset is a Natural Language SEARCH company not a Question-answering company.
Posted by James Simmons on February 13, 2008 at 9:02am
@Elder:
Traditionally search engines and question-answering systems have been very separate in their function. I believe that semantic search blurs the line considerably because of the semantic search engine's (at least, Hakia's as far as I know) ability to derive what information you may truly be looking for from your query.
Go to Google and type in "how old is Will Smith?" and you will see that Google has begun to blur the distinct line that separates SEs from QAs.
I often think about what the "perfect" search results would be, and often, it might be a mix of a question answered and some relevant Web documents.
Cheers
Posted by ABLVienna on February 15, 2008 at 3:19am
@Elder:
On Powerlabs users can find a lot of queries from many users like "who was the first to trade rooibos" (http://labs.powerset.com/queries/1869). From this page users can check out what Google, Ask, Yahoo etc. delivers.
So why should a strict distinction between QAs and SEs make any sense in the future?
Posted by John Bäckstrand on June 3, 2008 at 3:33am
True knowledge can answer the three first one correctly. I tried adding the knowledge for the fourth one, but failed.