Thursday, September 14, 2006
Mindset: Yahoo Applies Machine Learning to Search
Yahoo! Mindset is a very promising concept that utilizes machine learning to determing if pages are commercial or scholastic. Specifically, an automated text classifier was trained to distinguish between the two categories of pages and can dynamically sort through search results. The two researchers who developed it joined Yahoo! after its acquisition of Overture so this project has definite commercial search applications.
I performed a search on "harry potter" and initially got a page with standard Yahoo! search results. Topping the list were the offical Harry Potter movie site and J.K. Rowling's page.
I dragged the slider at the top of the search results to the left toward "shopping" and the results dynamically resorted themselves. Handily, the results also displayed the original rankings in parentheses to the left. 
The original number 30 and 20 search results jumped up to take the second and third spots, respectively. The Scholastic and EA games sites could be considered more commercial than the movie and author sites, and it's interesting to see the Swedish version of the official movie page displace the English one.Pulling the slider to the right toward "researching", I got fan fiction and fan sites near the top of the results. These sites are clearly not commercial so it appears that Mindset is at least good for a rough filtering of search results.
Based on just this one search, Yahoo! Mindset appears to be a keeper. It's certainly not perfect, but it is a good first-pass filter. For students, researchers, and others who need more objective search results, it'll help weed out shopping and other commercial sites. And for others, it'll help those who actually want to buy something or want to find "official" websites.Yahoo! Mindset
Technorati Tags:
Labels: Technology
