Chapter Seven: 

 

What is a Meta-Search engine?

 A meta-search, multi-threaded or multi-search engine searches other search engines' databases in parallel (at the same time) and collates the results (which usually consist of a predetermined maximum "number" such as 10, 20 or 30). The search engine does not maintain its own database. Major differences in the meta-search engines include:
  • Selection of databases searches
  • Submission of queries to the search engines
  • Manipulation and presentation of results
  • Response time

Selection of Databases Searched

Most meta-search engines will search several of the "major" search engines at once, usually: Altavista, Excite, Lycos, Webcrawler, Yahoo and Hotbot. Sometimes because of an agreement with one search engine, the meta-search service will not search another, specified, search engine. A number of directories such as Looksmart, Galaxy, and/or smaller search engines and specialized databases such as Deja News (newsgroups), stock quotes, or news wires may also be included. 

Sometimes this is wonderful -- but at other times a lot false hits from irrelevant sources can create a bad search set. This is why the ability to customize is so important -- being able to choose which search engines your query is sent to. But this also requires some experience on the part of the user to be able to determine which search engines or databases to include in the search.

How Queries are Submitted to the Search Engines

Queries are submitted to each search engine at once (in parallel) or one or several at a time. I tend to prefer the "simultaneous" search myself -- I mainly use meta-search engines because I want to see what the search engines find all at once (some meta-search services ask if you would like to "Search X more search engines" instead of sending all queries at once).

Remember how each search engine uses different search syntax and has different advanced features such as Boolean, field and proximity searching? Well here's one of the main problems of meta-search services -- unless specifically stated otherwise, the advanced techniques will be filtered out. This is because the interface of the meta-search engine must be programmed to interpret your search syntax for each and every search engine it queries. As we all know, search techniques frequently change. For the most part, however, simple Boolean (and, or, not) are honored as well as implied Boolean (+ meaning "and" or "require" and - "exclude").

Manipulation and Presentation of Results

Depending on the kind of research I am doing, I may wish to see which search engine retrieved which hit, or results grouped by web site, or results ranked according to relevance and almost always to have the duplicates removed. Unfortunately, all of these features are not available through all meta-search utilities, so you have to choose which is most important to you (and this often varies with the search....). Most annoying, is when relevancy rankings are stripped.

If Meta-Searchers are so great and search my favorite search engines simultaneously, then why wouldn't I always use one?

Well.....again, it depends on what you are looking for...consider
  • Meta-searchers, for the most part, strip the advanced search syntax and logic from your queries when they submit them to the search engines, so search statements with complex structures are not good candidates for these tools.
  • you will only see a few results from each of the search engines, which is also limiting.
  • the presentation of the results may be confusing.
  • sometimes the time it takes to conduct a search using a meta-search engine may be too long
  • sometimes relevancy rankings are removed from the listings

When would I use Meta Search Engine ?

Use a meta-search engine when:
  • when you can't find anything when you've queried one or two "major" search engines
  • when your topic is "obscure"
  • when you want to see the top hits (most relevant selections) from several databases at once
  • when you want to search a variety of sources on the same topic simultaneously
  • when you want to compare the indexing of several search engines (perhaps to pick one to search by itself)
  • when you know the web "literature" on the topic but want to see if you missed something
  • when you want to search several utilities at once and have all the results ranked by relevance (some engines will do this)
  • when you want to search using one interface and method of presentation

 

Meta-Search Engines

  • Ask Jeeves
    (http://www.askjeeves.com)
     
  • Cyber411
    (http://cyber411.com/)
     
  • Dogpile
    (http://www.dogpile.com/)
    Sends queries to a customizible list of up to 25 search engines including: Yahoo!, Lycos' A2Z, Excite Guide, Go2.com, HotBot, PlanetSearch, Reference.com, Thunderstone, What U Seek, Magellan, Lycos, WebCrawler, InfoSeek, Excite & AltaVista. Also, Newswires, FTP and Usenet.
     
  • Highway 61
    (http://www.highway61.com/)
     
  • Mamma
    (http://www.mamma.com/)
     
  • MetaCrawler
    (http://www.metacrawler.com/)
     Sends search queries to several search engines including: Lycos, Infoseek, WebCrawler, Excite, Alta Vista and Yahoo, then normalizes and ranks results. Of special interest: using Power Search, allows user to limit by continents and by U.S. educational, commercial or government sites.
     
  • ProFusion
    (http://www.profusion.com/)
     
  • Search.com
    (http://www.search.com/)
    Queries several specialized search engines (such as shareware.com, Internet Movie Database) as well as web indexes (such as Alta Vista, Lycos) and web directories (such as Yahoo and EINET Galaxy). User can specify/limit by source (Web Indexes, People, Entertainment, Software) and by "type" of information (images, technical reports, academic).