masthead for Beyond Google: The Invisible Web with a spiderweb next to it

How to Tame, Mine, or Harvest The “Invisible Web”

1. Directories:

Open Directory Project - "...largest human-edited directory of the Web"
www.dmoz.org

Librarian's Internet Index - Database of about 20,000 websites on a variety of topics. Offers a newsletter.
http://lii.org

Infomine: Scholarly Internet Research Collection from the University of California-Riverside
http://infomine.ucr.edu

Digital Librarian maintained by Margaret Vail Anderson, a librarian
www.digital-librarian.com

2. Pathfinders:

Boswell, Wendy. The Invisible Web: How To Find and Search the Invisible Web. About.com. 2009.
http://websearch.about.com/od/invisibleweb/a/invisible_web.htm

Lackie, Robert J. Those Dark Hiding Places: The Invisible Web Revealed. Rider University Libraries. November 2008.
New URL: http://library.rider.edu/scholarly/rlackie/Invisible/Inv_Web.html

Tool Kit for the Expert Web Searcher. The Invisible Web. Library and Information Technology Association (LITA) Wiki, maintained by Pat Ensor (14 July 2006).
http://wikis.ala.org/lita/index.php/The_Invisible_Web

A tutorial from the UC Berkeley Library. Invisible or Deep Web: What It Is, How to Find It, And Its Inherent Ambiguity. December 2008.
www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html

3. Search Tools:

www.completeplanet.com - BrightPlanet - Searches over 70,000 databases

www.closerlooksearch.com - Example of a vertical search engine

www.oaister.org - Union catalog of digital resources using the Open Archives Initiative Harvesting Protocl.

http://hakia.com - Example of semantic Web searching

http://turbo10.com - "Search the deep Net"

www.deepdyve.com - "Research engine for the Deep Web"

www.freepint.com/gary/direct.htm - Gary Price compilation of resources. No longer maintained. Last updated 6/26/2002.

BASE - Bielefeld Academic Search Engine - University of Bielefeld Academic Search Engine

4.  Specialized Databases:

www.findarticles.com - Now part of Bnet; free and premium articles in business

www.findlaw.com - information on federal and state laws, lawsuits, etc.

www.eric.ed.gov - Digital library of education research and information

http://worldwidescience.org - "The global science gateway"

www.science.gov - "Authoritative U.S. government science information"

www.scienceresearch.com - A free Web portal to science journals and science databases

www.archive.org/web/web.php - Wayback Machine - archive of websites

www.usa.gov - The official Web portal to the U. S. government

www.loc.gov - Library of Congress

www.ncbi.nlm.nih.gov/pubmed/ - PubMed: U.S. National Library of Medicine portal to government-funded medical research

www.imdb.com - Internet movie database

www.intute.ac.uk/ - British database of Web resources for education and research. Covers humanities, health sciences, natural sciences, and the social sciences.

www.hoovers.com/free - Listing of companies, people, and industries.

5. How to Keep Up-to-date:

http://searchenginewatch.com

www.marylaine.com/neatnew.html

www.researchbuzz.com/wp

6. Weblogs:

Deep Web Research - http://deepwebresearch.blogspot.com/

Invisible Web Weblog - http://invisibleweblog.blogspot.com/
Maintained by Yazdan Mansourian.

7. Foreign language sources:

http://europeana.eu/ - A digital European library available in 25 languages.

www.internetinvisible.com/ii/ - A Spanish directory of free databases

http://c.asselin.free.fr/french/invisible_web.htm - "Web invisible, web cache, web profound," a French tutorial on the Invisible Web.

Back to top

Last updated: June 2009