masthead for Beyond Google: The Invisible Web with a spiderweb next to it

Sample Suggestions for Mining or Harvesting the Invisible Web

1. Directories:

Open Directory Project - "...largest human-edited directory of the Web"
www.dmoz.org

Infomine: Scholarly Internet Research Collection from the University of California-Riverside - (Went offline 12/15/14)
http://infomine.ucr.edu

Digital Librarian maintained by Margaret Vail Anderson, a librarian
www.digital-librarian.com - (Last updated 4/20/13)

Internet Public Library (ipl2) - http://ipl2.org - (Ceased operations 6/30/15)

WWW Virtual Library http://vlib.org/ - Started by Tim Berners-Lee in 1991, oldest Web directory.


2. Guides:

Boswell, Wendy. The Invisible Web: How To Find and Search the Invisible Web. About.com. 2009.
http://websearch.about.com/od/invisibleweb/a/invisible_web.htm

A tutorial from the UC Berkeley Library. Invisible or Deep Web: What It Is, How to Find It, And Its Inherent Ambiguity. 2012.
www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html


3. Search Tools:

http://base.ub.uni-bielefeld.de/en/index.php - BASE - Bielefeld Academic Search Engine

www.closerlooksearch.com - Closer Look Search - Example of a vertical search engine

http://deep-web.org - Deep Web Search - A How-To Site

http://deeperweb.com - deeper web (powered by Google) -(Beta 2010)

http://mednar.com/mednar/ - Mednar

https://millionshort.com - Million Short will remove from a hundred to a million top websites from search results to reach new and original resources.

http://www.refseek.com - (in public beta 2014) - "is a web search engine for students and researchers that aims to make academic information easily accessible to everyone."

www.archive.org/web/web.php - Internet Archive Wayback Machine - archive of websites

http://www.wolframalpha.com - Wolfram Alpha - Computational knowledge engine

http://www.deepdotweb.com - Deep.Dot.Web - "provides the public with information about the dark net."

http://www.vocativ.com - Vocativ - a "new type of media company, bringing audiences hidden perspectives, unheard voices and original ideas from around the world via the Deep Web."


4.  Specialized Databases:

http://www.doaj.org/ - Directory of Open Access Journals

www.eric.ed.gov - Digital library of education research and information

www.findlaw.com - Information on federal and state laws, lawsuits, etc.

www.loc.gov - Library of Congress

https://pipl.com/ - People search

www.ncbi.nlm.nih.gov/pubmed/ - PubMed: U.S. National Library of Medicine portal to government-funded medical research

www.science.gov - Gateway to U.S. Federal Science

www.scienceresearch.com - A free Web portal to science journals and science databases

http://topsy.com - Searches the social web   (Closed December 2015)

www.usa.gov - The official Web portal to the U. S. government

http://worldwidescience.org - "The global science gateway". Available now in 10 languages.


5. Weblogs:

The Invisible Web Revisited - http://invisiblewebrevisited.wordpress.com/
Maintained by Francine Egger-Sider

Going Beyond Google Again - http://goingbeyondgoogleagain.wordpress.com/
Maintained by Francine Egger-Sider

Invisible Web Weblog - http://invisibleweblog.blogspot.com/
Maintained by Yazdan Mansourian.


6. LibGuides:

http://libguides.uky.edu/content.php?pid=53054&sid=1992304 - LibGuide created by the University of Kentucky Libraries in which the Invisible Web is depicted as a venn diagram

http://guides.stlcc.edu/deepweb - St. Louis Community College Library LibGuide, "Deep Web: Welcome"


Back to top

Last updated: December 2015