masthead for Beyond Google: The Invisible Web with a spiderweb next to it

Glossary

Database: A system of data organized in records and fields for fast retrieval. Databases have their own individual search tool.

Dark Web: This term usually refers to the "dark side" of the web experience which is also part of the Invisible Web. This dark side includes the world of illicit activities such as drug dealing, pornography, sexual predators and other crimes. It can offer anonymity to those who wish it. Reference to the dark web may include mention of the TOR Project or the Hidden Wiki.

Deep Web: This is another name favored by some for the Invisible Web.

Directory: This kind of search tool categorizes information sources. It does lead the user to the front door of sites that offer information. Directories are by their nature selective rather than exhaustive and can be limited in scope.

Dynamic (dynamically generated): This refers to information generated only in response to a specific database query. It ceases to exist afterwards.

Federated Search: A federated search tool gathers together several web resources such as specialized search engines and databases, and permits them all to be searched together from one search query. These tools usually have a subject orientation such as medicine or business. They may incorporate Invisible Web resources.

Filters: Google regularly "personalizes" searching by reviewing an individual's search patterns to base their future search results around their preferences. This may mean that an individual will only receive search results that favor their own biases and interests. Many people do not realize that his customization is happening and that they are only receiving a partial answer to their query or only what the search engine thinks that they want to receive. The information that has been screened or filtered out becomes part of their Invisible Web. Most researchers need to see the whole picture and all sides of an issue. Google does have a way for individuals to opt out of this personalization, but as most people don't realize it is happening, they won't seek the opt-out.

Harvesting: This term is often used to describe the process of finding information on the Web.

Hidden Web: This is another name favored by some for the Invisible Web.

Horizontal Search: This term is used to describe the kind of searching done when using a general-purpose search engine which draws on a broad number of surface Web resources.

Mining: This term is often used to describe the process of finding Information on the Web.

Mobile Web Searching: Mobile access to the web may be defined by the apps a person uses. Apps create their own search world which may leave a lot of untapped resources in an Invisible Web.

Niche Search: Niche searching refers to a very specific, narrow, but deep search into specialized resources from both the surface and Invisible Web. Niche is even more specialized than a vertical search (See definition below).

Opaque Web: This term refers to material that can be but is not included in search engine results. Some omissions may be due to new material being added and not yet picked up by web crawler programs or information being placed deeper within a site than web crawlers usually reach.

Personalized Searching: See Filters

Private Web: These sites have been intentionally excluded from web search engine results by their sponsors. They may have password protections or be programmed as a "Noindex" site, which tells web crawlers not to index the page.

Proprietary Web: This term refers to sites that require the user to be registered for use before access can be given.

Real-time searching: Search engines have never been very good about offering real time searching. Their spiders review the web constantly but can't keep track of every news item that occurs on a daily basis. This means that for a limited time new resources may not be found using a search engine and may reside in the Invisible Web while waiting for listing in the search engine indexing.

Search Engine: This refers to a computer program that can help users find information on the World Wide Web.

Search Engine Optimizers (SEOs): There is a whole industry that devotes its focus on helping websites establish good replacements in search results. It is important, for instance, for companies to see their products listed in the top ten search results as that may be as far as the majority of individuals want to look for their information. SEO's help organization design their web pages in a way that helps them gain top placements in search results.

Spider: Also referred to as a "Web Crawler," this program used by search engines, retrieves information from a web page and follows and web links for indexing purposes.

Split-Level Searching: General search engines such as Google can be used to find databases which reside in the Invisible Web. A searcher can frame a query using a subject and adding the term databases to the search (e.g. solar energy and databases.) The results will be a list of databases that will each then have to be searched individually for their content. This process is called split-level searching.

Static: Fixed and reliable links are referred to as "static."

Subscription Databases: These databases are fee based.

Surface Web: Sometimes referred to as the "Visible Web", it represents World Wide Web content found by using search engines.

Vertical Search: This term is used to describe searching which delves deeply into a subject. It implies the researcher is using specialized search tools and retrieving materials from both the surface and Invisible Web.

Visible Web: See "Surface Web."

Web Crawler: Sometimes called a "spider," this is a program used by search engines to locate and retrieve web pages and the references linked to them for indexing purposes.

Updated December 2014