the "surface" web is the part of the internet that the average surfer sees, i.e it appears on all the search engines. the size of the internet in 2000 was roughly 167 terabytes. (1 terabyte is 1,024 gigabytes)
But in contrast the "deep web" contains an estimated 91,000 terabytes these are websites that are private, only registered users can enter the deep web also includes sites not linked to search engines, or hyper-linked this is to prevent ''web crawling"
(source wikipedia)
this blew my mind when i found out the figures but it is interesting that most of the internet is hidden and unavailable to the public. People have started search engines that search the deep web, but it is far more difficult to sift through for information, this is because it also consists of documents, like e-books and large data bases.
below is a link to a deep web search engine, many other companies are attempting to harness the power of the deep web search.
http://www.closerlooksearch.com/index.aspx
1 comment:
Now this is something fascinating that I didn't know about, so thank you for bringing this issue to my attention.
I had wondered about the accessibiity of web pages which say you need to re-send the information in order to continue, but I had never reflected on how much data is deliberately kept from view.
I've tried the link to CloserLook Search and it's been a revealing experience.
I shall have to go away and do some more research of my own.
Post a Comment