A Free Database of the Entire Web May Spawn the Next Google

Common Crawl supplies a database of over five billion Web pages in the hope that it will inspire new research or online services.

Google famously started out as little more than a more efficient algorithm for ranking Web pages. But the company also built its success on crawling the Web—using software that visits every page in order to build up a vast index of online content.