A Birmingham-based company is nipping at the heels of Google after it became only the second search engine in the world to join the “webmap trillionaires club”.
The little-known Majestic-12 Distributed Search Engine, based at Birmingham Science Park in Aston, has found its 1,026,035,494,521st unique web page on the internet, making it the second organisation worldwide to publicly announce the one-trillion web page milestone - the first was Google.
Neither Yahoo, nor Microsoft Bing – two of the world’s best known search engines – has yet declared that they have reached this milestone.
The Majestic12.co.uk Distributed Search Engine project was founded five years ago with the aim of creating a community-driven search engine based on a distributed computing model.
Instead of one centralised computing centre trawling the net, internet users around the world can devote their machines to the project when they are not using them.
Majestic 12’s founding managing director, Alex Chudnovsky, said: “Compared to Google, we are very small but we are growing.
“We are trying to develop a community-driven project to build a search engine that’s comparable to Google but because we had limited resources, we decided to use a community model where members crawl the data on their own computers.
“We don’t have that many members - we have maybe 120 to 150 active users, although some of them have more than one computer.”
Mr Chudnovsky said the company was looking into how it can commercialise its data, given that the information on links between webpages it has discovered are of particular interest to search engine optimisation companies.
But Majestic 12 is aware that it has built up its huge banks of information thanks to volunteers working around the world, so the company has devised a partnership structure which rewards those people who have contributed their computing power to the product for free.
“We are a community-driven project,” he said.
“We can’t just go away and start selling data which has been collected by all these people - we don’t want to enrich ourselves with free labour, so we had to come up with a model of how we can give them a share.”
In the end, the company opted for a limited liability partnership which pays dividends to the volunteer “crawlers”.
“We feel we are doing the right thing by giving people a part of the company,” said Mr Chudnovsky.
“We are different from anyone else - for example, when YouTube was sold for a lot of money to Google, the people who uploaded video and who built the company didn’t get anything.
“We feel that people who contributed to our company should benefit from our activities.”
Majestic 12’s announcement provides an independent verification of the milestone announced in July last year that Google’s systems had detected one trillion - 1,000,000,000,000 - unique URLs on the web.
The project has been likened to the SETI@home initiative which saw more than three million people around the world download software that meant their computers’ processing power could be used during down-time to contribute towards the search for extra-terrestrial life.
Idle computers were given over to the task of analysing radio telescope data for signs of life in outer space.
Birmingham Science Park Aston business development manager Matthew Hidderley praised Majestic 12’s work in reaching the milestone.
“This is an excellent achievement by one of our entrepreneurial technology businesses,” he said.
“This further demonstrates the success the innovative business is enjoying.”