Editing WebCrawler

{{about|the [[search engine]]|web crawling programs in general|web crawler}}
{{pp-move-dispute|small=yes}}
{{short description|Web search engine}}
{{Infobox website
| name = WebCrawler
| logo = Webcrawler logo 2018.png
| logo_caption = Logo since 2018
| caption = Logo since 2018
| url = {{URL|webcrawler.com}}
| commercial = No
| type = [[Search engine]]
| language = [[English language|English]]
| registration = none
| owner = [[System1]]
| author = Brian Pinkerton
| current_status = Active
| launch_date = {{Start date and age|1994|04|20}}
}}

'''WebCrawler''' is a [[search engine]], and one of the oldest surviving search engines on the web today. For many years, it operated as a [[metasearch engine]]. WebCrawler was the first web search engine to provide full text search.<ref name=":0">{{Cite web |url=http://www.thehistoryofseo.com/The-Industry/Short_History_of_Early_Search_Engines.aspx |title=Short History of Early Search Engines |publisher=The History of SEO |access-date=2019-02-03 |archive-date=2019-01-21 |archive-url=https://web.archive.org/web/20190121213229/http://www.thehistoryofseo.com/The-Industry/Short_History_of_Early_Search_Engines.aspx |url-status=dead }}</ref>

==History==
[[File:Webcrawler screenshot 1995.png|thumb|200px|Screenshot of WebCrawler homepage in September 1995]]
Brian Pinkerton first started working on WebCrawler, which was originally a desktop application, on January 27, 1994 at the [[University of Washington]].<ref name=":1" /> On March 15, 1994, he generated a list of the top 25 websites.<ref name=":0" />

WebCrawler launched on April 21, 1994, with more than 4,000 different websites in its database<ref name=":1" /> and on November 14, 1994, WebCrawler served its 1 millionth search query<ref name=":1">{{Cite web|url=http://www.thinkpink.com/bp/webcrawler/history.html|title=WebCrawler's History|website=www.thinkpink.com|archive-url=https://web.archive.org/web/20051128091213/http://www.thinkpink.com/bp/WebCrawler/History.html|archive-date=2005-11-28|access-date=2019-01-09}}</ref> for "nuclear weapons design and research".<ref>{{Cite web|url=https://mashable.com/2012/03/16/90s-tech-icons-where-are-they-now/|title='90s Tech Icons: Where Are They Now?|last=Lammle|first=Rob|date=2012-03-16|website=Mashable|language=en|archive-url=https://web.archive.org/web/20120317115357/https://mashable.com/2012/03/16/90s-tech-icons-where-are-they-now/|archive-date=2012-03-17|access-date=2019-02-18}}</ref>

On December 1, 1994, WebCrawler acquired two sponsors, [[DealerNet]] and [[Starwave]], which provided money to keep WebCrawler operating.<ref name=":1" /> Starting on October 3, 1995, WebCrawler was fully supported by advertising, but separated the adverts from search results.<ref name=":1" />

On June 1, 1995, [[America Online]] (AOL) acquired WebCrawler.<ref name=":1" /> After being acquired by AOL, the website introduced its mascot "Spidey" on September 1, 1995.<ref name=":1" />

Starting in April 1996,<ref name=":1" /> WebCrawler also included the human-edited internet guide [[Global Network Navigator|GNN Select]], which was also under AOL ownership.<ref>{{Cite web|url=https://searchenginearchive.com/single.php?id=88|title=Se-En|website=searchenginearchive.com|access-date=2019-01-25}}</ref><ref>{{Cite web|url=http://www.webcrawler.com/select/|archive-url=https://web.archive.org/web/19961024000414/http://www.webcrawler.com/select/|url-status=dead|archive-date=1996-10-24|title=WebCrawler Select: Review Categories|date=1996-10-24|website=WebCrawler|access-date=2019-02-03}}</ref>

On April 1, 1997, [[Excite (web portal)|Excite]] acquired WebCrawler from AOL for $12.3 million.<ref name=":1" /><ref>{{Cite web|url=https://www.zdnet.com/home-and-office/networking/excite-buys-webcrawler-from-aol/|title=Excite buys WebCrawler from AOL|last=Keogh|first=Garret|website=ZDNet|language=en|access-date=2019-01-15}}</ref>

WebCrawler received a redesign on June 16, 1997, adding WebCrawler Shortcuts, which suggested alternative links to material related to a search topic.<ref>{{Cite web|url=https://searchenginewatch.com/sew/news/2047963/the-search-engine-update-june-1997-number|title=The Search Engine Update, June 17, 1997, Number 7|last=Sullivan|first=Danny|date=1997-06-16|website=Search Engine Watch|language=en-US|archive-url=https://web.archive.org/web/20160414221045/https://searchenginewatch.com/sew/news/2047963/the-search-engine-update-june-1997-number|archive-date=2016-04-14|access-date=2019-02-02}}</ref>

WebCrawler was maintained by Excite as a separate search engine with its own database until 2001, when it started using Excite's own database, effectively putting an end to WebCrawler as an independent search engine.<ref>{{Cite web|url=http://www.infotoday.com/online/may02/OnTheNet.htm|title=On the Net: Dead Search Engines|last=R. Notess|first=Greg|date=2002|website=InfoToday|archive-url=https://web.archive.org/web/20020525122755/http://www.infotoday.com/online/may02/OnTheNet.htm|archive-date=2002-05-25|access-date=2019-01-16}}</ref> Later that year, Excite (then called [[Excite@Home]]) went bankrupt and WebCrawler was bought by [[InfoSpace]] in 2001.<ref name=":1" />

[[File:WebCrawler Screenshot 6-7-2010.png|thumb|WebCrawler's homepage (June 2010)]]
Pinkerton, WebCrawler's creator, led the Amazon [[A9.com]] search division as of 2012.<ref name="before">{{Cite news|url=https://www.theregister.co.uk/2012/12/18/search_engines_we_have_known/?page=3|title=Search engines we have known ... before Google crushed them|author=Brid-Aine Parnell|date=December 18, 2012|work=The Register|access-date=November 17, 2016}}</ref><ref>{{Cite web |title= Leading Leaders |work= A9 Management web page |url= https://www.a9.com/about-us/management/ |archive-date= November 14, 2016 |archive-url= https://web.archive.org/web/20161114072240/https://www.a9.com/about-us/management/ |access-date= November 15, 2016 }}</ref>

In July 2016, InfoSpace was sold by parent company [[Blucora]] to [[OpenMail]] for $45 million, putting WebCrawler under the ownership of OpenMail.<ref>{{cite news | url=http://www.seattletimes.com/business/technology/blucora-to-sell-infospace-business-for-45-million/ |title=Blucora to sell InfoSpace business for $45 million | work=Seattle Times | date=July 5, 2016}}</ref> OpenMail was later renamed [[System1]].<ref>{{Cite web| title = System1 raises $270 million for 'consumer intent' advertising| work = L.A. Biz| access-date = 2017-12-01| url = https://www.bizjournals.com/losangeles/news/2017/09/20/system1-raises-270-million-for-consumer-intent.html}}</ref>

In 2018, WebCrawler was redesigned from scratch and its logo was changed.<ref>{{Cite web|url=http://www.webcrawler.com/|archive-url=https://web.archive.org/web/20180531135409/http://www.webcrawler.com/|url-status=dead|archive-date=2018-05-31|title=WebCrawler Search|date=2018-05-31|website=WebCrawler|access-date=2019-02-02}}</ref><ref>{{Cite web|url=http://www.webcrawler.com/|archive-url=https://web.archive.org/web/20181130131104/http://www.webcrawler.com/|url-status=dead|archive-date=2018-11-30|title=WebCrawler Search|date=2018-11-30|website=WebCrawler|access-date=2019-02-02}}</ref>

== Traffic ==

WebCrawler was highly successful early on.<ref>{{Cite web|url=http://www.wisegeek.com/what-was-the-first-search-engine.htm|title=What was the First Search Engine?|last=McGuigan|first=Brendan|date=2007|website=WiseGeek|archive-url=https://web.archive.org/web/20070427154159/http://www.wisegeek.com/what-was-the-first-search-engine.htm|archive-date=2007-04-27|access-date=2019-02-18}}</ref> At one point, it was unusable during peak times due to server overload.<ref>{{Cite web|url=http://www.searchenginehistory.com/|title=Search Engine History.com|website=www.searchenginehistory.com|access-date=2019-01-25}}</ref> It was the second most visited website on the internet in February 1996, but it quickly dropped below rival search engines and directories such as [[Yahoo!]], [[Infoseek]], [[Lycos]], and [[Excite (web portal)|Excite]] in 1997.<ref>{{Cite web|url=https://tech.co/news/top-20-popular-websites-1996-present-infographic-2014-12|title=Infographic: Top 20 Most Popular Websites (1996-2013)|date=2014-12-26|website=TechCo|language=en|access-date=2019-01-15}}</ref>

==See also==
{{Portal|Internet
}}
*[[Aliweb]]
*[[InfoSpace]]
*[[Dogpile]]
*[[MetaCrawler]]
*[[List of search engines]]
*[[Search engine]]
*[[Comparison of web search engines|Comparison of search engines]]

==References==
{{reflist}}

== External links ==
* {{official website|http://www.webcrawler.com/}}

{{Web search engines}}

{{DEFAULTSORT:Webcrawler}}
[[Category:Internet search engines]]
[[Category:Metasearch engines]]
[[Category:Internet properties established in 1994]]

[[pl:WebCrawler]]