A search engine or search service is a program designed to help find information stored on a computer system such as the World Wide Web, inside a corporate or proprietary network or a personal computer. The search engine allows one to ask for content meeting specific criteria (typically those containing a given word or phrase) and retrieves a list of references that match those criteria. Search engines use regularly updated indexes to operate quickly and efficiently.
Without further qualification, search engine usually refers to a Web search engine, which searches for information on the public Web. Other kinds of search engine are enterprise search engines, which search on intranets, personal search engines, which search individual personal computers, and mobile search engines. However, while different selection and relevance criteria may apply in different environments, the user will probably perceive little difference between operations in these.
Some search engines also mine data available in newsgroups, large databases, or open directories like DMOZ.org. Unlike Web directories, which are maintained by human editors, search engines operate algorithmically. Most web sites which call themselves search engines are actually front ends to search engines owned by other companies.
History
The very first tool used for searching on the Internet was Archie. (The name stands for "archive" without the "v", not the character from the 'Archie' comic book series). It was created in 1990 by Alan Emtage, a student at McGill University in Montreal. The program downloaded the directory listings of all the files located on public anonymous FTP (File Transfer Protocol) sites, creating a searchable database of filenames.
While Archie indexed computer files, Gopher indexed plain text documents. Gopher was created in 1991 by Mark McCahill at the University of Minnesota. (The program was named after the school's mascot). Because these were text files, most of the Gopher sites became Web sites after the creation of the World Wide Web.
Powerset joins Live Search livesearch Tue, 01 Jul 2008 10:30:00 -0000 We're excited to announce that we've reached an agreement to acquire Powerset, a San Francisco-based search and natural language company.
Powerset will join our core Search Relevance team, remaining intact in San Francisco. Powerset brings with it natural language technology that nicely complements other natural language processing technologies we have in Microsoft Research.
More importantly, Powerset brings to Live Search a set of talented engineers and computational linguists in downtown San Francisco. This is a great team with a wide range of experience from other search engines and research organizations like PARC (formerly Xerox PARC).
We're buying Powerset first and foremost because we're impressed with the people there. Powerset CTO and cofounder Barney Pell is a visionary and incredible evangelist. When he introduced our senior engineers to some of the most senior people at Powerset — Search engineers and computational linguists like Tim Converse, Chad Walters, Scott Prevost, Lorenzo Thione, and Ron Kaplan — we came away impressed by their smarts, their experience, their passion for search, and a shared vision.
That shared vision is to take Search to the next level by adding understanding of the intent and meaning behind the words in searches and webpages.
We know today that roughly a third of searches don't get answered on the first search and first click. Usually searchers find the information they want eventually, but that often requires multiple searches or clicks on multiple search results. Two specific problems are the most common reasons for this:
Differences in phrasing or context between a user's search and the way the same information is expressed on webpages. Search engines don't understand today that "shrub" and "tree" are similar concepts. We don't understand that "cancer" sometimes refers to a disease and sometimes refers to a horoscope and when a query or a webpage refers to which.
Lack of clarity in the descriptions for each webpage in the search results. Sometimes a result looks relevant from its short description on the results page but turns out to be not so relevant when you visit the actual page. As a result, searchers frequently click results and then rapidly click back when they realize they aren't what they're looking for.
These problems exist because search engines today primarily match words in a search to words on a webpage. We can solve these problems by working to understand the intent behind each search and the concepts and meaning embedded in a webpage. Doing so, we can innovate in the quality of the search results, in the flexibility with which searchers can phrase their queries, and in the search user experience. We will use knowledge extracted from webpages to improve the result descriptions and provide new tools to help customers search better.
Working with our existing Search team and other Microsoft teams that focus on natural language, Powerset will help us address all of those problems and opportunities.
We're looking to add even more talented engineers to the San Francisco team to accelerate our shared progress. If you're interested in joining the team, drop us a line.
We'll have more to say about the things we're doing in understanding searches and webpages through natural language technology in the coming months. In the meantime, please join me in welcoming Powerset to Microsoft!
Satya Nadella, Senior Vice President, Search, Portal, and Advertising
See also: Microsoft to acquire Powerset Live Search cashback now live on eBay livesearch Thu, 12 Jun 2008 14:42:00 -0000 Now all of eBay's "Buy it Now" offers will be eligible for cashback rewards. With eBay's expansive catalog of products, from jewelry to electronics, you'll start to see more cashback ads appearing in our search results. That means more ways for you to save.
You'll see three primary differences between this user experience and other cashback user experiences:
There's an eBay ad with a cashback gleam (the cashback icon in the image here).
Instead of going into the Live Search cashback experience, you now go directly to the advertiser's website, which in this case is eBay.
The cashback gleam follows you throughout your eBay shopping experience. This is good continuity for the consumer and something we want to do more of, but it takes a bit of work on the advertiser side to enable this.
We want to learn from two experiences in the cashback program. Depending on customer and advertiser feedback, we'll make the necessary changes to deliver the best user experience over time.
Destination site experience This is the experience that went live last month. Consumers research a product category on Live Search and then click a Live Search cashback ad to head over to cashback for the best deal. We've had a lot of feedback that we should do a better job integrating our product research capabilities with our cashback experience. So we'll work hard to do that over the next few releases.
Direct-to-merchant experience The other experience we envisioned for cashback would integrate directly with advertisers from the search results page. In this experience, advertiser's ads will appear with a cashback gleam. When shoppers click the gleam in the ad, they head directly to the advertiser's site. Obviously, this streamlines a bit of the purchasing process but requires some custom development on the partner website to enable it. We took this approach with eBay and are exploring it with other advertisers as well.
We're excited to offer consumers more money-saving cashback rewards with eBay and look forward to hearing from you on how best to implement new features and functionality to the cashback experience.
Paul Dillon, Director, Commercial Search Helping you Search AND Give livesearch Wed, 04 Jun 2008 08:00:00 -0000 One of the most exciting developments we've had in the last year was the success of our inaugural Search and Give program where your queries helped to raise more than $250,000 for local schools and non-profits. Over the course of the year the program generated cash donations for more than 20,000 organizations ranging from schools to non-profits such as Doctors Without Borders and ASPCA (The American Society for the Prevention of Cruelty to Animals). Today we've launched the latest version of the program that includes a new, easy-to-use design with enhanced features and more flexible donation capabilities.
Here's how it works
By signing up at http://www.searchandgive.com/, consumers can start donating one cent per search to more than 100,000 schools and 900,000 non-profit organizations worldwide every time they use the Live Search to find whatever interests them. People can also convert tickets they've earned playing games on Microsoft's Live Search Club, at http://www.club.live.com/, into donations for those same schools or charities.
What's new
More money donated You can earn one cent per search for your designated charity by just looking for things online — up to 500 searches per person per month.
More than 1 million organizations We've grown our list of eligible organizations to more than one million. We've also made it easier to find them with a new UI that allows for faster lookup by name or non-profit ID number.
Watch your community grow Track your donations, total donations, and total number of contributors for your chosen organization.
Tell a friend Search and Give brings communities together around causes that are important to them. People can now send instant emails to friends and family from within the tool letting them know how they too can make a difference.
We need your help
This program is powered by people through word-of-mouth or viral distribution. We're asking for your help to make this program an even greater success by sharing this information with your friends and family and recruiting them to start searching at http://www.searchandgive.com. If one person can earn $60 per year for their favorite charity just by searching the Web with Search and Give, imagine the power of 10 people = $600 per year; imagine the power of 25 people = $1,500 per year; imagine the power of 100 people = $6000 per year. Together, we can make every search count.
In the end, we hope this program will grow exponentially so we can redistribute more of the dollars coming into search to the local schools and charities that YOU are passionate about supporting. We hope you enjoy the new features and stay tuned as we start showcasing some of the best "Search and Give" campaigns you all have created!
Christine Andrews, Product Manager, Search and Give Managing search engine access to your content livesearch Tue, 03 Jun 2008 08:00:00 -0000 We at Live Search are pleased to announce another collaboration with Yahoo and Google aimed at making webmasters' lives easier. Webmasters have long used the Robots Exclusion Protocol (REP) to control how search engines access and display their content. The REP offers an easy and efficient way to communicate with search engines, and is currently used by millions of publishers worldwide.
Over the past few years, we have been working with Yahoo and Google to agree on common ways for webmasters to communicate with search engines. Our previous efforts include support for the Sitemaps protocol (see autodiscovery directive and cross-host sitemap submissions for more info). While most search engines already comply with the REP, this is the first time the three major search engines have come together to detail how we actually implement the protocol. This effort makes it easier for webmasters to know how REP directives will be handled by search providers.
You can view the details of how we implement the REP at Documentation for the Robots Exclusion Protocol.
If you are a publisher with feedback or suggestions for future enhancements to the REP, feel free to contact me at nbuggia@microsoft.com. We value your feedback as we continue to evolve Live Search.
Nathan Buggia, Live Search Webmaster team Finding the "page not found" livesearch Tue, 03 Jun 2008 07:00:00 -0000 Has this ever happened to you? You search online for information and click a promising link only to get an error page staring back at you:
These generic 404 error pages leave you stranded, with no option but to click the back button and start over, or just give up altogether — not very helpful.
We thought about this problem and realized that we had all the pieces to offer something much more useful. We could help website owners create error pages that actually suggest help even if the exact page you're looking for isn't available.
The Web Page Error Toolkit is a customizable Web application that extracts keywords from your search and gives you relevant search results in a custom error page.
So while you may not get the page you were originally looking for, maybe you'll find something even better.
You can find more details, including information on how to download the Toolkit, at Customize your 404 error pages with the Web Page Error Toolkit.
Go find the page not found!
Alessandro Catorcini, Lead PM for the Live Search API, and Amy Michaels, Group Product Manager for Live Search Live Search…coming to a toolbar near you livesearch Mon, 02 Jun 2008 07:22:00 -0000 With today’s HP announcement and the recent MSN toolbar release, I want to discuss our recent moves in the toolbar space. Overall, these distribution deals come down to three things:
1. Live Search is ready for primetime
These types of deals require a search engine that can stand on its own, as we all know how easy it is to switch to another search engine. Sure, Live Search has room to improve, but we’re confident that when consumers now try Live Search they’ll get relevant results, differentiated experiences, and a unique value proposition (see cashback).
2. Distribution can help with awareness and preference
We’re building a stronger consumer connection that starts with awareness and ends with preference. To be successful, we not only need to care about traditional consumer marketing campaigns to drive awareness. We also need to care about how other channels can help spread the word. We recognize that awareness for Live Search is low and that to crack into the consumer’s consciousness we need to be in front of them in more ways than one. Case in point: check out our new Search Share Facebook app.
3. Enabling partners is good for everyone
We need to provide publishers and our partners, like HP, with great tools and platforms to help them distribute content and reach their customers in new ways. The toolbar platform we have created exemplifies how we’re looking to extend customer service, brands, and content through great new experiences. With this platform we will be able to quickly build a branded feature and content-rich toolbar for HP’s entire line of US-based consumer PCs with just a few easy customizations. When the toolbar is released, HP will offer easy access to their online services like Snapfish and customer support, and bring their brand to life through Silverlight. Now imagine that type of customization for anyone and everyone on the Internet.
The HP toolbar highlights our excitement about the new developments we’re working on. Let us know what you think of the new toolbars.
Mikko Ollila, Product Manager, Live Search Partner Ecosystem Wikipedia gets big livesearch Fri, 30 May 2008 11:55:00 -0000 Check it out:
We realize that often you just need to get a sense of what your query is about. Wikipedia is great for that — you can learn enough from the first paragraph of a Wikipedia article to start you out on the right path.
For Wikipedia results, we now show a good portion of the first paragraph and a few links from the table of contents. You can see more about the topic right there and see what else the article offers.
We hope you learn more, faster with our expanded Wikipedia descriptions. Let us know what you think.
Kemp Peterson, Program Manager, Live Search Book search winding down livesearch Fri, 23 May 2008 06:45:00 -0000 Today we informed our partners that we are ending the Live Search Books and Live Search Academic projects and that both sites will be taken down next week. Books and scholarly publications will continue to be integrated into our Search results, but not through separate indexes.
This also means that we are winding down our digitization initiatives, including our library scanning and our in-copyright book programs. We recognize that this decision comes as disappointing news to our partners, the publishing and academic communities, and Live Search users.
Given the evolution of the Web and our strategy, we believe the next generation of search is about the development of an underlying, sustainable business model for the search engine, consumer, and content partner. For example, this past Wednesday we announced our strategy to focus on verticals with high commercial intent, such as travel, and offer users cash back on their purchases from our advertisers. With Live Search Books and Live Search Academic, we digitized 750,000 books and indexed 80 million journal articles. Based on our experience, we foresee that the best way for a search engine to make book content available will be by crawling content repositories created by book publishers and libraries. With our investments, the technology to create these repositories is now available at lower costs for those with the commercial interest or public mandate to digitize book content. We will continue to track the evolution of the industry and evaluate future opportunities.
As we wind down Live Search Books, we are reaching out to participating publishers and libraries. We are encouraging libraries to build on the platform we developed with Kirtas, the Internet Archive, CCS, and others to create digital archives available to library users and search engines.
In partnership with Ingram Digital Group, we are also reaching out to participating publishers with information about new marketing and sales opportunities designed to help them derive ongoing benefits from their participation in the Live Search Books Publisher Program.
We have learned a tremendous amount from our experience and believe this decision, while a hard one, can serve as a catalyst for more sustainable strategies. To that end, we intend to provide publishers with digital copies of their scanned books. We are also removing our contractual restrictions placed on the digitized library content and making the scanning equipment available to our digitization partners and libraries to continue digitization programs. We hope that our investments will help increase the discoverability of all the valuable content that resides in the world of books and scholarly publications.
Satya NadellaSenior vice president search, portal and advertising You talk. We listen. livesearch Thu, 15 May 2008 13:05:00 -0000 In our last release of Live Search for Windows Mobile we snuck in a little feature that allows users to send feedback directly to the engineering team. Since then, we've received tons of messages from users, many of which have been requests for new functionality. So what's the moral of the story? Ask, and you shall receive. Here's the list of some of the cool stuff we've built in response to user feedback.
Weather By far the most often-requested feature: get current weather conditions and a four-day forecast by clicking the Weather icon.
Web search Search the Web, news, images, and more just by clicking the Web icon.
Increased traffic coverage Piggy-backing on great work done by Live Search Maps, view up-to-the-minute traffic info for more cities, like Dallas, Indianapolis, and Baltimore.
Bluetooth headset support Now you can use the speech recognition feature with your Bluetooth headset (available on select devices).
Map a contact Open up a contact, press Menu, and then press Show On Map to view the contact on a map.Delete a recent location Simple, yet effective. Click the label showing your current location to display the list of recent locations, scroll to the item you want to delete, press Menu, and then press Delete.
These were the most common asks, but there were also a lot of requests for custom, personalized content. And since the customer is always right, we’ve built a way for you to access the content you care about when you’re on the go.
Collections Looking for pubs that offer WiFi in Seattle? Want to find a dog park in San Francisco? We can help! Search community-generated content such as Virtual Earth Collections and Google KML and find what you're looking for.
And we’re still listening. Send us more feedback at lsmobile@microsoft.com.
Gary Voronel, Program Manager, Live Search for mobile
ACM Queue: Why Writing Your Own Search Engine is Hard - Anna Patterson explains the problems of search engine design in practical terms in this lengthy article.
Meta Description: [ - Why Writing Your Own Search Engine is Hard: So you have a grand idea; are you ready for the execution? ]
Amfibi - General search engine with an uncluttered interface in a choice of Catalan, English, French, and Spanish. Also has a web directory using the Open Directory.
Meta Description: [ General search engine which includes its own index of websites, a directory (provided by the Open Directory Project) and a meta search feature for searching multiple search engines at once. Also includes a free traffic exchange for webmasters. ]
Ask - Offers search for web sites, news, weather reports, pictures, stocks, maps and directions, people or products. Also has a dictionary search.
Commercial Alert Files Complaint Against Search Engines for Deceptive Ads - A complaint filed with the United States Federal Trade Commission by Ralph Nader's Commercial Alert service, which charges the major search engines of inserting advertisements in search engine results without clear disclosure that the ads are ads.
Creative Commons Search - Powered by Nutch, it searches content which can be re-used (for somes uses) without having to pay or ask permission.
Dumbfind - Offers the option to specify a general topic as well as specific keywords for more precise results.
Entireweb - The English-language version of a Swedish search engine with a clean interface and large database.
Meta Description: [ Entireweb makes searching easy with a user friendly search engine. ]
FactBites.com - A cross between a search engine and an encyclopedia. The results return complete, informative sentences about the search topic. Related topics are suggested.
FirstGov Search - The U.S. Government's official search engine. Find information from federal, state, local, tribal and territorial governments. Meta-search special government databases.
Gigablast - A search engine with a clean interface, advanced search options including sorting by date, and a real time url submission feature. Includes PDF documents in its index.
Meta Description: [ A powerful, new search engine that does real-time indexing. ]
500Infosniff - A simple search engine from Cybersoft Infotek with advertising.
Meta Description: [ Infosniff search engine allows you to search for your favourtie things at once,
returning the most comprehensive and relevant results fast. Unleash the power of Infosniff Search engine! ]
Mojeek - Search engine with a simple interface and no advertising.
MSN Search - Microsoft provides search of the web, news, images and its own encyclopedia, Encarta. Also offers desktop search via a toolbar.
Meta Description: [ Find exactly what you are looking for – FAST! With Live Search. ]
NPR : The Search Engine Wars - In a five-part series, NPR's Rick Karr takes a look at the business of search engines.
Meta Description: [ Companies like Google, Yahoo and Microsoft are battling to be the main gateway to the Internet. These companies have gained unprecedented influence over what people see and learn, and have created a industry with brave new rules for business. In a five-part series, NPR's Rick Karr takes a look at... ]
Objects Search - An uncluttered interface offers options to search the web, news and blogs, includes cached pages and 'anchors', which are titles and descriptions from the Open Directory.
Meta Description: [ Search engine that allows users to search information, news,videos and images across the Web ]
500Pipeline - A clustering engine with a streamlined interface, which presents all results in one column and clusters by topic in another.
Scrub The Web - Search engine with sponsored links at the top of results. Also offering a meta tag builder and analyzer.
Meta Description: [ STW search engine provides quality search engine results for Internet searches. Submit your URL to Scrub The Web today! ]
Search Engines and Editorial Integrity - Article from the USC Online Journalism Review examines the issue of major search engines adopting deceptive, misleading advertising practices at the expense of editorial integrity and their users.
SearchHippo - Indexes the sites listed in the major Internet directories. Offers free web services, including a search toolbar, and code for webmasters to provide search boxes on their sites.
Meta Description: [ SearchHippo.com provides a crisp and clean
web search experience. ]
The Evolution of Web Searching - David Green's paper from Online Information Review explores the development of search engine technologies.
Meta Description: [ David Green's paper on 'The evolution of web searching' was voted the 'Most Outstanding paper' by the editorial advisory board of the Online Information Review journal. ]
ThreeMice - Natural-language answer engine, which queries the internet in real time, and will serve up full text from selected sites.
Meta Description: [ ThreeMice.com Natural Language Answer Engine ]
Ulysseek - A straightforward search engine with a simple interface.
Meta Description: [ A straightforward internet search engine with a simple interface. ]
Wotbox - Search engine with country specific searching. Options to display country flags, and include preview and translation features. Sponsored links appear in panels separate from the main results.
Ask Mobile - Offers search for web sites, news, weather reports, pictures, stocks, maps and directions, people or products. Also has a dictionary search.
Lycos Mobile - Mobile search powered by Windows Live.
MSN Search - Microsoft provides search of the web, news, images and its own encyclopedia.
Meta Description: [ Find exactly what you are looking for – FAST! With Live Search. ]
Yahoo: Mobile - Yahoo mobile seach, news and services.
. The search engines will become your friend. Search engines pay attention to reciprocal links (exchanging links with ...