Star Computer Network

Posts Tagged ‘Engines’

Search engines use something called an algorithm to determine which pages will be displayed higher for the keywords entered into the search engine. An algorithm is a step-by-step mathematical formula that produces results based on the data the search engine has compiled. Imagine an algorithm as a number-crunching machine. The algorithm is like a complicated formula that sorts information into a more useable format.           

After the user has entered all this data into the algorithm “machine,” sorted information begins coming out the other side. In this case, the data is a ranking of websites indexed by the search engine based on the keyword the user has already entered into the algorithm “machine.” By using a complicated formula, the search engines can determine the best quality websites online without having to rely on human editors to physically visit each site and determine that site’s quality. The job of the search engine is to provide the best searching experience for their users. To ensure that users keep coming back, they want to make sure they produce results that are relevant. This is an important part of SEO that we’ll be discussing at great length: relevancy.           

The algorithm helps determine which pages are most relevant. For example, if a user goes to a search engine and types in “snow tires,” the search engine uses their technology to determine which pages most closely relate to snow tires. The job of the algorithm becomes finding the best information related to a particular keyword. The search engine then displays the results in order of relevancy.

Remember that there are many factors that contribute to how search engines work, and some of them are closely-guarded secrets. Google, the most popular search engine, keeps parts of their operation under wraps to prevent fraud and dishonesty. 

Web Spiders and Bots 

In the old days of the Internet, a website would often have to be submitted directly to a search engine in order for listing on that search engine. This was often the only way to gain a listing on a site like Yahoo! because directory listing sites relied on human editors to select the topics and contents of directory listings. Some search engines had rudimentary ways to search pages on the Internet. The problem is that search engines hadn’t yet developed the ability to filter results very well. By using a search engine, the user couldn’t be assured that he or she would even come upon pages related to the keywords typed into the search engine.           

Today, however, no submission is necessary in order to be listed in search engines. Because technology has evolved, search engine listings are much more refined and certainly more user-friendly. Today’s search engines spend lots of time, money and research in order to provide a searching service that will give the user the best web experience possible. In fact, the search engine business model relies upon this fact. The more people that visit a search engine, the more web traffic that site will receive. The more web traffic, the more people will see advertising on that search engine. Since search engines frequently rely upon advertising to support the site, this works out for everyone. The user gets better search results and the search engine gets more users to their site.           

How do search engines provide an improved web experience for their users? One technology development that has improved the typical web search is the spider or bot. A web spider is a program run by the search engine. Instead of relying on the slow speed of a human editor, search engines can now rely on computer programs that never need to stop searching websites on the Internet. The sole purpose of these programs is to “crawl” the web all day, every day. These computer programs are, in essence, looking over every single website they encounter. These programs search through websites, checking links, and examining keywords. The spiders will also look at HTML elements within the pages like page descriptions, meta tags, and page titles. Basically, these spiders are used to compile a large amount of data from sites on the web. Here is a list of elements within a website that are scanned by search engine spiders: 

Text within the website Links within the website Page descriptions embedded within HTML Keywords embedded within HTML Photos and photo descriptions and alternate text 

All of this information is then compiled into a series of databases maintained by the search engine. This database is what eventually helps search engine users find the information they want. When a user goes to a search engine and searches for the term “kittens,” the search engine will then go to the database of all indexed information. The search engine will look for the keyword “kittens” within all of those databases, and then presents this information to the user. In order to give the best results to the user, the search engine will also sort this data by relevancy, putting the sites that are most related to the keyword or words at the top of the listing. By “indexing” all these sites, including their links and keywords, search engines can then provide a rank for each of these pages.

Search engines are an important part of the growing online economy, mainly because they are becoming an important part of how people use the Internet. Just 10 short years ago, Internet users weren’t using search engines very often, but favored index sites like Yahoo!, and for good reason. Index sites generally had human editors that chose websites based on topic and posted them in a hierarchical way where there were main categories and sub categories that got more specific. For example, under the heading “Arts and Humanities,” were more subtopics like “Artists, Art History, Visual Arts” and more. When users clicked on any of these subtopics, they would see even more subtopics. The directory of sites by topic allowed the user to find the exact topics they wanted and verity, with the help of human editors, that those pages contained good information. The sites listed were usually reliably accurate in regards to what the user was seeking and still exist to this day.           

On the other hand, search engine technology was far behind today’s standards ten years ago. By using older search engines, users would likely come up with search results unrelated to their keywords. Back then, the technology hadn’t evolved to accurately find and filter search engine results. By simply using some shady techniques, a webmaster could get his or her site displayed for totally unrelated keywords. For example, a site selling herbal supplements might be able to get their site to show up in search engines under search terms unrelated like “education” or “home mortgage.” The result was that search engines were fairly unreliable and not as many people used them. Today more than 85% of web users use search engines to find the pages they want.           

Currently there are several ways in which users can find the web pages they want to view:

First, they can use search engines. More than eighty-five percent of web surfers use this method in order to find what they’re looking for.
Users can go to websites they’ve heard about on TV, radio, and print advertising. Examples include traditional advertising that lists a website as part of normal advertising, like listing fedex.com in a Fed Ex ad. Another example is an advertisement for a specific website like Monster.com or eHarmony.com.
Users can click on ads they see online. Many sites display advertising on them, and users can follow those ads to find the sites they want to visit.
Users can find websites they learned about from viral marketing techniques, a topic that will be discussed in greater detail in a later chapter. This includes links that have been sent to users by a friend or recommended by someone they know, like word of mouth advertising.

By far, the most popular search engine is Google. There are several reasons for this. First of all, Google has invested a lot of money in developing algorithm technology that will give users the listings they want. Google has also developed a unique style, including text-only advertising. This makes for pages that load quickly and don’t contain offensive or obtrusive pictures. Because customers like this type of search experience, they’ve begun to overwhelmingly prefer Google as a search engine of choice.           

There are a few search engines that are based on Google. These sites use the Google technology in order to provide their searches. Some examples of these are AOL Search and MySpace search. These search engines work in almost the same way as Google at retrieving information, even if they don’t look the same. 

In the United States, Google takes up almost half of the search engine market. That means, for every search done on a search engine in the United States, nearly one in two is done with Google. Following Google is Yahoo! Search with about 24% of searches, and MSN with about 10%. AOL rounds out this list with about 6% of searches. Worldwide, Google takes an even bigger share of the market with up to 56% of worldwide searches.           

While Google does dominate both the domestic and foreign search market, there are other general search engines out there that aim to search the web keywords entered. There are many sites that are based on Yahoo! technology, which is the second most popular search engine domestically and worldwide. These sites include AltaVista and AllthWeb.           

There are also search engines based on MSN technology. While Windows Live (MSN) does have a smaller corner of the market, their technology is used by other search engines like A9. Ask also has several search engines based on their technology, including iWon and Lycos. Then there are whole subcategories of search engines that look at topics and information that is much more specific. After all, if a user is searching for a job, would he or she want to search on Google or on a site that only has job postings? Below are some common search engine types and for what information they search.           

Job search engines: Many people use job search engine sites to look for work or career-related information. These sites generally let users choose listings based on qualifying data like location, salary, and relevant keywords like “sales” or “management.” Some notable job search engines are Monster, HotJobs (which is owned by Yahoo!), and CareerBuilder.           

Multimedia search engines: While some people may want text-based information, some people may want to find pictures, videos, or sound files instead. High speed internet is growing and more people have access to more bandwidth as prices for this service have dropped. This increasing ability to handle information makes for more possibilities for multimedia. Many search engines also allow users to search for multimedia within their search engine.           

Google, for example, has an image and video search option. AltaVista allows users to search for images, videos, and sound files like MP3s. Ask allows users to search for images as well as blog posts. There are some sites specifically for multimedia searching. Notable examples are Blinkx which lets users search for video, FindSounds, which searches sound files, PodScope, which searches for video and sound files, and PicSearch, which lets the user search for pictures.           

Comparison shopping search engines: These search engines allow users to find the products they want to buy, then gives them tools to find the best products for them. These sites will generally let users compare prices on specific items from several different online stores. Some even calculate shipping and allow users to give feedback on online merchants to comment on their goods and service. Many will also recommend similar products, product accessories, or have other shopping-based tools that aim to serve the online shopper.           

Notable comparison shopping sites include Google Product Search, formerly known as Froogle. Google combined their searching strength with comparison shopping in order to create this search engine. Kelkoo is a Yahoo! company that specializes in online buying worldwide. ShopWiki allows users to browse for products by category and specialize their product searches. Other companies include MySimon and Shopzilla.           

There are many more kinds of search engines out there. Search engines can get fairly specialized. As long as there is a kind of data to be retrieved, someone will create a search engine to fill that demand. That way medical personnel, businesspeople, lawyers, accountants, and people in real estate all have specialized search engines devoted to them to help them find the information they need.