Leveraging Search Technologi

165

CHAPTER 6

Search, Semantic, and Recommendation Technology

C H A P T E R O U T L I N E

Case 6.1 Opening Case: Mint.com Uses Search Technology to Rank Above Established Competitors

6.1 Using Search Technology for Business Success

6.2 Organic Search and Search Engine Optimization

6.3 Pay-Per-Click and Paid Search Strategies

6.4 A Search for Meaning—Semantic Technology

6.5 Recommendation Engines

Case 6.2 Business Case: Deciding What to Watch— Video Recommendations at Netflix

Case 6.3 Video Case: Power Searching with Google

L E A R N I N G O B J E C T I V E S

6.1 Describe how search engines work and identify ways that businesses gain competitive advantage by using search technology effectively.

6.2 Explain how to improve website ranking on search results pages by optimizing website design and creating useful content.

6.3 Describe how companies manage paid search advertising campaigns to increase awareness and drive sales volume.

6.4 Describe how semantic Web technology enhances the accuracy of search engines results and how businesses can optimize their websites to take advantage of this emerging technology.

6.5 Describe how recommendation engines are used to enhance user experience and increase sales on e-commerce websites.

Introduction Every day, over 1.5 billion people around the world use what seems to be a simple tool to find information online—a search engine. We sometimes take for granted that behind a relatively simple user interface, an increasingly complex set of search engine technologies are at work,

166 C H A P T E R 6 Search, Semantic, and Recommendation Technology

helping us find the information we need to do our jobs, conduct research, locate product reviews, or find information about the television shows we watch. Because most search engine services are free, people are not generally aware that “Search” has become a multibillion- dollar- a-year business. More importantly, the way search engines work and how they rank-order the links displayed when we conduct a search have huge implications for millions of other busi- nesses. Because consumers typically don’t look past the first few pages of search results, hav- ing your business appear at the top of a search results page can make a big difference in how much traffic your website gets. In this chapter, you will read about how search engines work and how they determine which websites are listed at the top of search results. You will also read about the strategies companies use to increase their presence on search results pages includ- ing search engine optimization (SEO) and pay-per-click (PPC) advertising.

Semantic technologies are increasingly being used by search engines to understand Web page content. In this chapter you will read about the ways that search engines are using semantic technology to improve performance, increasing relevant pages and decreasing the number of irrelevant pages that appear in search results.

Finally, you will read about recommendation engines. These tools attempt to anticipate online information you might be interested in. Netflix uses recommendation engines to sug- gest movies you might like to watch and news organizations use them to recommend stories you might want to read on their websites. Amazon credits its recommendation technology for increasing sales by suggesting products that customers might want to buy.

Business managers need to understand search and recommendation technologies because their influence in directing potential consumers to business websites is already significant and expected to grow substantially in the future.

Case 6.1 Opening Case

Mint.com Uses Search Technology to Rank Above Established Competitors

Company Overview Mint is a popular, Web-based personal finance service that makes it easy for users to keep track of bank, credit card, and other financial accounts using a computer or mobile device. Customers can also use the service to create budgets and monitor progress toward financial goals. Since it began in 2006, the company has grown rapidly despite competition from more established companies. In 2009, Mint was acquired by Intuit, the maker of TurboTax and Quicken financial soft- ware. Today, over 20 million people use Mint’s free financial manage- ment service (Table 6.1).

The Business Challenge In the months leading up to the 2006 launch of Mint.com, a personal finance service, the leadership team faced a formidable challenge: How

to establish name awareness and brand equity in a market filled with established competitors, without spending a lot of money? Mint knew it would be competing in a market space already populated by familiar brands like Quicken Online and Microsoft Money Online. Since online platforms and communication channels tend to favor existing compa- nies with established audiences and reputations, the team knew they had to come up with a powerful strategy for overcoming the estab- lished brands.

Mint’s Content Marketing Strategy As a Web-based service, it was critical for Mint.com to rank high on search engine results pages (SERPs) when consumers used sites like Google or Bing to find information about personal finance services and related topics. Consumers are more likely to visit websites that appear at the top of SERPs. While the service was still in the beta (trial) stage of development, workers at Mint developed an aggressive strategy to optimize the brand’s ranking on popular search engines. Their strat- egy involved building the company’s Web presence on criteria used by

G eb

er 86

/E +

/G et

ty Im

ag es

Ia nD

ag na

ll C

om p

ut in

g /A

la m

y S

to ck

P ho

to

fil o/

D ig

ita lV

is io

n Ve

ct or

s/ G

et ty

Im ag

es

Introduction 167

TA B L E 6 . 1 Opening Case Overview

Company Mint

History Mint was launched in 2006 as a free, Web-based personal finance app by founder Aaron Patzer. In 2009, the company was acquired by the financial software company Intuit.

Growth Within two years of launch, Mint claimed over 1.5 million users. By 2012, the company claimed 10 million users and by 2016, the number of users rose to over 20 million.

Product lines Mint’s original service allowed users to track balances and transactions on credit card, investment, and bank accounts as well as to create budgets and establish financial goals. In addition, Mint now offers users a bill pay service and credit score monitoring.

Social technology Prior to the release of its flagship personal finance app, Mint created a large following of prospective users with MintLife, a blog that offered valuable advice targeted to young professionals.

Search technology Mint utilized an aggressive SEO strategy to rank highly on search results pages. Specific actions included the following: • Creation of useful personal finance content on its blog, MintLife • Use of targeted keywords in website content • Established audiences on popular social media sites such as Facebook and Twitter • Used various strategies, including sponsorship of third-party blogs, to generate links (or “backlinks”)

to Mint.com from other websites

Website Mint.com

search engines to determine SERP ranking. The strategy focused on the following:

• Increasing the number of other websites that linked back to Mint’s website (called “backlinks”)

• Creating interesting and useful content about personal finance topics that prospective customers would find helpful

• Identifying keywords and phrases used by prospective customers when searching for personal financial services, and creatively inserting these words and phrases into website content

• Regularly updating and adding to their collection of personal finance content

• Establishing a presence on popular social media sites, expanding their audience on those sites, and encouraging the audience to share links to Mint’s website content

Months prior to the launch of its personal finance service, Mint rolled out a personal finance blog called MintLife and quickly devel- oped a reputation for providing helpful financial advice targeted to young professionals. Blog posts on MintLife were creatively seeded with keywords and phrases the team had identified as likely to be used by prospective customers when conducting Internet searches for financial services. Mint also created landing pages on their web- site containing content optimized for keywords and phrases related to financial services. As search engines tracked this content, Mint began to lay a foundation for eventually being viewed by search engines as a credible authority for personal finance topics. New posts were regularly added to the blog, which further enhanced Mint’s ranking since search engines favor websites with lots of content (content depth) and regular updates. To further establish its position as a useful and authoritative site, Mint sponsored sev- eral third-party blogs and cultivated relationships with authors of established finance and money management blogs. Mint’s founder, Aaron Patzer, gave hundreds of interviews, resulting in print media and online articles about the start-up company. These and other actions resulted in more third-party websites posting links back to

Mint.com. These “backlinks” were tracked by search engines and resulted in additional increases to the site’s ranking on SERPs. Popu- lar search engines also track a company’s presence on social media and the extent to which users share information about the company and its products. Mint’s blog featured content in a variety of inter- esting formats: videos, podcasts, infographics, and so on. Users on social news sites like Reddit.com frequently shared and “upvoted” interesting infographics from Mint’s blog. Links to other types of blog content were shared by users on Facebook, Twitter, and other social media platforms. As a result, Mint’s expanding audiences on Face- book, Twitter, and other social media platforms further enhanced the new company’s SERP ranking.

Results In 2007, Mint launched its new financial services website into a market where it already enjoyed considerable name recognition and aware- ness. Within 2 years, the service acquired 1.5 million users and was purchased by Intuit for $170 million. The company continued its suc- cessful content marketing strategy, climbing to 10 million users in 2012 and over 20 million users today.

Questions 1. Why did Mint invest the time and effort to publish a financial

services blog almost two years before the launch of its service?

2. How did Mint use social media sites to increase its ranking on the search results pages of popular search engines?

3. Why did Mint use keywords and phrases associated with personal finance when creating content for its blog?

4. Why did Mint put so much emphasis on improving the rank of its website on SERPs?

5. Why did Mint use infographics, videos, and other types of rich media in its financial services blog?

Sources: Compiled from Sukhraj (2015), Bulygo (2013a), Obi-Azubuike (2016), Prince (2016), Greene (2016).

168 C H A P T E R 6 Search, Semantic, and Recommendation Technology

6.1 Using Search Technology for Business Success Search engines like Google, Bing, Yahoo, and others have traditionally been regarded as a consumer technology. But search technology has become an important business tool with many different uses and applications. In this section, you will learn how search engines work and the role they play in generating revenue and consumer awareness for organizations. You will also discover how businesses use enterprise search technology to unlock hidden content with their organizations. Finally, you will read about how search and Internet technology is evolving to provide more accurate and useful results.

How Search Engines Work People use the word search engine to refer to many different kinds of information retrieval (IR) services that find content on the World Wide Web. These services vary in significant ways. Understanding how these services differ can improve the quality of results obtained when con- ducting a search for online information. Listed below is a brief description of different IR ser- vices for finding Web content:

• Crawler search engines rely on sophisticated computer programs called spiders, crawlers, or bots that surf the Internet, locating Web pages, links, and other content that are then stored in the search engine’s page repository. The most popular commercial search engines, Google and Bing, are based on crawler technology.

• Web directories list Web pages organized into hierarchical categories. Originally, Web direc- tories were created and maintained by human editors who decided how a website would be categorized. Today, many Web directories use technology to automate new website list- ings. Web directories are typically classified as “general” directories that cover a wide-range topical categories, or “niche” directories that focus on a narrow range of topics. Examples of popular general directories include Best of the Web, JoeAnt, and LookSmart. Wikipedia maintains a list of general and niche Web directories.

• Hybrid search engines combine the results of a directory created by humans and results from a crawler search engine, with the goal of providing both accuracy and broad coverage of the Internet.

• Meta-search engines compile results from other search engines. For instance, Dogpile generates listings by combining results from Google and Yahoo.

• Semantic search engines are designed to locate information based on the nature and meaning of Web content, not simple keyword matches. The goal of these search engines is to dramatically increase the accuracy and usefulness of search results. Semantic search engines are described in more detail in Section 6.4.

Web Directories Before crawler search engines became the dominant method for finding Web content, people relied on directories created by human editors to help them find information. Web directories are typically organized by categories (for instance, see the categories listed on Best of the Web). Web page content is usually reviewed by directory editors prior to its listing in a category to make sure it is appropriate. This reduces the number of irrelevant links generated in a search. The review process, however, is very slow compared to the automated process used by crawlers (described in the following section). As a result, the listings in a Web directory represent a relatively small portion of the Web. Directories are particularly useful when conducting searches on a narrow topic, such as identifying suppliers of a specific type of product or service. Companies who need

Search engine an application for locating Web pages or other content (e.g., documents, media files) on a computer network. Popular Web-based search engines include Google, Bing, and Yahoo.

Spiders also known as crawlers, Web bots, or simply “bots,” spiders are small computer programs designed to perform automated, repetitive tasks over the Internet. They are used by search engines for scanning Web pages and returning information to be stored in a page repository.

Using Search Technology for Business Success 169

to identify vendors or suppliers may consult a niche Web directory created for just this purpose. For example, see the Web directory at business.com.

How Crawler Search Engines Work The two most popular commercial search engines on the Web, Google and Bing, are based on crawler technology. Behind the relatively simple interfaces of these two powerful search engines, a great deal of complex technology is at work (Figure  6.1). Because modern search engines use proprietary technology in the race to stay ahead of competitors, it is not possible to tell exactly how they decide what websites will appear in a SERP. While they each produce different results, it is possible to describe the basic process shared by most crawler search engines. The following description is based on publications by Grehan (2002) and Oak (2008).

1. The crawler control module assigns Web page URLs to programs called spiders or bots. The spider downloads these Web pages into a page repository and scans them for links. The links are transferred to the crawler control module and used to determine where the spi- ders will be sent in the future. (Most search engines also allow Web masters to submit URLs, requesting that their websites be scanned so they will appear in search results. These requests are added to the crawler control queue.)

2. The indexer module creates look-up tables by extracting words from the Web pages and recording the URL where they were found. The indexer module also creates an inverted index that helps search engines efficiently locate relevant pages containing keywords used in a search. (See Figure 6.2 for examples of an inverted index.)

3. The collection analysis module creates utility indexes that aid in providing search results. The utility indexes contain information about things such as how many pages are in a web- site, the geographic location of the website, number of pictures on a Web page, Web page length, or other site-specific information the search engine may use to determine the rel- evance of a page.

4. The retrieval/ranking module determines the order in which pages are listed in a SERP. The methods by which search engines determine website listing order varies and the spe- cific algorithms they use are often carefully guarded trade secrets. In some cases, a search engine may use hundreds of different criteria to determine which pages appear at the top of a SERP. Google, for instance, claims to use over 200 “clues” to determine how it ranks pages (Google.com, 2014).

Page repository a data structure that stores and manages information from a large number of Web pages, providing a fast and efficient means for accessing and analyzing the information at a later time.

Crawler control module a software program that controls a number of “spiders” responsible for scanning or crawling through information on the Web.

Page Repository

WWW

Text Utility

Indexer Module

Structure

Collection Analysis Module

Surfer–Client QueriesSpiders/ Crawlers

Crawler Control Indexes

URL Submissions

1. URL 2. URL 3. URL

. Query

Formulation

Results

Ranking ..

FIGURE 6.1 Components of crawler search engine (Adapted from Grehan, 2002).

170 C H A P T E R 6 Search, Semantic, and Recommendation Technology

5. Web pages retrieved by the spiders, along with the indexes and ranking information, are stored on large servers (see IT at Work 6.1).

6. The query interface is where users enter words that describe the kind of information they are looking for. The search engine then applies various algorithms to match the query string with information stored in the indexes to determine what pages to display in the SERP.

Each search engine utilizes variations and refinements of the aforementioned steps in an attempt to achieve superior results. The Web search industry is highly competitive and the proprietary advances in search technology used by each company are closely guarded secrets. For instance, even the first step in the process, crawling the Web for content, can vary greatly depending on the strategic goals of the search engine. Some search engines limit the number of pages scanned at each website, seeking instead to use limited computing power and resources to cover as many websites as possible. Other search engines program their spi- ders to scan deep into each website, seeking more complete coverage of each site’s content. Still other search engines direct their spiders to seek out websites that contain certain types of content, such as government sites or shopping (e-commerce) sites. Another decision that search engines make regarding spiders is the amount of resources directed at searching new websites versus devoting resources to exploring previously indexed pages for updates or changes.

One of the many challenges faced by large commercial search engines is storage. In the simplest sense, the crawler approach to search requires a company to store a copy of the Web in large data centers. In addition to the petabytes of storage required to maintain this copy of the Web, the search engine must also store the results of its indexing process and the list of links for future crawls.

Petabyte a unit of measurement for digital data storage. A petabyte is equal to one million gigabytes.

Document ID

1 To the heart, real love always endures.

Though passion may cool, love remains true. True love kindles the passion in my heart.

2

3

Content URL

Search Query:

Page Index

Inverted Index

ID Term Document: Position

1 1:3, 3:7

1:4

1:5, 2:5, 3:2

1:6

1:7

2:7, 3:2

2:1

2:2, 3:5

2:3

2:4

2:6

3:3

3:8

heart

real

love

always

endures

true

though

passion

may

cool

remains

kindles

heart

2

3

4

5

6

7

8

9

10

11

12

13

True love

Results Ranking (based on position)

True love kindles the passion in my heart.

Though passion may cool, love remains true.

Documents with both terms: 2 and 3

FIGURE 6.2 Search engines use inverted indexes to efficiently locate Web content based on search query terms.

Using Search Technology for Business Success 171

IT at Work 6.1

Google Data Centers Not only does Google maintain a copy of the Internet for its search engine services, it is also constantly updating a map of the entire planet for users of its popular Google Earth application. In addition, the company is making a full-text, searchable copy of over 129,864,880 known books, equal to 4 billion pages or 2 trillion words. And then there are applications like Gmail, serving roughly 425 million people and YouTube, where 300 hours of video are uploaded every minute! Add all this up, and Google is facing perhaps the biggest data storage challenge ever. So where does Google store all of these data?

Challenges: Energy, Performance, and Security Information collected by Google is housed on over 1 million servers spread across 12 different facilities worldwide. The facilities are large, factory-like installations containing row upon row of racked and stacked servers. Cooling systems, required to keep servers from overheating, are a significant component of any large data center (Figure  6.3). Google pioneered the software systems that connect the company’s servers and make it possible for various applications to access data stored on the machines. Unlike other companies that purchase servers from outside suppliers, Google builds its own. Based on its experience creating the hardware, software, and facil- ities necessary to power the company on a global scale, Google is recognized as a leader in data center operations.

The company’s data centers, including the servers, are built with energy efficiency, reliability, and performance in mind. As Google is a leading provider of Internet services, its data infrastruc- ture must keep up with growing consumer demand for speedy performance and reliability. A typical Google search delivers millions of pages of results in less than half a second. Consumer expectations for performance have grown so high that waiting more than a few seconds for an e-mail to load or a search to run can cause frustration.

More recently, Google has had to contend with revela- tions that the U.S. National Security Agency (NSA) breached its server network security. This follows cyberattacks in 2010 and 2011 by hackers suspected of being associated with the Chinese government. Protecting company data from criminals is a significant challenge in itself, but Google is understandably

frustrated by the fact that it must now fight off cyber-attacks from two world superpowers, one of which is its own government.

Environmental Impact Industrywide, data centers used 70 billion kilowatt-hours of electricity in 2014, representing a 4% increase from the amount used in 2012. Industrywide, data center energy use and the related environmental impact have become an issue of growing concern. Google is widely recognized as operating some of the most efficient data centers in the world, but many critics are disturbed by the industry’s overall level of energy consumption. According to some estimates, data centers account for about 2% of the world’s energy use and the fast rate of growth is cause for concern (see Figure 6.4). Google has taken an active approach to reducing its environmental footprint. Beginning in 2017, Google will source 100% of its energy needs for offices and data centers from renewable sources. See Google’s data center Web page https://www.google.com/about/ datacenters for additional information.

Google Data Center Statistics • Number of servers worldwide Over 1 million • Number of data centers Nine in North America, one in

South America, two in Asia, and four in Europe

• 2016 Capital investment in data centers Approximately $11 billion

• Data processing volume Over 100 petabytes a day • Average energy efficiency PUE* = 1.12 • Energy use Continual use of about 260 megawatts of

electricity, approximately 0.01% of global energy consumption

• Energy use comparisons Owns about 3% of servers world- wide, but only uses about 1% of data center industry energy

• Renewable energy Claims that 100% percent of its energy use comes from renewable sources

*PUE stands for Power Usage Effectiveness. A PUE of 2.0 means that for every watt of power devoted to computing, an additional watt is spent on cooling, power distribution, and overhead. The Data Center Industry average PUE falls between 1.8 and 1.89.

Sources: Jacobson (2010), Grifantini (2011), Newman (2011), Schneider (2011), Glanz (2011, 2012), Gallagher (2012), Venkatraman (2012), Anthony (2013), Miller (2013), Sverdlik (2016).

B lo

om b

er g

/G et

ty Im

ag es

FIGURE 6.3 Pipes pass through the chiller plant at the Google, Inc., data center in Changhua, Taiwan. Google doubled its spending plan for its new data center in Taiwan to $600 million amid surging demand from Asia for its Gmail and YouTube services.

© a

sh ar

ky u/

S hu

tte rs

to ck

FIGURE 6.4 New, large-scale data centers being constructed for companies like Google, Microsoft, and Facebook house thousands of servers and are creating concern among environmentalists over increases in energy consumption.

172 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Why Search Is Important for Business Search engines have become a part of our everyday life. They are free, easy to use, and become more powerful and effective every day. Most of us take them for granted and are generally unaware of the complex technologies that power these tools. For the average Web user, it may not be vitally important to understand how search technology is evolving. But for business managers, understanding the potential power of search technology is crucial and becoming more important every day. It has long been recognized that access to information is a competitive advantage. Search technology impacts business in each of the following ways:

• Enterprise search—finding information within your organization • Recommendation engines—presenting information to users without requiring them to

conduct an active search • Search engine marketing (SEM)—getting found by consumers on the Web • Web search—finding crucial business information online

Each of these important search technology applications are described in what follows.

Enterprise Search Enterprise search tools are used by employees to search for and retrieve information related to their work in a manner that complies with the organization’s information-sharing and access control policies. Information can come from a variety of sources, including publicly available information, enterprise information (internal records) found in company databases and intranets, as well as information on individual employee computers (Delgado, Renaud, & Krishnamurthy,  2014). Enterprise search tools allow companies to gain competitive advantages by leveraging the value of internal information that would otherwise remain hidden or “siloed.” Information can be inaccessible as a result of incompatible technol- ogies in various units, lack of coordination or cooperation between units, security concerns, and concerns about the cost of making information accessible (Thomas, 2013; Walker, 2014).

In most organizations today, a large portion of employees are “knowledge workers” (e.g., business analysts, marketing managers, purchasing agents, IT managers, etc.). Access to information has a significant impact on their productivity. Enterprise search tools allow workers to extract internal information from databases, intranets, content management sys- tems, files, contracts, policy manuals, and documents to make timely decisions, adding value to the company and enhancing its competitive advantage.

Structured versus unstructured data One of the challenges encountered by devel- opers of enterprise search tools is that information is not always in the same format. Data exist in two formats: structured or unstructured. Structured data can be defined as highly orga- nized information, which is easily searchable using simple search engine algorithms or related procedures. Unstructured data, sometimes called messy data, refers to information that is not organized in a systematic or predefined way. Unstructured data files are also more likely to con- tain inaccuracies or errors. Examples of unstructured data include e-mails, articles, books, and documents. Unstructured data accounts for a majority of all the data present on computers today. Originally, enterprise search tools worked only with structured data. Many newer sys- tems claim to work with unstructured information as well, although there is great variability in terms of how well they actually do this.

Security issues in enterprise search Unlike a Web search, enterprise search tools must balance the goal of making information widely available throughout the organization with the need to restrict access based on an employee’s job function or security clearance. Limiting access to certain documents or data is referred to as access control. Enterprise search tools introduce the potential for a number of security breaches or access of unauthorized information. Most of these can be addressed as long as the organization’s IT workers install and maintain the search system’s security features, including security integrations with other enterprise programs. An audit of requests logs should be conducted regularly to look for patterns or inconsistencies.

Using Search Technology for Business Success 173

Enterprise search vendors Market analysts Frost and Sullivan (Prnewswire.com,  2013) estimate that the global market for enterprise search tools was over $1.47 billion in 2012; it is predicted that the market will grow to over $5 billion by 2020. Clearly, organizations around the world recognize the value of this technology. Several different companies make and sell enterprise search systems, Autonomy, Google, Coveo, and Perceptive Software being the top contenders (Andrews & Koehler-Kruener, 2014). Vendors can be broken down into the following three categories:

• Specialized search vendors (for instance, Attivio, Endeca, Vivisimo): Software designed to target specific user information needs

• Integrated search vendors (for instance, Autonomy, IBM, and Microsoft): Software designed to combine search capabilities with information management tools

• Detached search vendors (for instance, Google, ISYS): Software designed to target flexi- bility and ease of use

With so many options available for enterprise search, it is important that organizations conduct a careful needs analysis prior to acquisition.

Recommendation Engines Recommendation engines represent an interesting twist on IR technology. Unlike Web search engines that begin with a user query for information, recommendation engines attempt to anticipate information that a user might be interested in. Recommendation engines are used by e-commerce sites to recommend products; news orga- nizations to recommend news articles and videos; Web advertisers to anticipate the ads people might respond to; and so on. They represent a huge potential for businesses and developers. While the use of recommendation engines is widespread, there is still much work to be done to improve the accuracy of these fascinating applications. You can read more about recommen- dation engines in Section 6.5.

Search Engine Marketing Most traditional advertising methods target customers who are not actively engaged in shopping for a product. Instead, they are watching television, listening to the radio, reading a magazine, or driving down the road, paying little attention to the billboards they pass. To most people, advertising represents an unwelcome interruption. On the other hand, people using search engines are actively looking for information. As a result, they are much more likely to be interested in product and service information found in SERPs as long as it is related to the topic they are searching for. Efforts to reach this targeted audience are much more likely to produce sales. That’s why search engine marketing (SEM) has become an important business strategy. Industry experts report that people generally engage in three basic types of searches:

1. Informational search Using search engines to conduct research on a topic. This is the most common type of search.

2. Navigational search Using a search engine to locate particular websites or Web pages. 3. Transactional search Using a search engine to determine where to purchase a product

or service.

You might think businesses would be primarily interested in transactional searches, but all three types are important and play a key role in the buying process. Say you are interested in purchasing a new tablet computer. Your first step is likely to engage in an informational search, attempting to learn about the product category of mobile tablet devices. Businesses should offer content on their websites and social media sites for consumers seeking general product information. An informational search also represents an opportunity to influence consumers early in the purchasing process.

After researching a product category, you might try finding websites of particular com- panies to learn more about individual tablet computer brands (navigational search). Com- panies need to design their websites so that they can be found easily by search engines.

Search engine marketing (SEM) a collection of online marketing strategies and tactics that promote brands by increasing their visibility in SERPs through optimization and advertising.

174 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Finally, you might try to determine where to buy your tablet computer by searching on terms like “lowest price,” “free shipping,” and so on. This is an example of a transactional search.

Search engine marketing (Figure  6.5) consists of designing and advertising a Web page, with the goal of increasing its visibility when consumers conduct the three types of searches just described. SEM strategies and tactics produce two different, but complementary outcomes:

1. Organic search listings are the result of content and website design features intended to improve a site’s ranking on SERPs that result from specific keyword queries. No payments are made to the search engine service for organic search listings.

2. Paid search listings are a form of advertising and are purchased from search engine companies. The placement and effectiveness of paid search ads on SERPs are a function of several factors in addition to the fees paid by advertisers. You will read more about these factors in Section 6.3.

Businesses utilize search engine optimization (SEO) to improve their website’s organic listings on SERPs. SEO specialists understand how search engines work and guide companies in designing websites and creating content that will produce higher organic SERP rankings than competitive websites.

Paid search listings are often referred to as pay-per-click (PPC) advertising because adver- tisers pay search engines based on how many people click on the ads. Typically, PPC ads are listed separately from organic search results. Managing an effective PPC ad campaign involves making strategic decisions about what keyword search queries you want to trigger the display of your ad. You will read more about PPC or paid search advertising in Section 6.3.

Social media optimization refers to strategies designed to enhance a company’s standing on various social media sites. Increasingly, search engines evaluate a company’s presence on social media to determine its reputation, which in turn influences how the company is ranked in SERPs. You will read more about social media strategies in Chapter 7.

Growth of search engine marketing As companies begin to realize the power of SEM, more money is being spent on this highly effective strategy. In 2016, businesses spent an esti- mated $65 billion on SEO services to improve the rank or listing order of their organic listings on SERPs. This figure is expected to rise to almost $80 billion in 2020 (Sullivan, 2016). In addition, the research firm eMarketer (2016) estimates that spending on PPC search advertising reached $86.25 billion in 2016, an increase of 15.4% from the year before. Both types of spending, SEO and PPC, illustrate how important search marketing is to businesses these days. Companies now spend more on SEM than they do on television or print advertising. Unlike most traditional advertising methods, return on investment (ROI) can be calculated for SEM by tracking click- through rates (CTRs), changes in site traffic, and purchasing behavior.

Click-through rates (CTRs) the percentage of people who click on a hyperlinked area of a SERP or Web page.

Search Engine MarketingSEO

Search Engine Optimization

SMO Social Media Optimization

PPC Pay-Per-Click Advertising

FIGURE 6.5 Search engine marketing integrates three different strategies: search engine optimization, pay-per-click advertising, and social media optimization.

Using Search Technology for Business Success 175

Mobile Search and Mobile SEO Mobile devices have become ubiquitous. With the emergence of smartphones and tablet computers, mobile devices now account for over half of all Web traffic. In some developing countries, mobile devices account for an even larger share of Internet use since they are less expensive than computers. Since more people are using mo- bile devices to surf the Web, it should come as no surprise that more Internet searches are conducted using mobile devices instead of computers.

With the dramatic increase in mobile device usage, companies need to make sure their websites and content can be found via mobile search. This means optimizing mobile websites differently from desktop sites. Two issues essential to mobile SEO include:

1. Properly configuring the technical aspects of the mobile site so that it can be crawled and indexed by search engines.

2. Providing content that is useful to people using mobile devices. Webmasters should con- sider how people use their mobile devices differently from computers and adjust content on their mobile websites accordingly. For instance, if consumers are likely to use their mobile device to check product reviews while shopping in a store, make this information easy to find on the mobile website.

When designing a mobile site for e-commerce, Web developers should make sure that information about store location, product reviews, and promotional offers is easily available and optimized so that it will appear in a mobile SERP. Mobile shoppers also use barcode scanning apps as a kind of mobile search engine for locating product reviews and price comparisons while shopping in stores. This practice, called showrooming, is becoming increasingly popular with con- sumers and creating a great deal of frustration and worry on the part of brick-and-mortar retailers.

Social Search Most major social media websites (i.e., Facebook, YouTube, Twitter, LinkedIn, etc.) have search engines designed to help users find content on their platforms. Of course, some search tools are better than others. It probably comes as no surprise that Face- book users have access to some advanced search features. People can search for friends by name or find information related to their friends using more complex queries such as “Movies liked by friends who liked The Godfather” or “Music liked by friends who liked Lady Gaga.” Face- book search can be used to find services, events, places, and groups. You can use it to find some place to eat with a search phrase like “Seafood restaurants in New Orleans.” Clearly, Facebook hopes to leverage the content and connections created by users to power a search tool that people will use instead of Google, Bing, or some other general Web search engine.

Recently, Facebook added a new image search feature powered by artificial intelligence that allows users to search for pictures using words that describe what’s in the picture instead of relying on tags and captions. For instance, you might search for “Santa Claus photo” and the search engine will be able to find photos with Santa Claus even if no tags or text associate the picture with Santa Claus. Developers say that eventually the image search will be able to recognize photos based on objects, actions (e.g., walking, running, dancing), and other descrip- tive terms. Eventually, this technology could be used to perform similar searches for video and other immersive formats (Candela, 2017).

Facebook undoubtedly can devote more resources to innovations of this nature than other social media platforms. However, while other platforms may take longer to develop sophisticated social search tools, most have the same motivations as Facebook when it comes enhancing user experience and providing a mechanism for highlighting content from individ- uals and organizations with commercial interests. With over 2 billion searches conducted on Facebook each day, businesses will undoubtedly be willing to pay for ways to reach this sizable social media audience in much the same way that they currently advertise on Google, Bing, and other Internet search engines (Kraus, 2015; Constine, 2016).

Personal Assistants and Voice Search Major Internet technology firms Apple, Amazon, Google, and Microsoft and a host of smaller firms have launched intelligent personal assistant (IPA) systems that threaten to disrupt conventional SEM paradigms. IPA software is typically designed to help people perform basic tasks like turning on/off lights and small

176 C H A P T E R 6 Search, Semantic, and Recommendation Technology

appliances, activating household alarm systems, and searching the Internet for music, videos, weather, and other types of information. While IPAs are still in the growth stages of the product life cycle, forecasted demand for the foreseeable future seems strong.

The typical IPA system is a voice-activated program that uses commands that approx- imate natural language. For instance, to learn about the weather, you might ask Amazon’s IPA, “Alexa, what’s the weather for this weekend?” To get Apple’s IPA to play a specific music genre, you might say, “Siri, play some R&B music.” In the not too distant future, we can expect voice-activated IPAs will be integrated with mobile devices, televisions, automobiles, and even hotel rooms.

Business that have become skilled at using SEO and PPC campaigns to drive traffic to their websites will have to go back to the drawing board to figure out how the rise in voice search will affect some fundamental marketing strategies. Currently, IPAs act like a kind of filter, screening search results and often basing answers on a single source. Just as businesses once faced the challenge of reformatting website content for smaller screens on mobile devices, they must now determine how to serve up information in a format optimized to make it attractive to a variety of IPAs acting as proxies for their owners.

Web Search for Business Commercial search engines and Web directories are use- ful tools for knowledge workers in business. To use search engines effectively, workers should familiarize themselves with all the features available on the search engine they use. Since Google is the most popular search engine, we highlight some of those features in the following list. Many of these features are also available on Bing.com.

• Focused search You can focus your search to information in different formats—Web pages, videos, images, maps, and the like—by selecting the appropriate navigation button on the SERP page.

• Filetype If you are looking specifically for information contained in a certain file format, you can use the “filetype:[file extension]” command following your keyword query. For in- stance, the search “private colleges filetype:xls” will produce links to MS Excel files with information related to private colleges. Use this command to find Adobe files (.pdf), MS Word files (.docx), MS PowerPoint files (.pptx), and so on.

• Advanced search To narrow your search, go to the Advanced Search panel. From this page, you can set a wide range of parameters for your search, including limiting the search to certain domains (e.g., .gov, .org, .edu), languages, dates, and even reading level. You can also use this to narrow your search to a particular website.

• Search tools button Allows you to narrow your results to listings from specific locations or time frames.

• Search history Have you ever found a page using a search engine, but later had trouble finding it again? If you are logged into your Google account while using the search engine, it’s possible to review your search history. It will show you not only your search queries but also the pages you visited following each query.

These are just a few of the many features you can use to conduct a power search. While you are in college, take the time to become proficient with using different search engine features. Not only will it help with your immediate research needs, it will help you in your career as well. At the end of this chapter, we include information for a free online Power Search course offered by Google. This is a good way to enhance your ability to find the information you need.

Finding intellectual property Your business may have an interest in protecting certain kinds of intellectual property being used without permission on the Web. This might include confidential reports, images, copyrighted blog posts, creative writing (e.g., poetry, novels, etc.), and so on. You can use search engines to find where someone may have posted your intellec- tual property on the Web without permission (see Osher, 2014). You can search for text-based work by simply using queries containing strings of text from the material you’re looking for. Images can be found by using Google’s reverse image search engine. Tin Eye is an alternative reverse image search engine with a number of interesting features.

Using Search Technology for Business Success 177

Real-time search Sometimes you need information about things as they happen. For in- stance, you may be interested in monitoring news stories written about your company or you might need to know what people are saying about your brand or a political candidate on Twit- ter. For these situations, you’ll need a real-time search tool.

Say your company wants to explore accepting Bitcoin payments. (Bitcoin is a digital currency that was launched in 2009.) After engaging in a traditional Web search to learn about the currency, you decide you want to learn about public interest in Bitcoin and find news stories that have recently been published about the currency. You might consider using the following tools:

• Google Trends This tool will help you identify current and historical interest in the topic by reporting the volume of search activity over time. Google Trends allows you to view the information for different time periods and geographic regions.

• Google Alerts Use Google Alerts to create automated searches for monitoring new Web content, news stories, videos, and blog posts about some topic. Users set up alerts by specifying a search term (e.g., a company name, product, or topic), how often they want to receive notices, and an e-mail address where the alerts are to be sent. When Google finds content that match the parameters of the search, users are notified via e-mail. Bing has a similar feature called News Alerts.

• Twitter Search You can leverage the crowd of over 650 million Twitter users to find information as well as gauge sentiment on a wide range of topics and issues in real time. Twitter’s search tool looks similar to other search engines and includes an advanced search mode. In addition to real-time search, the Twitter search tool is also an example of social search, which was explained earlier in the chapter.

Social bookmarking search Social bookmarking sites like Diigo provide a way for users to save links to websites they want to access at a later time. When saving page links, users tag them with keywords that describe the page’s content. The bookmarked links form a graph of content on the Web that can be used by others. Because the Web pages are tagged by humans, search results are often more relevant than results from commercial search engines. Pinterest is a variation on the social bookmarking idea, allowing users to save and share images they find online. You can find information about various topics by searching Pinterest to see what other users have collected on the subject.

Vertical search As described previously, large commercial search engines use indicators of popularity or reputation to determine website quality. This seems to work well for a gener- alized Web search, but it might not be effective when users search on very specific topics such as rare disease, which, by definition, does not generate a lot of activity on the Web. Crawlers do not often index pages in the lower levels of less popular websites. Vertical search engines are programmed to focus on Web pages related to a particular topic and to drill down by crawling pages that other search engines are likely to ignore. Vertical search engines exist for a variety of industries. Ironically, the best way to find a vertical search engine is to search for it on a commercial search engine like Google or Bing.

Questions

1. What is the primary difference between a Web directory and a crawler-based search engine? 2. What is the purpose of an index in a search engine? 3. Why are companies increasingly interested in enterprise search tools capable of handling unstruc-

tured data?

4. What is the difference between SEO and PPC advertising? 5. Describe three different real-time search tools.

178 C H A P T E R 6 Search, Semantic, and Recommendation Technology

6.2 Organic Search and Search Engine Optimization The goal of SEO practitioners is to help organizations increase traffic to their websites. They accomplish this by optimizing websites in an effort to increase visibility and ranking on SERPs. Using Web analytics programs like Google Analytics, companies can determine how many peo- ple visit their site, what specific pages they visit, how long they spend on the site, and what search engines are producing the most traffic (see Figure  6.6). More sophisticated SEO prac- titioners will also attempt to determine what keywords or phrases generated traffic to their website. These are just a few of the many metrics used to measure the effectiveness of SEO strategies. In the sections that follow, we will use the most popular search engine, Google, to explain the basics of SEO. Most of what we write, however, will also apply to other popular search engines.

Strategies for Search Engine Optimization As mentioned at the beginning of this chapter, all search engines use somewhat different pro- prietary algorithms for determining where a website will appear in search results. As a result, it is not possible to tell what specific factors will be used or how much weight they will carry in determining SERP ranking. Over time, there has been a significant increase in the number of factors that search engines like Google use to determine how a site is listed on a SERP. The general consensus among SEO experts is that Google probably uses over 200 different factors. To make things even more challenging, Google updates its algorithm hundreds of times a year. This presents somewhat of a moving target for SEO professionals hired to improve the organic SERP listings of their clients.

Why Does Google Keep Changing Its Algorithm? Google’s overall goal is to constantly improve the experience of people using its search engine. Over time, Google engi- neers have developed ways to predict if a website will provide a positive experience for people using its search engine. Whenever a new way to improve user experience is found, they imple- ment the change by updating the algorithm. Google also employs sophisticated technologies like artificial intelligence and semantic search algorithms to enhance the search experience.

© fr

an ck

re p

or te

r/ iS

to ck

p ho

to

FIGURE 6.6 Tools like Google Analytics are used to monitor changes in website traffic as a result of SEO practices.

Organic Search and Search Engine Optimization 179

Artificial intelligence constantly monitors how users respond to search results and modifies the listing algorithm to improve results. Semantic technology helps Google do a better job of understanding the content on a Web page and matching that content with the words and phrases people use to conduct a search.

Ranking Factors: On-Page and Off-Page SEO To understand how Google ranks website listings in search results, we begin by dividing ranking factors into on-page factors and off-page factors.

On-page factors are elements of the Web page that can be directly controlled by the pub- lisher or Web page creator. SEO professionals attempt to improve a website’s SERP listing by opti- mizing on-page factors related to content, functionality, and HTML programming (Sullivan, 2015).

Content Perhaps one of the biggest changes Google has made to its ranking algorithm over the years is an increased emphasis on high-quality content. Content marketing is a strategy that has gained popularity in recent years because of the significant weight assigned to high- quality content when determining search results and its role in attracting increased Web traffic. Some specific ways that Google determines if a website has high-quality content include the following:

• The quality of writing on the Web page • The presence of relevant keywords and phrases associated with the topic • How “fresh” or up-to-date the content is • Use of multiple content formats (i.e., news, video, podcast, blog, and social content) • Depth or quantity of topical content • Links that point to other well-respected and trustworthy websites • The proportion of relevant to irrelevant text about a topic • Barriers to content have a negative impact on user satisfaction. Examples include making

people register, provide names, or fill out forms to get to content.

Functionality and programming Website functionality has an impact on SERP rankings. Pages that don’t load quickly or display well on mobile devices are less likely to result in a positive user experience. Information in a page’s HTML (programming language) source code also influences ranking algorithms. Functionality and programming can be assessed by factors such as the following:

• How easily search engine programs can “crawl” the Web page. • How well the Web page works with mobile devices. • How quickly the Web page loads. • Availability of secure (https://) connections for visitors. • Minimal presence of duplicate content on the website. • Page URLs that contain keywords. • Use of topical words and phrases in source code metadata (e.g., title tags, page descrip-

tions, keywords). • How frequently users click on a listing in search results. Click-through rates (CTR) are

determined in part by how attractive a website’s listing is on a SERP. The way a SERP listing looks is the result of the Web page’s HTML source code.

• Hacked websites, sites that infect users with malware, and sites that fail to clean up spam or irrelevant content in comment sections are all factors that negatively impact user experience.

Off-page factors can be influenced but not directly controlled by SEO professionals. Many off-page factors are strongly related to a website’s relevance and credibility. Other off-page factors are related to personalized search, a relatively new effort by Google to improve user experience (Sullivan, 2015).

180 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Relevance and credibility Google uses a number of different metrics to determine if a website is a trustworthy source of information on a particular topic. Many of these metrics are based on user behavior and how the site is represented on other websites and social media platforms:

• Backlinks to the target website on other well-respected and trustworthy websites. The use of backlinks is based on the assumption that people who create website content are more likely to place links to high-quality websites than poor-quality sites on their Web pages. Google assigns a PageRank score to each Web page based on the quality and quantity of backlinks associated with the page. Since the PageRank score is believed to be a heavily weighted factor, SEO professionals have developed several creative strategies for increasing legitimate backlinks to their websites while avoiding certain tactics that Google disapproves of. Google downgrades websites that use methods that artificially inflate their backlink count.

• Click-through rate (CTR) is also an indicator of relevance. Users are more likely to click on SERP listings related to the information they’re searching for.

• Amount of advertising on the website—Too many ads detract from topical website content. • Dwell time—This is a measure of how long a user remains on a page. Users stay on pages

with useful content longer than pages that lack useful content. • Sites listed in respected Web directories are more likely to contain quality content because

they have been reviewed by human editors. Positive comments on review sites like Yelp.com and Zagat.com also have a positive impact on a website’s reputation.

• High-quality or helpful websites are more likely to be discussed on social media. Examples include comments on Facebook and Google+, shares, Tweets, Likes, and so on.

• Site traffic—Sites with high-quality content tend to get more traffic over time.

Personalized search Google uses information about the person conducting the search in an effort to enhance their experience:

• User location—the country, city or area the user is from • Past experience—Google SERPs can be influenced by search and Web browsing history • Social experience—the extent to which the user or people in their network engaged

with or discussed the website favorably on social media including Google+, Facebook, Twitter, and so on

Content and Inbound Marketing The ultimate goal of search engines is to help users find information. Sometimes it seems that SEO practitioners lose sight of this and spend too much time chasing down hundreds of factors they think are being used by search engine ranking algorithms. At worst, SEO can represent an attempt to “game the system” or trick search engines into ranking a site higher than its content deserves (see the discussion of black hat SEO in the next section).

Perhaps the most important action an organization can take to improve its website’s rank- ing and satisfy website visitors is to provide helpful content that is current and updated reg- ularly. When SEO practices are combined with valuable content, websites not only become easier to find but also contribute to building brand awareness, positive attitudes toward the brand, and brand loyalty.

Inbound marketing represents an alternative approach to traditional outbound marketing strategies (e.g., mass media advertising). Inbound marketers attract customers to their websites with content that is informative, useful, or entertaining. Inbound marketing campaigns are based on strategies that integrate content generation, SEO, and social media tactics. In Chapter  7, you will read more about how inbound marketers integrate content, SEO, and social media strategies in powerful marketing campaigns that deliver sales and profit. See Figure 6.7.

Organic Search and Search Engine Optimization 181

Black Hat versus White Hat SEO: Ethical Issues in Search Engine Optimization Search engines regularly update their algorithms to improve results. Two well-known Google updates called Panda (released in 2011) and Penguin (released in 2012) were designed to improve the ranking of websites with quality content and downgrade poor-quality sites. Both updates are designed to defeat what are commonly referred to as “black hat SEO” tactics. People who employ black hat SEO tactics try to trick the search engine into thinking a web- site has high-quality content, when in fact it does not. With stronger detection systems now in place, websites that use these tactics (or even appear to use them) will be severely down- graded in Google’s ranking system. Some examples of black hat SEO tactics are defined in the following list:

Link spamming—Generating backlinks for the primary purpose of SEO, not adding value to the user. Black hat SEOs use tricks to create backlinks. Some examples include adding a link to a page in the comments section of an unrelated blog post, or building sites called “link farms” solely for the purpose of linking back to the promoted page. Keyword tricks—Black hat SEOs will embed several high-value keywords on pages with unrelated content to drive up traffic statistics. For instance, an e-commerce site might embed words like “amazon” (a word that frequently shows up in search queries) in an attempt to get listed on SERPs of people looking for amazon.com. Ghost text—This tactic involves adding text on a page that will affect how a website is listed on SERPs. The text may not have anything to do with the real content of the page, or it may simply repeat certain words to increase the content density. The text is then hidden, usually by making it the same color as the background. Shadow pages—Also called “ghost pages” or “cloaked pages,” this black hat tactic involves creating pages that are optimized to attract lots of people. The pages, however, contain a redirect command so that users are sent to another page to increase traffic on that page.

These particular tactics are no longer effective as a result of updates to the Google ranking system. Most likely, other search engines have adopted similar measures. However, there will always be people who take shortcuts attempting to achieve higher SERP rankings. Businesses must be careful when hiring SEO consultants or agencies to make sure they do not use prohib- ited SEO techniques. When these actions are discovered, Google and other search engines will usually punish the business by dramatically reducing their visibility in search results.

Communication is interactive or two-way

Marketers promote

company by educating or entertaining

Typical strategies:

content, social media, and SEO

Customers seek

out the business

Inbound Marketing

Communication is one-way

Businesses broadcast

messages that interrupt

customers

Typical strategies: TV, radio,

print, outdoor, cold calls

Businesses seek out

customers

Outbound Marketing

FIGURE 6.7 Inbound marketers use valuable content, search engine optimization, and social media to attract customers.

182 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Questions

1. Search engines use many different “clues” about the quality of a website’s content to determine how a page should be ranked in search results. Explain how a search engine uses specific factors to deter- mine the quality of a website’s content.

2. SEO professionals strive to increase a Web page’s PageRank score which is based on the quality and quantity of backlinks. Explain what a backlink is and why search engines use the PageRank score to determine the order in which websites are listed in SERPs.

3. Explain why the so-called black hat SEO tactics are ultimately short-sighted and can lead to significant consequences for businesses that use them.

4. What is the fundamental difference between on-page and off-page SEO factors? 5. Explain why providing high-quality, regularly updated content is the most important aspect of any

SEO strategy.

6.3 Pay-Per-Click and Paid Search Strategies In addition to organic listings, most search engines display paid or sponsored listings on their SERPs. These advertisements provide revenue for the search engine and allow it to offer Web search services to the general public for free. They also provide a way for smaller organizations with new websites to gain visibility on SERPs while waiting for their SEO strategies to produce organic results. Most major search engines differentiate organic search results from paid ad list- ings on SERPs with labels, shading, and placing the ads in a different place on the page. Some critics have complained that paid advertisements receive preferential page placement and are not clearly distinguished from organic listings. However, at the time of this publication, it is easy to distinguish ads from organic results on Google and Bing SERPs. Defenders of the search engine companies argue that since the paid ads make it possible for everyone to use search services for free, the preferential page placement is justified.

Creating a PPC Advertising Campaign There are five steps to creating a PPC advertising campaign on search engines.

1. Set an overall budget for the campaign. 2. Create ads−most search engine ads are text only, but this is likely to change in the future. 3. Select keywords and other parameters associated with the campaign. 4. Set up billing account information. 5. Modify key words and ad copy based on results.

Search advertising allows businesses to target customers who are likely to purchase their products. They do this by selecting keywords that correspond to search queries that potentially identify someone as a customer. For instance, a company that sells women’s purses may want to appear on a SERP when someone conducts a search using any of the following terms or phrases:

• Purse • Handbag • Women’s purses • Ladies’ purses • Designer purses • Designer handbags

Pay-Per-Click and Paid Search Strategies 183

Google and other search engines provide advertisers with tools for evaluating the impact of different keywords or phrases. These tools typically display information about how often people use the word in a search and also recommend alternative words to consider using in the campaign. Advertisers “bid” on having their ads appear when someone searches on one of their keywords. Higher bids result in a greater probability that the ad will appear in search results associated with the keyword. However, this might also deplete the advertiser’s budget more quickly. On the other hand, if a bid is too low, the ad might not appear at all. Keyword tools usually provide information about typical bid prices for each keyword or phrase. Smart advertisers start with a modest bid and increase it over time to achieve the ad placement rate they desire.

The likelihood of ad placement is also influenced by a quality score representing the search engine’s estimate of how successful the ad will be. Quality scores are determined by factors related to ad relevance and user experience factors. Relevant ads closely match the intent of the user’s search. The expected CTR indicates how likely the ad will be clicked on. The user’s landing page experience is determined by things such as how relevant, transparent, and easy-to-navigate the page is. According to Google, quality scores are determined by sev- eral factors:

• Expected keyword CTR • The past CTR of your URL • Past effectiveness (overall CTR of ads and keywords in the account) • Landing page quality (relevance, transparency, ease of navigation, etc.) • Relevance of keywords to ads • Relevance of keywords to customer search query • Geographic performance—account success in geographic regions being targeted • How well ads perform on different devices (quality scores are calculated for mobile,

desktop/laptop, and tablets)

Relevant ads that produce sales are good for all parties. The search engine makes more money from clicked ads, the advertiser benefits from increased revenue, lower costs-per-click, and more favorable ad placement. When ads are relevant and landing pages are functional and contain relevant information, customers are more likely to find and purchase what they are looking for.

In addition to selecting keywords and setting bid prices, advertisers also set parameters for the geographic location and time of day they want the ad to appear. These factors allow for additional customer targeting designed to help advertisers reach the consumers most likely to purchase their products.

Companies need to consider the fit between ad content and landing page content and functionality. For instance, sometimes companies create product-oriented ads, but then link to the main page of their website instead of a page with information about the product in the ad. Other factors include landing page design, call to action (CTA) effectiveness, and quality of the shopping cart application. It does not make sense to spend money on a PPC campaign designed to drive consumers to an unattractive and dysfunctional website.

One of the attractive features of PPC ad campaigns is that managers can monitor results in real time and make adjustments to the campaign parameters if necessary. Advertisers fre- quently set up A/B tests to evaluate the relative effectiveness of two different ads. After a period of time, the advertiser checks to see which ad is producing better results and discontinues use of the less effective ad for the remainder of the campaign. Some advertisers run A/B tests throughout the campaign, constantly testing ad copy and other elements in the spirit of con- tinuous improvement. You can learn more about advertising on the major commercial search engines by visiting the following websites:

• Google adwords.google.com • Bing advertise.bingads.microsoft.com • Yahoo advertising.yahoo.com

184 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Metrics for Paid Search Advertising In addition to more effective targeting, one of the key benefits of online advertising is the ability to evaluate its contribution to sales revenue more effectively. PPC advertisers use the following metrics to gauge the effectiveness of their campaigns:

Click-through rates (CTRs) By themselves, CTRs do not measure the financial perfor- mance of an ad campaign. But they are useful for evaluating many of the decisions that go into a campaign, such as keyword selection and ad copy and ad attractiveness. Keyword conversion High CTRs are not always good if they do not lead to sales. Since the cost of the campaign is based on how many people click an ad, you want to select key- words that lead to sales (conversions), not just site visits. PPC advertisers monitor which keywords lead to sales and focus on those in future campaigns. Cost of customer acquisition (CoCA) This metric represents the amount of money spent to attract a paying customer. To calculate CoCA for a PPC campaign, you divide the total budget of the campaign by the number of customers who purchased something from your site. For instance, if you spent $1,000 on a campaign that yielded 40 customers, your CoCA would be $1,000/40 = $25 per customer. Return on advertising spend (ROAS) The campaign’s overall financial effectiveness is evaluated with ROAS (revenue/cost). For example, if $1,000 was spent on a campaign that led to $6,000 in sales, ROAS would be $6,000/$1,000 = $6. In other words, for every dollar spent on PPC ads, $6 was earned.

Questions

1. What would most people say is the fundamental difference between organic listings and PPC listings on a SERP?

2. What are the five primary steps to creating a PPC advertising campaign on search engines? 3. In addition to the “bid price” for a particular keyword, what other factor(s) influence whether or not

an advertisement will appear on a search results page? Why don’t search engines use just the adver- tiser’s bid to determine if an ad will appear on search results pages?

4. How do on-page factors influence the effectiveness of PPC advertisements? 5. What factors determine an ad’s quality score? 6. Describe four metrics that can be used to evaluate the effectiveness of a PPC advertising campaign.

6.4 A Search for Meaning—Semantic Technology If there is one thing history has taught us, it is that the future is hard to predict. It might seem silly to predict what the future Internet will look like when it’s clear so many people are having trouble understanding all the implications of the present Internet. However, forward-thinking businesses and individuals are beginning to plan for the next evolution which is sometimes called Web 3.0.

The current Web is disjointed, requiring us to visit different websites to get content, engage in commerce, and interact with our social networks (community). The future Web will use context, personalization, and vertical search to make content, commerce, and community more relevant and easier to access. With the addition of mobile technology, this Web will be always accessible.

• Context defines the intent of the user; for example, trying to purchase music, to find a job, to share memories with friends and family.

A Search for Meaning—Semantic Technology 185

• Personalization refers to the user’s personal characteristics that impact how relevant the content, commerce, and community are to an individual.

• Vertical search, as you have read, focuses on finding information in a particular content area, such as travel, finance, legal, and medical.

What Is the Semantic Web? Semantic refers to the meaning of words or language. The semantic Web is one in which computers can interpret the meaning of content (data) by using metadata and natural language processing to support search and retrieval, analysis and information amalgamation from both structured and unstructured sources. Semantic technologies are being developed that will create a new, richer experience for Web users.

Tim Berners-Lee, creator of the technology that made the World Wide Web possible, is director of the World Wide Web Consortium (W3C). This group develops programming standards designed to make it possible for data, information, and knowledge to be shared even more widely across the Internet. The result of these standards is a metadata language, or ways of describing digital information so that it can be used by a wide variety of applications.

Much of the world’s digital information is stored in files structured so they can only be read by the programs that created them. With metadata, the content of these files can be labeled with tags describing the nature of the information, where it came from, or how it is arranged. At the risk of sounding too dramatic, metadata transforms a connected, but largely uninterpreta- ble Web (network) of pages into a large database that can be searched, analyzed, understood, and repurposed by a variety of applications.

It is helpful to think about the semantic Web against the background of earlier Internet functionality (see Table  6.2). The early Internet allowed programmers and users to access information and communicate with one another without worrying about the details associ- ated with the machines they used to connect to the network and store the information. The semantic Web continues this evolution, making it possible to access information about real things (people, places, contracts, books, chemicals, etc.) without knowing the details associ- ated with the nature or structure of the data files, pages, and databases where these things are described or contained (Hendler & Berners-Lee, 2010). This will greatly expand the ways in which we search for and find information related to our needs and interests.

The Language(s) of Web 3.0 The early Web was built using hypertext markup language (HTML). Web 2.0 was made possible, in part, by the development of languages like XML and JavaScript. The semantic Web utilizes additional languages that have been developed by the W3C. These include resource description framework (RDF), Web ontology language (OWL), and SPARQL protocol and RDF query language (SPARQL). RDF is a language used to represent information about resources on the Internet. It will describe these resources using metadata uniform resource identifiers (URIs) like “title,” “author,” “copyright and license information.” It is one of the features that allow data to be used by multiple applications.

TA B L E 6 . 2 Evolution of the Web

Web 1.0 (The Initial Web) A Web of Pages

Pages or documents are “hyperlinked,” making it easier than ever before to access connected information.

Web 2.0 (The Social Web) A Web of Applications

New applications and technologies allow people to easily create, share, and organize information.

Web 3.0 (The Semantic Web) A Web of Data

Using metadata tags, artificial intelligence, natural language processing, and other semantic tools, computers can be used to access specific information across platforms and applications, regardless of the original structure of the file, page, or document. It turns the Web into a giant readable database.

186 C H A P T E R 6 Search, Semantic, and Recommendation Technology

As the acronym SPARQL implies, it is used to write programs that can retrieve and manip- ulate data stored in RDF format. OWL is the W3C language used to categorize and accurately identify the nature of things found on the Internet. These three languages, used together, will enhance the element of context on the Web, producing more fruitful and accurate information searches. The W3C continues its work, with input by programmers and the broader Internet community, to improve the power and functionality of these languages.

Semantic Web and Semantic Search As you have read, the semantic Web is described by metadata, making it easier for a broad range of applications to identify and utilize data. One of the barriers to creating a semantic Web based on metadata, however, is the tagging process. Who will tag all the data currently on the Web? How can we be sure that such data will be tagged correctly? Will people purposely tag data incorrectly to gain some kind of advantage in the same way that black hat SEO tactics are used to mislead search engines?

Semantic search engines can be programmed to take advantage of metadata tags, but their usefulness would be very limited if that was the only way they could understand Web content.

Metadata tags, therefore, are just one approach used by semantic search engines to under- stand the meaning of online content. In addition to metadata tags, semantic search engines use a variety of other strategies to find meaning:

• Natural language processing • Contextual cues • Synonyms • Word variations • Concept matching • Specialized queries • Artificial Intelligence

Semantic search will seek to understand the context or intent of users looking for information in an effort to increase the relevance and accuracy of results (DiSilvestro,  2013). For instance, if a search engine understood the proper context of a search query containing the word “Disneyworld,” it would know if the user was

• planning a vacation, or • looking for a job at the theme park, or • interested in the history of Disney World.

Semantic Search Features and Benefits So what can semantic search engines do that is so much better compared to search engines that work solely on keyword match- ing? Grimes (2010) provides a list of practical search features based on semantic search technology.

Related searches/queries The engine suggests alternative search queries that may pro- duce information related to the original query. Search engines may also ask you, “Did you mean: [search term]?” if it detects a misspelling.

Reference results The search engine suggests reference material related to the query, such as a dictionary definition, Wikipedia pages, maps, reviews, or stock quotes.

Semantically annotated results Returned pages contain highlighting of search terms, but also related words or phrases that may not have appeared in the original query. These can be used in future searches simply by clicking on them.

A Search for Meaning—Semantic Technology 187

Full-text similarity search Users can submit a block of text or even a full document to find similar content. Search on semantic/syntactic annotations This approach would allow a user to indi- cate the “syntactic role the term plays—for instance, the part-of-speech (noun, verb, etc.)— or its semantic meaning, whether it’s a company name, location, or event.” For instance, a keyword search on the word “center” would produce too many results. Instead, a search query could be written using a syntax such as the following:

<organization> center </organization>

This would only return documents where the word “center” was part of an organization’s name (e.g., Johnson Research Center or Millard Youth Center). Google currently allows you to do something similar to specify the kind of files you are looking for (e.g., filetype:pdf). Concept search Search engines could return results with related concepts. For instance, if the original query was “Tarantino films,” documents would be returned that contain the word “movies” even if not the word “films.” Ontology-based search Ontologies define the relationships between data. An ontol- ogy is based on the concept of “triples”: subject, predicate, and object. This would allow the search engine to answer questions such as “What vegetables are green?” The search engine would return results about “broccoli,” “spinach,” “peas,” “asparagus,” “Brussels sprouts,” and so on. Semantic Web search This approach would take advantage of content tagged with metadata as previously described in this section. Search results are likely to be more accu- rate than keyword matching. Faceted search Faceted search provides a means of refining or filtering results based on predefined categories called facets. For instance, a search on “colleges” might result in options to “refine this search by.  .  .” location, size, degrees offered, private or public, and so on. Many e-commerce websites provide users with faceted search features, allow- ing shoppers to filter search results by things like price, average rating, brand name, and product features. Clustered search This is similar to a faceted search, but without the predefined cat- egories. Visit Carrot2.org to better understand this concept. After conducting a search, click on the “foamtree” option to see ways to refine your search. The refining options are extracted from the content in pages of the initial search. Natural language search Natural language search tools attempt to extract words from questions such as “How many countries are there in Europe?” and create a semantic representation of the query. Initially, this is what people hoped search engines would evolve toward, but Grimes wonders if we have become so accustomed to typing just one or two words into our queries that writing out a whole question may seem like too much work.

You may recognize some of these search enhancements when using popular commercial search engines like Google or Bing. That is because they have been building semantic tech- nologies into their systems to improve user experience. You are encouraged to explore other search engines with semantic search features like DuckDuckGo and SenseBot.

Semantic Web for Business What opportunities and challenges does the semantic Web hold for businesses? Perhaps the most immediate challenge faced by businesses is the need to optimize their websites for semantic search. Because search engines are responsible for directing so much traffic to business websites, it will be important that companies take advantage of semantic technologies to ensure they continue to remain visible to prospective customers who use search engines. While

188 C H A P T E R 6 Search, Semantic, and Recommendation Technology

the details of semantic SEO are beyond the scope of this book, we can illustrate one important benefit of semantic website optimization. Websites optimized for semantic technology with metadata produce richer, more attractive listings on SERPs. Google calls these rich snippets (see Figure 6.8).

Note how detailed the organic search listing in Figure  6.8 is compared to a basic listing. These enhanced search listings are more visually attractive and produce greater CTRs.

Businesses need to stay up to date with advances in semantic search so that they can continuously optimize their sites to increase traffic from major search engines.

Questions

1. List five different practical ways that semantic technology is enhancing the search experience of users. 2. How do metadata tags facilitate more accurate search results? 3. Briefly describe the three evolutionary stages of the Internet? 4. Define the words “context,” “personalization,” and “vertical search.” Explain how they make for better

information search results.

5. What are three languages developed by the W3C and associated with the semantic Web?

6.5 Recommendation Engines

A lot of times, people don’t know what they want until you show it to them. —Steve Jobs (quoted in Business Week, May 12, 1998)

Think about the challenge faced by large e-commerce websites like Amazon or Netflix. Brick-and-mortar retailers can capture people’s attention in the store with eye-catching point- of-purchase displays or suggestive selling by store employees. However, these are not options

FIGURE 6.8 The Google search listing for this New York-based grocery chain is more attractive because it uses metadata from the business’s website. (Google and the Google logo are registered trademarks of Google, Inc., used with permission.)

Recommendation Engines 189

for retail websites. They need an effective way of recommending their vast array of products to customers. Most e-commerce sites provide website search tools based on the technologies previously discussed in this chapter. Relying on customers to find products through an active search, however, assumes customers know what they want and how to describe it when form- ing their search query. For these reasons, many e-commerce sites rely on recommendation engines (sometimes called recommender systems). Recommendation engines proactively identify products that have a high probability of being something the consumer might want to buy. Amazon has long been recognized as having one of the best recommendation engines. Each time customers log into the site, they are presented with an assortment of products based on their purchase history, browsing history, product reviews, ratings, and many other factors. In effect, Amazon customizes their e-commerce site for each individual, leading to increased sales. Consumers respond to these personalized pages by purchasing products at much higher rates when compared to banner advertisements and other Web-based pro- motions. At Amazon, the recommendation engine is credited with generating 35% of sales (Arora, 2016).

Recommendation Filters There are three widely used approaches to creating useful recommendations: content-based filtering, collaborative filtering, and hybrid strategies (Asrar, 2016).

Content-Based Filtering Content-based filtering recommends products based on the product features of items the customer has interacted with in the past (Figure 6.9). Inter- actions can include viewing an item, “liking” an item, purchasing an item, saving an item to a wish list, and so on. In the simplest sense, content-based filtering uses item similarity to make recommendations. For instance, the Netflix recommendation engine attempts to recom- mend movies that are similar to movies you have already watched (see IT at Work 6.2). Music- streaming site Pandora creates its recommendations or playlists based on the Music Genome Project©,a system that uses approximately 450 different attributes to describe songs. These detailed systems for describing movies and songs enhance Netflix’s and Pandora’s positions in highly competitive industries because of their ability to offer superior recommendations to their customers.

3. Recommendation: “Based on your rating of fruity cocktail umbrella drink you may also like…”

1. Customer likes fruity cocktail umbrella drink

2. Computer searches products for fruity cocktail umbrella drink

FIGURE 6.9 Content-based filtering produces recommendations based on similarity of product features.

190 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Collaborative Filtering Collaborative filtering makes recommendations based on a user’s similarity to other people. For instance, when a customer gives a product a high rating, he or she may receive recommendations based on the purchases of other people who also gave the same product a high rating. Sometimes, websites will explain the reason for the rec- ommendations with the message “Other people who liked this product also bought. . .” Many collaborative filtering systems use purchase history to identify similarities among customers. In principle, however, any customer characteristic that improves the quality of recommendations could be used (see Figure 6.10).

In an effort to develop increasingly better recommendation engines, developers are exploring a number of creative ways to predict what consumers might like based on patterns of consumer behavior, interests, ratings, reviews, social media contacts and conversations, media use, financial information, and so on.

In addition to content filtering and collaborative filtering, two other approaches to rec- ommendation engines are mentioned in the literature: knowledge-based systems and demographic systems. Knowledge-based systems use information about a user’s needs to rec- ommend products. This kind of system is useful for developing recommendations for products

IT at Work 6.2

Violent Nightmare-Vacation Movies and Other Fun Movie Genres at Netflix Alexis Madrigal (2014) reverse-engineered Netflix’s list of movie genres and was surprised to learn the company uses approxi- mately 76,897 different ways to describe movies, creating the potential for some unusually specific movie recommendations. Christian Brown (2012) compiled a list of humorous and some- times disturbing movie categories, a few of which are listed below:

10. Cerebral Con-Game Thrillers 9. Visually Striking Father–Son Movies 8. Violent Nightmare-Vacation Movies

7. Understated Independent Workplace Movies 6. Feel-Good Opposites-Attract Movies 5. Witty Dysfunctional-Family TV Animated Comedies 4. Period Pieces about Royalty Based on Real Life 3. Campy Mad-Scientist Movies 2. Mind-Bending Foreign Movies 1. More like Arrested Development

The fact that Netflix went to the trouble of creating so many detailed and descriptive labels suggests that a content-based fil- tering strategy is at use in the company’s recommendation system.

3. Recommendations: “Other customers who like pink beverage also like…”

1. Customer likes pink beverage 2. Other customers like pink beverage

FIGURE 6.10 Collaborative filtering bases recommendations on similarity to other customers.

Recommendation Engines 191

that consumers do not shop for very often. For instance, an insurance company may ask a cus- tomer a series of questions about his or her needs, and then use that information to recom- mend policy options. Demographic systems base recommendations on demographic factors corresponding to a potential customer (i.e., age, gender, race, income, etc.). While similarity to other customers might play a role in developing these recommendations, such systems are different from collaborative filtering systems that typically rely on information about a person’s behavior (i.e., purchase, product ratings, etc.).

Systems are being developed that leverage big data streams from multiple sources to refine and enhance the performance of current systems.

Limitations of Recommendation Engines While recommendation engines have proven valuable and are widely used, there are still challenges that must be overcome. Four commonly cited limitations are described as follows:

Cold start or new user Making recommendations for a user who has not provided any information to the system is a challenge since most systems require a starting point or some minimal amount of information about the user (Adomavicius & Alexander,  2005; Burke,  2007). Tiroshi and colleagues (2011) have suggested consumers’ existing social media profiles from sites like Facebook and Twitter could be used in situations where a website did not have sufficient information of its own to make recommendations. Sparsity Collaborative systems depend on having information about a critical mass of users to compare to the target user in order to create reliable or stable recommendations. This is not always available in situations where products have only been rated by a few people or when it is not possible to identify a group of people who are similar to a user with unusual preferences (Burke, 2007). Limited feature content For content filter systems to work, there must be sufficient information available about product features and the information must exist in a struc- tured format so it can be read by computers. Often feature information must be entered manually, which can be prohibitive in situations where there are many products (Adoma- vicius & Alexander, 2005). Overspecialization If systems can only recommend items that are highly similar to a user profile, then the recommendations may not be useful. For instance, if the recommen- dation system is too narrowly configured on a website that sells clothing, the user may only see recommendations for the same clothing item he or she liked, but in different sizes or colors (Adomavicius & Alexander, 2005).

Hybrid Recommendation Engines Hybrid recommendation engines develop recommendations based on some combination of the methodologies described above (content-based filtering, collaboration filtering, knowledge-based and demographic systems). Hybrid systems are used to increase the quality of recommendations and address shortcomings of systems that only use a single methodology. Burke (2007) identified various ways that hybrid recommendation engines combine results from different recommender systems. To illustrate the potential complexity and variation in hybrid systems, four approaches are listed below:

• Weighted hybrid Results from different recommenders are assigned a weight and combined numerically to determine a final set of recommendations. Relative weights are determined by system tests to identify the levels that produce the best recommendations.

• Mixed hybrid Results from different recommenders are presented alongside of each other. • Cascade hybrid Recommenders are assigned a rank or priority. If a tie occurs (with two

products assigned the same recommendation value), results from the lower-ranked sys- tems are used to break ties from the higher-ranked systems.

• Compound hybrid This approach combines results from two recommender systems from the same technique category (e.g., two collaborative filters), but uses different algo- rithms or calculation procedures.

192 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Recommendation engines are now used by many companies with deep content (e.g., large product inventory) that might otherwise go undiscovered if the companies depended on cus- tomers to engage in an active search.

To simplify our description of recommendation engines, most of the examples above have been based on the e-commerce sites recommending products to customers. However, this technology is used by many different kinds of business organizations, as illustrated in Table 6.3.

Questions

1. How is a recommendation engine different from a search engine? 2. Besides e-commerce websites that sell products, what are some other ways that recommendation

engines are being used on the Web today?

3. What are some examples of user information required by recommendation engines that use collab- orative filtering?

4. Before implementing a content-based recommendation engine, what kind of information would website operators need to collect about their products?

5. What are the four limitations or challenges that recommendation systems sometimes face? 6. What is a recommendation engine called that combines different methodologies to create recom-

mendations? What are three ways these systems combine methodologies?

TA B L E 6 . 3 Examples of Recommendation Engine Applications

Company How It Uses Recommendation Engines. . . Amazon Recommends products using multiple filtering methods.

Netflix Approximately 75% of Netflix movies are selected as a result of its recommendation system.

Pandora This streaming music site creates playlists based on similarity to initial songs or artists selected by the user.

CNN, Time, Fast Company, Rolling Stone, NBCNews.com, Reuters, Us Weekly

These news and entertainment companies all use a recom- mendation engine (or “content discovery system”) created by Outbrain.com to suggest additional articles related to the one site visitors initially viewed.

YouTube YouTube uses a variation of Amazon’s recommendation engine to suggest additional videos people might like to watch.

Goodreads This social website for readers recommends books based on user ratings of books they have read.

Samsung Uses recommendation engines built into its “smart TVs” to sug- gest television programming to viewers.

Facebook and LinkedIn These social networking services use recommendation engines to suggest people that users may want to connect with.

Apple Helps users find mobile apps they might enjoy.

Microsoft Xbox 360 Suggests new games based on what users have previously shown an interest in.

Tripadvisor Recommends travel destinations and services based on destina- tions people have viewed or rated.

Stitch Fix This fashion start-up uses a recommender system in conjunction with human stylists to select and ship clothing products to customers, before customers viewed or ordered them!

Assuring Your Learning 193

Assuring Your Learning

Key Terms access control 172 backlinks 180 click-through rates (CTRs) 174 collaborative filtering 190 collection analysis module 169 content-based filtering 189 crawler control module 169 crawler search engines 168 dwell time 180 enterprise search 172 ghost text 181 hybrid recommendation engines 191 hybrid search engines 168 inbound marketing 180 indexer module 169 informational search 173 keyword conversion 184 keywords 169

link spamming 181 metadata 185 meta-search engines 168 navigational search 173 organic search listings 174 page repository 169 PageRank 180 paid search listings 174 pay-per-click (PPC) 174 petabyte 170 quality score 183 query interface 170 recommendation engines 173 resource description framework (RDF) 185 retrieval/ranking module 169 rich snippets 188 search engine 168 search engine marketing (SEM) 173

search engine optimization (SEO) 174 search engine results page (SERP) 166 semantic search engines 168 semantic Web 185 shadow pages 181 showrooming 175 social media optimization 174 SPARQL protocol and RDF query language (SPARQL) 185 spiders 168 structured data 172 transactional search 173 uniform resource identifiers (URIs) 185 unstructured data 172 vertical search engines 177 Web directories 168 Web ontology language (OWL) 185

Discuss: Critical Thinking Questions

1. Why is it important that businesses maintain a high level of visibil- ity on SERPs?

2. Why are organic search listings more valuable than paid search listings for most companies over the long term? Even though organic search listings are more valuable, what are some reasons that com- panies should consider using PPC advertising as part of their search marketing strategies?

3. Why is relevant and frequently updated content a significant factor for companies concerned about their visibility on popular search en- gines? Does the quality of content impact organic results, paid results, or both? Explain.

4. Explain the differences between Web directories, crawler search en- gines, and hybrid search engines.

5. Why do search engines consider their algorithms for rank ordering Web page listings on SERPs to be trade secrets? What would be the consequences of publicizing detailed information about how a search engine ranks its results?

6. Why do consumer search engines like Google and Bing require vast amounts of data storage? How have they addressed this need? What environmental issues are associated with the way large technology companies operate their data storage facilities?

7. Explain why enterprise search technology is becoming increasingly important to organizations. Describe how enterprise search applica- tions are different from consumer search engines in terms of their func- tionality, purpose, and the special challenges they must overcome.

8. Explain why people are much more likely to view and pay attention to product and service information in SERPs compared to traditional

mass media advertising? What strategies are businesses adopting to take advantage of this trend?

9. Why is it easier to measure the return-on-investment of re- sources spent on search engine marketing compared to mass media advertising?

10. How has the widespread adoption of mobile devices impacted the SEO practices?

11. Identify at least five ways that Google has changed its algo- rithms in recent years to encourage website developers to do more than simply list keywords in an attempt to improve their ranking on search results.

12. The ultimate goal of Google, Bing, and other consumer search en- gines is to provide users with a positive user experience. What recom- mendations would you make to a website owner with regard to using website content to improve the site’s rank on search result listings?

13. Why are “black hat” SEO techniques (see Section 6.2) considered unethical? Who is harmed by the use of such techniques? What are the consequences of using these questionable SEO tactics?

14. Explain how search engines determine if websites contain infor- mation relevant to a user’s search inquiry.

15. Identify and describe the five steps to creating a PPC ad campaign. 16. How does an advertiser’s bid and quality score determine the like- lihood of PPC ad placement on SERPs? What are the factors that Google uses to determine an advertiser’s quality score? Why does Google use the quality score instead of relying solely on the advertiser’s bid?

17. Describe three metrics used by PPC advertisers to evaluate the ef- fectiveness of their search ad campaign.

194 C H A P T E R 6 Search, Semantic, and Recommendation Technology

18. Identify the three things that SEO practitioners can optimize by making changes to on-page factors. What three things can SEO prac- titioners attempt to optimize by making changes to off-page factors?

19. Describe five ways that semantic search engines could enhance functionality for users. How will businesses benefit from the develop- ment of semantic search functions?

20. Recommender systems use different approaches to generating recommendations. Explain the difference between content-based

filtering and collaborative filtering. Describe the kind of information required for each approach to work.

21. What are the alternatives to content-based filtering and collabora- tive filtering recommender systems? When is it most useful to use these alternatives?

22. Hybrid recommendation engines utilize two or more filtering strategies to create recommendations. Describe the four different ap- proaches to creating a hybrid system.

Explore: Online and Interactive Exercises

1. Select a search query term or phrase based on a class assignment, a product you plan to purchase, or some area of personal interest. Use the query at each of the following search engines:

a. Google.com b. Bing.com c. Yahoo.com d. DuckDuckGo.com

For each site, make the following observations: a. How relevant or useful are the websites listed on the first two pages of search results?

b. What differences do you observe in terms of how the search engines list websites on the search results page?

c. Do you see any indication that the search engine is using semantic technology to generate results (see “Semantic Search Features and Benefits” in Section 6.4)?

2. Visit the website for fitbit products at fitbit.com and familiarize yourself with the products and website content.

a. Make a list of nonbranded keywords and phrases (i.e., doesn’t contain the word “fitbit”) that you would recommend fitbit use to optimize its pages so they show up in organic search listings.

b. Based on your list of keywords and phrases, make a list of recommendations for content (i.e., articles, blog posts, information, etc.) that fitbit should add to its website to increase the chances that it will show up in organic search results. What keywords should be emphasized in the content you recommend?

3. Use an existing account, or sign up for an account, at one of the websites listed in Table  6.3. Make a list of the ways the website

recommends its content, goods, or services to you. Based on your observations, are you able to determine what kind of recommendation system is in use by the website?

4. Pretend you are going to purchase an expensive item like a large flat-screen television or a major appliance from a national retailer like Best Buy or Sears. Using your mobile phone, attempt to find store locations, product information, and customer reviews. Next, install one of the popular shopping and price comparison apps listed below on your phone:

• Red Laser

• Amazon Price Check, or Flow (also by Amazon)

• Barcode Scanner

• Or, find a similar price-checking app at your mobile app store

Now, go shopping (visit the store). While shopping, use your mobile app to find product reviews and make price comparisons of the products you find. Briefly describe your mobile shopping experience. How did the mobile technology help or hinder your shopping experience? What challenges does mobile technology pose for traditional retailers?

5. Pretend you are an SEO consultant for a local business or not- for-profit organization. Visit the organization’s website to familiarize yourself with the brand, mission, products, services, and so on. Next, make a list of keywords or phrases that you think should be used to optimize the site for search engines. Rank-order the list based on how frequently you think the words are used in searches. Finally, go to google.com/trends/explore and enter your keywords or phrases, creating a graph that illustrates how often they have been used in search queries. Based on what you learn, what keywords or phrases would you recommend the organization use to optimize its site?

Analyze & Decide: Apply IT Concepts to Business Decisions

1. Perform a search engine query using the terms “data center” + “environmental impact.” Describe the environmental concerns that large-scale data centers are creating around the globe and steps that companies are taking to address these concerns. Read about Google’s efforts at environment.google. In your opinion, is Google making a satisfactory effort to minimize the negative impact of its business on the environment? Explain your answer.

2. Review the information in Section  6.1 about the three types of searches (informational, navigational, and transactional) that people conduct on search engines. Put yourself in the role of an SEO consultant for your college or university. Create a set of content and/or keyword strategies that you would recommend to your institution’s leaders to increase the chances of appearing on SERPs resulting from prospective students conducting each kind of search.

Case 6.2 195

3. Review the information about website relevance and credibility in Sec- tion  6.3. Next, generate a list of strategies or ways that a website owner might use to improve its ranking on search results pages by optimizing the site for relevance and credibility. For instance, if one of your factors is “site traffic,” you might recommend that the website owner post links to the website on the company Facebook page to increase traffic. Or, you might recommend the website run a contest that requires people to visit the site to enter. This would increase traffic during the contest.

4. Traditional brick-and-mortar stores are increasingly frustrated by competition with online retailers. Online websites often have a cost advantage because they do not have to maintain physical storefronts or pay salespeople, and can use more efficient logistical and opera- tional strategies. This sometimes allows them to offer better prices to consumers. With the emergence of recommendation engines, they appear to be gaining another advantage—the ability to suggest prod- ucts to customers based on their past shopping history and personal characteristics. Pretend you are a senior manager for a national retail chain. How could your company make use of recommendation sys- tems to suggest products to customers shopping in your store? Outline a creative approach to this problem that identifies the information you would need to collect, the in-store technology required, and the man- ner in which you would inform customers about the personalized rec- ommendations generated by your system.

5. Select a consumer product or service for which there are at least three popular brand names. For example, you might choose the cat- egory “cell phone carriers,” which includes Verizon, AT&T, Sprint, and T-Mobile. On the Google.com/trends page, type the brand names,

separated by commas, into the search field at the top of the screen (e.g., Verizon, AT&T, Sprint, T-Mobile). The resulting chart will display the search query volume by brand, an indicator of how much interest each brand has received over time. Using the Google Trends data, an- swer the following questions.

Tip: Before answering the questions below, use Google’s search engine to find articles on “how to interpret Google trends.” This will help you better understand the Google trends report and make it easier to answer the following questions.

a. Using the date setting at the top of the Google Trends page, ex- plore different periods of time. Briefly summarize how interest in each brand has changed over the last four years.

b. In the Regional Interest section, you can see how interest in each brand varies by country or city. In which countries and cities is each brand most popular?

c. In the Related Searches section, you will see a list of topics and query terms of interest to people who used one of the brand names in a search. How does the list of related topics change from one brand name to another? Do the topic and query term lists give you any insight into what kind of information people may be inter- ested in relative to each brand?

d. Using a search engine, see if you can find market share data for the product or industry you researched on Google Trends. If you find this information, does there seem to be any relationship between search volume and market share for the brand names you explored?

Case 6.2 Business Case: Deciding What to Watch—Video Recommendations at Netflix Netflix is the undisputed leader of video streaming services, account- ing for more than half (53%) of U.S. video streaming subscriptions. Amazon Prime Video (25%) and Hulu (13%) are the company’s largest competitors. Netflix is also the oldest company in this group, having originally started as a DVD by mail rental service. Unlike other com- panies that dominated the DVD rental business, Netflix successfully made the transition to on-demand video streaming by investing in new technology and redefining its business model. The service is now available in 190 countries and claims over 90 million subscrib- ers globally.

Netflix executives credit the company’s recommendation system for driving the “Netflix experience” and boosting profitability (Gomez-Uribe & Hunt,  2015; Raimond & Basilico,  2016). Surprisingly, the origin of the recommendation system dates back to 2000, when Netflix was still a DVD rental service. Recommendations during these early days were based largely on members’ movie ratings. Ratings often reflect how people want to be perceived as opposed to how they act. For instance, rating data will tend to overemphasize how much people like documentaries and foreign language films, whereas behavioral metrics provide more accurate meas- ures of how subscribers use the service. Today, when Netflix subscribers use the online service, they see recommendations generated by multiple

algorithms that use descriptive information about the subscriber and their past viewing behavior (Gomez-Uribe & Hunt,  2015). Netflix claims that 75% of the activity on the service is a result of the recommendations it offers subscribers.

Netflix Analytics Netflix enjoys a significant advantage over traditional television chan- nels because the company collects information about how subscrib- ers use the service. Netflix can make marketing and product decisions based on several behavioral metrics. You might be surprised at the details Netflix collects:

• The device you use (tablet, Roku, smart TV, etc.) • Where (zip code) you watch from • The days and times you watch • When you pause, rewind, or fast-forward during viewing • How you search—the words and phrases used, how long you

search, etc.

• Whether or not you watch the credits following a show • How many episodes of a series you watched • Whether or not you watch all episodes in a series • How long it takes you to watch all episodes in a series

196 C H A P T E R 6 Search, Semantic, and Recommendation Technology

• How many hours you spend using the service • What movies and television shows you watch • How often you use the service

In addition to making recommendations, Netflix uses the informa- tion to do the following:

• Identify subscribers who are likely to cancel the service • Select new movies to add to their catalog • Decide if a television show should be renewed for another season • Identify movies and television shows to drop from the catalog • Determine the days and times to recommend certain movies

or shows

• Determine what to recommend immediately following the view- ing of another movie or show

• Determine how to describe movies and shows (i.e., long vs. short descriptions)

Recommendation Algorithms at Netflix

The Netflix home screen can offer up to 40 rows of recommendations to a subscriber. Each row is generated by a different algorithm designed to personalize recommendations as well as determine the order in which movies and shows are listed. Each row is based on a different theme or rationale for the titles appearing in the row. Netflix even uses a Page Generation Algorithm to personalize the type of row-level recommendations and their order when creating the page. Some examples of the different recommendation rows include the following:

Genre Rows Several of the rows appearing on the home page are based on movie or television show genres that Netflix believes the subscriber will be interested in based on past viewing behavior. Genre rows are generated by what Netflix calls its Personalized Video Ranker (PVR). The rows reflect three levels of personalization: (1) the selection of the genre, (2) the selection of specific titles within the genre, and (3) the ordering of the titles.

Continue Watching Titles appearing in the Continue Watching row highlight episodic content that Netflix thinks a subscriber might want to return to. The Continue Watching ranker evaluates recently viewed videos for signals that a subscriber intends to resume watching or is no longer interested in the title. These signals include things like time since last viewing, point of abandonment (mid-program, end

of program), if other titles have been viewed since, and type of device used.

Because You Watched The Because you watched (BYW) row is based on the similarity of recommended videos to past videos watched by the subscriber. The BYW row is determined by the Sims Ranker, which generates an ordered list of videos, based on similarity, for every title in the catalog. Various personalization cues are then used to further refine the subset of videos that actually appear in the row on the home page.

Top Picks The goal of the Top Picks row is to feature Netflix’s best guess as to the videos in its catalog that are most likely to be of interest to the subscriber. The Top Picks algorithm uses cues from the individual subscriber along with viewing trend information to recommend titles from among the most popular or top-ranked videos in the catalog.

Netflix believes that its recommendation system plays a significant role in user satisfaction and customer retention. A team of workers regularly updates the system with new algorithms and modifications to existing ones. Their ultimate goal is to generate such high-quality recommendations that subscribers will rarely have to search for videos to watch.

Questions 1. You read about four different types of recommendations that Netf-

lix features on their home page. Think of a new type of recommen- dation row that Netflix could use and the kind of information or behavioral metrics that would be needed to generate your recom- mendations.

2. Based on the information in this case, would you say that Netflix primarily uses content-based filtering, collaborative filtering, or both? Explain your answer.

3. Netflix is expanding globally. When Netflix first enters a market, the recommendation system can face “cold start” or “sparsity” problems. Explain why this happens and suggests ways that Netf- lix might deal with these challenges.

4. What metrics do you think Netflix could use to identify subscribers who are likely to cancel the service?

5. Visit Netflix’s Technology Blog http://techblog.netflix.com. Iden- tify three challenges that the company faces in generating recom- mendations for its subscribers.

Sources: Compiled from Bulygo (2013b), Alvino and Basilico (2015), Gomez- Uribe and Hunt (2015), Arora (2016), Cheng (2016), Lubin (2016), Nicklesburg (2016), Raimond and Basilico (2016).

Case 6.3 Video Case: Power Searching with Google This video case is a bit different from what you have seen in other chap- ters. Google has created two easy-to-follow video courses designed to teach you how to use search engines more effectively: Power Search- ing and Advanced Power Searching. Each course contains a series of videos that you can view at your own pace. Following each video, you

are shown a set of activities and small quizzes that you can use to test your knowledge. Start with the Power Searching course. Once you have mastered the basic skills discussed in that course, move on to the Advanced Power Searching course.

Visit Google’s Search Education Online page powersearching- withgoogle.com. On this page, you will see links for the two self-guided courses: Power Searching and Advanced Power Searching. Select

References 197

the Power Searching link and begin viewing the course videos. After each video, do the related activities and test your knowledge with any online quizzes or tests that are provided. After you have completed the Power Searching course, go back and take the Advanced Power Search- ing course.

While it may take several days to complete both courses, we encour- age you to do so. The time you invest in learning these power search

techniques will pay off next time you need to use a search engine for a class- or work-related research project.

Question 1. Describe two or three search techniques you learned from these

tutorial videos that you think will be particularly helpful.

References Adomavicius, G. and T. Alexander. “Toward the Next Generation of

Recommender Systems: A Survey of the State-of-the-Art and Possi- ble Extensions.” IEEE Transactions on Knowledge and Data Engineer- ing 17, no. 6, 2005, 734−749.

Alvino, C. and J. Basilico. “Learning a Personalized Homepage.” tech- blog.netflix.com, April 9, 2015.

Andrews, W. and H. Koehler-Kruener. “Magic Quadrant for Enterprise Search.” gartner.com, July 16, 2014.

Anthony, S. “Microsoft Now Has One Million Servers−Less Than Google, But More Than Amazon, Says Ballmer.” extremetech.com, July 19, 2013.

Arora, S. “Recommendation Engines: How Amazon and Netflix Are Winning the Personalization Battle.” martechadvisor.com, June 28, 2016.

Asrar, S. “A Quick Look at Recommendation Engines and How the New York Times Makes Recommendations.” knightlab.northwestern.edu, March 28, 2016.

Brown, C. “43 Increasingly Precise Netflix Custom Genre Recommen- dations.” TheAwl.com, March 16, 2012.

Bulygo, Z. “How Mint Grew to 1.5 Million Users and Sold for $170 Mil- lion in Just 2 Years.” blog.kissmetrics.com, November, 2013a.

Bulygo, Z. “How Netflix Uses Analytics to Select Movies, Create Con- tent, and Make Multimillion Dollar Decisions.” blog.kissmetrics. com, 2013b.

Burke, R. “Hybrid Recommender Systems.” In Brusilovsky, P., A. Kobsa, and W. Nejdl (eds.), The Adaptive Web, pp. 377−408. Heidel- berg: Springer-Verlag Berlin, 2007.

Candela, J. “Building Scalable Systems to Understand Content.” code. facebook.com, February 2, 2017.

Cheng, R. “Netflix Leads a Streaming Video Market That’s Close to Peaking.” cnet.com, May 25, 2016.

Constine, J. “Facebook Sees 2 Billion Searches per Day, but It’s Attack- ing Twitter not Google.” techcrunch.com, July 27, 2016.

Delgado, J., L. Renaud, and V. Krishnamurthy. “The New Face of Enter- prise Search: Bridging Structured and Unstructured Information.” Information Management Journal 39, no. 6, 2005, 40−46. Business Source Premier. Online. February 28, 2014.

DiSilvestro, A. “The Difference Between Semantic Search and Seman- tic Web.” Search Engine Journal, July 10, 2013.

eMarketer. “Yahoo Ad Revenue to Drop Nearly 14% This Year.” eMar- keter.com, March 23, 2016.

Gallagher, S. “The Great Disk Drive in the Sky: How Web Giants Store Big—and We Mean Big—Data.” Arstechnica.com, January 26, 2012.

Glanz, J. “Google Details, and Defends, Its Use of Electricity.” The New York Times, September 8, 2011.

Glanz, J. “The Cloud Factories: Power, Pollution and the Internet.” The New York Times, September 22, 2012.

Gomez-Uribe, C. and N. Hunt. “The Netflix Recommender System: Algorithms, Business Value, and Innovation.” ACM Transactions Man- agement Information Systems 6, 4, Article 13, December 2015.

Google.com. “Algorithms.” Accessed March 24, 2014. Greene, K. “Mint Introduces Bill Pay, Helping Millions to Never Miss a

Bill.” blog.mint.com, December 13, 2016. Grehan, M. “How Search Engines Work.” Excerpted from Search

Engine Marketing: The Essential Best Practice Guide. New York: Inci- sive Media, 2002.

Grifantini, K. “What It Takes to Power Google.” MIT Technology Review, September 9, 2011.

Grimes, S. “Breakthrough Analysis: Two + Nine Types of Semantic Search.” InformationWeek.com, January 1, 2010.

Hendler, J. and T. Berners-Lee. “From the Semantic Web to Social Machines: A Research Challenge for AI on the World Wide Web.” Arti- ficial Intelligence 174, 2010.

Jacobson, J. “Google: 129 Million Different Books Have Been Pub- lished.” PCWorld.com, August 6, 2010.

Kraus, J. “The Advanced Guide to Facebook Graph Search.” sitepoint. com, August 18, 2015.

Lubin, G. “How Netflix Will Someday Know Exactly What You Want to Watch as Soon as You Turn Your TV On.” Businessinsider.com, September 21, 2016.

Madrigal, A. “How Netflix Reverse Engineered Hollywood.” The Atlan- tic, January 2, 2014.

Miller, R. “Google Has Spent $21 Billion on Data Centers.” Datacenter- knowledge.com, September 17, 2013.

Newman, J. “6 Things You’d Never Guess About Google’s Energy Use.” Techland.time.com, September 9, 2011.

Nicklesburg, M. “Study: Amazon Video Is Now the Third-largest Stream- ing Service, Behind Netflix and YouTube.” geekwire.com, June 22, 2016.

Oak, M. “How Does a Search Engine Work?” buzzle.com, June 5, 2008. Obi-Azubuike, G. “Five SEO Strategies That Will Grow Your Business.”

Linked.com/pulse, February 4, 2016 Osher, M. “Finding Copyright Infringements of Your Artwork on the

Internet.” MarianOsher.com, February 9, 2014. Prince, K.T. “Mint by the Numbers: Which User Are You?” blog.mint.

com, April 6, 2016. Prnewswire.com. “Growth of Big Data in Businesses Intensifies Global

Demand for Enterprise Search Solutions, Finds Frost & Sullivan.” January 24, 2013.

Raimond, Y. and J. Basilico. “Recommending for the World.” techblog. netflix.com, February 17, 2016.

198 C H A P T E R 6 Search, Semantic, and Recommendation Technology

Schneider, D. “Under the Hood at Google and Facebook.” spectrum. ieee.org, May 31, 2011.

Sullivan, D. “The Periodic Table of SEO Success Factors: 2015 Edition Now Released.” SearchEngineLand.com, June 1, 2015.

Sullivan, L. “Report: Companies Will Spend $65 Billion on SEO In 2016.” MediaPost.com, April 21, 2016.

Sukhraj, R. “How Mint Acquired Over 1.5 Million Users Without a Sin- gle Growth Hack.” Impactbnd.com, June 17, 2015.

Sverdlik, Y. “Here’s How Much Energy All US Data Centers Consume.” datacenterknowledge.com, June 27, 2016.

Thomas, V. “Six Reasons to Break Out of Your Organization’s Silos.” mangoapps.com, September 24, 2013.

Tiroshi, A., T. Kuflik, J. Kay, and B. Kummerfeld. “Recommender Systems and the Social Web.” In Proceedings of the International Workshop on Augmenting User Models with Real World Experiences to Enhance Personalization and Adaptation (AUM), Girona, Spain, July 15, 2011. Ardissono, L. and T. Kuflik (eds.), Springer-Verlag Berlin, Heidelberg, 2012.

Venkatraman, A. “Global Census Shows Data Centre Power Demand Grew 63% in 2012.” Computerweekly.com, October 8, 2012.

Walker, M. “Data Silos Obstruct Quest for Competitive Advantage.” datasciencecentral.com, February 11, 2014.

I teach college students how to succeed in their programs by leveraging the power of academic writing, without

cheating or cutting corners.

Hey there! I'm Chelsea.

I'm an online educator and academic writing expert.

Thank you for downloading this free guide on APA formatting 7th edition for STUDENT PAPERS!

Let's get started!

Page number

Paper title Bolded Capital Case

Author(s)

Affiliation line

Course info

Instructor

Due date

Title page

Inset page break

For full APA formatting tutorial click here!

Paper title No "introduction" heading before the first

paragraph. Title of your paper

signifies the introduction.

Parenthetical citation, one author

Narrative citation, one author

Parenthetical citation, multiple authors

Parenthetical citation, direct quote

Parenthetical citation, two authors

Having Trouble Meeting Your Deadline?

Get your assignment on Leveraging Search Technologi completed on time. avoid delay and – ORDER NOW

Get a 20% Discount

For full in-text citation tutorial click here!

First page/ in-text citations

Main section heading

Subsection of Level 1

Subsection of Level 2

Centered Bolded Title Case

Flush left Bolded Title Case

Flush left Bolded italic Title Case

Indented Bolded Title Case End with period

Indented Bolded italic Title Case End with period

Subsection of Level 3

Subsection of Level 4

For full section heading tutorial click here!

Section headings

Inset page break

Reference List

For full reference list tutorial click here!

Centered & Bolded

4 elements all references Author. (Date). Title. Source

Inset page break

Centered & bolded

DO NOT indent first line

For full ABSTRACT tutorial click here!

ABSTRACT/ Running Head

If you're serious about mastering academic writing once and for all, save your seat for my FREE academic writing workshop. I'm breaking the

ENTIRE writing process down so you know exactly what you need to improve your college paper writing skills – See you inside!

The exact powerful method that is helping students all over the world do the same?

Join my FREE FACEBOOK GROUP where I dive DEEPER into the #SmartStudentWriting strategy

Want to learn my step-by-step proven method on how I streamlined the academic writing process?

The same powerful method that enabled me to scored 95% and above on all my academic writing assignments? Yes, even in my hardest classes!

WANT MORE HELP?

Join Smart Student!

Yes!

Yes!

Yes!

FREE Academic Writing Training -Stay

smar t