Search Engine Optimization (SEO)
Search engine optimization (SEO) is the process of improving
the quality and quantity of website traffic to a website or
a web page from search engines. SEO targets unpaid traffic
(known as "natural" or "organic" results) rather than
direct traffic or paid traffic. Unpaid traffic may originate from
different kinds of searches, including image search, video
search, academic search, news search, and
industry-specific vertical search engines.
As an Internet marketing strategy, SEO
considers how search engines work, the
computer-programmed algorithms that dictate search engine behavior,
what people search for, the actual search terms or keywords typed
into search engines, and which search engines are preferred by their targeted
audience. SEO is performed because a website will receive more visitors from a
search engine when websites rank higher on the search engine results
page (SERP). These visitors can then potentially be converted into
customers.
History
Webmasters and content providers began
optimizing websites for search engines in the mid-1990s, as the first search
engines were cataloging the early Web. Initially, all webmasters only
needed to submit the address of a page, or URL, to the various engines
which would send a web crawler to crawl that page,
extract links to other pages from it, and return information found on the page
to be indexed. The process involves a search engine spider
downloading a page and storing it on the search engine's own server. A second
program, known as an indexer, extracts information about the page, such as
the words it contains, where they are located, and any weight for specific
words, as well as all links the page contains. All of this information is then
placed into a scheduler for crawling at a later date.
Website owners recognized the value of a
high ranking and visibility in search engine results, creating
an opportunity for both white hat and black hat SEO
practitioners. According to industry analyst Danny Sullivan, the phrase
"search engine optimization" probably came into use in 1997. Sullivan
credits Bruce Clay as one of the first people to popularize the term.
Early versions of
search algorithms relied on webmaster-provided information such as
the keyword meta tag or index files in engines like ALIWEB. Meta
tags provide a guide to each page's content. Using metadata to index pages was
found to be less than reliable, however, because the webmaster's choice of
keywords in the meta tag could potentially be an inaccurate representation of
the site's actual content. Inaccurate, incomplete, and inconsistent data in
meta tags could and did cause pages to rank for irrelevant searches. Web
content providers also manipulated some attributes within
the HTML source of a page in an attempt to rank well in search
engines. By 1997, search engine designers recognized that webmasters were
making efforts to rank well in their search engines and that some webmasters
were even manipulating their rankings in search results by stuffing
pages with excessive or irrelevant keywords. Early search engines, such
as Altavista and Infoseek, adjusted their algorithms to prevent
webmasters from manipulating rankings.
By heavily relying on factors such
as keyword density, which were exclusively within a webmaster's control, early
search engines suffered from abuse and ranking manipulation. To provide better
results to their users, search engines had to adapt to ensure
their results pages showed the most relevant search results, rather
than unrelated pages stuffed with numerous keywords by unscrupulous webmasters.
This meant moving away from heavy reliance on term density to a more holistic
process for scoring semantic signals. Since the success and popularity of
a search engine are determined by its ability to produce the most relevant
results to any given search, poor quality or irrelevant search results could
lead users to find other search sources. Search engines responded by developing
more complex ranking algorithms, taking into account additional factors that
were more difficult for webmasters to manipulate.
Companies that employ overly aggressive
techniques can get their client websites banned from search results. In
2005, the Wall Street Journal reported on a
company, Traffic Power, which allegedly used high-risk techniques and
failed to disclose those risks to its clients. Wired magazine
reported that the same company sued blogger and SEO Aaron Wall for writing
about the ban. Google's Matt Cutts later confirmed that Google
did in fact ban Traffic Power and some of its clients.
Some search engines have also reached out to The SEO industry, and are frequent sponsors and guests at SEO conferences,
webchats, and seminars. Major search engines provide information and guidelines
to help with website optimization. Google has a Sitemaps program
to help webmasters learn if Google is having any problems indexing their
website and also provides data on Google traffic to the website. Bing
Webmaster Tools provides a way for webmasters to submit a sitemap and web
feeds, allows users to determine the "crawl rate", and track the web
pages index status.
In 2015, it was reported
that Google was developing and promoting mobile search as a key
feature within future products. In response, many brands began to take a
different approach to their Internet marketing strategies.
In 1998, two graduate students at Stanford
University, Larry Page and Sergey Brin, developed
"Backrub", a search engine that relied on a mathematical algorithm to
rate the prominence of web pages. The number calculated by the algorithm, PageRank,
is a function of the quantity and strength of inbound links. PageRank
estimates the likelihood that a given page will be reached by a web user who
randomly surfs the web, and follows links from one page to another. In effect,
this means that some links are stronger than others, as a higher PageRank page
is more likely to be reached by the random web surfer.
Page and Brin founded Google in 1998. Google
attracted a loyal following among the growing number of Internet users,
who liked its simple design. Off-page factors (such as PageRank and
hyperlink analysis) were considered as well as on-page factors (such as keyword
frequency, meta tags, headings, links and site structure) to enable Google
to avoid the kind of manipulation seen in search engines that only considered
on-page factors for their rankings. Although PageRank was more difficult
to game, webmasters had already developed link-building tools and schemes
to influence the Inktomi search engine, and these methods proved
similarly applicable to gaming PageRank. Many sites focused on exchanging,
buying, and selling links, often on a massive scale. Some of these schemes,
or link farms, involved the creation of thousands of sites for the sole
purpose of link spamming.
By 2004, search engines had incorporated a wide
range of undisclosed factors in their ranking algorithms to reduce the impact
of link manipulation. In June 2007, The New York Times' Saul
Hansell stated Google ranks sites using more than 200 different signals. The
leading search engines, Google, Bing, and Yahoo, do not disclose the
algorithms they use to rank pages. Some SEO practitioners have studied
different approaches to search engine optimization, and have shared their
personal opinions. Patents related to search engines can provide
information to better understand search engines. In 2005, Google began
personalizing search results for each user. Depending on their history of
previous searches, Google crafted results for logged-in users.
In 2007, Google announced a campaign against
paid links that transfer PageRank. On June 15, 2009, Google disclosed that
they had taken measures to mitigate the effects of PageRank sculpting by use of
the Nofollow attribute on links. Matt Cutts, a well-known
software engineer at Google announced that Google Bot would no longer treat
any no-follow links, in the same way, to prevent SEO service providers from
using no follow for PageRank sculpting. As a result of this change, the
usage of no follow led to the evaporation of PageRank. To avoid the above,
SEO engineers developed alternative techniques that replace no-followed tags
with obfuscated JavaScript and thus permit PageRank sculpting.
Additionally, several solutions have been suggested that include the usage
of iframes, Flash, and JavaScript.
In December 2009, Google announced it would be
using the web search history of all its users to populate search
results. On June 8, 2010, a new web indexing system called Google
Caffeine was announced. Designed to allow users to find news results,
forum posts, and other content much sooner after publishing than before, Google
Caffeine was a change to the way Google updated its index to make
things show up quicker on Google than before. According to Carrie Grimes, the software
engineer who announced Caffeine for Google, "Caffeine provides 50 percent
fresher results for web searches than our last index..." Google
Instant, real-time-search, was introduced in late 2010 in an attempt to make
search results more timely and relevant. Historically site administrators have
spent months or even years optimizing a website to increase search rankings.
With the growth in popularity of social media sites and blogs, the leading
engines made changes to their algorithms to allow fresh content to rank quickly
within the search results.
In February 2011, Google announced the Panda update,
which penalizes websites containing content duplicated from other websites and
sources. Historically websites have copied content from one another and
benefited in search engine rankings by engaging in this practice. However,
Google implemented a new system that punishes sites whose content is not
unique. The 2012 Google Penguin attempted to penalize websites
that used manipulative techniques to improve their rankings on the search
engine. Although Google Penguin has been presented as an algorithm aimed
at fighting web spam, it really focuses on spammy links by gauging the quality of the sites the links are coming from. The
2013 Google Hummingbird update featured an algorithm change designed
to improve Google's natural language processing and semantic understanding of
web pages. Hummingbird's language processing system falls under the newly
recognized term of "conversational search" where the system pays more
attention to each word in the query to better match the pages to the
meaning of the query rather than a few words. With regards to the changes
made to search engine optimization, for content publishers and writers,
Hummingbird is intended to resolve issues by getting rid of irrelevant content
and spam, allowing Google to produce high-quality content and rely on them to
be 'trusted' authors.
In October 2019, Google announced they would
start applying BERT models for English language search queries in the US. Bidirectional Encoder Representations from Transformers (BERT) was another
attempt by Google to improve their natural language processing but this time to better understand the search queries of their users. In terms of
search engine optimization, BERT intended to connect users more easily to relevant
content and increase the quality of traffic coming to websites that are ranking
in the Search Engine Results Page.
Methods
Search engine crawlers may look at some different factors when crawling a site. Not every page is indexed
by search engines. The distance of pages from the root directory of a site
may also be a factor in whether or not pages get crawled.
Today, most people are searching on Google using
a mobile device. In November 2016, Google announced a major change to the
way crawling websites and started to make their index mobile-first, which means
the mobile version of a given website becomes the starting point for what
Google includes in their index. In May 2019, Google updated the rendering
engine of their crawler to be the latest version of Chromium (74 at the time of
the announcement). Google indicated that they would regularly update the Chromium rendering
engine to the latest version. In December 2019, Google began updating the A user-Agent string of their crawler to reflect the latest Chrome version used by
their rendering service. The delay was to allow webmasters time to update their
code that responded to particular bot User-Agent strings. Google ran
evaluations and felt confident the impact would be minor.
Preventing crawling
To avoid undesirable content
in the search indexes, webmasters can instruct spiders not to crawl certain
files or directories through the standard robots.txt file in the root
directory of the domain. Additionally, a page can be explicitly excluded from a
search engine's database by using a meta tag specific to robots
(usually <meta name="robots" content="no index"> ).
When a search engine visits a site, the robots.txt located in the root
directory is the first file crawled. The robots.txt file is then parsed
and will instruct the robot as to which pages are not to be crawled. As a
search engine crawler may keep a cached copy of this file, it may on occasion
crawl pages a webmaster does not wish crawled. Pages typically prevented from
being crawled include login-specific pages such as shopping carts and
user-specific content such as search results from internal searches. In March
2007, Google warned webmasters that they should prevent indexing of internal
search results because those pages are considered search spam. In 2020
Google sunsetted the standard (and open-sourced their code) and now treats it
as a hint, not a directive. To adequately ensure that pages are not indexed a
page-level robot's meta tag should be included.
Increasing prominence
A variety of methods can increase the prominence
of a webpage within the search results. Cross-linking between pages
of the same website to provide more links to important pages may improve its
visibility. Writing content that includes frequently searched keyword phrases,
to be relevant to a wide variety of search queries will tend to increase
traffic. Updating content to keep search engines crawling back frequently
can give additional weight to a site. Adding relevant keywords to a web page's
metadata, including the title tag and meta description, will tend to
improve the relevancy of a site's search listings, thus increasing
traffic. URL canonicalization of web pages accessible via multiple
URLs, using the canonical link element or via 301 redirects can
help make sure links to different versions of the URL all count towards the
page's link popularity score.
Also, in recent times Google is giving more
priority to the below elements for SERP (Search Engine Ranking Position).
·
HTTPS version (Secure Site)
·
Page Speed
·
Structured Data
·
Mobile Compatibility
·
AMP (Accelerated Mobile Pages)
·
BERT
White hat versus black hat techniques
SEO techniques can be classified into two broad
categories: techniques that search engine companies recommend as part of good
design ("white hat"), and those techniques of which search engines do
not approve ("black hat"). The search engines attempt to minimize the
effect of the latter, among them spamdexing. Industry commentators have
classified these methods, and the practitioners who employ them, as
either white hat SEO, or black hat SEO. White hats
tend to produce results that last a long time, whereas black hats anticipate
that their sites may eventually be banned either temporarily or permanently
once the search engines discover what they are doing.
An SEO technique is considered a white hat if it
conforms to the search engines' guidelines and involves no deception. As the
search engine guidelines are not written as a series of rules or
commandments, this is an important distinction to note. White hat SEO is not
just about following guidelines but is about ensuring that the content a search
engine indexes and subsequently ranks is the same content a user will see.
White hat advice is generally summed up as creating content for users, not for
search engines, and then making that content easily accessible to the online
"spider" algorithms, rather than attempting to trick the algorithm
from its intended purpose. White hat SEO is in many ways similar to web
development that promotes accessibility, although the two are not identical.
Black hat SEO attempts to improve rankings
in ways that are disapproved of by the search engines, or involve deception.
One black hat technique uses hidden text, either as text colored similar to the
background, in an invisible div, or positioned off-screen. Another method
gives a different page depending on whether the page is being requested by a
human visitor or a search engine, a technique known as cloaking. Another
category sometimes used is grey hat SEO. This is in between black hat and
white hat approaches, where the methods employed avoid the site being penalized
but do not act in producing the best content for users. Grey hat SEO is
entirely focused on improving search engine rankings.
Search engines may penalize sites they discover
using black or grey hat methods, either by reducing their rankings or
eliminating their listings from their databases altogether. Such penalties can
be applied either automatically by the search engines' algorithms, or by a
manual site review. One example was the February 2006 Google removal of
both BMW Germany and Ricoh Germany for use of deceptive practices. Both
companies, however, quickly apologized, fixed the offending pages, and were
restored to Google's search engine results page.
As marketing strategy
SEO is not an appropriate strategy for every
website, and other Internet marketing strategies can be more effective, such as
paid advertising through pay per click (PPC) campaigns, depending on
the site operator's goals. Search engine marketing (SEM) is the
practice of designing, running, and optimizing search engine ad campaigns. Its
difference from SEO is most simply depicted as the difference between paid and
unpaid priority ranking in search results. SEM focuses on prominence more so
than relevance; website developers should regard SEM with the utmost importance
with consideration to visibility as most navigate to the primary listings of
their search. A successful Internet marketing campaign may also depend
upon building high-quality web pages to engage and persuade internet users,
setting up analytics programs to enable site owners to measure
results, and improving a site's conversion rate. In November 2015,
Google released a full 160-page version of its Search Quality Rating Guidelines
to the public, which revealed a shift in their focus towards
"usefulness" and mobile local search. In recent years the mobile market has exploded, overtaking the use of desktops, as shown by StatCounter in
October 2016 where they analyzed 2.5 million websites and found that 51.3% of
the pages were loaded by a mobile device. Google has been one of the
companies that are utilizing the popularity of mobile usage by encouraging
websites to use their Google Search Console, the Mobile-Friendly Test,
which allows companies to measure up their website to the search engine results
and determine how user-friendly their websites are.
SEO may generate an adequate return on
investment. However, search engines are not paid for organic search traffic,
their algorithms change, and there are no guarantees of continued referrals. Due
to this lack of guarantee and uncertainty, a business that relies heavily
on search engine traffic can suffer major losses if the search engines stop
sending visitors. Search engines can change their algorithms, impacting a
website's search engine ranking, possibly resulting in a serious loss of
traffic. According to Google's CEO, Eric Schmidt, in 2010, Google made over 500
algorithm changes – almost 1.5 per day. It is considered a wise
business practice for website operators to liberate themselves from dependence
on search engine traffic. In addition to accessibility in terms of web
crawlers (addressed above), user web accessibility has become
increasingly important for SEO.
International markets
As of 2009, there are only a few large markets
where Google is not the leading search engine. In most cases, when Google is
not leading in a given market, it is lagging behind a local player. The most
notable example markets are China, Japan, South Korea, Russia, and the Czech
Republic where respectively Baidu, Yahoo! Japan, Naver, Yandex, and Seznam are
market leaders.
Successful search optimization for international markets may require professional translation of web pages, registration of a domain name with a top-level domain in the target market, and web hosting that provides a local IP address. Otherwise, the fundamental elements of search optimization are essentially the same, regardless of language.
0 Comments