MENU

SEO: How Search Engines Work – Franklin TN

search engines

Search engines crawl the web, sifting through billions of data points to serve up information in a fraction of a second. The access to instant information we’ve come to take for granted is based on an enormous system of data retrieval and software.

Google has been the most forthcoming about how its search engine works, so I’ll use it as an example.

At the simplest level, search engines do two things.

  • Index information.Discover and store information about 30 trillion individual pages on the World Wide Web.
  • Return results.Through a sophisticated series of algorithms and machine learning, identify and display to the searcher the pages most relevant to her search query.

Crawling and Indexation

How did Google find 30 million web pages? Over the past 18 years, Google has been crawling the web, page by page. A software program called a crawler — also known as a robot, bot, or spider — starts with an initial set of web pages. To get the crawler started, a human enters a seed set of pages, giving the crawler content and links to index and follow. Google’s crawling software is called Googlebot, Bing’s is called Bingbot, and Yahoo uses Slurp.

When a bot encounters a page, it captures the information on that page, including the textual content, the HTML code that renders the page, information about how the page is linked to, and the pages to which it links.

As Googlebot crawls, it discovers more and more links. The image below shows a very simplistic diagram of a single, three-page crawl path on Jerry’s Artarama, a discount art supplies E-commerce site. Screen Shot 2016-03-30 at 2.51.35 PM

The logo at left indicates a starting point at the site’s home page, where Googlebot encounters 184 links: the 10 listed and 174 more. When Googlebot follows the “Acrylic Paints and Mediums” link in the header navigation, it discovers another page. The “Acrylic Paints and Mediums” page has 135 links on it. When Googlebot follows the link to another page, such as “Winsor & Newton Acrylics and Mediums,” it encounters 108 links. The example ends there, but crawlers continue accessing pages via the links on each page they discover until all of the pages considered relevant have been discovered.

In the process of crawling a site, bots will encounter the same links repeatedly. For example, the links in the header and footer navigation should be on every page. Instead of recrawling the content in the same visit, Googlebot may just note the relationship between the two pages based on that link and move on to the next unique page.

All of the information gathered during the crawl — for 30 trillion web pages — is stored in enormous databases in enormous data centers. To get an idea of the scale of just one of its 15 data centers, watch Google’s official tour video “Inside a Google data center.”

As bots crawl to discover information, the information is stored in an index inside the data centers. The index organizes information and tells a search engine’s algorithms where to find the relevant information when returning search results.

But an index isn’t like a dark closet that everything gets stuffed into randomly as it’s crawled. Indexation is tidy, with discovered web page information stored along with other relevant information, such as whether the content is new or an updated version, the context of the content, the linking structure within that particular website and the rest of the web, synonyms for words within the text, when the page was published, and whether it contains pictures or video.

Returning Search Results

Results are displayed after you search for something in a search engine. Every web page displayed is called a search result, and the order in which the search results are displayed is known as ranking.

But once information is crawled and indexed, how does Google decide what to show in search results? The answer, of course, is a closely guarded secret.

How a search engine decides what to display is loosely referred to as its algorithm. Every search engine uses proprietary algorithms that it has designed to pull the most relevant information from its indices as quickly as possible in order to display it in a manner that its human searchers will find most useful.

For instance, Google Search Quality Senior Strategist Andrey Lipattsev confirmed that Google’s top three search ranking factors are content, links, and RankBrain, a machine learning artificial intelligence system. Regardless of what each search engine calls its algorithm, the basic functions of modern search engine algorithms are similar.

Content determines contextual relevance. The words on a page, combined with the context in which they are used and to pages they are linked to, determines how the content is stored in the index and which search queries it might answer.

Links determine authority and relevance. In addition to providing a pathway for crawling and discovering new content, links also act as authority signals. Authority is determined by measuring signals related to the relevance and quality of the pages linking into each individual page, as well as the relevance and quality of the pages to which that page links.

Search engine algorithms combine hundreds of signals with machine learning to determine the match between each page’s context and authority and the searcher’s query to serve up a page of search results. A page needs to be among the top seven to 10 most-highly-matched pages algorithmically, in both contextual relevance and authority, to be displayed on the first page of search results.

Contact us about our SEO services.


It’s remarkable working with the team that treats your business they would their own with drive, innovation, humility and a shared love of bringing great things to the wonderful Franklin & Nashville community.

 

RED DOG WINE & SPIRITS – Nashville

 

“In the search of looking high and low for a website design firm, we came across JLB. They are now like a part of our family and strive to do all they can to take our sales and growth to a whole new level. We recommend JLB to everyone we come across looking for a solution to their website needs.”

 

Chad, BOWADX – Franklin TN


“JLB has been the driving force  that has launched our company into the media spotlight, gaining us publicity through a broad cross-section of business related media including new apps and interactive activity for our clients. We are moving forward with our second web design with JLB and will continue to make them part of our team.”

 

John, The Perch – Brentwood TN


“I started with JLB when they first opened fifteen years ago, when they redesigned a website for me.  They are my go-to company for anything to do with web sites. These guys are professional, talented, creative and an energetic team. I will always go to JLB first for my web design work. Thanks JLB.”

 

Phyllis, Carneros Winery – Nashville TN


We were having problems with our online brand and digital marketing. JLB was great. Worked through it with us, created an amazing presence and business is great. The staff at JLB is fantastic too. Friendly and always there for you.

 

Life Line Repairs – Nashville TN


I and all of us at Sports & Entertainment Nashville TN have been thrilled with the website you guys did for us, and especially the professional service and communication you guys have shown.

 

Sports and Entertainment – Nashville


We interviewed several web design firms, but found that JLB’s talented team was the most creative, innovative, and professional. Choosing JLB has been a great decision! We are fortunate to have JLB be a part of the BMC Metalwork’s future as we continue to enhance our business.

 

Jimmy, BMC – Franklin TN