Put in the simplest terms, the internet is like a huge collection of pages that are connected by links. If we think of it as a city – your website is like your house, where each room in the house is a different page on the site, with internal links that act like hallways and stairs connecting the different rooms. There are different houses on the street, but everything in the neighbourhood is related. By adding even more houses, we get a city.
Outbound links act as roads – they connect the different buildings and neighbourhoods together. A big, popular, website is like a large shopping mall–with lots of roads leading to it. It’s easy to see, and there are plenty of different ways to get there.
If you’re building a new site, it’s like you’re adding a new building to the city. Where would you rather have that building? If you’re right on a major intersection and several cross streets, it’ll be easy to find. But if you build it in the outskirts of town in the middle of a huge field, no one will know it’s there.
When the World Wide Web first started, there weren’t that many sites, so you could only reach a website by directly typing in the address of the page or by clicking a link from another website.
In the early 90s, the web had outgrown this simple navigation and discovery process. The first search engines were born – now users could type keywords into the search engine and receive a list of websites related to the search phrase. Excite, AltaVista, and Yahoo! quickly changed the way that people were using the internet.
In the late 90s, Larry Page and Sergey Brin were working on their own search engine, and Google was the first to use links as a signal of authority and trust. When they launched in 1998, it quickly became the favoured search engine because users found the quality of the search results better than other options.
Google’s search engine created advanced algorithms that analyze hundreds of factors for each website – both on and off the website. The algorithms are constantly updated and adjusted to ensure that search results are the highest quality they can be.
- Learn the basics of SEO according to Google’s best practices with the ‘SEO Starter Guide‘ by Google
Crawling, Indexing and Ranking
Now let’s talk about how search engines actually work. Yes, Google’s algorithm is extremely complex but put simply, Google is a pattern detection program. When you search for a keyword or phrase, Google is going to provide you with a list of websites that match patterns related to your search.
Most people don’t realize that when they do a search on Google, they’re not actually searching the live web. What they’re actually searching is Google’s index of the web… the stored copy of the sites that Google has crawled.
Google uses little programs called spiders to crawl the web. The spiders are how Google actually discovers content. Basically, a spider will start on a page and check out all the content on that page, and then it follows the links on that page and looks at the content on those pages. Then it follows the links on those pages to discover even more content, and then the links on those pages lead to more content… The spiders keep crawling at a mind-blowingly massive scale until most of the internet has been crawled.
As the spiders crawl all of these pages, a copy is stored on Google’s servers. This is Google’s index, and it’s stored in a way that allows Google to quickly search through the billions of pages in the index. When you search for a certain phrase, Google checks the index to get a list of every page that’s related to your search phrase.
Here’s where ranking comes into play – this is the part that SEOs work to influence. Google uses an algorithm to look at that list of pages it just pulled from the index and rank the pages based on relevancy.
Let’s say you did a search for “Caesar cocktail recipe” – Google’s going to search the index and come up with a list of every website with a recipe for a Caesar cocktail. Then it’s going to use the algorithm to sort that list so that the most relevant sites will be at the top.
There are hundreds of ranking factors, each with a different assigned weight or value. The algorithm will look at a few hundred different factors that influence relevancy, like the content on the page, the number of other sites that link to that page, and the overall quality of the website.
When we do SEO, we’re hoping to influence those relevancy scores. We know that if we optimize the right signals, Google’s algorithm will decide that the page is more relevant – that it’s a better answer to the question being asked – and show that page higher than the other options in the index.
Continue learning by using this list of educational resources:
- Get a deeper understanding of crawling and indexing, search algorithms, and useful responses from ‘How Google Search Works’
- Check if your website is easy to crawl and index with the ‘What are the Crawlability and Indexability of a Website?’ blog post
- Learn about factors that affect your site’s ‘crawlability’ with ’18 Reasons Your Website is Crawler-Unfriendly: Guide to Crawlability Issues’