In 1992, Gopher entered the scene, becoming the first search engine to use a hypertext paradigm.
Web Crawler, which arrived two years later, was the first search engine to provide full text search and remains the oldest surviving search engine on the internet today.
Yahoo! also came on the scene in 1994 and quickly came to dominate the space.
In those days, the folks in charge of websites would submit their page’s URL to sites like Web Crawler and Gopher.
At that point, a “spider” would “crawl” through the page, extracting links to other pages as it went, and returning those links and other information to be indexed.
But this was no easy undertaking. These efforts were undertaken in the days before concepts like “cloud computing” were even a glimmer in the collective eye of the internet.
The search engine spider would need to download these pages and store them on the search engine’s own server. Once there, a second program – known as an indexer – would extract information about the page like the words it contained, where it was located, and any links the page contained.
The term “search engine optimization” is thought to have come into existence in 1997, according to industry analyst Danny Sullivan.
In these early days of SEO, algorithms relied on webmaster-provided information like keyword meta tags or index files. Meta tags would provide a guide to each page’s content.
This proved rather unruly though, as webmaster’s could easily manipulate the keywords in the meta tag to misrepresent their site’s content.
This made it very difficult for these nascent search engines to create an accurate overview of the internet.
On the flipside, webmasters suffered from a drought of data. Unless you had BIG BUCKS to spend, there weren’t really analytics available for websites.
What few options did exist to help provide insight certainly didn’t come cheap.
Enter Google
Even as search engines and web developers continued to bump up against issues, it was becoming increasingly clear that there was a lot of money to be made on this crazy newfangled thing called the internet.