Help search engines find and understand the pages
Find out why an important page might be missing from Google, and know what to check before changing the website.
noindex setting; prepare a sensible investigation and verification plan.The page works. So why can't people find it?
In the last lesson, Sam planned a better mobile experience for ClearFlow's water heater repair page. The draft is easier to read, and the phone and contact-form links are easier to use.
But Sam remembers something from the earlier website review.
The earlier review showed that the water heater repair page opened normally, but was sending a setting that told Google to leave it out of search results.
That's important because ClearFlow wants more relevant inquiries about water heater repair from Austin homeowners. Even an excellent page design won't solve a setting that tells Google to leave the page out.
Sam's next job isn't to change the design again. It's to find out what the website is telling search engines.
Three things a search engine needs to do
Imagine a homeowner looking for a plumber in Austin. Before Google can consider showing a page in its regular search results, it needs to process that page.
Here's a simple way to think about that process:
| Step | What it means | Why a designer should care |
|---|---|---|
| Find | Google discovers a page address, perhaps through a link on another page or a sitemap. | Visitors and search engines need sensible paths to important pages. |
| Visit | Google's crawler requests the page so it can read the content and relevant instructions. This is called crawling. | A page must be accessible to the crawler for Google to read its content and page-level instructions. |
| Consider for the index | Google processes what it finds and decides whether to keep the page in its search index. The index is the collection Google uses to serve results. | A page can be found and visited without being indexed. |
Being indexed is different from ranking. Indexing makes a page eligible to be considered for relevant searches; it doesn't mean Google must show it for a particular search or in a particular position.
Now Sam checks which part of this process matters for ClearFlow.
Worked example: the water heater page
Sam opens the notes from ClearFlow's website review on 20 October.
That short line of code is the important clue:
<meta name="robots" content="noindex">
noindex is an instruction telling search engines that support it not to include this page in their search index.
A visitor with the page address can still open it. A link on the homepage can still take someone there. But those things don't cancel the noindex instruction.
Sam doesn't need to rewrite the SEO title or make the heading longer to solve this. Those are different parts of the website, and he has already drafted them in Lesson 7.
The next decision is to check whether the instruction is still there, and arrange the right correction if it is.
What Sam checks before suggesting a fix
Sam already knows ClearFlow's owner did not mean to keep the water heater repair page out of Google. The owner has agreed that the issue can be investigated.
Before anyone changes the live website, Sam needs two things: current evidence and permission for the actual change.
Here's how Sam would organize that work with the right access or with the client's developer.
1. Inspect the exact page.
Open the existing address /water-heater-repair, not a new URL made just for SEO. If authorized access to Google Search Console is available, use URL Inspection to review what Google last recorded about the page.
Sam also needs a live check: what instruction is the page sending now? Google's recorded indexing information and a live test can differ because Google may not have visited the latest version yet. A developer can help check the page's search settings and the instructions actually sent to crawlers.
2. Find where the instruction comes from.
The setting might be controlled by a website platform, SEO plugin, page configuration or code. Sam shouldn't assume which system ClearFlow uses. The responsible person can inspect the actual setup.
3. Get approval before changing it.
The owner has permitted an investigation, but removing a search-indexing restriction changes how the page may become visible. Sam should explain the finding, confirm the intended result and obtain approval for the specific change.
4. Correct and check it.
Once authorized, the developer or site administrator can remove the unintended noindex instruction from the intended public page, then verify what the live page sends. Google Search Console can be used to request another crawl when appropriate and to monitor later indexing information.
An important point: a live test showing that Google can index a page is not the same as the page already being indexed. Those are two different checks.
Sam's job as a designer is to recognize the issue, describe it clearly and follow it through with the right person. He doesn't need to guess at the fix or make changes outside his access and approval.
What about sitemaps, robots.txt and canonical URLs?
A few other technical terms come up during website projects. Sam gives each one a simple job instead of treating them as interchangeable fixes.
| Term | What it does | What it means for ClearFlow |
|---|---|---|
| Internal link | Lets people and crawlers follow a path between pages. | The homepage already links to water heater repair. Sam has also drafted a useful article-to-service link, but it hasn't been added to the live page. |
| Sitemap | Provides search engines with a list of site URLs the business wants them to discover. | Sam can check that the existing service URL is listed when reviewing the site setup. A sitemap doesn't cancel noindex or guarantee indexing. |
| robots.txt | Gives crawlers instructions about which site URLs they may request. | Sam should not change it without evidence of a crawl-access problem. It is not the same as a noindex instruction. |
| Canonical URL | Helps signal which URL is preferred when very similar versions exist. | ClearFlow already has one main water heater repair URL. There is no evidence here of a canonical problem or a reason to create another page. |
There's one detail worth understanding: blocking a crawler in robots.txt isn't a reliable way to keep a web address out of Google results. In some situations Google can know about a blocked address from links elsewhere, even if it cannot read the page. And if a crawler cannot access a page, it cannot read a page-level noindex instruction there either.
This is why Sam checks the actual issue before choosing a technical setting.
You may also hear about structured data: extra, organized information in a website's code that helps supported search features interpret certain page details. It must reflect the real, visible page content. It isn't a substitute for indexability, and there's no reason to invent a special tag to solve ClearFlow's noindex problem.
The same foundation matters when people ask about AI-powered search. For Google's AI features to use a page as a supporting link, the page must meet the usual search eligibility requirements. No special AI-only setting guarantees that a page will be included. We'll explore AI visibility in more detail later in the course.
Sam's short handover
Sam now prepares something the owner and developer can actually use.
Notice what Sam has not done: he hasn't moved the water heater page, published the redesigned content, created a second service page or claimed that Google now shows the page. He's identified the right next action.
Try this on your own studio website
Before publishing Cedarline's new service copy, Sam can perform a similar check on the existing /services page.
He can confirm that the page opens, that important internal links point to it, and, with the right access, what Google Search Console says about its indexing status. If he finds an issue, he can record it before changing settings.
Sam doesn't assume Cedarline has the same noindex problem as ClearFlow. He checks his own site on its own evidence.
That's a useful habit for every designer: check what the website is actually doing before recommending a technical change.
Help Sam make the right technical decisions
ClearFlow's page looks usable, but the earlier review found an indexing problem. Choose Sam's next action each time. You can change your answer to see how the reasoning changes.
What matters first?
The water heater repair page opens in a browser. The earlier inspection shows Excluded by noindex.
Plan a simple indexing check
Start with ClearFlow's water heater repair page, or use the same questions for a client website you're reviewing. Keep your answers short enough to hand to a developer or client. Then choose one check for your own studio website.
See Sam's completed example
- Which page matters, and why?
- https://clearflow.example/water-heater-repair: existing service page for Austin homeowners. ClearFlow wants more relevant water heater repair inquiries. Keep this URL rather than creating another near-duplicate page.
- What evidence do you have?
- The October 20 website review showed this page loading directly and sending
<meta name="robots" content="noindex">. The URL Inspection example reported Excluded by noindex. The homepage linked to the page. This evidence is dated; we need a current live check. - What will you check next?
- With authorized site and Search Console access, inspect the exact URL, compare Google's last recorded state with a live test, and identify whether the CMS/page response still sends noindex. Ask the technical owner to check other crawl or canonical signals only if the evidence points there.
- Who approves and makes the change?
- The client authorized investigation, not a live edit. Explain any confirmed restriction and ask the owner to approve removing it from the intended public service page. The site administrator or developer makes the authorized technical change; Sam documents it.
- How will you verify the outcome?
- After an approved edit, inspect the live page and confirm the unwanted noindex instruction is gone. Use Search Console to request indexing if appropriate, and check the recorded indexing status after Google revisits the page. Don't report that indexing or rankings are guaranteed.
- What should you check on your studio website?
- Check Cedarline's existing /services URL. Make sure it is reachable from the homepage; review live indexing settings and any available Search Console information with the right access. Log findings before publishing the drafted copy.
Good technical SEO starts with the right question
Sam has learned something important: improving a page's design and explaining its service clearly are valuable, but they don't replace checking whether Google is allowed to index it.
He now has a specific technical issue to recheck, a clear way to ask for approval and a plan to verify any future correction. The existing water heater repair page stays in place, and the rest of the redesign remains a draft.
Next part: Protect visibility at launch
Next: Understand local search for client websites
Sam will review how ClearFlow presents its Austin service area and which business details need checking before they appear on the site or a Google Business Profile.