radar IndexRise Website monitor
Join the beta

Technical SEO, explained

Short, practical explanations: the answer first, then how to check and fix it. We write about what our monitoring catches every day.

Guides

SEO monitoring: what to watch on your site before Google notices SEO monitoring is the daily, automated check of the page elements that decide your visibility in Google: status codes, noindex, canonical, robots.txt, redirects, meta tags and sitemaps. Unlike an audit, it doesn't list every issue once — it compares today with the baseline and tells you when something changed. Website migration SEO checklist: migrate without losing traffic A migration — a new domain, platform, template or URL structure — is when sites most often lose Google traffic. Three things matter most: a complete map of 301 redirects from old URLs to new ones, a record of the SEO state before the change, and daily checks for several weeks afterwards.

Search Console statuses

Alternate page with proper canonical tag Usually not an error. Google found a page whose canonical tag points to another URL, agreed with that choice and indexed the canonical page instead. You only need to act if the list contains a page that should rank on its own. Blocked by robots.txt A Disallow rule in robots.txt stops Googlebot from crawling this URL, so Google doesn't fetch it. For the cart, internal search or filters that's intended. If the list includes pages that should rank, fix the rule — the robots.txt checker shows exactly which line blocks the URL. Crawled – currently not indexed Google visited the page but decided not to add it to the index for now. It isn't a technical error but a judgement: the content looks thin, too similar to other pages or poorly connected to the rest of the site. Improving content and internal links usually helps more than re-submitting the URL. Discovered – currently not indexed Google knows the URL (from a link or a sitemap) but hasn't visited it yet. It usually postponed the crawl because the server responded slowly, there are very many URLs, or the page doesn't look important. A faster server, fewer pointless URLs and better internal linking help. Duplicate without user-selected canonical Google decided the page is a copy of another one and picked which of them to index on its own, because the page has no canonical tag. The fix: choose the canonical yourself — this page if it should rank, or the main version if it's a duplicate. Duplicate, Google chose different canonical than user The page has a canonical, but Google decided another URL is the better main version and indexed that one. A canonical is a hint, not a directive — Google overrides it when other signals (redirects, internal links, sitemap, content) point elsewhere. The fix is to make those signals consistent. Excluded by ‘noindex’ tag Google fetched the page, found a noindex directive — in meta robots or the X-Robots-Tag header — and, as instructed, left it out of the index. If that's intended (cart, account pages, test pages), all good. If not, remove noindex and request indexing. Indexed, though blocked by robots.txt Google can't crawl the page because robots.txt blocks it, but it indexed the URL anyway — usually because links point to it. It then appears in results without a description. The fix depends on your goal: if the page should rank, unblock it; if not, unblock it and add noindex. Not found (404) Google hit a URL that returns 404 — the page doesn't exist. 404s don't harm the rest of the site and are the right response for removed content. Fix the 404s that had traffic or links, and the ones your internal links or sitemap point to. Page with redirect The URL redirects to another one, so Google indexes the target instead. Usually that's normal — old URLs after a migration, or http and non-www versions. It becomes a problem when redirecting URLs sit in your sitemap, internal links or canonicals, or when a page that should rank starts redirecting. Redirect error Googlebot tried to follow a redirect and never reached the destination page. Causes include a redirect loop, a chain longer than 10 hops, a URL that's too long, or an empty or invalid URL in the Location header. It's a real failure — the page won't load for users either. Sitemap could not be read / Couldn't fetch Google couldn't fetch or read the sitemap file. Most often the file returns an error, is blocked in robots.txt or by a firewall (such as Cloudflare), redirects, or isn't valid XML. For a freshly submitted sitemap the status is often temporary — if the file opens correctly, wait a day. Soft 404 A soft 404 is a page that returns 200 but looks like an error to Google: it's empty, says “not found”, or is a redirect to an unrelated page such as the homepage. Google treats it as a 404 and doesn't index it. Fix it by returning a real 404/410, redirecting to an equivalent, or adding content.

Technical SEO glossary

301 vs 302 redirect A 301 redirect means the page has moved permanently — Google passes signals to the new URL and replaces it in results. A 302 means the move is temporary — Google keeps the old URL in the index for longer. For migrations and permanent URL changes, use 301 (or 308). Canonical tag (rel=canonical) A canonical tag is <link rel="canonical" href="…"> in the page head that tells search engines which URL is the main version of a page. When the same content is available at several URLs, Google usually indexes the one named in the canonical. It's a hint, not a directive — Google can choose differently. H1 tag The H1 is the main heading of the page content (<h1>) — usually the article title, product name or category name. It tells readers and search engines what the page is about. Good practice: one visible H1 per page, matching the topic and the title tag. HTTP status codes and SEO An HTTP status code is the three-digit number a server sends back with every response. For SEO the rule is simple: a page that should appear in results must return 200. 3xx codes redirect, 4xx say the page doesn't exist, and 5xx say the server has a problem — during a longer outage Google starts removing pages from the index. Indexability A page is indexable when nothing technically prevents Google from adding it to the index: it returns 200, has no noindex, isn't blocked in robots.txt and its canonical points to itself. Indexability is necessary but not sufficient — whether the page actually gets indexed also depends on Google's assessment of the content. JavaScript rendering (raw HTML vs rendered HTML) Many pages get an HTML skeleton from the server, and the content, title, canonical or meta robots are added later by JavaScript in the browser. Google renders JavaScript, but in a second step and not always completely. When the raw HTML and the rendered version disagree — for example a different canonical or a noindex in only one of them — the result can be unpredictable. Meta description A meta description is a <meta name="description" content="…"> tag with a short summary of the page. It doesn't directly affect rankings, but Google often shows it under the title in results, so it decides whether people click. A good description is unique, specific and fits in about 150–160 characters. Noindex (meta robots) Noindex is a directive that tells search engines: don't show this page in results. It goes in a <meta name="robots" content="noindex"> tag or in the X-Robots-Tag: noindex HTTP header. Google has to fetch the page to see the noindex — so the page must not be blocked in robots.txt at the same time. Redirect chain and redirect loop A redirect chain is when a URL redirects to another URL, which redirects to the next, and so on before reaching the final page. A loop is a chain that returns to a URL it already visited — the page never loads. Googlebot follows at most 10 hops; ideally there should be one. Robots.txt Robots.txt is a text file at the root of a domain (/robots.txt) that tells search engine crawlers which URLs not to crawl. Disallow and Allow rules control crawling, not indexing — a blocked page can still appear in results. A mistake in robots.txt can block an entire site with one line. Soft 404 A soft 404 is a page that returns a 200 (“OK”) status but is actually an error or has no content — such as “Product not found”, an empty category, or everything redirected to the homepage. Google detects these pages and treats them as 404s. It's better to return a real error status from the start. Title tag The title tag is the <title> element in the page head — the page title shown in the browser tab and usually as the blue headline in Google results. It's one of the strongest signals of what a page is about. A good title is unique, starts with the main keyword and fits in roughly 50–60 characters. X-Robots-Tag X-Robots-Tag is an HTTP response header that carries the same directives as meta robots — such as noindex or nofollow. It's set by the server, not the page code, so it also works for PDFs, images and other non-HTML files. It's invisible in the page source, which makes it easy to overlook. XML sitemap (sitemap.xml) An XML sitemap is a file listing the URLs you want search engines to know about and crawl. It helps Google find new and updated pages faster, especially on large sites. It should contain only canonical URLs that return 200 — no redirects, errors or noindex pages.