Technical SEO is the layer of your site that Google touches before it reads a single word of your content. Get it wrong and your pages won't rank — not because the writing is bad, but because Googlebot couldn't crawl them, or crawled them and couldn't index them, or indexed them under the wrong URL.
This guide covers what technical SEO actually includes, the eight categories that matter most, the right order to tackle them, and where each cluster article fits.
What Technical SEO Actually Is
Technical SEO is every decision that affects how search engine crawlers discover, access, render, and store your pages — independent of what those pages say.
That definition matters because it separates two failure modes:
- Content problems: the page exists and is indexed, but it doesn't rank because it's thin, off-topic, or outcompeted.
- Technical problems: the page can't rank because Google can't find it, won't index it, or is indexing the wrong version of it.
Technical SEO fixes failure mode 2. It doesn't fix failure mode 1. Clean technical configuration is necessary but not sufficient — a perfectly crawlable, indexable, fast-loading page still needs to be genuinely useful to rank.
The Eight Categories
1. Crawling Controls
Googlebot starts with your robots.txt. That file tells crawlers which paths they're allowed to fetch. If a path is disallowed, Googlebot won't crawl it — which means it won't see any noindex tags, canonical tags, or content on that URL.
The most common mistake: blocking a URL in robots.txt and also expecting the noindex directive on that page to be respected. Google can't read a noindex tag on a page it's not allowed to fetch.
Key articles:
- Indexed, Though Blocked by Robots.txt — How to Actually Fix It
- Shopify Robots.txt: What the Default Blocks and What to Change
- Robots.txt Syntax Rules
- Faceted Navigation SEO
2. Indexing Controls
Once Google can crawl a page, you can still tell it not to index that page. The noindex directive — delivered either via the <meta name="robots"> tag or the X-Robots-Tag HTTP header — is the right mechanism. The disallow directive in robots.txt is not.
Key articles:
3. Redirects
Redirects consolidate link equity, move users to the right URL, and prevent duplicate content from spreading across multiple addresses. A 301 passes most of a page's authority to the destination. A redirect chain (A→B→C) wastes crawl budget and dilutes that signal. A redirect loop (A→B→A) breaks the page entirely.
Key articles:
4. Canonicalization
When multiple URLs serve equivalent content — http vs https, www vs non-www, trailing slash vs none, URL parameters — Google has to pick one to index. The rel="canonical" tag is your vote. If you don't cast it, Google will make its own choice, which may not be the one you want.
5. International (Hreflang)
If you publish content in multiple languages or target multiple regions, hreflang tags tell Google which version of a page to show to which audience. Misconfigured hreflang causes the wrong language version to rank in the wrong country — a problem that's invisible until you check rankings by region.
Key article:
6. Sitemaps
An XML sitemap is a list of URLs you want Google to know about, with optional metadata like last-modified dates. It's not a crawl guarantee — Google can ignore sitemap URLs and discover others through links — but it's useful for large sites, new sites, and any URL that doesn't have many inbound links.
Key articles:
7. Page Experience
Page experience covers the signals Google uses to evaluate whether a page is pleasant to use: Core Web Vitals (LCP, INP, CLS), HTTPS, mobile-friendliness, and the absence of intrusive interstitials. These are ranking factors — confirmed, weighted lightly, but capable of being a tiebreaker between otherwise similar pages.
8. Structured Data
Schema markup is machine-readable metadata embedded in your HTML. It doesn't directly cause rankings, but it enables rich results (review stars, FAQ accordions, recipe cards, sitelinks search boxes) that improve click-through rates and, in some cases, trigger AI Overview citations.
Priority Order for a New Site
If you're starting from zero or doing a site audit, tackle categories in this order:
1. Crawlability first. If Googlebot can't access your pages, nothing else matters. Check robots.txt, verify no accidental disallow rules exist, confirm your server returns 200s for important URLs.
2. Indexability second. Confirm your important pages don't have accidental noindex tags. Check for <meta name="robots" content="noindex"> in templates that might be applied too broadly.
3. Canonicalization third. Pick a canonical URL format (with or without www, with or without trailing slash, always HTTPS) and enforce it consistently. Add canonical tags. Set up 301s for all variants.
4. Sitemaps fourth. Submit an accurate sitemap through Google Search Console. Include only canonical, indexable URLs.
5. Performance fifth. Run a Core Web Vitals report. Fix LCP first — it's usually the highest-impact metric and often fixable with image optimization and better server response times.
6. Schema last. Add schema once the above is clean. Schema on a page Google can't crawl doesn't help.
Why Technical SEO Is Necessary but Not Sufficient
A technically perfect site with no useful content will not rank. Google's systems are designed to surface pages that answer queries better than their competitors. Technical SEO removes the obstacles that prevent Google from evaluating your content — it doesn't improve the content itself.
The flip side is also true: excellent content on a technically broken site gets little traction. Googlebot has a crawl budget. If it's wasting that budget on blocked paths, soft 404s, and redirect chains, your genuinely good pages may be crawled infrequently or not at all.
Think of technical SEO as clearing the runway. The plane still has to be able to fly.
Common Audit Findings by Site Type
WordPress sites
- Accidental noindex left on from development (Settings → Reading → "Discourage search engines")
- Yoast or RankMath canonical tags conflicting with theme-level head output
- Redirect loops from HTTP→HTTPS handled in both .htaccess and a plugin
- Search result pages (
?s=) indexed and competing with real content
Shopify stores
- Default robots.txt blocking
/collectionsfaceted URLs that should be indexed - Canonical tags pointing to collection pages instead of product pages for duplicate products
- Thin tag and vendor pages indexed without enough content to justify it
Static sites (Next.js, Astro, etc.)
noindexleft in<head>from a staging environment variable that wasn't cleared on deploy- Sitemaps generated at build time with stale
lastmodvalues that erode Google's trust - Trailing-slash handling inconsistent between dev and production causing canonical splits
The RankCrab Audit
RankCrab's 80-point on-page audit runs a Lighthouse check via Google PageSpeed Insights and layers on content, metadata, and technical checks — crawlability signals, canonical consistency, noindex detection, redirect health, Core Web Vitals, and structured data presence — all in a single report.
The audit won't replace a manual crawl of a large site, but for individual URLs or a focused content review it covers the technical bases alongside content quality in one place. If you're diagnosing a page that isn't ranking, the audit is the fastest starting point.
Start with the most common fixable problem: Indexed, Though Blocked by Robots.txt. It's the technical SEO warning that affects the most sites and has a clear decision tree for resolving it.