Check this page with an assistantOpens a chat asking it to summarise this article and name the evidence behind each claim.
Claude opens with the prompt on your clipboard: Anthropic does not support prefilled prompts on the web, and we would rather copy it than ship a button that drops it.
The Status Line Is the Page’s First Contract
A browser can style an error page beautifully, replace a route with JavaScript, or navigate after a client-side check. A crawler still begins with the server response: success, redirection, missing resource, rate limit, or server failure.
Google uses that response to decide whether content proceeds to later systems, a target should be fetched, a URL should leave the index, or crawling should slow down. The status does not guarantee ranking or indexing, but a false status can prevent the intended processing before content quality is even evaluated.
Audit status codes by page state and template. The homepage, a live product, a removed product, an empty filter, a redirect, a private account page, and a maintenance response should not all return 200.
Use the Smallest Truthful Status Vocabulary
| Situation | Response | Search handling |
|---|---|---|
| Real public page | 200 | Content can proceed to processing; indexing is not guaranteed |
| Permanent move | 301 or 308 | Strong signal to process the target |
| Temporary move | 302 or 307 | Target is followed with a weaker canonical signal |
| Unchanged cached resource | 304 | Reuse the prior content; this is not a redirect |
| Missing or removed resource | 404 or 410 | Content is ignored and an indexed URL leaves over time |
| Rate limit or server outage | 429 or 5xx | Crawling slows; persistent failures risk eventual removal |
Google treats 301 and 308 as permanent and 302 and 307 as temporary. It may follow up to ten redirect hops for general web crawling, but long chains waste requests, add failure points, and slow the final response. Send each old URL directly to the final useful destination.
A 200 Means “Process This,” Not “Index This”
A successful response passes the received content into the next product-specific system. Google explicitly says a 2xx status does not guarantee Search indexing. The page must still be renderable, indexable, canonical, useful, and selected.
A 204 response contains no content to process. A 200 with an empty body, a “not found” message, or a thin placeholder may be classified as a soft 404. The solution is not adding more generic words to every empty state; it is deciding whether a real destination exists.
Client-only applications require extra care. A router that returns the same 200 app shell for every path can hide missing routes from the server. The JavaScript SEO guide explains how to compare raw response and rendered state.
Redirect Only When the User’s Task Survives
A permanent redirect is appropriate when a page moved, a duplicate was consolidated, or several older pages now have one genuinely equivalent destination. Preserve query-independent value, update internal links, and remove the old URL from sitemaps.
Do not redirect a discontinued product, expired job, deleted profile, and retired article to one generic homepage merely to avoid 404s. If the target does not satisfy the old intent, Google may treat the redirect as a soft 404. A relevant parent category can work when it clearly helps the visitor continue; otherwise, let the URL be gone.
Use temporary redirects for genuinely temporary routing. During redesigns and migrations, maintain a one-to-one mapping and validate the final destination with the website redesign SEO checklist.
Treat 404 and 410 as Inventory Hygiene
Google treats ordinary 4xx responses, except 429, as a signal that the content does not exist. It ignores the response body for indexing and gradually reduces crawling. The page can still provide useful navigation, search, and support to a person while returning the correct error status.
Not every observed 404 needs repair. External sites mistype URLs, bots invent paths, and old resources disappear. Prioritize submitted URLs, broken internal links, high-value backlinks, common misspellings, and pages that should still be live. Remove dead URLs from internal links and sitemaps.
Do not block missing URLs in robots.txt. Google needs the response to recognize the missing state. The crawl and indexing controls comparison covers why a robots rule is not a removal mechanism.
Use 429 and 5xx as Incident Signals, Not SEO Tools
Google treats 429 as server overload and groups it operationally with 5xx failures. A significant number of 500, 503, or 429 responses can reduce crawl rate across the hostname. Once healthy responses return, crawling increases gradually.
This protects overloaded infrastructure, but it is not a sustainable way to “save crawl budget.” Google warns against maintaining these errors for more than a short emergency window because persistent failures can remove URLs from Search. Fix the capacity, caching, deployment, or rate-limiting cause.
Monitor status distribution, DNS failures, timeouts, response time, and robots.txt availability during an incident. A robots.txt outage can postpone crawling widely because Google cannot confirm which URLs are permitted.
Keep Transport Status Separate From Indexing Directives
A status code describes whether the server fulfilled the request. A robots directive describes whether an accessible page may appear in a search index. A canonical suggests which accessible duplicate should represent a set. These controls are not substitutes. A 200 page with noindex is still a successful, crawlable page; a blocked URL may hide its noindex directive; and a 404 does not need a canonical pointing elsewhere.
Authentication is another distinct boundary. A private account, invoice, or workspace should require real access control rather than returning sensitive content with 200 and relying on noindex. Test the response without a session and ensure error templates do not leak personal data, internal navigation, or cached content.
Record all three layers in the audit (response, directives, and canonical target) so a team does not “fix” a truthful 404 with a meta tag or troubleshoot an indexing exclusion by changing server status.
Build a Status-Code Release Matrix
Encode expected responses as automated tests rather than checking a handful of browser screens after deployment. Use representative URLs for every state a template can produce.
- Live, indexable examples return 200 with substantive content.
- Moved URLs make one permanent hop to a 200 destination.
- Temporary experiments preserve temporary redirect semantics.
- Unknown and impossible states return 404, not an empty 200.
- Private pages enforce access without exposing protected content.
- Maintenance and overload responses are observable and short-lived.
- Sitemaps contain only intended canonical 200 destinations.
In Search Console, pair Crawl Stats response-code trends with Page indexing examples and live URL Inspection. The indexing incident manual keeps fetch failures separate from rendering, directives, canonicalization, and selection.
Questions About HTTP Status Codes and SEO
- Is a 404 bad for SEO?
A legitimate 404 is normal and does not harm the rest of the site. Google advises fixing 404s that come from broken internal links or submitted sitemap URLs. If a page is permanently gone with no useful replacement, a truthful 404 or 410 is the correct result.
- Is 410 better or faster than 404?
Google currently treats 404 and 410 the same for Search. Use 410 when the resource was intentionally removed and 404 when it is simply not found, because those semantics help other clients and your own operations. Do not expect a ranking or guaranteed removal-speed advantage.
- Should every deleted URL redirect to the homepage?
No. Redirect only when a close replacement preserves the user’s task. Redirecting unrelated old URLs to the homepage can be treated as a soft 404 and gives visitors an unexpected destination. Return 404 or 410 when no equivalent replacement exists.
- What status should planned maintenance return?
Use a temporary server-error response such as 503 when the service genuinely cannot fulfill requests. Google reduces crawling after significant 5xx or 429 responses and can preserve indexed URLs initially, but persistent errors may lead to removal. Keep the outage short and restore healthy 2xx responses promptly.
Primary Sources Reviewed
- Google Crawling Infrastructure: HTTP status-code handling
- Google Search Console: 404 and soft-404 guidance
- Google Search Central: Redirects and Google Search
- Google Crawling Infrastructure: Reducing crawl rate
- Google Search Console: Crawl Stats report
Reviewed July 28, 2026. HTTP semantics apply beyond Google; choose accurate responses for browsers, APIs, accessibility tools, caches, and other search engines as well.
