Where must the `robots.txt` file be located on a web server according to the Robots Exclusion Standard (RFC 9309)?

In any subfolder referenced by an HTML meta link tag.

Inside the `/assets/seo/` directory.

At the root of the host domain (e.g. `https://example.com/robots.txt`).

In the `.well-known/robots/` directory only.

How does case sensitivity apply to directives and URL path patterns in a `robots.txt` file?

Both directives and URL path patterns are completely case-insensitive.

Directive names are case-sensitive, but path patterns are case-insensitive.

The entire robots.txt file is strictly case-sensitive for all tokens.

Directive names (like `User-agent` and `Disallow`) are case-insensitive, but URL path patterns are case-sensitive.

If a `robots.txt` file contains `Disallow: /catalog/` and `Allow: /catalog/shoes`, how does Googlebot evaluate `/catalog/shoes`?

It allows crawling `/catalog/shoes` because the most specific (longest) matching directive pattern wins.

It blocks crawling because `Disallow` always overrides `Allow` regardless of length.

It evaluates whichever directive appears first in the file.

It produces a parsing error and ignores the entire robots.txt file.

What is the crawling effect of including a `Disallow:` directive with an empty value in `robots.txt`?

It blocks crawlers from accessing any page on the entire website.

It allows crawlers to access all URLs on the entire website.

It deletes the website from the search index immediately.

It instructs crawlers to crawl at a maximum rate of 1 page per day.

What happens when a web server returns an HTTP 500 Internal Server Error when Googlebot attempts to fetch `/robots.txt`?

Googlebot treats the 500 error as a full crawl block and pauses crawling the entire website until robots.txt is accessible.

Googlebot assumes there are no restrictions and crawls all pages aggressively.

Googlebot deletes the domain from the search index within 5 minutes.

Googlebot automatically switches to HTTP/1.0 and retries every millisecond.

What happens when a web server returns an HTTP 404 Not Found status code for `/robots.txt`?

Googlebot halts all crawling and marks the domain as broken.

Googlebot concludes there are no crawling restrictions and proceeds to crawl the entire site normally.

Googlebot submits an automated spam report to the Webmaster team.

Googlebot automatically generates a default robots.txt file on the server.

Does Googlebot support or obey the `Crawl-delay` directive inside `robots.txt`?

Yes, Googlebot strictly delays crawling by the exact number of seconds specified.

Only if `Crawl-delay` is set to an integer between 1 and 5.

No, Googlebot completely ignores `Crawl-delay`; crawl speed is configured in Search Console or calculated algorithmically.

Yes, but only for mobile smartphone user-agents.

Does Googlebot support using `Noindex:` directly inside `robots.txt` files?

No, Google officially deprecated and completely removed support for `Noindex` in `robots.txt` in September 2019.

Yes, `Noindex:` in `robots.txt` is Google's preferred de-indexing method.

Only if paired with a valid XML sitemap link.

Only if written in all lowercase letters (`noindex:`).

SEO & Speed / MEDIUM

Technical SEO Fundamentals Quiz: 100 Crawling, Indexing & Audit Questions

Validate your technical search engine optimization skills with 100 practice questions covering robots.txt, XML sitemaps, canonical tags, and status codes.

100Questions

90mDuration

+1Marks / Q

NoneNegative Marks

Technical SEO Fundamentals & Search Architecture Practice Exam

Demonstrate your mastery of modern technical search engine optimization with this 100-question practice test. Built for SEO specialists, technical marketers, web developers, and content strategists, this assessment evaluates your practical knowledge of search engine crawling, indexation controls, canonicalization rules, status code routing, and search engine rendering.

Core Technical SEO Disciplines Evaluated

The questions test deep knowledge of how search engine bots (Googlebot, Bingbot) discover, process, and rank web content:

Crawling & Bot Access Controls: The Robots Exclusion Protocol, authoring robust robots.txt files (directives: User-agent, Disallow, Allow, Sitemap, wildcard * and end-of-string $ matching), crawl budget optimization for enterprise websites, and HTTP header crawl controls (X-Robots-Tag).
Indexation Directives & Meta Robots: directives (noindex, nofollow, noarchive, nosnippet, max-snippet, max-image-preview:large), understanding why blocking a page in robots.txt prevents Google from seeing a noindex tag, and removing indexed pages safely via 410 Gone or noindex.
Canonicalization & Duplicate Content Elimination: Self-referential canonical tags, cross-domain canonicalization, resolving duplicate content across faceted navigation and URL parameters (?sort=, ?color=), trailing slash normalization, and protocol/subdomain consolidation (HTTP to HTTPS, non-www to www).
XML Sitemaps Architecture: XML sitemap protocol standards, sitemap index files, , format (W3C Datetime format), image sitemaps, video sitemaps, hreflang sitemaps, avoiding non-canonical or redirected URLs in sitemaps, and sitemap pinging.
International SEO & Hreflang Tags: Multi-regional and multilingual architecture (ccTLDs vs subdirectories vs subdomains), hreflang link attributes (ISO 639-1 language and ISO 3166-1 country codes), return tag reciprocity requirements, and the x-default fallback selector.
Rendering & JavaScript SEO: How search engines render modern single-page applications (CSR vs SSR vs Static Site Generation SSG), the two-wave indexing model, ensuring content and internal links exist in the initial HTML response, and debugging client-side rendering bottlenecks using Google Search Console URL Inspection.

Realistic Technical SEO Audits

Evaluate real audit scenarios: diagnosing why new blog posts fail to index, resolving canonicalization conflicts that cause search engines to choose the wrong landing page, and eliminating faceted navigation crawl traps that exhaust crawl budgets.

Assessment Format

Standardized testing rules & scoring benchmarks

Questions 100 multiple-choice questions.

Pass Threshold 70% passing grade.

Explanations Comprehensive technical rationales referencing Google Search Central official documentation.

Keep Practicing

Related Quizzes & Challenges

Strengthen your understanding with more interactive technical tests in this domain.

Showing 4 of 8 related tests

SEO & Speed•100 Qs

Website Speed & Frontend Optimization Challenge: 100 Performance Questions

Challenge your web performance expertise with 100 questions covering CDN caching, critical rendering path, code splitting, lazy loading, and asset minification.

90m

Take Quiz

Cybersecurity•100 Qs

Web Security Headers Quiz: 100 CSP, HSTS, CORS & Cookie Questions

Validate your browser security knowledge with 100 practice questions on Content Security Policy (CSP), HSTS, X-Frame-Options, CORS, and cookie prefixes.

90m•1

Take Quiz

Cybersecurity•100 Qs

Web Security & HTTP Headers Implementation Quiz: 100 Technical Questions

Test your web security deployment skills with 100 practice questions on Nginx and Apache header directives, TLS ciphers, SRI, and browser policies.

90m

Take Quiz

General Knowledge•100 Qs

Web Design Principles & UI/UX Architecture Test: 100 Questions

Validate your UI/UX and web design knowledge with 100 questions covering visual hierarchy, design systems, responsive typography, and cognitive ergonomics.

90m

Take Quiz

Technical SEO Fundamentals & Search Architecture Practice Exam

Core Technical SEO Disciplines Evaluated

Realistic Technical SEO Audits

Assessment Format

Related Quizzes & Challenges

Website Speed & Frontend Optimization Challenge: 100 Performance Questions

Web Security Headers Quiz: 100 CSP, HSTS, CORS & Cookie Questions

Web Security & HTTP Headers Implementation Quiz: 100 Technical Questions

Web Design Principles & UI/UX Architecture Test: 100 Questions

Supercharge Your Linux & Cloud Workloads with Hostinger