A meta directive that tells search engines not to include a page in their index. Set via
<meta name="robots" content="noindex"> in the page <head>, or via the X-Robots-Tag: noindex HTTP header (which is the only option for non-HTML responses like PDFs). Crawlers must be able to fetch the page to see the directive — so don't combine noindex with a Disallow in robots.txt, or the directive will never be read. Commonly used on thin pages, faceted-navigation duplicates, internal search results, and staging sites.