TL;DR
To validate an llms.txt file, check five things: it lives at the root of your domain (https://yourdomain.com/llms.txt), it returns plain text with a 200 status rather than your HTML homepage, it starts with a single H1, it uses Markdown link lists under H2 headings, and every link is an absolute URL that resolves. You can run all five checks in one go with the free Llms.txt Checker. Whether you need the file at all depends on your site: documentation and knowledge-heavy sites gain most, and Google has said it does not use llms.txt for Search.
| Check | What passes | What fails |
|---|---|---|
| Location | https://yourdomain.com/llms.txt |
A file only at /blog/llms.txt or on a subdomain you did not mean |
| Response | 200 status, plain text or Markdown | A redirect to the homepage, a soft 404, or an HTML page |
| Title | One H1 line with the site or project name | No H1, or several H1s |
| Summary | A short blockquote under the H1 | Missing (allowed, but tools lose context) |
| Sections | H2 headings with Markdown link lists | Plain paragraphs of URLs with no structure |
| Links | Absolute https:// URLs that return 200 |
Relative paths, broken links, redirect chains |
What llms.txt is
llms.txt is a proposed file format, put forward by Jeremy Howard on 3 September 2024, that gives AI tools a clean Markdown map of a website. Instead of a model scraping menus, scripts and cookie banners, it reads a short file that says what the site is and which pages matter most.
The proposal sets out a simple structure:
- An H1 with the site or project name. This is the only required element.
- A blockquote with a one or two sentence summary.
- Optional paragraphs of background.
- H2 sections containing Markdown links, each with a short note on what the page covers.
- An optional section titled "Optional" for links a tool can skip when it is short on space.
A companion file, llms-full.txt, can sit beside it. Where llms.txt is an index of links, llms-full.txt bundles the full text of the key pages into one document.
Where does llms.txt go?
It goes in the root of your domain, so it loads at https://yourdomain.com/llms.txt, in the same place as robots.txt. Tools looking for general site context request that exact path. The proposal also allows a file inside a subpath for a distinct project, such as documentation at https://yourdomain.com/docs/llms.txt, but that is an addition to the root file, not a replacement.
If your site runs on a hosted builder, check before you write anything that it lets you serve a file at the root. Some platforms only let you add pages, and a "page" called llms.txt usually comes back as HTML, which fails validation.
How to validate your llms.txt file
1. Fetch it like a tool would
Open the URL in a browser, then check the raw response. A command such as curl -I https://yourdomain.com/llms.txt shows the status code and content type. You want a 200 status and a text/plain or text/markdown content type. A 301 to your homepage, or a 200 that returns your full HTML template, means the file is not really there.
2. Check the Markdown structure
The first non-empty line should be a single # heading. Below it, a > blockquote, then ## section headings with bulleted links in the form - [Page name](https://yourdomain.com/page): short description. Avoid tables, images and raw HTML in the file.
3. Check every link
Each link should be absolute and should return a 200 status directly. Relative links like /pricing break when a tool reads the file out of context, and links that redirect two or three times waste a tool's limited budget.
4. Run an llms.txt checker
The free Llms.txt Checker runs these checks for you. It confirms the URL returns a real text file rather than a catch-all page, checks for the H1 and blockquote, reads the H2 sections and link lists, samples the linked pages to see whether they resolve, and looks for an llms-full.txt companion file. It needs no sign-up.
A minimal llms.txt example
# Northfield Bakery
> Northfield Bakery is an independent bakery in Leeds selling sourdough, pastries and celebration cakes, with local delivery and click and collect.
## Ordering
- [Celebration cakes](https://northfieldbakery.co.uk/cakes): sizes, flavours, lead times and prices
- [Delivery and collection](https://northfieldbakery.co.uk/delivery): postcodes covered, cut-off times and fees
## About
- [Allergens](https://northfieldbakery.co.uk/allergens): allergen information for every product
- [Contact](https://northfieldbakery.co.uk/contact): opening hours, address and phone number
## Optional
- [Blog](https://northfieldbakery.co.uk/blog): recipes and news
Keep it short. A small business file with ten to thirty links is plenty, and every link should point to a page that will still exist in a year.
Common errors a validator finds
- The file returns HTML. Many content systems send any unknown URL to the homepage with a 200 status. The fix is to upload a real file to the root, or use your host's static file feature.
- No H1. Files that start with body text or a link list fail the one required rule.
- Relative links. Change
/servicestohttps://yourdomain.com/services. - Volatile pages. Product variants, cart pages, search results and login screens change or break. Link to stable category, policy and guide pages instead.
- Blocked by robots.txt. If your robots.txt blocks AI crawlers, the pages your llms.txt points to may be unreachable for those tools. Check with the free AI Crawler Checker, and read how to check your robots.txt for AI crawlers.
Do you need an llms.txt file?
Treat it as a low-cost, experimental extra rather than a ranking factor. Google has said it does not use llms.txt for Search, and a Google search advocate said in 2025 that he knew of no AI system using it. Documentation and coding tools are the clearest users today, and some assistants may read it when they visit a site directly.
It is most worth the twenty minutes for:
- Software products and documentation sites.
- Service businesses with clear packages, prices and processes.
- Sites with large guide libraries or knowledge bases.
For a small shop or local business, it does no harm, but your robots.txt, sitemap and indexability matter far more. For the bigger picture, see how AI search is changing SEO.
| File | Read by | Status | Job |
|---|---|---|---|
| robots.txt | All crawlers | Standard (RFC 9309) | Says which paths crawlers may fetch |
| XML sitemap | Search engines | Established protocol | Lists the URLs you want indexed |
| llms.txt | AI tools and agents | Community proposal | A curated Markdown index of key pages |
| llms-full.txt | AI tools and agents | Community proposal | The full text of key pages in one file |
Check the rest of your technical setup
An llms.txt file cannot fix pages that crawlers cannot reach. Run the free Sitemap Checker and Indexability Checker alongside it, and read why Google might not be indexing your pages.
Public checkers show what your server publishes. To see how Google actually treats your pages, connect Google Search Console to Ergora, and its Content specialist reads the queries that bring clicks, the pages losing clicks and whether a page is in Google's index.
Verdict: Publish an llms.txt if your site has documentation, guides or clear service pages, keep it at the root, and validate it with the free Llms.txt Checker after every change. Do not expect it to move your Google rankings.
Frequently asked questions
How do I validate an llms.txt file?
Fetch https://yourdomain.com/llms.txt and confirm it returns plain text with a 200 status, starts with one H1, uses H2 sections with Markdown link lists, and links only to absolute URLs that resolve. A free llms.txt checker runs all of these checks at once.
Where should llms.txt be placed on a website?
At the root of the domain, so it loads at /llms.txt, the same place as robots.txt. A project can add another file in a subpath, such as /docs/llms.txt, but tools looking for site context request the root file.
Does Google use llms.txt?
Google has said it does not use llms.txt for Search. Adding one will not change your Google rankings.
What is the difference between llms.txt and llms-full.txt?
llms.txt is a short index of links with one line descriptions. llms-full.txt contains the full text of those key pages in a single file, for tools that want all the context in one request.
Is submitting an llms.txt file a benefit?
There is nowhere to submit it: tools find it by requesting /llms.txt on your domain. The benefit is a clean, accurate summary for the AI tools that do read it, at very little cost.
Does llms.txt replace robots.txt or a sitemap?
No. robots.txt controls what crawlers may fetch and a sitemap lists the pages you want indexed. llms.txt sits alongside both.