llms.txt: What It Is, Whether You Need One and How to Validate It

The emerging standard known as llms.txt provides a lightweight way for websites to serve structured Markdown directly to language models. Marketing teams and online store operators need a realistic view of what the file achieves, what it does not, and whether publishing one is worth the effort. This guide also shows how to run an llms.txt validator check on a file you have made.

What is llms.txt and who proposed it?

At its core, llms.txt is a text-based roadmap for AI tools. It is a proposed file format that lives at the root of a domain (or within a specific project subpath) to help AI systems read and process authoritative content without scraping through navigational chrome, scripts, or interface clutter.

The format was proposed by Jeremy Howard on 3 September 2024. The initiative is an open proposal to standardise how websites expose clean, human-curated context to automated tools. AI coding and documentation tools make wider use of it than other kinds of tools.

The structure of an llms.txt file is clean Markdown containing specific elements:

• An H1 title containing the project or business name. This is the only strictly required element.
• A blockquote directly below the H1 that provides a short, plain-English summary of the site purpose.
• Optional introductory paragraphs offering brief background context.
• Multiple H2 sections organising clean Markdown hyperlinks, with each link accompanied by a short note explaining the target content.

A companion file called llms-full.txt may also be published alongside it. While llms.txt provides a curated directory of links, llms-full.txt provides the complete text of the key pages bundled into one file.

Does llms.txt work for search engines and AI chatbots?

It is vital to treat llms.txt as an experimental standard rather than an established search signal. It is an open proposal with variable community adoption, not an official internet protocol.

Google has said it does not use llms.txt for Search. A Google search advocate also said in 2025 that he knew of no AI system using it. Support elsewhere varies: documentation and coding tools read it, and some assistants may read it when they visit a site.

Treat llms.txt as a tidy, low-overhead way to serve stripped-down context to AI agents and development assistants that visit your pages directly. It will not replace classical discovery pipelines or magically boost your positions in regular web search. For a broader look at how conversational platforms discover content, read our guide on how AI search changes traditional SEO.

Should a small business or online store add one?

Because drafting the file requires minimal maintenance, many teams choose to publish one preemptively. However, not every website needs it.

Websites that benefit most include:

• Software and technical documentation portals.
• Service firms with clearly defined service packages and methodology notes.
• Sites with deep knowledge bases, research libraries, or extensive tutorials.

If you operate an ecommerce business, building an llms.txt for Shopify or similar platforms requires selectivity. The file should only feature permanent, high-value resources:

• Core brand history, delivery areas, and support contact details.
• Evergreen category guides, sizing resources, and warranty details.
• Primary service offerings or flagship product categories.
• Frequently asked questions about shipping policies and returns.

Avoid cluttering the file with volatile inventory, shopping cart paths, customer login portals, search results pages, or thin duplicate tags.

How to write your file in five steps

Creating the file takes only a few minutes with a basic text editor or an automated llms.txt generator.

1. Create a plain text file named llms.txt using standard UTF-8 encoding.
2. Add your site or brand name as the primary H1 header at the top of the file.
3. Write a single blockquote sentence directly under the title that summarises what your business does.
4. Organise your core pages into logical H2 categories such as Core Services, Guides, or Company Policies.
5. Add bullet points under each section featuring full absolute URLs, followed by a concise sentence explaining what a reader finds on that page.

Once saved, upload the file to your web server root directory so it resolves cleanly at /llms.txt.

Three common mistakes that break the file

Small structural errors will prevent tools from parsing the contents effectively:

• Missing H1 element: The official specification requires an H1 heading with the site name. Files that start directly with body copy fail basic parsing validation.
• Relative links: Tools reading the file out of context may fail to resolve links written as /services or ../pricing. Every link must be an absolute URL starting with https://.
• Soft 404 and catch-all pages: Many content management systems silently redirect missing URLs back to the homepage while returning a 200 HTTP status code. If an agent requests /llms.txt and receives your full HTML homepage instead of plain text, the file cannot be processed.

Using an llms.txt validator to check your setup

Before assuming your file is functional, test it with an automated parser. You can inspect your published configuration using our free web-based llms.txt validation tool.

The validator reviews several technical factors:

• Confirms the URL returns a real text file rather than an HTML catch-all page.
• Verifies the presence of the mandatory H1 project title.
• Checks for the blockquote summary and proper Markdown structure.
• Validates that H2 sections contain cleanly formatted lists of absolute links.
• Samples linked destinations to confirm they resolve correctly.
• Checks whether an optional llms-full.txt companion file exists at the root.

To understand where this file fits alongside existing web infrastructure, review the differences across technical protocols:

Hosting considerations for online stores and local businesses

Before investing time into drafting structured files, verify how your web host or ecommerce platform manages files hosted at the root directory.

Some hosted platforms and website builders limit what you can serve from the site root, so check how yours handles a file at /llms.txt before you plan on it. The proposal allows files in subpaths for distinct projects, but tools looking for general site context expect /llms.txt.

If your platform makes root file hosting difficult, do not compromise your core technical setup to force implementation. Ensure your fundamental technical foundation, such as robots.txt directives and canonicalisation, remains solid first. You can explore how assistants evaluate businesses in our article on monitoring brand visibility across AI engines.

Audit your wider technical discoverability

A clean llms.txt file is a helpful addition, but it cannot fix broken crawl rules or missing indexing instructions. Review your complete technical architecture with our collection of free online website auditing utilities.

Begin by testing crawler access with our robots.txt parser for AI bots, which evaluates nineteen automated agents against your access directives. Next, confirm that traditional search engines can navigate your architecture using our XML sitemap testing tool. Finally, verify that your priority URLs are accessible to web robots with the page-level indexability utility.

Public checkers confirm what is published on your server, while Google Search Console records how search engines interact with those pages. Once your Google Search Console account is connected, Ergora's SEO specialist can read it: the queries that drive clicks, pages losing clicks, and whether a page is in Google's index.

Frequently asked questions

What is llms.txt?

It is a proposed file format that provides a lightweight, Markdown-based directory of a website's most important pages and summaries. Proposed by Jeremy Howard in 2024, it aims to help AI tools read concise context without parsing complex web code.

Do I need an llms.txt file?

No, it is not mandatory or an official internet requirement. Websites with complex documentation, coding libraries, or rich educational resources benefit most, while standard websites can safely operate without one.

Does Google use llms.txt?

Google has said it does not use llms.txt for Search.

How do I validate my llms.txt file?

You can validate your file using our free online validator to confirm the URL returns plain text rather than an HTML redirect. The tool checks that your Markdown contains the required H1 header, proper link structures, and working destination URLs.

What is the difference between llms.txt and llms-full.txt?

An llms.txt file is an index containing short descriptions and hyperlinks to key pages across your domain. An llms-full.txt file bundles the full written text of those authoritative resources into a single document for extensive context processing.

Does llms.txt replace robots.txt or a sitemap?

No, it serves an entirely different function and does not replace existing web standards. The robots.txt file controls crawler permissions, while an XML sitemap provides search engines with a comprehensive catalog of pages to index.