llms.txt

llms.txt is a proposed file format designed to give AI agents and large language model tools a concise, structured overview of a website and its most useful resources.

It is usually written in Markdown and commonly published as:

/llms.txt

The current version of the proposal also allows an llms.txt file to sit within a specific section of a website, such as:

/docs/llms.txt

where it can describe the content beneath that path.

The idea is straightforward.

Instead of making an AI agent work through navigation, JavaScript, advertising and other HTML around a page, the website can provide a simpler map pointing towards useful, machine-readable resources.

That makes sense in some environments, particularly developer documentation.

What it does not currently provide is a proven shortcut to better visibility in Google AI Overviews, AI Mode or other major AI search products.

For most ordinary business websites, that distinction matters more than the file itself.

What is llms.txt?

The original llms.txt proposal was published in 2024 as a way to make website information easier for language models and AI agents to find and consume.

A basic file can contain:

• The name of the website or project
• A short description
• Additional context
• Groups of important resources
• Links to useful pages or Markdown versions

A simplified example might look like:

# Example Roofing

> Roofing company serving Kent.

## Services

- [Roof Repairs](https://example.com/roof-repairs/): Emergency and scheduled roof repairs.

- [Flat Roofing](https://example.com/flat-roofing/): Flat roof installation and repair.

## Guides

- [Roof Replacement Guide](https://example.com/guides/roof-replacement/): What homeowners should know before replacing a roof.

The format is intended to be readable by both humans and machines.

The current v2 proposal also recommends linking to clean Markdown versions of pages where those versions exist.

How llms.txt is supposed to work

The proposed workflow is simple.

An AI agent discovers the llms.txt file.

The file gives it a concise description of the site and points it towards resources relevant to the task.

The agent can then fetch those resources rather than trying to process the whole website blindly.

For example, a coding assistant trying to answer a question about an API could use:

/docs/llms.txt

to find:

• API references
• Tutorials
• Configuration documentation
• Examples

That can reduce unnecessary retrieval and give the agent a cleaner route through a large documentation site.

This is where the idea makes the most practical sense.

llms.txt v2

The proposal was revised in August 2026.

The newer version reflects how AI agents and documentation platforms have begun using web content in practice.

Among the changes, v2 allows llms.txt files at subpaths and recommends standard link relationships so an agent can discover the appropriate file or Markdown alternative without simply guessing that it exists.

For example, a page can advertise a Markdown alternative using:

<link

rel="alternate"

type="text/markdown"

href="/docs/example.md"

>

and identify the relevant llms.txt file through a describedby relationship.

This is a meaningful development for agent-friendly documentation.

It is not evidence that the file improves Google rankings or AI citations.

Those are separate questions.

llms.txt vs robots.txt

llms.txt and robots.txt perform completely different jobs.

robots.txt communicates crawling instructions to compliant automated crawlers.

llms.txt provides information and links intended to help an AI agent understand or navigate content.

An llms.txt file does not tell a crawler:

Do not access this directory.

It is not an access-control mechanism.

If you want to control crawler access, use the appropriate robots.txt directives and the crawler's documented user agent.

Do not rely on llms.txt to allow or block AI bots.

llms.txt vs XML sitemap

An XML sitemap lists URLs that a website wants search engines to discover and understand as part of the site's indexable content.

llms.txt has a different purpose.

It is intended to provide a curated overview, rather than necessarily listing every URL.

For example, an ecommerce website might have 20,000 indexable URLs in its XML sitemap but only point an agent towards a small number of key documentation, policy or product resources from llms.txt.

The comparison is useful conceptually.

But the two files do not have the same level of search-engine support.

XML sitemaps are an established search standard.

llms.txt remains a community proposal.

Does Google use llms.txt?

Google has been explicit on this point.

As of June 2026, Google says llms.txt files are not needed for Google Search and have neither a positive nor negative effect on Google Search visibility or rankings.

That means an llms.txt file is not required for:

• Google Search
• AI Overviews
• Google AI Mode

It is also not a Google ranking factor.

Adding one does not make a website more eligible for Google's generative Search features.

Removing one does not make the website less eligible.

For Google visibility, focus on the normal search fundamentals instead.

Does llms.txt improve AI citations?

There is currently no convincing evidence that simply publishing the file increases the likelihood of being cited by major AI answer engines.

One large Ahrefs study examined 137,210 domains using its analytics products.

Among the sites publishing valid llms.txt files, 97% received no requests to those files during May 2026. Ahrefs also found that AI bots did not appear to probe for nonexistent llms.txt files across the sites in its sample.

A separate SE Ranking analysis of nearly 300,000 domains reported no measurable relationship between the presence of llms.txt and how frequently domains were cited by LLMs in its dataset.

Neither study proves that the file can never matter.

They do show why claims such as:

“Add llms.txt and your ChatGPT visibility will improve.”

are not supported by the evidence available today.

Do AI crawlers read llms.txt?

Some do.

But usage appears limited and inconsistent.

Ahrefs found that of the small minority of llms.txt files that received requests, some traffic came from named AI-related bots and tools. It also found substantial activity from developer tooling, research tools and other automated systems rather than mainstream AI-search retrieval alone.

That is an important distinction.

The correct conclusion is not:

Nobody reads llms.txt.

It is:

Publishing llms.txt does not currently mean major AI search systems will automatically discover, use or reward it.

Is llms.txt an official web standard?

No.

It is an open proposal rather than a formal search-engine or web-platform standard.

The current specification is maintained by the llms.txt project and has evolved through community adoption and experimentation.

That means its future could go several ways.

It could:

• Gain wider agent adoption
• Remain useful mainly for developer tooling
• Become integrated into more CMS platforms
• Be replaced by another approach
• Remain optional indefinitely

Do not treat its current status as permanent.

But do not treat anticipated future adoption as though it already exists either.

What happened to llms-full.txt?

Early discussion around the format often included llms-full.txt, intended to provide much more of a site's content in a single Markdown document.

The current v2 proposal has shifted towards a smaller llms.txt file that points agents towards relevant Markdown resources rather than requiring one enormous combined document.

For new implementations, follow the current specification rather than old tutorials written around the original proposal.

This is particularly important because the format is still evolving.

Where llms.txt can genuinely be useful

The strongest current use case is not local SEO.

It is agent-friendly documentation.

Developer documentation

A large software documentation site may contain:

• API references
• Setup guides
• Tutorials
• Examples
• Version-specific documentation

An agent helping a developer does not necessarily need the whole site.

A curated llms.txt file can point it towards the resources that matter.

Several documentation platforms now automatically generate the format, and the v2 proposal specifically describes software documentation as one of its strongest current use cases.

Coding assistants

Coding agents frequently retrieve documentation while helping users write or debug software.

A clean machine-readable index can reduce the amount of irrelevant page content the agent needs to process.

This is very different from saying the file improves Google rankings.

It is a retrieval-efficiency use case.

Internal AI systems

If your company operates its own:

• RAG system
• AI assistant
• Agent
• MCP-based workflow

you control what the system reads.

You can deliberately configure it to use an llms.txt file.

In that situation, the file's value is not speculative.

You built the system consuming it.

Documentation-heavy products

Products with large technical knowledge bases may also benefit from exposing clear Markdown resources to agents.

Again, this is about improving machine access to documentation.

Not about earning search citations automatically.

Does a local business need llms.txt?

For most ordinary local business websites, it should be a low priority.

A plumber, dentist, solicitor or landscaping company will normally gain more from improving:

• Service pages
• Location information
• Internal linking
• Crawlability
• Structured data
• Google Business Profile
• Reviews
• Original content
• Business information consistency

than from manually maintaining an llms.txt file.

That does not mean adding the file is harmful.

If your CMS generates one automatically at little or no cost, there is usually no urgent reason to remove it.

The issue is opportunity cost.

Do not spend hours maintaining a speculative AI file while important parts of the website remain weak.

Should you create an llms.txt file?

For a standard business website, it is usually optional rather than necessary.

A sensible decision looks like this:

Consider it if:

• You operate a large documentation site
• AI agents are part of your actual user base
• Your own software consumes the file
• Your CMS generates it automatically
• You have a specific agent workflow that benefits from it

Do not prioritise it simply because:

• You want to rank in AI Overviews
• You want ChatGPT to cite you
• A vendor says every AI-optimised site needs one
• You think it controls AI crawler access

There is currently no evidence supporting those uses.

What should you do instead for AI visibility?

The practical work is covered more fully in generative engine optimisation.

For most business websites, focus on:

• Publishing genuinely useful information
• Keeping important pages crawlable and indexable
• Answering customer questions clearly
• Adding first-hand expertise
• Keeping factual information current
• Making your entity information clear
• Using accurate structured data
• Building a credible presence across the web

Those improvements benefit traditional Search as well as generative search systems.

Ranksphere's website audit checks the technical fundamentals that do affect AI visibility, including crawlability, internal structure and on-page issues.

llms.txt and crawler control

Do not confuse making content easier for agents to understand with giving them permission to crawl it.

Those are different decisions.

If you want to control access from:

• Search crawlers
• Training crawlers
• AI search crawlers
• Other automated agents

check the provider's current crawler documentation and use the appropriate robots.txt rules or other access controls.

llms.txt does not override them.

It is not an AI permissions file.

llms.txt and Markdown pages

The v2 proposal puts greater emphasis on providing clean Markdown alternatives for content intended for agent use.

This can make sense for:

• API documentation
• Technical references
• Developer guides

where reducing page chrome and formatting noise can make machine retrieval simpler.

That does not mean every local service page needs a second Markdown copy.

Creating parallel versions of an entire business website solely for supposed GEO benefits adds complexity without established search benefit.

Use Markdown alternatives where there is an actual consumer for them.

Is llms.txt harmful?

There is no evidence that simply having a normal llms.txt file harms Google rankings.

Google specifically says the file has neither a positive nor negative effect on Google Search visibility.

The main risks are practical rather than algorithmic.

You may:

• Spend time maintaining something nobody uses
• Publish outdated links
• Duplicate work already handled elsewhere
• Pay unnecessarily for implementation
• Assume the file solves AI visibility problems that remain unresolved

The cost is often the attention diverted from higher-value work.

Should an existing llms.txt file be removed?

Usually, there is no need to rush to delete it.

If it is:

• Accurate
• Automatically maintained
• Not exposing anything inappropriate
• Costing nothing meaningful to operate

it can remain.

Google says it does not hurt Search visibility.

The important thing is not to mistake its presence for evidence that the website has been “optimised for AI”.

A file existing at /llms.txt tells you very little about the quality or visibility of the underlying site.

Common llms.txt mistakes

• Treating llms.txt as an AI ranking factor. Google explicitly says it is not needed for Search.
• Claiming it guarantees AI citations. Current evidence does not support that.
• Saying nobody uses it at all. Some documentation platforms and agent tooling do.
• Using old v1 instructions without checking the current specification. The proposal moved to v2 in August 2026.
• Believing it controls AI crawlers. Use appropriate crawler controls instead.
• Confusing it with an XML sitemap. The purposes and support levels are different.
• Creating Markdown duplicates of an entire business site without a real use case.
• Maintaining the file manually when nothing consumes it.
• Paying substantial recurring fees simply to keep it updated.
• Treating Google's lack of support as proof that no other tool can ever use it.
• Assuming future adoption is guaranteed. The proposal remains optional and evolving.

llms.txt best practices

• Treat it as optional, not an SEO requirement.
• Follow the current v2 specification if you implement it.
• Consider it primarily for documentation and agent workflows.
• Keep listed resources accurate.
• Use Markdown alternatives only where they serve a real purpose.
• Use robots.txt or appropriate controls for crawler permissions.
• Do not sell or report the file as evidence of improved Google AI visibility.
• Prioritise content quality, crawlability and entity clarity first.
• Ask vendors what measurable problem their implementation solves.
• Revisit the evidence periodically because adoption may change.

Example

“Hadleigh Financial receives a proposal to implement llms.txt as part of an AI search optimisation package.

The setup costs £900, followed by a monthly maintenance fee.

The pitch says the file will make the firm's content easier for Google AI Overviews and other AI assistants to cite.

The practice asks a simple question:

Which of those platforms has documented that this file improves citation or search visibility?

For Google, the answer is clear.

Google says llms.txt is not required and does not positively or negatively affect Search visibility or rankings.

Independent research also provides little evidence of a citation benefit. Ahrefs found that most files in its dataset were not requested at all during the period studied, while SE

Ranking found no relationship between file adoption and AI citation frequency in its sample.

The firm declines the package.

Instead, it uses the budget to:

• Update outdated financial guides
• Add qualified authors to specialist pages
• Improve internal linking
• Correct inconsistent business information
• Strengthen page structure
• Add useful first-hand explanations

Those improvements have value whether the visitor arrives through:

• Google Search
• AI Overviews
• ChatGPT
• Perplexity
• A normal referral

That is the right way to think about llms.txt today:

it is a legitimate and evolving proposal with useful applications in agent-friendly documentation, but it is not a proven shortcut to AI search visibility. For most ordinary business websites, strengthen the underlying content and technical foundation before worrying about the file.”

See also

• Generative engine optimisation — improving visibility across generative search systems
• Robots.txt — controlling crawler access
• XML sitemap — helping search engines discover important URLs
• LLM visibility — measuring appearances in generative answers
• AI Overviews — Google's generative Search feature
• Entity — how search and AI systems distinguish a business or organisation