llms.txt is a text file published at the root of a website that aims to give large language models (the AI models behind systems like ChatGPT, Claude and Gemini) a plain summary of the site and its most important pages. The idea was proposed in September 2024 by Jeremy Howard of Answer.AI. The reasoning in short: web pages are full of menus, ads and code, and offering an AI system a short, clean summary that says "here's what matters" could make its job easier.
The most important point here is this: llms.txt is a proposal, not an official standard. It isn't a rule everyone has followed for years like robots.txt, and as of this writing there is no official statement from the major AI providers that they regularly read this file and use it in their answers.
What goes in llms.txt?
The file is written in a plain text format called Markdown and sits at the root address of the site, that is at yourdomain.com/llms.txt. The suggested structure is:
- At the very top, the name of the site or company as a one-line heading.
- Right below it, a short summary paragraph describing what the company does.
- Important notes if needed: service area, fields of work, product groups.
- Then lists of links grouped under headings, for example "Services," "Products," "Guides," each with a one-sentence description.
- Optionally, a list of less important pages under a heading called "Optional."
The proposal also has an extended version called llms-full.txt, which gathers the full text of the pages into a single file. This is mainly intended for long technical content such as software documentation.
Who uses it, and who doesn't?
The group that has adopted the file most is companies publishing software documentation. Some AI coding tools used by developers can make use of an llms.txt file to read a library's documentation. In that area it has real value.
On the general search side the picture is different. Google representatives have said in public statements that they don't use this file for search or AI summaries. Other major providers haven't officially announced that their crawlers use llms.txt for ranking or source selection either. Seeing occasional requests for this file in your server logs doesn't mean it is being used in answers.
Should you add it?
The short answer: adding it does no harm and costs very little, but don't expect a visibility miracle from it.
| Situation | Our recommendation |
|---|---|
| Corporate brochure site | Add it if you like, but keep it near the bottom of the priority list |
| You publish a software product, API or technical documentation | Worth adding; it makes your docs easier for AI tools to read |
| A site with unresolved basic SEO problems | Indexing, speed and content first; llms.txt later |
| A large e-commerce catalog that changes very often | Summarize category and information pages instead of listing products |
Let's also be clear about what llms.txt can't do: it doesn't replace robots.txt, meaning it doesn't open or close bots' access to your site. If your pages themselves can't be read, this file won't fix that. And writing a summary that contradicts the site's content creates more confusion than benefit.
What actually matters?
For visibility in AI search, there are tasks whose impact is better understood: having your pages indexed by search engines, not blocking AI search bots unnecessarily in robots.txt, making content readable without JavaScript, and having pages give clear answers to concrete questions. We cover these in our post on appearing in AI answers, and bot access decisions in our post on AI bots and robots.txt.
It is best to think of llms.txt not as a replacement for these basics but as a small add-on on top of them.
Practical checklist
- Does the file open at the root address as plain text?
- Is the company summary consistent with the About information on the site?
- Do the listed links really exist, and are they current?
- Is the file free of fast-aging information such as prices and campaigns?
- Is it clear who will update the file when the site structure changes?
How we do it at Globya
On the new sites we build, we can add an llms.txt file generated automatically from the site's own page structure, so the file updates whenever a new page is added and needs no manual upkeep. However, we don't present it to clients as a ranking tool. In our SEO and GEO work we always prioritize tasks with known impact. If you'd like to see where to start with your current site, a free preliminary analysis is a good first step.
If I add llms.txt, will ChatGPT read my site?
This file is neither an access permission nor an invitation. ChatGPT's search feature finds your site through its own crawlers and search infrastructure; there is no official statement that it uses the llms.txt file.
Does llms.txt replace robots.txt?
No. robots.txt tells bots which areas they may enter, and most major crawlers follow it. llms.txt is only a summary and content map; it contains no access rules.
If it becomes a standard later, will I have fallen behind?
Preparing the file is a short job. If it becomes a standard, it can be added in a few hours; adding it today carries no risk either. So the decision is a matter of priority, not urgency.
The Globya assistant is online 24/7; it answers right away and passes your question to the team if needed.