What is llms.txt and why we added it to our site
· 5 min read · CLICKO
Last updated:
A small text file that explains your site to AI assistants. What it does, what it does not do and whether it is worth the effort.
More and more people ask an AI assistant before they open a search engine. "Who builds e-shops in Thessaloniki?" is now a question that can be answered without a single link being clicked.
How fast this is happening, in three numbers:
45%
of US consumers asked an AI tool to recommend a local business, up from 6% a year earlier.
Source: BrightLocal, Local Consumer Review Survey 2026, March 2026
What it is
llms.txt is a proposal published in 2024 at llmstxt.org. It is a plain Markdown file at the root of your site, at /llms.txt, that says in a few words who you are, what you offer and where your important pages are.
The idea is simple. A web page is full of menus, buttons and code. A language model reads a clean text with headings and lists more easily and more accurately.
What it looks like: our own llms.txt
The structure is always the same: a heading with the name, a one sentence summary, then sections with lists of links. An excerpt from ours:
# CLICKO
> We create websites, e-shops and digital products that are not just beautiful. They are designed to deliver results.
## Services
- **E-shops**: ...
## Projects
- [Creatorshop](https://creatorshop.gr): ...
## Blog
- [Shopify or WooCommerce?](https://clicko.gr/en/blog/shopify-i-woocommerce): ...You can see the whole file at clicko.gr/llms.txt. In it we have included:
- One paragraph on who we are and what we build.
- Our services, with one sentence for each.
- All our projects, with a link, type and technologies.
- The blog articles and how to get in touch.
The file is not written by hand. It is generated from the same data the site shows, so when we add a project or an article it updates itself and never says anything different from the site.
What it does
- It gives an assistant a clean, accurate summary of your business, without having to dig through menus and marketing copy.
- It reduces the chance of being described wrongly: wrong services, wrong city, projects you did not do.
- It shows which pages are the important ones, so that if the assistant reads only a few, it reads the right ones.
What it does not do
- It is not an official standard. No company has committed to its assistants reading it.
- It does not replace robots.txt. That file says who is allowed to read the site; llms.txt says what is worth reading.
- It does not replace SEO. If the content of the site is thin, a text file will not save it.
- It does not bring mentions or visits on its own. It is an aid, not a channel.
Do AI bots get into your site?
Before you bother with llms.txt, check something more basic: whether the assistants' bots are even allowed to read your site. That is set by robots.txt, which you can see by opening yourdomain.gr/robots.txt. The names you care about:
| Bot | Company | What it does |
|---|---|---|
| GPTBot | OpenAI | Collects content to train the models |
| OAI-SearchBot | OpenAI | Builds the index for ChatGPT search |
| ChatGPT-User | OpenAI | Opens a page when a user asks for it |
| ClaudeBot | Anthropic | Collects content to train the models |
| Claude-SearchBot | Anthropic | Builds the index for Claude's search |
| PerplexityBot | Perplexity | Builds the index for Perplexity's answers |
| Google-Extended | Not a separate bot; it says whether what Googlebot reads can be used for Gemini |
If you see "Disallow: /" under any of these, that assistant does not read your site. Check your hosting provider or CDN too: some, such as Cloudflare, have a setting that blocks AI bots, and it is often switched on without you knowing. To see whether the bots actually visit, search for these names in your server logs or your hosting statistics.
In our own robots.txt we let them all in, except for the internal endpoints:
User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ClaudeBot
User-agent: PerplexityBot
Allow: /
Disallow: /api/If you want assistants to recommend you but not to train models on your content, leave the search bots open (OAI-SearchBot, Claude-SearchBot, PerplexityBot) and block the training bots (GPTBot, ClaudeBot).
Is it worth it?
It costs very little and does no harm. We do not promise anyone that it will bring customers, because it cannot yet be measured reliably. We see it as good housekeeping: if an assistant decides to read your site, it is better that it finds a clean summary than has to guess.
Frequently asked questions
What is llms.txt?
A Markdown file at the root of a site (yourdomain.gr/llms.txt) with a short description of the business and links to the important pages, written so that AI assistants can read it easily.
Do ChatGPT and other assistants read llms.txt?
No major company has publicly committed to its assistants reading it. That is why we see it as a small, cheap addition and not a guarantee.
How do I make an llms.txt for my site?
Write a heading with the business name, a one sentence summary and lists of links to the important pages, then upload it to the root of your site. On WordPress, some SEO plugins generate it automatically.
How do I know whether AI bots visit my site?
Open your site's robots.txt and check that GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot are not disallowed. Then search for the same names in your server logs or your hosting statistics.
Got something similar in mind?
Tell us what you want to build and we will reply with a clear proposal.