- Home
- Artificial intelligence
- llms.txt Generator
llms.txt Generator
Build the file that tells AI assistants what your site is — and what matters in it.
Generated locally from what you enter, following the convention published at llmstxt.org. Nothing is fetched from your site and nothing is stored. The file that comes out is yours to review before publishing — and it should be reviewed.
What this tool is for
When someone asks an AI assistant about your company, the model has to work out what you do from whatever it can reach. Left to itself, it infers that from your navigation, your footer and whichever pages happen to rank — a picture that is often outdated and sometimes simply wrong.
llms.txt is an attempt to fix that at the source. A short Markdown file at the root of your domain, written for a machine to read directly: here is what this site is, here are the pages that matter, in this order. No crawling to infer, no guessing.
Be clear-eyed about its status. It is a convention proposed in 2024, not a standard, and no major provider has committed to honouring it. What makes it worth doing anyway is the ratio: half an hour of thought, a few kilobytes, no downside. And the exercise of writing it — deciding what genuinely defines your site — tends to be useful regardless.
How to read your results
The summary line carries most of the weight. It is the sentence a model is most likely to reuse when describing you. Write what you do and for whom, in concrete terms. "Innovative solutions for your digital transformation" tells a machine nothing; "free SEO tools in French, no sign-up" tells it everything.
The page list is a curation exercise, not an inventory. Each entry should earn its place by defining something about the site. If you cannot say why a page is there, remove it. Short files are read; long ones are skimmed like any other list.
The descriptions after each address matter as much as the addresses. A bare URL forces the model to fetch and interpret. A URL followed by "step-by-step guide for first-time users" is immediately usable, even without a visit.
The content type is the technical detail that breaks most deployments.
The file must be served as text/plain. Some hosts serve unknown extensions
as a download or wrap them in HTML, and either makes the file unreadable. Load it in a
browser and check that you see raw text.
The most common mistakes
Turning it into a second sitemap. Four hundred URLs in llms.txt defeats the purpose entirely. The value of the file is the selection. If everything is important, nothing is, and the model falls back to guessing — exactly what the file was meant to prevent.
Writing it in marketing language. Superlatives and slogans are noise to a model, and they crowd out the facts that would actually help. Write it the way you would brief a new colleague on their first morning: plain, specific, no adjectives that cannot be checked.
Publishing it and forgetting it. A file describing a discontinued offer teaches models something false, indefinitely. This is the one genuine risk of llms.txt: it is more harmful stale than absent. Put a reminder in the calendar.
Expecting it to replace robots.txt. The two do unrelated jobs. llms.txt describes; robots.txt permits. A site with a beautiful llms.txt and a robots.txt that blocks AI crawlers has written a letter and burned the envelope — check both with the AI crawler checker.
Believing it affects rankings. Nothing supports that claim. Treat any service selling llms.txt as an SEO technique with suspicion. Its purpose is to be described accurately, which is worth something on its own.
Related tools
- AI Crawler Access Checker Check whether GPTBot, ClaudeBot and PerplexityBot can read your pages
- Robots.txt Tester Test whether a URL is allowed or blocked, and by which rule
- Googlebot Simulator See a page exactly as a search engine crawler sees it
- Meta Tags Analyzer Title, description, Open Graph and indexing directives of a page
Frequently asked questions
Is llms.txt an official standard?
No. It is a convention proposed by Jeremy Howard of Answer.AI in September 2024, not a specification ratified by any standards body. No major AI provider has publicly committed to reading it. Treat it as a low-cost bet rather than a requirement: a few kilobytes of Markdown that may help and cannot hurt.
How is it different from robots.txt or a sitemap?
robots.txt says what a crawler may not read. A sitemap lists every URL for machines that already crawl well. llms.txt does something else: it explains, in prose a model can read directly, what the site is and where the important things are. It is curation, not enumeration — which is why a good llms.txt is short.
Where does the file go?
At the root of the domain, exactly like robots.txt: https://yoursite.com/llms.txt. It must be served as plain text. If your server sends it as HTML or forces a download, the convention is broken and no model will parse it.
Should I list every page?
No, and that is the most common misuse. A file listing four hundred URLs is a sitemap with extra steps and helps nobody. Keep to the pages that genuinely define your site: what you offer, your reference documentation, your key guides. Ten to thirty entries is a healthy range for most sites.
Will it improve my rankings?
There is no evidence that it affects classic search rankings, and you should distrust anyone who claims otherwise. Its plausible benefit is different: helping a model summarise your site correctly when someone asks about it. That is a positioning question, not a ranking one.
Do I need to keep it up to date?
Yes, and this is the real cost. A file describing an offer you dropped a year ago is worse than no file at all, because it actively teaches models something false about you. Review it whenever your main offer changes — twice a year is a reasonable rhythm for most sites.