Want to follow updates from Weeblr.com about our extensions, but too many news, too many blogs to read?

Many of our blog posts are also produced as podcast episodes so paste this link into your favorite podcast app, or search for The Weeblr blasts on Apple Podcasts, Spotify, Google Podcasts, PocketCasts and several other platforms and you won't miss an update detail at most convenient time for you!

4SEO pages list with new refreshed UI design

AI Search in 4SEO

Written by Weeblr on .

 

Hi all,

In its last major release, version 7, 4SEO has just gained a new configuration panel called AI search.

Unfortunately, there are many "AI search features" being promoted by various Joomla extensions, when in fact AI search isn't really a thing in itself, at least concerning your website.

Still, there are a few use cases where we can implement technical solutions on the site. That's why 4SEO now fully supports generating Markdown versions of your website pages, as well as creating an llms.txt file.

And then there's also the Content Signals policy, which may or may not become a standard, but we are supporting it for now.

Two of the most vaunted features that supposedly help you gain more visibility in ChatGPT, Gemini, or Claude are having Markdown versions of your pages and listing them in an llms.txt file at the root of your site. Let's discuss that:

Are Markdown and llms.txt useful for a Joomla website SEO/AI search?

No. Having a Markdown version of your website pages and listing them in an llms.txt file offers no benefit for SEO or for AI search.

All major search engines (Google, Bing) and AI assistants (ChatGPT, Gemini, Perplexity) have stated they do not use the Markdown versions of pages to train or read data. They also do not read any file such as llms.txt.

This has also been confirmed by large-scale studies. Currently, Markdown versions and llms.txt are simply not relevant.

Will they become useful in general? Likely not. There are several reasons for this, the top two being:

  • Few sites have Markdown versions, so AI services would still need to be able to read and understand normal HTML anyway. That's double the work for them.
  • Markdown versions are not visible to humans, so, as usual, Markdown versions are likely to be filled with spam and cannot be trusted.

Now, if they are useless but also don't cost much, why not use them anyway? Because they actually incur costs:

  • Search engines and other bots will read them (read, not use them), because these versions are linked from your regular page with a <link rel="alternate"> tag. On larger sites, this wastes some of your crawl budget.
  • This adds a significant load to your server, as generating Markdown from content requires considerable effort (caching helps).

Can Markdown and llms.txt be useful?

Well, yes, otherwise why would I have taken the time to add them to 4SEO??

AI agents want to learn your software and product documentation.

If you create software or other products and have documentation on your site, it's likely helpful to provide Markdown versions to AI agents tools such as Claude Code, Claude Cowork, Codex, Hermes, OpenClaw, GrokAI, or Gemini CLI.

And coding and AI agents makers have actually stated that they want access to Markdown, and at least for some, an llms.txt file.

Markdown generation config in 4SEO

Because this is an absolutely worthwhile use case, the 4SEO implementation is exhaustive:

  • Conversion is fully automatic and extracts only the important, meaningful content of a page.
  • Enable Markdown for all or only parts of your site.
  • Cache Markdown to essentially remove the load on your server.
  • Optionally include structured data.
  • Optionally include an llms.txt file, for which you can configure a description.

You can certainly enable Markdown generation for your entire website if you wish, just be aware that the real use case is documentation and AI agents.

Content Signals policy

Content Signals policy is an initiative by Cloudflare to allow website owners to precisely inform AI tools of how they accept – or refuse – their content to be used.

Content Signals configuration page in 4SEO admin

It separates acceptance into three areas:

  • AI training: Can my content be used to train AI models?
  • Search: Can my content be used for regular search (Google, Bing,...)?
  • AI input: Can my content be used when a user asks ChatGPT or Gemini to check a specific website page?

It is, of course, entirely up to the AI companies to read and use these signals, but with Cloudflare being Cloudflare, there is some weight to it.

And contrary to Markdown or llms.txt, they cost absolutely nothing to implement, so 4SEO now provides you with Content Signals policy support.

Learn more

I have spoken about this very topic at the latest JoomlaDay USA, and the videos will very soon be available (registration required), so make sure to watch that if you want to get deeper on what you can do today about AI search. 

That's all for now; more coming your way soon!

Cheers,

Yannick