AI and performance

How we prepare a PrestaShop store for AI assistants without opening it to every bot

AI assistants already answer "where can I buy this?". To recommend a store they need to read it quickly and understand prices, stock and promotions. At the same time, other bots download entire stores just to train models, using server resources without bringing a single customer. Blocking everything protects the server but removes the store from the answers. That is the problem that led us to build AI Friendly.

Why it is not trivial in PrestaShop

The first temptation is to add a few lines to robots.txt. In PrestaShop that does not last: the file is regenerated from the back office and any manual change is lost. And even if it stayed, it would not be enough:

  • Not all AI bots are equal. Some collect content to train models; others build the index the assistant uses to answer and cite; others read a page on the spot because a person has just asked. Treating them all the same forces a choice between server load and visibility.
  • Some bots ignore robots.txt and others impersonate well-known ones.
  • Faceted navigation creates almost endless combinations of filters and sort orders. A crawler that gets in there can request thousands of heavy pages.
  • The PrestaShop webservice is not designed for this: it needs a key, it is verbose and it is not built for an assistant to query the catalogue in seconds.
  • Promotions live in cart rules and are invisible outside the cart.
  • It has to work from PrestaShop 1.6 to 9.2.1, on very different themes and servers.

The approach: tell them apart before deciding

The core technical decision was to classify bots by what they do, not by who publishes them, and apply a different policy to each group:

Bot typeWhat it doesDefault policy
TrainingCollects content to train modelsBlock
AI searchIndexes so the assistant can answer and citeFast paths
User requestReads a page because someone askedFast paths
Classic search enginesGooglebot and BingbotNever touched

Googlebot and Bingbot are deliberately left out of every rule: they also feed Google's AI answers, Copilot and ChatGPT search.

Technical decisions

A robots.txt that repairs itself

Instead of editing the file by hand, the module manages its own delimited block and inserts it again whenever PrestaShop regenerates robots.txt, using the hook the core fires when it writes the file. A scheduled task also checks that it is still there. The merchant can regenerate the file at any time without losing anything.

Block as early as possible

The earlier an unwanted request is stopped, the less it costs. The module offers three levels: robots.txt only; plus a PHP response at the start of PrestaShop's dispatch, before the page is built; and, where the server allows it, .htaccess rules or an nginx snippet so the request never reaches PHP. It includes per-bot rate limiting and redirects heavy filter URLs to clean ones.

Verify instead of trusting

A bot's name can be faked. Where the provider publishes its IP ranges in a stable format, the module downloads them and blocks impersonators. Where it does not, we say so honestly: today an agent browsing with a normal browser cannot be reliably identified without third-party services.

Fast paths for the bots that matter

For assistants, the module publishes an llms.txt with the store map (categories, best sellers, best offers and policies), an extended catalogue file and a read-only JSON or Markdown API described with OpenAPI, with search, products, offers and public promotions. Responses are cached, validated with ETag and invalidated through the catalogue hooks when a product changes. Prices are always those of an anonymous visitor, so the cache stays valid and private price lists never leak.

Agentic browsing

It also exposes the agent resource manifest and registers read-only WebMCP tools in the browser, the checks PageSpeed Insights currently measures under agentic browsing. We ruled out running an MCP server inside PrestaShop: it needs persistent connections that are not viable on shared hosting, and the OpenAPI description already covers most agents.

Diagnostics with real data

The dashboard calculates a readiness score from around twenty checks and analyses real traffic from the last week: bots ignoring the rules, aggressive crawling, impersonation or traffic on filters. Each proposal explains why and, when it is safe, can be fixed in one click. It also attributes visits and orders coming from assistants.

What we learned

The hard part was not writing rules but deciding what not to block, and making everything survive the way PrestaShop regenerates its files, four generations of the platform and servers where the configuration cannot always be changed. That same judgement is what we apply when a store asks us for something the core does not offer.

What this development demonstrates

  • Using core hooks to extend PrestaShop without modifying its files
  • Real compatibility with PrestaShop 1.6, 1.7, 8 and 9 in a single module
  • Traffic control in several layers: robots.txt, PHP and web server
  • Read-only API design with caching, ETag and OpenAPI documentation
  • Performance: stopping requests before the page is built
  • Practical security: bot verification, no private prices exposed
  • Diagnostic tools and guided fixes in the back office