For years, we have optimised websites for humans: UX, usability, emotion. Now a second dimension has been added: websites must also be understood by machines, not only by search engine crawlers but also by AI assistants that fetch pages on behalf of their users and by agents that fill in forms and complete tasks. This article separates what is documented from what is hype, and shows which measures are worthwhile for Swiss SMEs.
What is Machine Experience (MX)?
Machine Experience (MX) describes how the meaning, structure and hierarchy of a website become readable for machines. It is about how design and technical decisions influence not only what people see, but also what machines understand and pass on. MX is a working term, not a standard. It does, however, help to distinguish three kinds of machine visitors:
Crawlers for search and training: Googlebot crawls pages for Google Search; to appear in its AI features, a page must be indexed and eligible to be shown with a snippet. OpenAI distinguishes between OAI-SearchBot, which surfaces websites in ChatGPT's search features, and GPTBot, which crawls content that may be used to train its models.
Fetches on behalf of users: when someone asks ChatGPT a question, the ChatGPT-User agent may visit a page directly. OpenAI notes that robots.txt rules may not apply to such user-initiated requests.
Agents that operate websites: they navigate, click and fill in forms. For this group, Lighthouse, Chrome's auditing tool, now has its own experimental category called "Agentic Browsing".
The term marks a shift in perspective: a layer of systems that summarise, compare and act on content is moving in between users and websites. How well they do this depends heavily on how cleanly a website is built.
How AI systems read websites
Machines see a website differently from humans. Google processes JavaScript web apps in three phases: crawling, rendering and indexing. Pages are placed in a rendering queue, where they may stay for a few seconds, but it can take longer. Google still recommends server-side rendering or pre-rendering, because it makes websites faster for users and crawlers, and because not all bots can run JavaScript.
In practice, if your most important content only appears in the browser via JavaScript, for example in a single-page application without server-side rendering, there is no guarantee that every system will see it. For its AI features, Google also lists making sure important content is available in textual form as a best practice. Anything that exists only in images, videos or behind interactions needs a text equivalent, such as meaningful alt text, transcripts or visible descriptions.
Practical MX optimisations
Server-side rendering: deliver all relevant content in the initial HTML. This helps bots that cannot run JavaScript and makes the page faster for people too. Our article on server-first architecture explains why this approach is gaining ground.
Semantic HTML and accessibility: use the correct HTML elements (header, nav, main, article, section) instead of generic div containers, a logical heading hierarchy, real buttons and links, and labelled form fields. Screen readers and machines benefit equally. Among other things, the new Lighthouse category checks names and labels, the integrity of the accessibility tree and the visibility of elements, and Google describes the accessibility tree as the primary data model for agents. Making your website accessible is therefore direct work on its MX.
Control access deliberately: Google lists as a basic requirement that crawling is allowed in robots.txt and by any CDN or hosting infrastructure. Check whether a firewall or bot protection is blocking crawlers unintentionally. With OpenAI, search and training settings are independent: you can allow OAI-SearchBot so that your content appears in ChatGPT search results, and block GPTBot if you do not want your content used for training.
Put llms.txt in perspective: the /llms.txt file is a proposal published by Jeremy Howard on 3 September 2024. It is meant to help agents use a website by providing a short Markdown overview: the only required section is an H1 with the name of the site, and the format also provides for a short summary and lists of links to key pages. Contrary to frequent claims, llms.txt is not a counterpart to robots.txt: robots.txt controls access, while llms.txt provides context and, according to the proposal, can complement robots.txt. Google states that you do not need to create new machine-readable files, AI text files or markup to appear in its AI features. An llms.txt file takes little effort and does no harm, but it does not replace any of the other measures.
Use structured data selectively: schema markup such as Organization, LocalBusiness, Product or Article describes your offering in machine-readable form. Two caveats matter. First, since August 2023 Google has only shown FAQ rich results for well-known, authoritative government and health websites, and HowTo rich results were deprecated in September 2023. Second, Google's AI features do not require any special schema, but your structured data must match the visible text on the page.
Clear positioning: state clearly on every important page which problem you solve, for whom and in which region. What a person understands in a few seconds, a machine can summarise correctly. Vague marketing language without concrete information on services, locations or prices, on the other hand, is hard to cite in any meaningful way.
A quick test for your website
Open an important page with JavaScript disabled, or view the source. Are headings, body text and prices in the HTML?
Check robots.txt and your CDN or firewall settings: which crawlers are allowed, which are blocked, and is that intentional?
Run Lighthouse and start by fixing missing labels on buttons, links and form fields.
Validate your structured data and compare it with the visible content.
Ask several AI assistants about your company and services, and note which details are wrong or outdated. This shows which information is missing or unclear on your website.
UX and MX are not a contradiction
Good MX design is good design. Clear hierarchies, semantic structures, fast loading times and precise content help people and machines alike. The key difference is that MX requires these qualities to be visible in the code and data, not just in the visual design. It is worth being cautious about promises, though: according to Google, the Agentic Browsing category in Lighthouse is based on proposed standards and currently focuses on gathering data. If you invest in clean HTML, accessibility and clear content today, you are well prepared for search engines, AI assistants and future agents alike. How this affects visibility in AI answers is covered in our article on Generative Engine Optimization.
As part of SEO and GEO optimisation, Sidora AG reviews how well websites can be read by machines and implements the findings in web design and UX and website development.
Sources
Google Search Central: Understand the JavaScript SEO basics (last updated 4 March 2026)
Google Search Central: AI features and your website (last updated 10 December 2025)
Chrome for Developers: Lighthouse agentic browsing scoring (last updated 5 May 2026)
Jeremy Howard: The /llms.txt file, v2 (published 3 September 2024, modified 10 August 2026)
Google Search Central Blog: Changes to HowTo and FAQ rich results (8 August 2023)
