Our own research
What do AI crawlers miss when they skip JavaScript?
We loaded each homepage that served HTML to our bot twice: as plain HTML and with JavaScript. On October 20, 2026, structured data hardly ever depended on JavaScript: only 1.3% of these homepages among the 10,000 most popular sites got their JSON-LD from JavaScript. Text is another story. On 26.4% of them, less than half of the words are in the HTML that a bot without JavaScript receives.
Measured on October 20, 2026
- of top 10,000 homepages have JSON-LD only after JavaScript
- 1.3%
- have most of their text only after JavaScript
- 26.4%
- show their main heading (h1) only after JavaScript
- 6.5%
What we tested
A web page arrives in two steps. First the server sends HTML. Then the browser runs the page's JavaScript, which can add text, headings and structured data.
Not every bot does that second step. Vercel's 2024 log study found that the big AI crawlers fetch JavaScript files but don't run them. Anthropic's web fetch tool for developers says it doesn't support pages built with JavaScript. Google is different: it renders pages.
So we loaded each homepage twice, seconds apart and with the same bot name: once as plain HTML, the way a bot without JavaScript sees it, and once in a real Chrome browser that runs the JavaScript. Then we compared the two.
Structured data that only appears with JavaScript
51.2% of the top 10,000 homepages have JSON-LD in the raw HTML. Only 1.3% get it from JavaScript alone, and on 1.3% JavaScript adds extra types. For structured data, the raw HTML is usually the whole story.
| Measure | Top 10,000 | 10,001–100,000 | 100,001–1 million | Dutch-language | German-language |
|---|---|---|---|---|---|
| Has JSON-LD in the raw HTML | 51.2%46.1%–56.3% | 51%45.9%–56% | 49%43.8%–54.2% | 43.2%38.2%–48.3% | 33%28.2%–38% |
| Has JSON-LD after JavaScript | 52.3%47.2%–57.3% | 52.6%47.5%–57.6% | 49.9%44.6%–55.1% | 43.2%38.2%–48.3% | 33.5%28.8%–38.6% |
| JSON-LD only after JavaScript | 1.3%0.6%–3.1% | 1.6%0.8%–3.5% | 0.9%0.3%–2.5% | 0.3%0%–1.5% | 0.6%0.2%–2.1% |
| JavaScript adds extra types | 1.3%0.6%–3.1% | 0.3%0%–1.5% | 0.6%0.2%–2.1% | 0.5%0.1%–2% | 0.6%0.2%–2.1% |
| JavaScript removes it again | 0.3%0%–1.5% | 0%0%–1% | 0%0%–1.1% | 0.3%0%–1.5% | 0%0%–1.1% |
| Homepages compared (n) | 371 | 367 | 349 | 368 | 349 |
Broken JSON-LD is rare too: 0.5% of the homepages with JSON-LD in the raw HTML have a block that is not valid JSON. Such a block can't be read as structured data at all. Our schema validator reads the raw HTML the same way and shows the exact line.
| Measure | Top 10,000 | 10,001–100,000 | 100,001–1 million | Dutch-language | German-language |
|---|---|---|---|---|---|
| At least one block is not valid JSON | 0.5%0.1%–2.9% | 1.6%0.5%–4.6% | 2.9%1.3%–6.7% | 1.3%0.3%–4.5% | 0.9%0.2%–4.8% |
| Homepages with JSON-LD in raw HTML (n) | 190 | 187 | 171 | 159 | 115 |
Under each share you see its range: with 95% confidence the real share lies between those two numbers. Columns 1-3 are popular sites by rank in the Tranco list; columns 4-5 are Dutch- and German-language sites from the Curlie directory.
Title, description and main heading
The page title is almost always in the raw HTML: only 1.1% of the top 10,000 homepages add it with JavaScript. The main heading is a different matter: on 6.5%, the h1 only appears after JavaScript.
| Measure | Top 10,000 | 10,001–100,000 | 100,001–1 million | Dutch-language | German-language |
|---|---|---|---|---|---|
| Page title | 1.1%0.4%–2.7% | 1.6%0.8%–3.5% | 0.9%0.3%–2.5% | 0.5%0.1%–2% | 0.3%0.1%–1.6% |
| Meta description | 1.1%0.4%–2.7% | 1.6%0.8%–3.5% | 0.9%0.3%–2.5% | 1.1%0.4%–2.8% | 0.6%0.2%–2.1% |
| Main heading (h1) | 6.5%4.4%–9.4% | 8.4%6%–11.7% | 5.7%3.7%–8.7% | 1.1%0.4%–2.8% | 0.9%0.3%–2.5% |
| Homepages compared (n) | 371 | 367 | 349 | 368 | 349 |
The text itself
On the typical top 10,000 homepage, 87.5% of the words a browser shows are also in the raw HTML. But on 26.4% of the homepages, less than half of the text is there without JavaScript. A bot that skips JavaScript only sees part of those pages.
| Measure | Top 10,000 | 10,001–100,000 | 100,001–1 million | Dutch-language | German-language |
|---|---|---|---|---|---|
| Less than half of the words are in the raw HTML | 26.4%22.1%–31.1% | 23.9%19.8%–28.6% | 16.3%12.8%–20.5% | 13.2%10.1%–17.1% | 8%5.6%–11.4% |
| Homepages compared (n) | 368 | 360 | 344 | 363 | 348 |
| Measure | Top 10,000 | 10,001–100,000 | 100,001–1 million | Dutch-language | German-language |
|---|---|---|---|---|---|
| Words in the raw HTML | 948 | 710 | 830 | 453 | 505 |
| Words after JavaScript | 1,409 | 1,058 | 1,044 | 523 | 552 |
Want to see what a bot without JavaScript gets from your page? Open the page source (Ctrl+U, or Cmd+Option+U on a Mac) instead of the page itself. Or let our free GEO analysis check the page.
What this does not prove
- It doesn't show that AI tools use structured data to choose sources, or that adding it gets you quoted.
- We are not an AI crawler. Sites can send AI bots other HTML than they sent us, based on the bot's name or IP address.
- Homepages only. Product pages and articles can behave differently.
- We count all text in the HTML, also text that is hidden on screen, on both sides. We don't check what a visitor actually sees.
- We clicked nothing. Where a cookie or consent banner shows up, the page after JavaScript can contain the banner's text, or less of the page itself.
- Text inside shadow DOM (used by some web components) is not part of the page HTML we read, so we did not count it after JavaScript either.
- It says nothing about Google: Google renders JavaScript before it indexes a page.
How we did it
We drew 1,500 sites at random from each of five groups: three rank bands of the Tranco list 467YX of popular websites (ranks 1-10,000, 10,001-100,000 and 100,001-1,000,000), plus the Dutch- and German-language sections of the Curlie web directory (47,502 and 309,045 sites listed with their homepage). A fixed random seed means anyone with the same two files draws the same sites.
From the sites whose homepage opened normally in our robots.txt test, we took the first 400 per group in random order. Before loading a page, we fetched robots.txt again and only continued when it still allowed our bot.
The browser was Chrome/141.0.7390.76 (headless), with the same bot name as the plain request: GEOScoreBot/1.0 (+https://geo-score.online). It waited for the page to finish loading plus up to 10 seconds of network quiet, then read the page. It skipped images, video and fonts, which don't change the text, and followed robots.txt for the site's own files. The run took about 32 minutes.
Both versions went through the same code: JSON-LD from our schema validator's parser, the page title, meta description and h1 when they are not empty, and the words in the page body without scripts, styles and comments. A page counts as "most text needs JavaScript" when less than half of the words after JavaScript are also in the raw HTML.
We wrote down the questions, the sample and every measure before the crawl started, and saved that plan. Anything we decided after seeing the data is marked as exploratory.
| Step | Top 10,000 | 10,001–100,000 | 100,001–1 million | Dutch-language | German-language |
|---|---|---|---|---|---|
| Homepages tried | 400 | 400 | 400 | 400 | 400 |
| robots.txt now said no to our bot | 2 | 2 | 2 | 1 | 0 |
| Plain HTML request failed | 0 | 2 | 0 | 0 | 0 |
| Browser could not load the page | 6 | 11 | 24 | 24 | 47 |
| JavaScript sent the browser to another page | 21 | 18 | 25 | 7 | 4 |
| Compared: counted in the shares | 371 | 367 | 349 | 368 | 349 |
| … of which still loading when we stopped waiting | 46 | 37 | 24 | 12 | 22 |
When JavaScript sends a browser to another page (a language page, a cookie wall, a bot check), the two versions are no longer the same page, so we leave them out. A bot without JavaScript stays on the first page.
Other studies on the same question
Vercel (vendor data, December 2024) studied AI crawler traffic on its own network. It found that none of the big AI crawlers rendered JavaScript at the time: they fetched JavaScript files but did not see the content that JavaScript adds. Gemini was the exception, because it uses Google's crawler, which renders pages.
Anthropic's documentation for its web fetch tool, which developers use to let Claude open a page, says it does not support websites that are built with JavaScript. That covers the developer tool; the documentation says nothing about the Claude app.
Check your own site
These are averages. Your own site is either in or out. Our free tools check it in seconds.
Download the data
Every number on this page is in the CSV file, one row per number, with its count, its base and its 95% range. It contains totals only, no site names.
Download the numbers (CSV)You may reuse the numbers under the CC BY 4.0 licence. Please name GEO-Score and link to this page.
Sources of the site lists
Popular sites: the Tranco list 467YX (Le Pochat et al., NDSS 2019).
Dutch- and German-language sites: with content from Curlie.org - the largest human-edited directory of the web. Contribute by submitting a website or becoming an editor. Curlie data is licensed under CC BY 3.0.
October 20, 2026: first measurement published.