Omnibus AI enhancements

[alexander-sei]

What is the purpose of the change?

Improve how our docs are served to LLMs and clean up old scraping code we're no longer using.

Describe the changes

LLM-friendly Markdown access for every page

• Added an API route and middleware so any doc page can be fetched as clean Markdown by appending .md to the URL (e.g. docs.sei.io/evm/evm-general.md). Strips JSX wrappers, converts HTML tables, keeps frontmatter.
• Also generates static .md files at build time for CDN caching.

Overhauled llms.txt / llms-full.txt

• Rewrote the generation to follow the llmstxt.org spec more closely — pages are now grouped into sections (Learn, AI Tooling, EVM Dev, Node Ops) with overviews, quick-reference info (chain IDs, RPCs, token addresses), and key external resources.
• Output moved to public/ root so they're served directly instead of redirected from _scraped-docs/.
• Excluded /cosmos-sdk pages from LLM output.

Removed dead scraping scripts & deps

• Deleted scrape-docs-direct.js, scrape-docs-rendered.js, and scrape-docs.js (~1,700 lines total). All superseded by scrape-docs-html.js.
• Dropped puppeteer, puppeteer-core, and @sparticuz/chromium* from devDependencies.

Config

• Added proper Content-Type and caching headers for .md/.txt files in next.config.mjs.
• Updated vercel.json redirects to point old paths to the new canonical locations.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
sei-docs	Ready	Preview, Comment	Feb 19, 2026 1:15am