# How to build an agent-first website, from our first six weeks and 5 million page views (slim)

> Quynh Tran Thanh on what she and Vlad Cealicu learnt building Anchor Terminal for agents and with them, from their first podcast. Short pages for agents, llms.txt and the URLs agents guess, files instead of a database, how to tell people from agents and crawlers, and what 5 million page views were made of.

- Full: https://www.anchorterminal.com/blog/how-to-build-an-agent-first-website.md (~3,800 tokens) · this version ~1,680 tokens · JSON https://www.anchorterminal.com/blog/how-to-build-an-agent-first-website.json · canonical https://www.anchorterminal.com/blog/how-to-build-an-agent-first-website
- Index: https://www.anchorterminal.com/llms.txt · API: https://www.anchorterminal.com/api/v1/index.json · MCP: https://www.anchorterminal.com/mcp · Updated: 2026-11-19
- Licence: CC BY 4.0, credit Anchor Terminal and link this page (https://www.anchorterminal.com/terms/#data)

By Quynh Tran-Thanh, 19 November 2026. 1,908 words, about 8 min to read.

Vlad and I recorded our first podcast, two conversations about how we build Anchor Terminal.

[Watch the podcast on YouTube](https://www.youtube.com/watch?v=4SyThXImQNo) (34 minutes).

We launched on 26 September 2026. When we recorded, the dashboard's all-time view showed over 5 million page views from over a million IP addresses, and 1.5 million of those requests were for the Markdown and JSON versions we publish for agents [[2]](#ref-2).

## Key takeaways

1. Ask the agent you already work with what it wants. It will tell you which formats it prefers and which cost fewer tokens.
2. Keep a short version of every page. Agents picked our short Markdown files with the facts and the ranking over the long pages with every review on them.
3. Give agents a way in. An llms.txt that says where everything is does for a website what a CLAUDE.md does for a codebase.
4. Name URLs the way an agent would guess them, and answer a wrong guess with the closest pages you have.
5. Keep everything in files for as long as you can. An agent finds a file in the repository without needing an MCP server or a database login.
6. Use fewer frameworks and libraries. Fewer ways to do a thing keep an agent focused on what to do.
7. Split your traffic into crawlers, agents and people before you celebrate a number. Most of our launch-day page views weren't people.
8. Search still decides whether agents find you. An agent mostly reads what Google or Bing gave it.

## Building for the agents who read the site

People use "agent-first" for two things, building for agents as your users and using agents to build.

The best place to start is the agent you already work with.

We learnt this as the pages grew.

Then help them find their way around.

Structure matters on both sides, folders inside and URLs outside.

Being independent helps here.

## Building with agents inside the repository

The other half is how we build, and three choices made the most difference.

There's no database.

There are no frameworks and few libraries.

Most features have a CLI.

Then we wrote down how we work.

It moves fast.

## How each of us works

My setup started differently.

I use both Claude and Codex.

I think that's where things are heading.

Vlad works in Claude on the web, in the sandbox Anthropic gives you, a big virtual machine that cleans itself up.

## What 5 million page views are made of

Vlad thought nobody would look at the site, so we launched without analytics and only added them on the Sunday.

- Crawlers fetch pages for a search engine or an AI index, such as Googlebot, Bingbot, Applebot and Meta's crawler. They say who they are in their User-Agent, some publish the IP ranges they crawl from, Google among them [[9]](#ref-9), and they fetch a page and leave without running JavaScript.
- Agents fetch a page because a person asked something, such as ChatGPT, Claude or Perplexity fetching for a user. They also say so in their User-Agent, and they mostly land on comparison pages, because the person asked what the difference is between two tools.
- People are browsers that ran our page script and kept a page in view for at least five seconds [[8]](#ref-8). There are far fewer of them than crawlers or agents, in the hundreds, because we build for agents first [[1]](#ref-1).

Crawlers are good news.

Agents come after crawlers.

The stats are our own.

Here's what the dashboard showed when we went through it [[2]](#ref-2).

- The most requested page of all time is the about page. Crawlers come back to see whether it changed, and people read it first to find out who we are.
- The MCP page and the reviews come next, and agents read the MCP page far more than people do, which tells us we're reaching the right audience.
- robots.txt is near the top, from crawlers checking what they're allowed to read.
- The page people read most was [Google Veo](/tools/google-veo), Google's video model.
- Google still sends the most people, then Reddit and X when we've been posting there, then Bing and LinkedIn.
- Most visits are from desktop, because crawlers and agents ask for the desktop page.
- Applebot, Meta and Amazon index us a lot, and ChatGPT fetching for a user is high.

## What we're trying next

We'd like more reviews and more people using the site, because the reviews are something a competitor can't copy.

Vlad's post [Agent first, inside and out](/blog/agent-first-inside-and-out) has more on the files our agents read before they start [[6]](#ref-6).

---

More in [Inside Anchor Terminal](https://www.anchorterminal.com/blog/inside/index.md) · Previous [Agent first, inside and out](https://www.anchorterminal.com/blog/agent-first-inside-and-out.md)

- Older: [Gemini 4 Neon, and more horses in the race](https://www.anchorterminal.com/blog/gemini-4-neon-more-horses-in-the-race.md)
- Newer: [OpenAI's agents now finish half its two-to-four-hour research tasks](https://www.anchorterminal.com/blog/openai-agents-half-the-research.md)
- All posts: https://www.anchorterminal.com/blog/index.md

Anchor Terminal grades the tools AI agents run on, with a source for every score.

## References

Read on 19 November 2026.

[1] Anchor Terminal, How to Build an Agent-First Website: Lessons From Our Launch Hitting 5M views after 6 weeks, YouTube, 19 November 2026, 33 minutes 52 seconds.

[2] Anchor Terminal admin dashboard, all-time view, read out in the podcast in November 2026, not published.

[3] Anchor Terminal, for agents.

[4] Anchor Terminal, the not-found page, read on 19 November 2026 at /compare/openai-vs-vertex-ai, a comparison URL that doesn't exist.

[5] Anchor Terminal, the benchmark, "Vendors can't pay for placement.

[6] Vlad Cealicu, Agent first, inside and out, Anchor Terminal, 15 November 2026. Source for CLAUDE.md, the voice files, the facts file and the other files the agents read before they start.

[7] Anchor Terminal, CLAUDE.md and the deploy, 19 November 2026, not published.

[8] Anchor Terminal, the session logs and their report (internal/sessions), 19 November 2026, not published.

[9] Google, Verify requests from Google crawlers and fetchers, Google for Developers.

[10] Anchor Terminal, privacy.
