Pulse

analytics built for AI products.

Traditional product analytics tools weren't built for AI products. PostHog tracks button clicks. Mixpanel tracks user flows. Neither was designed for the metrics that actually matter when you're shipping an AI product — token spend, tool call latency, model error rates, prompt regressions.

Pulse is a concept for what analytics could look like when the product you're measuring is an AI agent or LLM-powered tool. Different metrics, different visualizations, different alerts.

Three design decisions:

01 — Spend lives in the top row of stat cards. Most AI products go over budget within their first month of growth. Most teams don't notice until they get the invoice. Putting spend up there alongside conversations and tool calls forces the engineer to see the cost of every feature decision.

02 — Tool call breakdown is the second-most important chart on the page. When something goes wrong with an AI product, it's almost always a tool — not the model. The model just generates strings. The tool is what actually hits Stripe or Postgres or Slack. Surfacing tool call volume + ranking by usage lets engineers spot anomalies before they become incidents.

03 — Latency by model, not by endpoint. Traditional APM tools group latency by route. AI products need latency grouped by model. A 2.8-second Claude Opus response is fine. A 2.8-second Haiku response is a fire. Same number, completely different operational meaning.

Built as Case Study 009. Most curious about the model split donut — I went back and forth on whether to show it as percentage or absolute call count. Open to feedback.

On putting spend at the top

The biggest hidden problem with AI products right now is that engineers ship features without seeing the cost. A new tool gets added to a workflow, it runs 50,000 times that week, suddenly you're paying $4,000 in API fees you didn't budget for. Pulse forces the cost number into the same visual hierarchy as the engagement metrics — they're equally important, so they should look equally important.

Open to work

I'm a full-stack designer-developer — I design product UI and ship working code. If you're building developer analytics, AI observability, or any data-heavy SaaS and need someone who can do both, get in touch and DM me here.