Why we build inference for everyone

"Access to intelligence is a fundamental human right. We exist to make it universal."

That is our mission. Our product changed a lot this year, so I want to explain what the mission means to me, what it means in practice and why we changed course.

Why this matters to me

My personal mission is free education for every child in the world.

Education transformed my own life. I want that to happen for everyone, because when it does, it transforms whole societies.

A great personal tutor is one of the most valuable things a child can have, and today only some children get one. The child of a billionaire and a homeless child should have access to the same level of intelligence, the same level of education and the same personal tutor. AI can be that tutor. It can only reach every child if it costs almost nothing to run.

So that is where we want to take the cost of a token: as cheap as water.

What the mission means in practice

In practice it means one thing: faster, cheaper inference. The best open models should be fast, cheap and easy for anyone to use. A student, a two-person startup and a large company should all be able to call a frontier model without a procurement process or a GPU cluster of their own. Every time a model gets faster and the price of a token falls, more people can build with intelligence. That is the work we care about most.

Why we started on-device

Shubham and I started RunAnywhere to run AI on the device in your hand. I spent years building mobile SDKs at Intuit. Shubham built infrastructure at Microsoft Azure and AWS. We went through Y Combinator in the Winter 2026 batch.

We started there because the goal is free intelligence, and the freest intelligence anyone can get runs on a device they already own. There is no round trip to the cloud, no GPU to rent and no bill for each token. Models were also getting smaller fast: quantization and better training kept shrinking the number of parameters a useful model needs. We bet that a decent phone or laptop would soon run a model good enough to matter.

We shipped a lot of on-device work:

• An open-source SDK with one C++ core and six SDKs: Swift, Kotlin, React Native, Flutter, TypeScript for the web, and a C++ CLI.
• MetalRT, our engine for Apple silicon.
• QHexRT, our engine for Qualcomm Hexagon NPUs.

To build those engines faster, we built an agentic pipeline that writes kernels and checks them. That pipeline turned out to matter more than we expected.

What we learned

The engines got fast. The models that fit on a phone or a laptop are getting better, but they are not there yet.

People want models that write real code, plan across many steps and hold a long conversation. Those models do not fit on the hardware most people own today. To run a useful model locally, you need something like a 64 GB Mac. Most phones and laptops are a long way from that.

We had a choice. We could wait for small models to catch up, or we could go where the useful models are today. We picked the second, and we keep working on the first.

Why we host frontier open models

The open models released this year are very good. Anyone can download the weights. Serving a very large mixture-of-experts model well is the hard part, and it is the part we know how to do.

So we built Wally. Wally by RunAnywhere is one CLI and one API key for frontier open models. The API is OpenAI-compatible, so most existing code works once you change the base URL. The CLI connects coding agents like OpenCode, Claude Code and Codex to the model you pick. You sign in with wally account login and start working.

Wally launched on 21 September. On 22 October we published its catalog of ten models. Production Wally runs with zero data retention by default: it does not store your prompts or the replies.

Why our AI agents build the stack

If we want models to be fast and cheap, we have to control the stack that serves them. Speed and the cost per token both come from the kernels, the serving engine and the hardware under them. When you rent someone else's stack, you take their speed and their costs as given.

We already had a pipeline of AI agents that writes kernels and checks them on real hardware. Those agents now build the kernels and the end-to-end serving stack for data center chips, and our engineers direct and verify their work. We will write about it when it is ready, with the method and the raw data behind every number. Until then we publish none.

What happens to the on-device work

The SDK stays open source, and we keep maintaining it. If you ship it in an app today, nothing changes for you.

MetalRT and QHexRT stay up as research. The kernel pipeline behind them is the base of the engine work we do now, so none of it is wasted.

On-device is still part of where we are going. Next we will work on hybrid inference, so a request can run on your device when a local model is good enough and on a larger model in the cloud when it is not. The cheapest token is the one your own device can serve.

Who should talk to us

If your team runs coding agents or a product on a closed model and wants to move to an open one, I want to hear from you. Tell me what you run, what it costs and where open models fall short for you. That tells us what to fix next.

Try it

Install Wally, sign in and point your agent at a model. Usage runs on prepaid credits you buy in the console. The Wally docs cover setup, and runwally.com has the rest.

Want to talk first? Talk to us. The link goes straight to my calendar.

Sanchit