How to run an AI model right in the browser — no server, no API key

The usual path to AI in an app: get a key, stand up a backend, pay for tokens, explain to your users where their data went.
There's a second path far fewer people know about. You can download the model into the tab and compute on the user's machine. No key, no server, no bill — and nothing leaves the device, because there's nowhere for it to go. Let's build exactly that: semantic search, entirely in the browser.
What you'll need
A recent Chrome, Edge or Safari, a project folder, and any local server (npx serve is fine). A file opened as file:// won't do — you need http://localhost or HTTPS.
Step 1. Check whether there's a GPU
Open the browser console and run:
console.log(!!navigator.gpu);
const adapter = navigator.gpu && (await navigator.gpu.requestAdapter());
console.log(adapter);
Two checks, not one. navigator.gpu says the browser knows about WebGPU. But requestAdapter() can still return null — in Firefox on macOS, say, or on an unsupported card. If you get an object, you'll compute on the GPU. If it's null, that's fine too, just slower. What to do about it: why WebGPU isn't available.
Step 2. Add the library
Transformers.js does what Python's transformers does, in JavaScript. Install via npm:
npm i @huggingface/transformers
Or, if you want a single HTML file with no build step, straight from a CDN:
import { pipeline } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@4/+esm";
Step 3. Find out how much you're about to download
This is the step almost everyone skips, and shouldn't: ask for the size before downloading. A user on mobile data will not enjoy a surprise 400 megabytes.
import { ModelRegistry } from "@huggingface/transformers";
const modelId = "onnx-community/all-MiniLM-L6-v2-ONNX";
const options = { dtype: "q8" };
const files = await ModelRegistry.get_pipeline_files("feature-extraction", modelId, options);
const meta = await Promise.all(files.map((f) => ModelRegistry.get_file_metadata(modelId, f)));
const bytes = meta.reduce((sum, m) => sum + m.size, 0);
console.log(Math.round(bytes / 1e6), "MB");
While you're there, check whether it's already downloaded: await ModelRegistry.is_pipeline_cached("feature-extraction", modelId, options). If it is, the second run is instant.
Step 4. Run the model
The whole point. One function builds a ready pipeline: download the model, prepare the input, compute, parse the output.
import { pipeline } from "@huggingface/transformers";
const device = (await navigator.gpu?.requestAdapter()) ? "webgpu" : "wasm";
const embed = await pipeline("feature-extraction", "onnx-community/all-MiniLM-L6-v2-ONNX", {
device,
dtype: "q8",
progress_callback: (p) => {
if (p.status === "progress") console.log(Math.round(p.progress) + "%");
},
});
Two parameters carry the weight here:
device— where to compute."webgpu"on the graphics card,"wasm"on the CPU via WebAssembly. That switch is why we did step 1.dtype— weight precision."q8"is 8-bit quantization: a far smaller file at nearly the same quality. There's"q4"too, and even leaner"q2".
Step 5. Compute meaning
The model turns text into a vector — an embedding. Two texts about the same thing land close together, even with different words.
const texts = ["how do I cancel my subscription", "I want my money back", "borscht recipe"];
const out = await embed(texts, { pooling: "mean", normalize: true });
const v = out.tolist();
const cos = (a, b) => a.reduce((s, x, i) => s + x * b[i], 0);
console.log(cos(v[0], v[1])); // close — both are about money and quitting
console.log(cos(v[0], v[2])); // far — soup is unrelated
Those three lines are semantic search: compute vectors for your notes once, store them alongside the text, compare on each query. Not a single network request.
Step 6. Don't download it twice
Transformers.js puts downloaded weights in the browser cache — the same shared storage where IndexedDB lives. On the second visit nothing is fetched.
Three things so this doesn't break:
- serve over HTTPS — otherwise you get neither WebGPU nor a proper cache;
- show progress from
progress_callback— a silent 200-megabyte download looks like a hung site; - compute in a web worker for larger models, or the UI stutters while it runs.
What you get
An HTML page that downloads a model on first open and then works even without internet. It compares texts by meaning in tens of milliseconds, has never heard the phrase "API key", and sends nothing anywhere.
From here you change one string — the task name in pipeline. "automatic-speech-recognition" for speech, "image-classification" for what's in a photo, "translation" for translation. The library supports around two hundred architectures, and nearly all of them run on the GPU.
Which model to pick
In the browser the main criterion isn't "smart", it's "how much to download". The user is waiting for a page, not installing software, so here's the rough map.
- Embeddings and semantic search. The most rewarding case: MiniLM-class models weigh tens of megabytes and run instantly even without a GPU. Start here.
- Text classification — sentiment, topic, spam. Same size class, works on any hardware.
- Speech recognition. Whisper-family models come in sizes: the small one downloads in half a minute, the large one runs to hundreds of megabytes. Small is plenty for voice input.
- Images — what's in the photo, object detection, background removal. Medium weight, brisk on a GPU, tolerable on a CPU.
- A chat language model. The heaviest option: even a small language model at 4 bits means hundreds of megabytes of weights. Doable, but only with an explicit download warning and only with WebGPU.
General rule: lower the dtype before you downgrade the model. Going from "fp32" to "q8" shrinks the file several times over, and on these tasks you usually can't tell the difference in the answers.
Is it really free?
Yes — the user's hardware does the work, so there's no token bill. You pay in bandwidth for the first download, and in speed on weak machines. Which is why browsers get small models and heavy jobs stay on the server.
How much slower is this than the cloud?
On a GPU a small model answers almost instantly. The gap shows up with large models: local versus cloud is its own discussion. Rough guide: embeddings, classification and speech recognition already belong in the browser; a full chat assistant comes with caveats.
How is this different from running a model with Ollama?
Running locally on your own machine is for you: you install software and download weights yourself. The browser version is for your user: they open a link, install nothing, and everything runs on their side. Different audiences, different cost of entry.
The AI Architect 2027 course: step by step from computer basics to your own app and first customers. 543 short lessons on your phone.





