Run local AI in your browser with the unified AiBrow JavaScript API. The API allows you to make the best use of the devices hardware to run local AI in the browser in the most performant way possible. It's based around the Chrome built-in AI APIs, but adds support for new features such as custom/HuggingFace models, grammar schemas, JSON output, LoRa Adapters, embeddings, and a fallback to a self-hosted or public server for lower powered devices.
| Engine | Targets | Custom models | HuggingFace models | Runs on-device | Grammar output | LoRA Adapters | Embeddings | GPU Required | Performance |
|---|---|---|---|---|---|---|---|---|---|
| Chrome AI | Chrome Desktop | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ✅ | ⭐️⭐️ |
| llama | Desktop Browsers | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ⭐️⭐️⭐️ |
| WebGPU | Desktop & Android Browsers | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ⭐️ |
| Self-hosted Server | Any browser | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ⭐️⭐️⭐️ |
| Public Server | Any browser | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ⭐️⭐️ |
When using AiBrow through the llama extension or WebGPU, it runs small language models completely on your machine, making it fast and keeping your data private. You can complete sentences, improve your writing, rephrase text, or sum up complex information using the latest small LLMs such as Llama 3.2, Phi 3.5, Gemma 2 and Qwen 2.5.
The AiBrow API follows the current proposals for the browser machine learning APIs under development in Google Chrome, namely the
You can find more information about the AIBrow implementation along with docs & API reference following the links below...
Install the dependencies:
npm install @aibrow/webYou can use the languageModel API to have a conversation with the AI, using whichever backend you choose.
import AI from '@aibrow/web'
// WebGPU
const webGpu = await AI.AIBrowWeb.LanguageModel.create()
console.log(await webGpu.prompt('Write a short poem about the weather'))
// Llama.cpp
const ext = await AI.AIBrow.LanguageModel.create()
console.log(await ext.prompt('Write a short poem about the weather'))
// Chrome AI
const browser = await AI.Browser.LanguageModel.create()
console.log(await browser.prompt('Write a short poem about the weather'))
// Server
const server = await AI.Server.LanguageModel.create()
console.log(await server.prompt('Write a short poem about the weather'))
Using the AiBrow extension gives the best on-device performance with the broadest feature-set. It's a browser extension that leverages the powerful node-llama-cpp and llama.cpp and can giv 8000 e great performance on all kinds of desktop computers either leveraging the GPU or CPU. Downloaded models are stored in a common repository meaning models only need to be downloaded once. You can use models provided by AiBrow, or any GGUF model hosted on HuggingFace.
import AI from '@aibrow/web'
if (AI.AIBrow) {
const { ready } = await AI.AIBrow.capabilities()
if (ready) {
const session = await AI.AIBrow.LanguageModel.create()
console.log(await session.prompt('Write a short poem about the weather'))
} else {
// Here are some tips to help users install the AiBrow extension & helper https://docs.aibrow.ai/guides/helping-users-install-aibrow
console.log(`Install the extension from https://aibrow.ai/install?redirect_to=${window.location.href}`)
}
} else {
// Here are some tips to help users install the AiBrow extension & helper https://docs.aibrow.ai/guides/helping-users-install-aibrow
console.log(`Install the extension from https://aibrow.ai/install?redirect_to=${window.location.href}`)
}Additional info
When the extension is installed, it's directly usable on any page via `window.aibrow` or if installed on a browser other than Google Chrome it automatically polyfills `window.ai`.Typescript types
Types for window.aibrow can be added to your project by using the npm install --save-dev @aibrow/dom-types package. Then to expose them, place the following either in your global.d.ts or the entry point to your code
import type AI from "@aibrow/dom-types"
declare global {
interface Window {
readonly aibrow: typeof AI;
}
}Other extensions
Other extensions can make use of the AiBrow extension by using the extension library library using npm install @aibrow/extension
import aibrow from '@aibrow/extension'
const { ready } = await aibrow.capabilities()
if (ready) {
const session = await aibrow.LanguageModel.create()
const stream = await sess.promptStreaming('Write a poem about AI in the browser')
for await (const chunk of stream) {
console.log(chunk)
}
} else {
console.log(`Install the extension from https://aibrow.ai/install?redirect_to=${window.location.href}`)
}WebGPU provides a good middle-ground for performance and feature set, but it comes with some memory usage restrictions and performance overheads. If you only need to use small models or want to provide a fallback for when the extension isn't installed this can provide a great solution. Under the hood it uses transformers.js from HuggingFace. Models are downloaded through an AiBrow frame which means models only need to be downloaded once. You can use models provided by AiBrow, or any ONNX model hosted on HuggingFace.
import AI from '@aibrow/web'
const session = await AI.AIBrowWeb.LanguageModel.create()
console.log(await session.prompt('Write a short poem about the weather'))The Chrome built-in AI is a great option for simple tasks such as summarization, writing etc. It has a smaller feature-set compared to the AiBrow extension and WebGPU and has reasonable on-device performance.
import AI from '@aibrow/web'
if (AI.AIBrow) {
const session = await AI.AIBrow.LanguageModel.create()
console.log(await session.prompt('Write a short poem about the weather'))
} else {
console.log(`Your browser doesn't support browser window.ai`)
}npm run watchornpm startto build the extension- Run
out/native/boot.cjs --installto install the native manifest hooks - Install the extension from
out/extensioninto the browser - Logs from the native binary are available in
out/native/aibrow.log