Google released LiteRT.js,：https://developers.google.com/edge/litert/web a JavaScript binding of LiteRT. LiteRT is Google’s on-device inference library, previously called TensorFlow Lite.
It is not a new model format. Rather, Google compiled its existing native runtime to WebAssembly and exposed it to JavaScript.
Earlier web AI solutions, including TensorFlow.js, relied on JavaScript-based kernels. Google describes those as less performant. LiteRT.js instead ships the native cross-platform runtime with its optimizations intact.
Consequently, web apps inherit work done elsewhere. Performance upgrades, quantization improvements, and hardware optimizations built for Android, iOS, and desktop arrive on the web too.
Under that runtime, LiteRT.js targets three backends:
Two related rules govern dispatch. First, LiteRT.js does not support partial delegation. A graph cannot split across CPU and GPU.
Second, delegation is all-or-nothing per model. If a model cannot be fully delegated to the chosen accelerator, LiteRT falls back to wasm execution. The CPU path has the widest operator coverage.
Given those backends, Google team reports two distinct results.
Against other web runtimes, LiteRT.js is up to 3x faster across CPU and GPU inference. That figure covers classical computer vision and audio processing models.
Against its own CPU execution, GPU or NPU delivers a 5–60x speedup . That applies to demanding real-time work like object tracking and audio transcription.
Both benchmarks ran in a controlled browser environment on a 2024 MacBook Pro with M4 Apple Silicon. Google notes results vary with local GPU, thermal throttling, and driver optimization. A “10x” figure circulating alongside the launch does not appear in the announcement.
LiteRT Torch converts PyTorch models to .tflite in a single step.
However, the prerequisites are strict. Your model must be exportable with torch.export.export , meaning TorchDynamo-exportable. It cannot contain Python conditional branches that depend on runtime tensor values. It also cannot have dynamic input or output dimensions, including the batch dimension.
For size, AI Edge Quantizer configures quantization schemes across different model layers. Pretrained .tflite models are also available on Kaggle and the LiteRT Hugging Face Community.
Once converted, the runtime code is short. This is the WebGPU path, verified against @litertjs/core v2.5.2:
That last block deserves attention. LiteRT.js does not garbage-collect tensors. Every Tensor must be deleted explicitly, or the app leaks device memory. The snippet in Google’s announcement post omits this step.
WebNN needs one extra flag. LiteRT.js requires JSPI, which bridges synchronous kernel scheduling with asynchronous device polling:
Before writing pre-processing, test with fake inputs. Run npm i @litertjs/model-tester , then npx model-tester . It runs your model on WebNN, WebGPU, and CPU using random inputs. Use model.getInputDetails() to read input names and shapes.
Those APIs back four demos Google shipped at launch:
Those demos raise an obvious question for existing web ML teams.
Importantly, the two are not mutually exclusive. Google positions LiteRT.js as a replacement for TF.js Graph Models specifically, not the whole library.
TensorFlow.js remains the recommended tool for pre- and post-processing. The @litertjs/tfjs-interop package passes tensors between them via runWithTfjsTensors . Avoid tensor.dataSync , which carries a significant penalty on the WebGPU backend.
The embed below animates the six pipeline stages across each backend.
Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.
Scan to open this article