github.com/pytorch/executorch @v1.3.1

ExecuTorch is a PyTorch platform that provides infrastructure to run PyTorch programs everywhere from AR/VR wearables to standard on-device iOS and Android mobile deployments. One of the main goals for ExecuTorch is to enable wider customization and deployment capabilities of the PyTorch programs.

The executorch pip package is in beta. * Supported python versions: 3.10, 3.11, 3.12, 3.13 * Compatible systems: Linux x86_64, Linux aarch64, macOS aarch64

The prebuilt executorch.runtime module included in this package provides a way to run ExecuTorch .pte files, with some restrictions: * Only core ATen operators are linked into the prebuilt module * Only the XNNPACK backend delegate is linked into the prebuilt module. * [macOS only] Core ML and MPS backend are also linked into the prebuilt module. * [Linux x86_64] QNN backend is linked into the prebuilt module. * [Linux] OpenVINO backend is also linked into the prebuilt module. OpenVINO requires the runtime to be installed separately: pip install executorch[openvino]

Please visit the ExecuTorch website for tutorials and documentation. Here are some starting points: * Getting Started * Set up the ExecuTorch environment and run PyTorch models locally. * Working with local LLMs * Learn how to use ExecuTorch to export and accelerate a large-language model from scratch. * Exporting to ExecuTorch * Learn the fundamentals of exporting a PyTorch nn.Module to ExecuTorch, and optimizing its performance using quantization and hardware delegation. * Running etLLM on iOS and Android devices. * Build and run LLaMA in a demo mobile app, and learn how to integrate models with your own apps.