TensorRT conversion tools and an RGB-D camera live demo for LingBot-Depth.
LingBot-Depth is not vendored in this repository. Please refer to and cite the upstream project.

cd python
uv venv --python 3.12
uv sync
RealSense support is optional. Install the realsense extra when using Intel RealSense:
cd python
uv sync --extra realsense
Orbbec support is optional. Install the orbbec extra when using Orbbec Gemini 335:
cd python
uv sync --extra orbbec
To install both camera backends:
cd python
uv sync --extra realsense --extra orbbec
If you are installing from requirements.txt instead of uv sync, install pyrealsense2 and/or pyorbbecsdk2 separately.
If you want to use a local LingBot-Depth checkout instead of the GitHub dependency:
cd python
uv pip install -e ../../lingbot-depth
Download the pretrained model from Hugging Face:
cd python
mkdir -p ../output/models
uv run hf download \
robbyant/lingbot-depth-pretrain-vitl-14-v0.5 \
model.pt \
--local-dir ../output/models
The model page is:
https://huggingface.co/robbyant/lingbot-depth-pretrain-vitl-14-v0.5/blob/main/model.pt
Convert the model to a fixed-shape TensorRT engine. The default shape is 640 x 480.
cd python
uv run python ../tools/export_trt.py \
--model ../output/models/model.pt \
--precision fp16 \
--num-tokens 1200 \
--work-dir ../output/trt_nt1200_fp16
The generated TensorRT engine is written to:
output/trt_nt1200_fp16/lingbot_depth_nt1200.engine
To build another fixed input size, pass --width and --height.
You may pass --capture /path/to/capture for smoke validation with real RGB-D data. The capture directory should contain rgb.png and raw_depth.png. It is optional and is not required for conversion.
Requires the realsense extra.
List RealSense and Orbbec devices:
cd python
uv run python scripts/live_demo.py --list-devices
Run RealSense with an OpenCV display window:
cd python
uv run python scripts/live_demo.py \
--model ../output/trt_nt1200_fp16/lingbot_depth_nt1200.engine \
--camera realsense \
--realsense auto \
--show-display
Run with a viser point cloud viewer:
cd python
uv run python scripts/live_demo.py \
--model ../output/trt_nt1200_fp16/lingbot_depth_nt1200.engine \
--camera realsense \
--realsense auto \
--show-viser
Requires the orbbec extra.
Run Orbbec Gemini 335 with an OpenCV display window:
cd python
uv run python scripts/live_demo.py \
--model ../output/trt_nt1200_fp16/lingbot_depth_nt1200.engine \
--camera orbbec \
--orbbec auto \
--show-display
Run with a viser point cloud viewer:
cd python
uv run python scripts/live_demo.py \
--model ../output/trt_nt1200_fp16/lingbot_depth_nt1200.engine \
--camera orbbec \
--orbbec auto \
--show-viser
Open http://localhost:8080 in a browser. In the default --viser-mode both mode, this is a comparison dashboard with two independent viser viewers. Raw and refined point clouds each have their own coordinate axes, and camera movement in either viewer is synchronized to the other. The raw/refined viewer ports are printed at startup; with the default dashboard port they are normally 8081 and 8082. Use --viser-mode raw or --viser-mode refined to start with only one point cloud.
Click a visible point to inspect the matching pixel in both viewers. The Selection field shows x/y/z coordinates from the axis origin and distance in meters. Axis size adjusts the coordinate frame size and is synchronized between the two viewers in both mode.
Run browser-only tracking with raw depth vs TensorRT-filtered depth. The default tracking engine is the smaller nt100 engine:
output/trt_nt100_fp16/lingbot_depth_nt100.engine
Start the viser viewer:
cd python
uv run python scripts/orbbec_tracking_demo.py \
--orbbec auto \
--metrics-csv ../output/orbbec_tracking_metrics.csv
Open the printed URL, normally http://localhost:8090. The viewer shows raw and filtered point clouds, their camera trajectories, an IMU orientation frame, and a 10 cm XYZ grid over a 2 m workspace.
@article{lingbot-depth2026,
title={Masked Depth Modeling for Spatial Perception},
author={Tan, Bin and Sun, Changjiang and Qin, Xiage and Adai, Hanat and Fu, Zelin and Zhou, Tianxiang and Zhang, Han and Xu, Yinghao and Zhu, Xing and Shen, Yujun and Xue, Nan},
journal={arXiv preprint arXiv:2601.17895},
year={2026}
}
@article{oquab2023dinov2,
title={DINOv2: Learning Robust Visual Features without Supervision},
author={Oquab, Maxime and Darcet, Timothée and Moutakanni, Theo and Vo, Huy and Szafraniec, Marc and Khalidov, Vasil and Fernandez, Pierre and Haziza, Daniel and Massa, Francisco and El-Nouby, Alaaeldin and others},
journal={Transactions on Machine Learning Research},
year={2024}
}
$ claude mcp add lingbot-depth-trt \
-- python -m otcore.mcp_server <graph>