Segment Anything Model (SAM) in Napari

Segment anything with our Napari integration of Meta AI's new Segment Anything Model (SAM)!

SAM is the new segmentation system from Meta AI capable of one-click segmentation of any object, and now, our plugin neatly integrates this into Napari.

We have already extended SAM's click-based foreground separation to full click-based semantic segmentation and instance segmentation!

At last, our SAM integration supports both 2D and 3D images!

---

Everything mode	Click-based semantic segmentation mode	Click-based instance segmentation mode

---

SAM in Napari demo

https://user-images.githubusercontent.com/3471895/236152620-0de983db-954b-4480-97b9-901ee82f8edd.mp4

---

Installation

The plugin requires python>=3.8, as well as pytorch>=1.7 and torchvision>=0.8. Please follow the instructions here to install both PyTorch and TorchVision dependencies. Installing both PyTorch and TorchVision with CUDA support is strongly recommended.

Install Napari via pip:

pip install napari[all]

You can install napari-sam via pip:

pip install git+https://github.com/facebookresearch/segment-anything.git
pip install napari-sam

To install latest development version :

pip install git+https://github.com/MIC-DKFZ/napari-sam.git

Usage

Start Napari from the console with:

napari

Then navigate to Plugins -> Segment Anything (napari-sam) and drag & drop an image into Napari. At last create, a labels layer that will be used for the SAM predictions, by clicking in the layer list on the third button.

You can then auto-download one of the available SAM models (this can take 1-2 minutes), activate one of the annotations & segmentation modes, and you are ready to go!

Contributing

Contributions are very welcome. Tests can be run with tox, please ensure the coverage at least stays the same before you submit a pull request.

License

Distributed under the terms of the Apache Software License 2.0 license, "napari-sam" is free and open source software

Issues

If you encounter any problems, please file an issue along with a detailed description.

Acknowledgements

napari-sam is developed and maintained by the Applied Computer Vision Lab (ACVL) of Helmholtz Imaging and the Division of Medical Image Computing at the German Cancer Research Center (DKFZ).