Fluid

The Fluid Framework is a library for building distributed, real-time collaborative web applications using JavaScript or TypeScript.

Getting started using the Fluid Framework

You may be here because you want to...

• Learn more about the Fluid Framework
• Build a Fluid object

Documentation and guides can be found at https://fluidframework.com/.

Hello World repo can be found at https://github.com/microsoft/FluidHelloWorld.

Core Examples repo can be found at https://github.com/microsoft/FluidExamples.

Have questions? Engage with other Fluid Framework users and developers in the Discussions section of our GitHub repo.

Using Fluid Framework libraries

When taking a dependency on a Fluid Framework library's public APIs, we recommend using a ^ (caret) version range, such as ^1.3.4. While Fluid Framework libraries may use different ranges with interdependencies between other Fluid Framework libraries, library consumers should always prefer ^.

If using any of Fluid Framework's unstable APIs (for example, its beta APIs), we recommend using a more constrained version range, such as ~.

Code structure

The core code for both the Fluid client packages and the reference ordering service is contained within this repo.

The repo structure is somewhat unique because it contains several pnpm workspaces: some for individual packages and some for larger collections which we call "release groups". The workspaces are versioned separately from one another, but internally all packages in a workspaces are versioned together.

These workspaces do not align with package namespaces, and also don't always correspond to a single directory of this repo.

Here's the list of release group workspaces:

• client (previously known as "Fluid Framework Client" or "core") (Rooted in ./. Configured by ./pnpm-workspace.yaml)
  • ./packages (Published in the @fluidframework/ namespace, but some in @fluid-tools and unpublished packages in @fluid-internal/)
  • ./experimental (Published in the @fluid-experimental/ namespace)
  • ./examples (Not published, live in the @fluid-example/ namespace)
  • ./azure. (Published in the @fluidframework/ namespace)
• routerlicious (Reference Fluid Ordering Service) (Rooted in ./server/routerlicious. Configured by ./server/routerlicious/pnpm-workspace.yaml)
  • Packages (Published in the @fluidframework/ namespace)
• gitrest (Rooted in ./server/gitrest. Configured by ./server/gitrest/pnpm-workspace.yaml)
  • Packages (Published in the @fluidframework/ namespace)
• historian (Rooted in ./server/historian. Configured by ./server/historian/pnpm-workspace.yaml)
  • Packages (Published in the @fluidframework/ namespace)
• build-tools (Rooted in ./build-tools. Configured by ./build-tools/pnpm-workspace.yaml)
  • Packages (Published in a mix of @fluidframework/ and @fluid-tools/ namespaces)

Here's a list of other sets of other packages (each package within these groups is versioned independently, forming its own release group):

• "Common" Packages: miscellaneous packages in the ./common directory and published under the @fluidframework/ namespace. Most of these (but not all) have "common" in their package name. Packages which are used by multiple other groups of packages (such as built tools, linter configs and protocol definitions) live here.
• "Tools" Packages: miscellaneous packages in the ./tools directory and published under a variety of namespaces. Logically about the same as "Common", but most of the names include "tools" instead of "common".
• Auxiliary Microservice Packages (supporting Routerlicious)
  • ./server excluding routerlicious, gitrest and historian (Published in the @fluidframework/ namespace)
• ./docs: The code and content for https://fluidframework.com.

Dependencies between packages in various layers of the system are enforced via a build step called layer-check. You can view the full list of packages and layers in PACKAGES.md.

• Note: to update the contents of PACKAGES.md for local package changes, run pnpm layer-check --md ..

Setup and Building

Install the required tools:

• Git.
  • + Git LFS
• Node.js: install the version noted in in the .nvmrc file. See NodeJs Installation for details.

Clone a copy of the repo and change to the repo root directory:

git clone https://github.com/microsoft/FluidFramework.git
cd FluidFramework

Enable NodeJs's corepack:

corepack enable

Run the following to build the client packages:

pnpm install
npm run build

You can use the worker mode to get faster build time as well: npm run build:fast

See also: Contributing

Build in VSCode

To build Fluid Framework within VSCode, open the Fluid Framework repo folder as a work space and use Ctrl-Shift-B to activate the build task. It is the same as running npm run build on the command line.

NodeJs Installation

We recommend using nvm (for Windows or MacOS/Linux) or fnm to install Node.js. This ensures you stay at the correct version while allowing other uses of NodeJS to use the (possibly different) versions they need side-by-side.

Because of a transitive dependency on a native addon module, you'll also need to ensure that you have the prerequisites for node-gyp. Depending on your operating system, you'll have slightly different installation requirements (these are largely copied from node-gyp's documentation):

On Windows

The node installer should ask if you want to install "Tools for Native Modules." If you check the box for this nothing further should be needed. Otherwise, you can follow the steps listed here

On Unix

1. Python v3.7, v3.8, v3.9, or v3.10
2. make
3. A C/C++ toolchain (like GCC)

On MacOS

If you've upgraded your Mac to Catalina or higher, you may need to follow these instructions.

1. Python v3.7, v3.8, v3.9, or v3.10
2. XCode Command Line Tools, which will install make, clang, and clang++
   • You can install these by running xcode-select --install from a command line.

Other Build Requirements

• Building server/Routerlicious
  • Refer to that package's README for additional requirements.
  • Note that these requirements do not affect all workflows (e.g. the one noted above), but will affect workflows that include the packages under server.

On Windows

• Ensure that you have enabled running Powershell scripts by setting your environment's Execution Policy.

Other Build Commands

Building our docs

There are a few different areas in which we generate documentation content as a part our overall build.

1. fluidframework.com
   • We build the contents of our public website from the docs directory under the root of this repo. See its README for more details.
2. Generated README contents
   • We leverage a local tool (markdown-magic) to generate / embed contents in our various package-level READMEs. This is done as a part of a full build, but it can also be executed in isolation by running npm run build:readme from the repo root.
3. API reports
   • We leverage API-Extractor to generate summaries of our package APIs. This is done as a part of a full build, but it can also be executed in isolation by running npm run build:api from the repo root.

Common Workflows and Patterns

This section contains common workflows and patterns to increase inner dev loop efficiency.

Build

From the root of the repository:

• pnpm install to install dependencies. This is necessary for new clones or after pulling changes from the main branch.
• pnpm build:fast to perform an incremental build that matches the CI build process. Incremental builds tend to leave extra files laying around, so running a clean is sometimes needed to cleanup ghost tests.
• pnpm build:compile:esm:packages to perform an incremental build for production ESM code only. This a fast (minimal) build that can be used to check broad-reaching changes without noise from tests that might need adjusted.
• pnpm build:compile:esm to perform an incremental build for ESM production code and tests.
• pnpm [build:fast|build:compile:esm*] <path|name-regex>... to build only a specific part of the repository.

From root or within a package:

• pnpm build to build that package.
• pnpm build:compile for cross-package compilation.

• pnpm format to format the code.

• fluid-build --vscode to output error message to work with default problem matcher in vscode. If fluid-build is not installed, please install it with npm i -g @fluidframework/build-tools.

Multi package setup

• fluid-build -t build <NAME_OF_PACKAGES> to build a multiple space-separated packages along with all their dependencies. If fluid-build is not installed, please install it with npm i -g @fluidframework/build-tools.

Debug

• You can also use the VSCode JS debug terminal, then run the test as normal.

• Sometimes, uncommitted changes can cause build failures. Committing changes might be necessary to resolve such issues.

Troubleshooting

• pnpm clean if random build failures, especially with no changes
• git clean -Xdf to remove extraneous files if debugging becomes slow or hangs.
  • git clean -xdf (lowercase x) can be used (after -Xdf for performance reasons) to clean up additional untracked (not staged) files, if there are any.

Repo maintenance

It's a good idea to periodically run git maintenance run in the folder where you cloned the repository (by default this will just run git's garbage collection) to keep its size down and operations on it as snappy as possible.

Depending on your environment you might want to run git maintenance start instead, which will schedule maintenance to run periodically. - It should work on Windows because it leverages Task Scheduler. - It won't work in WSL unless you configure it to use systemd, which could have other implications for your system, so proceed with caution. - It won't work on Codespaces because it doesn't run cron.

Testing

You can run:

1. all of our tests from the root of the repo using pnpm test
2. a scoped set of tests by running the pnpm test command from the package you're interested in
3. incrementally build and test using pnpm build-and-test <optional name-regex> (see specific package.json scripts). Note that test output will only be shown if there is a test error.

Note: Some of the tests depend on test collateral that lives in a submodule here: https://github.com/microsoft/FluidFrameworkTestData. You may choose to fetch that collateral into your local repository, which is required to run all the tests - otherwise some will be skipped.

First, ensure you have installed Git LFS. Then, from the repo root:

git lfs install
git submodule init
git submodule update

Run the tests

Before running the tests, the project has to be built. Depending on what tests you want to run, execute the following command in the package directory or at the root:

npm run test

• To run a single test within a module, add .only to it or describe. To exclude a test, use .skip.

• You can use ts-mocha to quickly run specific test files without needing to make the whole project compile. For more details on test filtering using CLI arguments, refer to the Mocha documentation.

• Our test setup generally requires building before running the tests.

• Incremental builds may leave extra files, which can result in ghost tests. To avoid this, consider running a clean build with the following command:

pnpm clean <package>

This removes any leftover files from previous builds, providing a clean testing environment.

Include code coverage

npm run test:coverage

Mimic the official CI build

Our CI pipelines run on Linux machines, and the npm scripts all have the ci prefix. To replicate the test st