Important note: This is still in the making but as this was requested, I'm open-sourcing the first version.

A practical introduction to the whys of Effect

This introduction comes from Effect workshops I gave in which the main objective was to explain in few hours the whys of Effect coming from raw JavaScript/TypeScript.

This introduction is not about how to write Effect code but rather focuses on why Effect might be an interesting pick for writing softwares using TypeScript as of today taking into account all the common problems we face as developers. As a rule of thumb, each developer should be aware of the problems a tool is solving before even trying to take a look at the implementation details. Hopefully with that short introduction you will first become aware of the existing problems and then understand how elegantly and efficiently Effect solves them.

Effect is a well-rounded tool solving a lot of well-known software engineering problems. Let's first talk about problems before solutions.

Inspiration

This is highly inspired by both excellent talks from Michael Arnaldi (@mikearnaldi) at the WorkerConf and Mattia Manzati at React Alicante.

N.B: If you're already comfortable with the problems Effect tries to solve and wish to jump straight into the hows of Effect, I suggest you to take a look at the official Effect documentation (still in the works) and the excellent crashcourse from Stefano Pigozzi (@pigoz).

Samples and source code

In the src/ folder you will be able to find some samples used alongside the introduction. There is also a TypeScript version if you prefer to both read the content and have the ready-to-be-run and type-checking samples.

Note: TypeScript files are still in the making so they might be incomplete/outdated as of now.

Outcomes you can expect from the introduction

• Understanding most commons problems we're facing as developers
• Understanding limits we're facing as JavaScript/TypeScript developers
• Basic understanding of Effect
• Basic understanding of an "Effect System"

Before diving into Effect, let's take a step back talking about what problems we commonly face as developers. Effect is a tool in the same way as TypeScript is a tool. Our responsibility is to first understand the problems as it would help us finding the good solutions.

What are the most common challenges we are facing when developing softwares?

• Explicitness

• Testing

• Resilience

• Composability

• Concurrency

• Efficiency & Performance

• Tracing & Logging

We'll show examples using TypeScript, but this is not only related to JavaScript/TypeScript concern. It concerns every ecosystem, language, for instance Effect was initially heavily inspired by ZIO, its Scala counterpart, because most of the problems also apply to Scala.

Hopefully, you'll realise that Effect is just a tool that addresses hard problems that we will always face, regardless the underlying ecosystem/language.

Before diving into these problems and the solutions Effect brings, let me just do a pretty quick prelude that will help you understand right away the approach.

0. Prelude

Let's talk a little bit about the Effect datatype in itself with a bit of background history.

Effect is the core datatype of the ecosystem, but what if I tell you that it could have been called Program instead? The reason for that is that Effect tries to model exactly what a program is, that is something that requires an environment to run, that can fail with an error or succeed with a value.

You can see the original conversation started by @mikearnaldi just there in Effect's Discord

Consequently, Effect is a generic datatype with 3 type parameters: A: represents the value that can be produced by the program E: represents the error that can be produced by the program R: represents the environment required to run the program

Resulting in: Effect<A, E, R>.

import type { Effect } from "effect";

type Program<Environment, Error, Success> = Effect.Effect<
  Success
  Error,
  Environment,
>;

Let's just model a simple command-line interface program (with a very high level of abstraction). We can say that our command line program requires a process to run, that will be granted by the OS. It's represented by the first generic type parameter R. Then, our program can fail with an error of type E (Standard Error) or succeed with a value of type A (Standard Output).

type Stdout = any;
type Stderr = any;
type Process = any;

type CommandLineProgram = Program<Stdout, Stderr, Process>;

Having these 3 generic parameters explicitly defined in the type signature of our program allows us to have a very precise understanding of what our program is doing and what it can produce as a result. In addition to explicitness, Effect provides us a very strong type safety guarantee that the constraints of the generic type parameters will be respected by the implementation of the program.

1. Explicitness

<a href="https://github.com/antoine-coulon/effect-introduction/blob/main/src/01-explicitness.ts" target="_blank">
  <img src="https://skillicons.dev/icons?i=ts" width="25" />
  Go to source file (01-explicitness.ts)
 </a>

The ability of making a program self-describing, allowing to have a clear vision and understanding what outcomes the program can produce without having to run it.

Ideally, what we want is:

• explicit errors
• explicit dependencies
• explicit outcomes

Let's see few examples using TypeScript first, then with Effect

Synchronous computations

function multiplyNumber() {
  const generatedNumber = NumberGeneratorLibrary.generateRandomNumber();
  //    ^ number
  return number * 2;
}

Unfortunately when running the code our program crashes: Error at <anonymous> Without taking a look at the implementation of the generateRandomNumber(), we don't even know that this thing might throw an error. The consequence of that is having runtime defect makes the process just die. Think of that in a wider scope of a program, where this can be very hard to properly handle.

export function generateRandomNumber(): number {
    const randomNumber = Math.random();

    if (randomNumber > 0.9) {
      // RIP
      throw new Error();
    }

    return randomNumber;
}

This behavior can be the root cause of many problems including defensive coding, for instance:

function defensiveMultiplyNumber() {
  try {
    const number = NumberGeneratorLibrary.generateRandomNumber();
    return number * 2;
  } catch {
    // Just in case
  }
}

Or we need to deal with runtime errors the hard way:

function blindlyCatch() {
  try {
    const random = Math.random();

    if (random > 0.9) {
      throw new SomeError();
    }

    if (random > 0.8) {
      throw new SomeOtherError();
    }

    return random;
  } catch (exception: unknown) {
    if (isSomeErrorException(exception)) {
      // do something
    } else if (isSomeOtherErrorException(exception)) {
      // do something else
    }
  }
}

The solution that we just found is not ideal and even if there is only thirty lines of code, compromises must already be done because we simply lack of explicitness.

Asynchronous operations

One way to model an async computation with JavaScript is using a Promise whose results is always delivered asynchronously.

function doSomething(): Promise<number> {
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve(3)
    });
  });
}

doSomething().then(
  // Callback will be executed at some point in time (generally as soon as possible)
  () => {

  }
);

However, Promises are both conceptually limited and lacking a lot of important features to deal with common problems that we face.

Drawbacks of a Promise 😥

• Eagerly executed, hence is impure, referentially-opaque and is running computation (already a value). Consequently, can't be used around for writing functional programs.

You might already know this eager nature of a Promise, but you might not know that it prevents many interesting rules to be applied.

Purity and referential transparency are important concepts in Functional Programming because they allow you to make assumptions about the behavior of your program levaraging mathematical laws (compositions and substitutions of expressions, etc). Moreover, it helps reasoning about the behavior of your program by just looking at the types, which is what we also target with explicitness. By leveraging compilers, in our case TypeScript, we will be constrained to a set of well-behaved types and principles, allowing us to eliminate whole classes of bugs and unexpected behaviors.

A more detailed version of the explanation is available in the 01-explicitness.ts source file

• Implicit memoization of the result (either success or failure).

As we already said, a Promise is eagerly executed. It means that as soon as you create a Promise, the computation is already running and might have already completed with a value. That value produced by the Promise is implicitly memoized meaning that when the Promise is settled, the internal state of the Promise is frozen and can't be changed anymore, whether the Promise is fulfilled or rejected. Consequently if you want to run the same computation again, you'll need to recreate the Promise from scratch. Altough this is convenient because it allows subscribers to receive the value even when registering for it after the Promise produced its value, this makes the behavior of a Promise non-reusable and does not favor retries and compositions.

• Has only one generic parameter: Promise<A>. The error is non-generic/non-polymorphic.

Promise has only one generic parameter, which is the type of the value produced. This is not really convenient because it means that the error is not reflected by default in the type of the Promise. This highly restricts the type-level expressiveness and forces us to deal with untyped and unknown failures. We could say that only generic parameter can be used to represent the error using Either/Result representations, but this model has its own limitations when it comes to combining many operations together and when trying to inferthe type of the errors of the whole chain.

• Can't depend on any contextual information.

A Promise can't explicitely encode the fact of depending on some contextual information. It means that if you want to run a Promise that depends on some input context, dependencies can not be explicitely modeled hence it is impossible to statically constrain the Promise to only be run in a valid context i.e. with all the requirements satisfied.

This is a problem because this means that Promises can implicitely rely on hidden dependencies and does not offer any flexibility when it comes to composition and dependency injection. By nesting Promises, that implicit layer of dependencies will grow and it will be harder to reason about the behavior and the requirements of the program.

A more detailed version of the explanation is available in the 01-explicitness.ts source file

• No control over concurrency.

Natively, a Promise does not offer any control execution over concurrency so when composing many Promises together, you can't control how many Promises can be spawned and run in parallel (unbounded concurrency). This is a problem because in most cases you will end up either spawning too many Promises and overloading the system or constrain Promises to run sequentially and not taking advantage of the asynchronous nature of the platform.

This is talked in more details in the Concurrency section of the introduction.

• Not much built-in combinators (then, catch, finally) and static methods (all, allSettled, race, any, resolve, reject).

By default, Promises don't have much combinators to work with nor Promise constructors and are lacking some important features, for instance all and allSettled are unbounded concurrency-wise, race and any are working as expected but are unsafe because "race losers" are not cleanly interrupted hence underlying resources can not be released (it's also the case for Promise.all).

• No builtin interruption model.

Following what was said just before, Promises unfortunately don't have a built-in interruption model. There was one attempt to introduce cancellation to Promises that was withdrawn for some unclear reasons. My 2 cents is that it was because adding cancellation into Promises would have introduced too many changes, and the initial design constrained the evolution of the builtin features around Promises.

In any case as of now, we are not able to cancel a Promise using the standard API.

• No builtin retry logic.

Another feature which won't never see the light of day is the built-in retry policies. Given that a Promise already r