TypeScript - Type Safe Plugins

2024-10-14

A recent question in the TypeScript Discord needs more space than is available there. It happens to be a pretty good question for providing a walkthrough for how I create complex systems, so I am roughly reproducing it here to be able to reference it in the future.

The question, stripped down from the back and forth to arrive here:

How can I create a type safe plugin system which requires that plugin dependencies are loaded? I'm looking for an API similar to the following:

const ext1 = ExtensionLoader(
  () => app, // Dependencies
  { name: "Ext1" }, // Extension name
  {
    // Commands contributed by this extension
    commandFromExt1: {
      function: () => {
        return "";
      },
      name: "a",
    },
  }
);

const ext2 = ExtensionLoader(
  () => app.add(ext1),
  { name: "Ext2" },
  {
    commandExt2: {
      // app parameter here should be the same type as returned by
      // the dependency function above
      function: (app) => {
        // This should be type safe, note we don't have to pass `app`
        // to run()
        const result: string = app.commands.commandFromExt1.run();
      },
      name: "b",
    },
    commandWithArgument: {
      // Commands can accept additional arguments
      function: (app, arg: number, arg2: string) => {
        // The loaded extensions should also be available on app
        app.extensions.Ext1.commands.commandFromExt1.run();
      },
      name: "c",
    },
  }
);

In addition to this, when adding plugins to the app, there should be a type error if any dependencies are not met.

app.add(ext1); // ok, ext1 has no dependencies
app.add(ext1).add(ext2); // ok, ext2 depends on ext1, which is loaded
app.add(ext2); // error, ext2 requires that ext1 has been loaded

The asker also provided a playground link with their attempt, but I've decided to build this from the ground up rather than work from it.

The proposed API is fairly complicated so we'll start with a simplified version. (just looking at the provided code, it definitely seems overly complicated, but good questions are usually heavily simplified, so I generally assume the additional complexity is important for reasons outside of the scope of the question)

Let's start with the following version:

const ext1 = {
  name: "Ext1",
  commands: {
    commandFromExt1(app: Application<{}>) {
      return "";
    },
  },
} as const;

const ext2 = {
  name: "Ext2",
  commands: {
    commandFromExt1(app: Application<{ Ext1: typeof ext1 }>) {
      const result: string = app.commands.commandFromExt1(app);
      const ext1Name: "Ext1" = app.extensions.Ext1.name;
    },
  },
} as const;

declare const app: Application<{}>;
app.add(ext1).add(ext2); // ok, dependencies met
app.add(ext2); // compile error

This version is missing quite a few features of the original:

• implicit app parameter passing for properties of app.commands
• inferred parameter types
• factory functions for plugins (they're just objects here)
• additional members of app.commands (command names)
• a working implementation

This is a good thing, as it lets us focus on figuring out a good base from which the additional features can gradually be added on. When developing the types, being able to actually run the code isn't important, so we'll start with interfaces everywhere and replace them with classes at the end.

To start, we'll define the Plugin and Application interfaces:

interface Plugin<App extends Application<any>> {
  name: string;
  commands: Record<string, (app: App, ...args: any) => any>;
}

interface Application<Plugins extends Record<string, Plugin<any>>> {
  extensions: Plugins;
  commands: PluginCommands<Plugins>;
  add<P extends Plugin<Application<Plugins>>>(plugin: P): Application<Plugins & Record<P["name"], P>>;
}

There are a couple things to note about these types:

1. Plugin doesn't try to define specific types, it is just used as a constraint.
2. Application has a single type parameter which describes all dynamic content.

The definition for PluginCommands will change in the future, but for now let's just get the intersection of the commands property of all plugins, which can be done with a mapped type and a union to intersection helper

type U2I<U> = (U extends U ? (_: U) => 1 : 2) extends (_: infer I) => 1 ? I : never;
type PluginCommands<P extends Record<string, Plugin<any>>> = U2I<{ [K in keyof P]: P[K]["commands"] }[keyof P]>;

With this, everything nearly works! In the playground we only have one undesirable compiler error due to the Plugins type parameter being invariant.

The current variance of the Application class means that Application<{ Ext1: typeof ext1 }> is not assignable to Application<{}> (undesirable) and that Application<{}> is not assignable to Application<{ Ext1: typeof ext1 }> (desirable). This is a problem since it means that if we have two plugins without dependencies, attempting to add them both to an application will be a compiler error. (playground with variance issue)

declare const ext0: Plugin<Application<{}>>;
declare const ext1: Plugin<Application<{}>>;
declare const app: Application<{}>;

app.add(ext0); // ok
app.add(ext1); // ok
app.add(ext0).add(ext1); // error

Thankfully, we can fix this by updating the commands property to exclude the app parameter. (playground with fixed variance issue)

type Shift<T extends any[]> = T extends [any, ...infer R] ? R : [];
type PluginCommands<P extends Record<string, Plugin<any>>> = U2I<
  {
    [K in keyof P]: {
      [C in keyof P[K]["commands"]]: (
        ...args: Shift<Parameters<P[K]["commands"][C]>>
      ) => ReturnType<P[K]["commands"][C]>;
    };
  }[keyof P] | {} // extra | {} here to make PluginCommands<{}> be {} instead of never
>;

Next, let's look at getting inferred parameter types working. With lots of commands, it's annoying to have to repeat the type of the app parameter for each command.

In TypeScript 4.9 and later, we can achieve this with the satisfies operator without the need for a function call.

const ext1 = {
  ...
} as const satisfies Plugin<Application<{}>>;

const ext2 = {
  name: "Ext2",
  commands: {
    commandFromExt1(app) {
      const result: string = app.commands.commandFromExt1();
    },
  },
} as const satisfies Plugin<Application<{ Ext1: typeof ext1 }>>;

The duplication of the Ext1 name when defining the constraint for ext2 is unfortunate, but easily fixed with a helper to apply multiple dependent plugins. We can add a default to Plugin to also make plugins which don't require dependencies simpler to define: (playground with satisfies)

interface Plugin<App extends Application<any> = Application<{}>> {
  name: string;
  commands: Record<string, (app: App, ...args: any) => any>;
}

type PluginWithDeps<Deps extends Plugin> = Plugin<Application<U2I<Deps extends Deps ? Record<Deps["name"], Deps> : never>>>

// can be used as...

const ext0 = { ... } as const satisfies Plugin;
const ext2 = {
  name: "Ext2",
  commands: {
    commandFromExt1(app) {
      const result: string = app.commands.commandFromExt1();
      const ext1Name: "Ext1" = app.extensions.Ext1.name;
    },
  },
} as const satisfies PluginWithDeps<typeof ext1>;

// If both ext1 and ext2 were needed, could use PluginWithDeps<typeof ext1 | typeof ext2>

Now that we have a working system with simple types, we can start extending it to support the more complex version in the original question. Let's start with the factory function for plugins.

interface Application<Plugins extends Record<string, Plugin<any>>> {
  extensions: Plugins;
  commands: PluginCommands<Plugins>;
  add<P extends Plugin<Application<Plugins>>>(plugin: (app: Application<Plugins>) => P): Application<Plugins & Record<P["name"], P>>;
}

With this change, it probably makes sense to update PluginWithDeps to operate on factory functions too. (playground with factory)

type PluginWithDeps<Deps extends (_: any) => Plugin> = Plugin<Application<U2I<Deps extends Deps ? Record<ReturnType<Deps>["name"], ReturnType<Deps>> : never>>>

Personally, this is as far as I would take this system. Without a good reason for it, I might even stop earlier, before adding the factory function for plugins... but for completeness, let's extend this to meet the original API requested.

Let's start with the commands, they aren't just functions in the original API, but also have a name.

interface Command<Name extends string, Fn extends (app: Application<any>, ...args: any) => any> {
  readonly name: Name;
  run(...args: Shift<Parameters<Fn>>): ReturnType<Fn>;
}

This is a different shape than is configured when creating commands, so we need to introduce different types for the configured commands and the runtime commands. The ExtensionLoader is responsible for making the transformation from the configure shape to the runtime shape.

To make this work I just threw code at the wall until TS stopped complaining... Without knowing the why for this additional complexity, it's hard to tell what exactly should be required. (playground with commands)