ExpressWebJS

# Multithreading

# Introduction

In today's world, the performance and complexity of systems and applications are increasing due to the use of GPUs and multi-core CPUs. To maximize performance and responsiveness, applications often use threads, which are independent units of execution within a process.

ExpressWebJs Thread provides a simple way to create a pool of workers to manage dedicated workers and perform offloaded computations using a thread pool pattern. In this pattern, new tasks are added to a queue and a worker executes them one at a time, picking up a new task from the queue once it has completed the previous one.

💡 In Node.js everything runs in parallel, except your code. What this means is that all I/O code that you write in Node.js is non-blocking, while all non-I/O code that you write in Node.js is blocking.

This means that CPU heavy tasks will block other tasks from being executed. The server will not respond to any new request while executing a single, heavy request. CPU intensive tasks should be offloaded from the main event loop into dedicated workers.

# Multithreading Setup

To setup multithreading in ExpressWebJs, there are some configurations we need to make in the APP.start method in app.ts file in the root directory. We need to set the minimun workers:

App.start({
  minWorkers: 3,
});

This will create a minimun of three (3) workers. There are other configuration options avaliable which we will discuss later in this document.

# Configuration options

The following options are available:

Name Value Function
minWorkers number | 'max' The minimum number of workers that must be initialized and kept available. Setting this to 'max' will create maxWorkers
maxWorkers number The default number of maxWorkers is the number of CPU's minus one. When the number of CPU's could not be determined , maxWorkers is set to 3.
maxQueueSize number The maximum number of tasks allowed to be queued. Can be used to prevent running out of memory. If the maximum is exceeded, adding a new task will throw an error. The default value is Infinity.
workerType 'auto' | 'web' | 'process' | 'thread' 1. In case of 'auto' (default), thread will automatically pick a suitable type of worker: worker_threads will be used if available (Node.js >= 11.7.0), else child_process will be used.
2. In case of 'process', child_process will be used.
3. In case of 'thread', worker_threads will be used. If worker_threads are not available, an error is thrown.
forkArgs String[] For process worker type. An array passed as args to child_process.fork
forkOpts Object For process worker type. An object passed as options to child_process.fork. See nodejs documentation for available options.
workerThreadOpts Object For worker worker type. An object passed to worker_threads.options. See nodejs documentation for available options.

# Enabling Multithreading

In Config/app.ts file, a new configuration option has been added called enable_multithreading. By default, it is switched to false.

  /*
    |--------------------------------------------------------------------------
    | MultiThreading
    |--------------------------------------------------------------------------
    |
    | ExpressWebJs MultiThread offers an easy way to dynamically offload computations
    | to a worker as well as manage a pool of dedicated workers. It implements the
    | thread pool pattern, it executes one task at a time, and once finished,
    | picks a new task from the queue.
    |
    */

  enable_multithreading: env("ENABLE_MULTITHREADING", false), 👈 enable_multithreading set to false

You will switch it to true only if you've configured the number of workers to run with.

# Running Threads

Lets run a simple example by offloading a heavy task to thread. We will run two examples to really understand the use case.

First, we will run a heavy function without thread and later run it with thread.

# Without thread

export function heavyFuntion() {
  let total = 0;
  for (let index = 0; index < 900000; index++) {
    total++;
  }
  return total;
}

Let's create two routes in our api route file.

import { heavyFuntion } from "./heavyFuntion";

Route.get("/heavy-funtion", async (req: Request, res: Response) => {
  return res.send({ message: "Process heavy function", total: heavyFuntion() });
});

Route.get("/normal-function", async (req: Request, res: Response) => {
  return res.send("Welcome to our platform");
});

If you call these two endpoints, you will notice that the heavy-function will execute and block the pool which stops other requests from executing.

# With thread

Let's execute the heavyFunction with a thread and see the difference.

import { heavyFuntion } from "./heavyFuntion";
import { Thread } from "Elucidate/Thread";

Route.get("/heavy-funtion", async (req: Request, res: Response) => {
  let total = await Thread.execute(heavyFuntion, []);
  return res.send({ message: "Process heavy function", total });
});

Route.get("/normal-function", async (req: Request, res: Response) => {
  return res.send("Welcome to our platform");
});

By pushing the heavy function to a thread, our application can now go on to handle other requests. Our created thread will return result to the main thread when its done executing.

# Thread Api

The API of thread consists of :

  • Thread.execute(method: Function | string, params: Array | null [, options: Object]) : Promise.<*, Error>

    Execute a function on a worker with given arguments.

    • When method is a string, a method with this name must exist at the worker and must be registered to make it accessible via the pool. The function will be executed on the worker with given parameters.
    • When method is a function, the provided function fn will be stringified, send to the worker, and executed there with the provided parameters. The provided function must be static, it must not depend on variables in a surrounding scope.
    • The following options are available:
      • on: (payload: any) => void. An event listener, to handle events sent by the worker for this execution.

  • Thread.proxy() : Promise.<Object, Error>

    • Create a proxy for the worker pool. The proxy contains a proxy for all methods available on the worker. All methods return promises resolving the methods result.

  • Thread.statistics() : Object Retrieve statistics on workers,active and pending tasks. Returns an object containing the following properties:

     {
  totalWorkers: 0,
  busyWorkers: 0,
  idleWorkers: 0,
  pendingTasks: 0,
  activeTasks: 0
}

  • Thread.terminate([force: boolean [, timeout: number]])

    • If parameter force is false (default), workers will finish the tasks they are working on before terminating themselves. Any pending tasks will be rejected with an error 'Pool terminated'. When force is true, all workers are terminated immediately without finishing running tasks. If timeout is provided, worker will be forced to terminate when the timeout expires and the worker has not finished.

← ExpressWebJs Authentication →