Create NestJS health check library

Reading Time: 6 minutes

 89 total views,  1 views today

Introduction

NestJS applications in my company implement health checking for k8s liveness probe. These applications check the following items are up and running: backend, database, redis, bull queue and external servers. Eventually, different backend teams encapsulate similar endpoint and services in a heath check module and import it to the application. This approach is not DRY and I decided to create a heath check library and reuse it in applications to replace the custom health check module.

In this blog post, I show the codes to create a nestjs health check library and use it in a sample application to check the health of backend, Postgres database, redis, bull queues and doc website of nestjs and angular.

let's go

Create health check library

First, create a new project called nestjs-health. Rename src/ to lib/ and update tsconfig.json and package.json respectively.

// Directory tree
lib
├── controllers
│   ├── health.controller.ts
│   └── index.ts
├── health-module-definition.ts
├── health.module.ts
├── index.ts
├── indicators
│   ├── bull-queue.health.ts
│   └── index.ts
├── interfaces
│   ├── health-module-options.interface.ts
│   └── index.ts
├── services
│   ├── health.service.ts
│   └── index.ts
└── types
    ├── index.ts
    └── redis-connection-options.type.ts
// tsconfig.json
{
  "compilerOptions": {
    "module": "commonjs",
    "declaration": true,
    "removeComments": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "target": "es2017",
    "sourceMap": true,
    "outDir": "./dist",
    "rootDir": "./lib",
    "incremental": true,
    "skipLibCheck": true,
    "strictNullChecks": false,
    "noImplicitAny": false,
    "strictBindCallApply": false,
    "forceConsistentCasingInFileNames": false,
    "noFallthroughCasesInSwitch": false,
    "esModuleInterop": true,
    "strict": true,
    "strictPropertyInitialization": false,
    "noImplicitThis": true,
    "noUnusedParameters": true,
    "noUnusedLocals": true,
    "noUncheckedIndexedAccess": true
  },
  "include": ["lib/**/*"],
  "exclude": ["dist", "test", "node_modules"]
}

Update outDir to ./dist and rootDir to ./lib. Include ['lib/**/*'] and excludes ["dist", "test", "node_modules"] from compilation

Add "main": "./dist/index.js" to package.json

Install dependencies

npm i --save-exact --save-dev @nestjs/terminus
npm i --save-exact @nestjs/microservices @nestjs/axios @nestjs/bull bull @nestjs/typeorm ts-node

Then, move some “dependencies” to “peerDependencies”

"peerDependencies": {
    "@nestjs/common": "^9.0.0",
    "@nestjs/core": "^9.0.0",
    "@nestjs/microservices": "^9.1.5",
    "@nestjs/axios": "^0.1.0",
    "@nestjs/bull": "^0.6.1",
    "bull": "^4.10.1",
    "@nestjs/typeorm": "^9.0.1",
    "ts-node": "^10.0.0"
}

Define HealthModuleOptions interface

First, we define HealthModuleOptions interface that enables us to pass configurations from application to the library

// redis-connection-options.type.ts

import { RedisOptions } from '@nestjs/microservices';

export type RedisConnectOptions = RedisOptions['options'];
// health-module-options.interface.ts

import { HealthIndicatorFunction } from '@nestjs/terminus';
import { RedisConnectOptions } from '../types';

export interface HealthModuleOptions {
  app: string;
  backendUrl: string;
  shouldCheckDatabase?: boolean;
  queueNames?: string[];
  redisOptions?: RedisConnectOptions;
  indicatorFunctions?: HealthIndicatorFunction[];
}
  • app – key of the backend application
  • backendUrl: URL of the backend application, for example, http://localhost:3000
  • shouldCheckDatabase: determine whether or not ping the database that uses TypeORM
  • queueNames: Name array of bull queues
  • indicatorFunctions: custom indicator functions designed to check components’ health

Define Health Module Definition

Define a ConfigurableModuleBuilder to export ConfigurationModuleClass and MODULE_OPTIONS_TOKEN.

Invoke setClassMethodName to override the static methods of HealthModule to forRoot and forRootAsync

import { ConfigurableModuleBuilder } from '@nestjs/common';
import { HealthModuleOptions } from './interfaces';

export const { ConfigurableModuleClass, MODULE_OPTIONS_TOKEN } = new ConfigurableModuleBuilder<HealthModuleOptions>()
  .setClassMethodName('forRoot')
  .build();

Create module for the health check library

Create HealthModule that extends ConfigurableModuleClass

import { Module } from '@nestjs/common';
import { TerminusModule } from '@nestjs/terminus';
import { ConfigurableModuleClass } from './health-module-definition';

@Module({
  imports: [TerminusModule],
  controllers: [],
  providers: [],
  exports: [],
})
export class HealthModule extends ConfigurableModuleClass {}

controllers, providers and exports are empty arrays but we will update them after generating services and controller.

Generate custom health indicator

The following code perform health check on arbitrary number of bull queues. A queue is up and can accept jobs when it connects to redis and redis status is ready.

// indicators/bull-queue.health.ts

import { getQueueToken } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { ModuleRef } from '@nestjs/core';
import { HealthIndicator, HealthIndicatorResult, HealthCheckError } from '@nestjs/terminus';
import { Queue } from 'bull';

@Injectable()
export class BullQueueHealthIndicator extends HealthIndicator {
  constructor(private moduleRef: ModuleRef) {
    super();
  }

  async isHealthy(queues: string[]): Promise<HealthIndicatorResult> {
    const promiseResults = await this.checkQueuesReady(queues);
    const errorResults = this.filterErrors(promiseResults);

    if (errorResults.length) {
      throw new HealthCheckError('Bull queue failed', this.getStatus('bull', false, { errors: errorResults }));
    }

    return this.getStatus('bull', true);
  }

  private filterErrors(promiseResults: PromiseSettledResult<boolean>[]): string[] {
    const errorResults: string[] = [];
    for (const promiseResult of promiseResults) {
      if (promiseResult.status === 'rejected') {
        if (promiseResult.reason instanceof Error) {
          errorResults.push(promiseResult.reason.message);
        } else {
          errorResults.push(promiseResult.reason);
        }
      }
    }
    return errorResults;
  }

  private async checkQueuesReady(queues: string[]) {
    const promises = queues.map(async (name) => {
      const queueToken = this.moduleRef.get(getQueueToken(name), { strict: false });
      if ((queueToken as Queue).isReady) {
        const queue = await (queueToken as Queue).isReady();
        const isEveryClientReady = queue.clients.every((client) => client.status === 'ready');
        if (!isEveryClientReady) {
          throw new Error(`${name} - some redis clients are not ready`);
        }
        return true;
      }
      throw new Error(`${name} is not a bull queue`);
    });

    return Promise.allSettled(promises);
  }
}

Create health checking service

In the health service, I inject MODULE_OPTIONS_TOKEN injection token to obtain a reference to HealthModuleOptions. Then, I examine the options to determine the components to perform health check.

// services/health.service.ts

import { MODULE_OPTIONS_TOKEN } from '../health-module-definition';
import { HealthModuleOptions } from '../interfaces';
import { BullQueueHealthIndicator } from '../indicators';

@Injectable()
export class HealthService {
  constructor(
    @Inject(MODULE_OPTIONS_TOKEN) private options: HealthModuleOptions,
    private health: HealthCheckService,
    private http: HttpHealthIndicator,
    private db: TypeOrmHealthIndicator,
    private bull: BullQueueHealthIndicator,
    private microservice: MicroserviceHealthIndicator,
  ) {}

  check(): Promise<HealthCheckResult> {
    const indicatorFunctions: HealthIndicatorFunction[] = [];

    indicatorFunctions.push(() => this.http.pingCheck(this.options.app, this.options.backendUrl));

    if (this.options.shouldCheckDatabase) {
      indicatorFunctions.push(() => this.db.pingCheck('database'));
    }

    if (this.options.redisOptions) {
      indicatorFunctions.push(() =>
        this.microservice.pingCheck<RedisOptions>('redis', {
          transport: Transport.REDIS,
          options: this.options.redisOptions,
          timeout: 5000,
        }),
      );
    }

    if (this.options.queueNames?.length) {
      indicatorFunctions.push(() => this.bull.isHealthy(this.options.queueNames));
    }

    if (this.options.indicatorFunctions?.length) {
      indicatorFunctions.push(...this.options.indicatorFunctions);
    }

    return this.health.check(indicatorFunctions);
  }
}

Create health controller

The controller has one endpoint, /health, that injects HeathService to check the readiness of the components.

import { Controller, Get } from '@nestjs/common';
import { HealthCheck, HealthCheckResult } from '@nestjs/terminus';
import { HealthService } from '../services';

@Controller('health')
export class HealthController {
  constructor(private health: HealthService) {}

  @Get()
  @HealthCheck()
  check(): Promise<HealthCheckResult> {
    return this.health.check();
  }
}

Register controller and services

Now, we register new controller and services in HealthModule

// heath.module.ts

import { Module } from '@nestjs/common';
... other import statements ...
import { HealthService } from './services';
import { HealthController } from './controllers';
import { BullQueueHealthIndicator } from './indicators';

@Module({
  imports: [TerminusModule],
  controllers: [HealthController],
  providers: [HealthService, BullQueueHealthIndicator],
  exports: [HealthService],
})
export class HealthModule extends ConfigurableModuleClass {}

Finally, add index.ts to export controllers, interfaces, services, types and the health module

./index.ts

export * from './interfaces';
export * from './services';
export * from './health.module';
export * from './types';

I finished the implementation of the library and will publish it to npmjs.com. Npm publish is not part of the scope of this article.

Use case of health checking

I would continue to use nestjs-health-terminus project to demonstrate how to health check the following resources:

  • backend app
  • Postgres database that uses TypeORM
  • redis
  • bull queues
  • just for fun, ping docs.nestjs.com and angular.io/docs are alive

Clone or fork the Nestjs project

You can find the sample code of nestjs bull queue in this repo: https://github.com/railsstudent/nestjs-health-terminus

First, copy .env.example to .env to load environment variables of redis and Postgres database

REDIS_PORT=6379
REDIS_HOST=localhost
PORT=3000
NODE_DEV=development
BACKEND_DOMAIN=http://localhost:3000
DATABASE_CONNECT=postgres
DATABASE_HOST=localhost
DATABASE_USER=postgres
DATABASE_PASSWORD=postgres
DATABASE_NAME=testDB
DATABASE_PORT=5432

Second, run docker-compose to create and run redis and Postgres

docker-compose up -d

Install dependencies

Install nestjs-health library

npm install --save-exact nestjs-health

Import health module to AppModule

First, create health.config.ts in src/configs folder

import { ConfigService } from '@nestjs/config';
import { HttpHealthIndicator } from '@nestjs/terminus';
import { HealthModule } from 'nestjs-health';

export const healthConfig = HealthModule.forRootAsync({
  inject: [ConfigService, HttpHealthIndicator],
  useFactory: (configService: ConfigService, http: HttpHealthIndicator) => {
    return {
      app: 'nestjs-health-terminus',
      backendUrl: configService.get<string>('BACKEND_DOMAIN', ''),
      shouldCheckDatabase: true,
      queueNames: ['fibonacci', 'prime'],
      redisOptions: {
        host: configService.get<string>('REDIS_HOST', 'localhost'),
        port: configService.get<number>('REDIS_PORT', 0),
      },
      indicatorFunctions: [
        () => http.pingCheck('nestjs-docs', 'https://docs.nestjs.com'),
        () => http.pingCheck('angular-docs', 'https://angular.io/docs'),
      ],
    };
  },
});

heathConfig variable configures dependent resources that require health checking.

Next, import healthConfig in AppModule

@Module({
  imports: [
    ... other modules...
    healthConfig,
  ]
})
export class AppModule {}

Invoke health check endpoint

Run docker-compose to start redis. Since both fibonacci and prime queues can connect to redis, the health check status should be “up”

http://localhost:3000/health

Kill redis and call the same endpoint to display a different response.

docker stop <container id of redis>

redis and bull display health check error messages.

Kill Postgres and call the same endpoint to display a new response.

docker stop <container id of Postgres>

Final Thoughts

In this post, I show how to author a NestJS library to encapsulate health check services and controller in a dynamic module. Then, I publish the library to npmjs and install it into NestJS application to check the health of backend application, database, redis, bull queues and external servers. Finally, the library provides a defined /health endpoint that displays the status of components.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in NestJS and other technologies.

Resources:

  1. NestJS Heath Library: https://github.com/railsstudent/nestjs-health
  2. Repo: https://github.com/railsstudent/nestjs-health-terminus
  3. npm package: https://www.npmjs.com/package/nestjs-health
  4. Terminus: https://docs.nestjs.com/recipes/terminus
  5. Dynamic Module: https://docs.nestjs.com/fundamentals/dynamic-modules#dynamic-modules

Check health of nestjs bull queue with terminus

Reading Time: 5 minutes

 71 total views

Introduction

It is common for enterprise applications to connect to external resources to process requests and get back responses. Therefore, application should know that the components it depends on are up and running all the times. Otherwise, the application behaves erroneously when serving client requests. To stay aware of the availability of connected resources, we can perform health check on them and wait for the results.

In this blog post, I describe the use case of health check bull queue in nestjs with terminus. First, it requires a custom health indicator to verify that the queue is connected to redis and redis status is ready. Next, I add a /health endpoint that calls the health indicator to return the status.

let's go

Clone or fork the Nestjs project

You can find the sample code of nestjs bull queue in this repo: https://github.com/railsstudent/nestjs-health-terminus

nest generate nestjs-health-terminus

Install dependencies

Install nestjs terminus library to the project

npm install --save @nestjs/terminus

Create health module for health check

nest g mo health

Import HealthModule in AppModule

@Module({
  imports: [
    ... other modules...
    HealthModule,
  ]
})
export class AppModule {}

Add a custom health indicator to health check bull queue

@nestjs/terminus does not provide built-in health indicator for bull queue; therefore, I create a custom one in the health module.

First, Use nestjs CLI to generate a health indicator

nest g s health/health/bullQueue --flat

My convention is to rename the filename suffix from .service.ts to .health.ts.

src/health
├── controllers
│   ├── health.controller.ts
│   └── index.ts
├── health
│   ├── bull-queue.health.ts
│   └── index.ts
├── health.module.ts
└── index.ts

Then, open bull-queue.health.ts and extends BullQueueHealthIndicator from HealthIndicator.

import { getQueueToken } from '@nestjs/bull';
import { Injectable } from '@nestjs/common';
import { ModuleRef } from '@nestjs/core';
import { HealthIndicator, HealthIndicatorResult, HealthCheckError } from '@nestjs/terminus';
import { Queue } from 'bull';

@Injectable()
export class BullQueueHealthIndicator extends HealthIndicator {
  constructor(private moduleRef: ModuleRef) {
    super();
  }
}

Next, import TerminusModule into HealthModule and verify that the providers array has the entry of BullQueueHealthIndicator.

import { Module } from '@nestjs/common';
import { TerminusModule } from '@nestjs/terminus';
import { HealthController } from './controllers';
import { BullQueueHealthIndicator } from './health';

@Module({
  imports: [TerminusModule],
  controllers: [HealthController],
  providers: [BullQueueHealthIndicator],
})
export class HealthModule {}

Add health check logic in custom health indicator

In BullQueueHealthIndicator, I implement isHealthy() method to check the health of fibonacci and prime queues. When the queue connects to redis and redis status is ready, it is considered up. Otherwise, the queue is down and the indicator returns error messages.

async isHealthy(queues: string[]): Promise<HealthIndicatorResult> {
    const promiseResults = await this.checkQueuesReady(queues);
    const errorResults = this.filterErrors(promiseResults);

    if (errorResults.length) {
      throw new HealthCheckError('Bull queue failed', this.getStatus('bull', 
         false, { errors: errorResults }));
    }

    return this.getStatus('bull', true);
 }

checkQueuesReady accepts a list of queue names and examines their readiness.

private async checkQueuesReady(queues: string[]) {
    const promises = queues.map(async (name) => {
      const queueToken = this.moduleRef.get(getQueueToken(name), 
          { strict: false });
      if ((queueToken as Queue).isReady) {
        const queue = await (queueToken as Queue).isReady();
        const isEveryClientReady = queue.clients.every((client) => client.status === 'ready');
        if (!isEveryClientReady) {
          throw new Error(`${name} - some redis clients are not ready`);
        }
        return true;
      }
      throw new Error(`${name} is not a bull queue`);
    });

    return Promise.allSettled(promises);
}

const queueToken = this.moduleRef.get(getQueueToken(name), { strict: false });

The code uses the queue token to obtain the queue instance. When the queue is ready, I get queue.clients that is an array of redis clients. When every redis client has ready status, the queue is ready to accept jobs. On the other hand, when redis store is not set up, the method throws an error message.

private filterErrors(promiseResults: PromiseSettledResult<boolean>[]): string[] {
    const errorResults: string[] = [];
    for (const promiseResult of promiseResults) {
      if (promiseResult.status === 'rejected') {
        if (promiseResult.reason instanceof Error) {
          errorResults.push(promiseResult.reason.message);
        } else {
          errorResults.push(promiseResult.reason);
        }
      }
    }
    return errorResults;
 }

The result of checkQueuesReady is a collection of resolved and rejected promises. filterErrors iterates the promise results, finds those that are rejected and stores the error messages.

When health check returns error messages, isHealthy throws HealthCheckError.

throw new HealthCheckError('Bull queue failed', this.getStatus('bull', false, 
    { errors: errorResults }));

When health check does not find error, isHealthy returns

this.getStatus('bull', true);

Define health controller

nest g co health/controllers/health --flat

import { Controller, Get } from '@nestjs/common';
import { HealthCheck, HealthCheckService } from '@nestjs/terminus';
import { BullQueueHealthIndicator } from '../health/bull-queue.health';

@Controller('health')
export class HealthController {
  constructor(private health: HealthCheckService, private bull: BullQueueHealthIndicator) {}

  @Get()
  @HealthCheck()
  check() {
    return this.health.check([() => this.bull.isHealthy(['fibonacci', 'prime'])]);
  }
}

First, the GET endpoint is annotated with a HealthCheck decorator. Second, the HealthCheckService expects some health indicators to execute. Therefore, I provide BullQueueHealthIndicator to the check method to process.

this.health.check([() => this.bull.isHealthy(['fibonacci', 'prime'])])

checks the health of fibonacci and prime queues and maps the health status to the object with key “bull”.

Invoke health check endpoint

Run docker-compose to start redis. Since both fibonacci and prime queues can connect to redis, the health check status should be “up”

http://localhost:3000/health

Kill docker-compose and call the same endpoint to see a different response.

Both queues are down because redis has not launched yet.

Suppose I also check ‘dummy’ that is not a bull queue, isHealthy outputs bad injection token error message.

return this.health.check([() => this.bull.isHealthy(['fibonacci', 'prime', 'dummy'])]);

Final Thoughts

In this post, I show how to use terminus package to write a custom health indicator to check the health of bull queue. When terminus package does not provide predefined indicators and there is no open source library, we have the ability to write our own indicators to perform the appropriate testing.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in NestJS and other technologies.

Resources:

  1. Repo: https://github.com/railsstudent/nestjs-health-terminus
  2. Queue: https://docs.nestjs.com/techniques/queues
  3. Bull: https://github.com/OptimalBits/bull
  4. Terminus: https://docs.nestjs.com/recipes/terminus

Queue and job processing in nestjs

Reading Time: 5 minutes

 99 total views

Introduction

The responsibility of backend application is to handle client requests and return responses back to the client. However, backend performance can degrade when the request is resource intensive; UI thread waits endlessly and user interface is blocked for further user actions. To avoid the unresponsiveness, we can place jobs in queue and delegate the processor to manage job processing.

In this blog post, I describe how to register bull queues in a Nestjs project and add jobs to queue to run in the backend. When processor consumes the job from the queue and finishes job processing, the queue listener logs message to terminal.

let's go

Create a new Nestjs project

nest generate nestjs-health-terminus

Install dependencies

Install bull queue package for nestjs and other dependencies for configuration and validation.

npm i @nestjs/bull bull
npm i @nestjs/config class-validator class-transformer
npm i --save-dev @types/bull

Add Docker Compose to install redis

version: '3.1'

services:
  redis:
    container_name: redis
    image: redis:6-alpine
    restart: always
    ports:
      - '6379:6379'
    volumes: 
      - cache:/data
    networks:
      - terminus

volumes:
  cache:

networks:
  terminus:
    driver: bridge

When we launch docker compose, it will install and run Redis 6 on port number 6379.

Import Bull module used by queues

//.env
REDIS_PORT=6379
REDIS_HOST=localhost
@Module({
  imports: [
    ConfigModule.forRoot({
      isGlobal: true,
    }),
    BullModule.forRootAsync({
      inject: [ConfigService],
      useFactory: (configService: ConfigService) => ({
        redis: {
          host: configService.get<string>('REDIS_HOST'),
          port: configService.get<number>('REDIS_PORT'),
        },
      }),
    }),
  ]
})
export class AppModule {}

Register bull queues to process jobs

To enable configService to find the values of REDIS_HOST and REDIS_PORT, the project needs to provide a .env with the environment variables.

// .env

REDIS_HOST=localhost
REDIS_PORT=6379

In tutorial, it is tempting to hardcode the values but professional developers should not do it in production application.

First, we create a queue module to demonstrate some examples of queue and process jobs.

import { BullModule } from '@nestjs/bull';
import { Module } from '@nestjs/common';
import { FibonacciService, PrimeService } from './services';
import { QueueController } from './controllers';
import { PrimeProcessor, FibonacciProcessor } from './processors';

@Module({
  imports: [
    BullModule.registerQueue({
      name: 'fibonacci',
    }),
    BullModule.registerQueue({
      name: 'prime',
    }),
  ],
  providers: [FibonacciService, FibonacciProcessor, PrimeProcessor, PrimeService],
  controllers: [QueueController],
})
export class QueueModule {}

The module is consisted of the following

  • Two bull queues, fibonacci and prime, respectively
  • Queue controller that adds jobs to fibonacci and prime queues
  • FibonacciService that calculates the first Nth fibonacci numbers
  • FibonacciProcessor that processes jobs in fibonacci queue
  • PrimeService that solves two problems of prime factors
  • PrimeProcessor that processes “prime-factors” and “distinct-prime-factors” jobs

Add jobs to queue and wait for job processing

I inject queues to add job in the controller in my example but queuing can also occur in services. If the background job depends on the previous results in the service, then we will inject the queue into the service and append jobs to it inside the method.

@Controller('queue')
export class QueueController {
  constructor(
    @InjectQueue('fibonacci') private fibonacciQueue: Queue<{ order: number }>,
  ) {}

  @Post('fib')
  async getFibonacci(@Query('order', ParseIntPipe) order: number): Promise<void>   {
    console.log(`${new Date()} - Job submitted to queue`, order);
    await this.fibonacciQueue.add({ order }, { delay: 1000 });
  }
}

Fibonacci queue has a single job that is to calculate the first Nth fibonacci numbers a

await this.fibonacciQueue.add({ order }, { delay: 1000 });

adds job with data { order } and delays by one second ({ delay: 1000 }).

Add named jobs to queue for job processing

@Controller('queue')
export class QueueController {
  constructor(@InjectQueue('prime') private primeQueue: Queue) {}


  @Post('prime-factors')
  async getPrimeFactors(@Query('input', ParseIntPipe) input: number): Promise<void> {
    console.log(`${new Date()} - Prime factors job submitted to prime queue`, input);
    await this.primeQueue.add('prime-factors', { input }, { delay: 1000 });
  }

  @Post('distinct-prime-factors')
  async getDistinctPrimeFactors(@Body() arrayDto: ArrayProductsDto): Promise<void> {
    console.log(`${new Date()} - Distinct prime factor job submitted to prime queue`, arrayDto.products);
    await this.primeQueue.add('distinct-prime-factors', {
       products: arrayDto.products,
    });
  }
}

On the other hand, prime queue manages two types of jobs. Therefore, we specify the job name when appending job to the queue.

await this.primeQueue.add('prime-factors', { input }, { delay: 1000 });

The code snippet appends ‘prime-factors’ job to queue to find all prime factors of an integer

await this.primeQueue.add('distinct-prime-factors', {
   products: arrayDto.products,
});

The other code snippet appends ‘distinct-prime-factors’ job to queue to find distinct prime factors of an integer.

Next, we define job processors to process the jobs such that they don’t stay in idle status in the queues.

Define job processors for job processing

It is really easy to create job processors in nestJs. Job processor is a class annotated by @Processor() decorator and queue name. Each method has a @process() decorator and an optional job name to consume queue job.

@Processor('fibonacci')
export class FibonacciProcessor {
  constructor(private fibonacciService: FibonacciService) {}

  @Process()
  calculateNFibonacciNumbers({ data }: Job<{ order: number }>): void {
    const fibSequences = this.fibonacciService.fibonacci(data.order);
    console.log(`${new Date()} Calculating ${data.order + 1} fibonacci numbers...`);
    for (let i = 0; i < data.order; i++) {
      console.log(`${new Date()} - Fib(${i}) = ${fibSequences[i]}`);
    }
  }
}

@Processor('fibonacci') listens to fibonacci queue registered in queue module. The method, calcualteNFibonacciNumbers, is responsible for all jobs of fibonacci queue because @Process() decorator does not specify a job name. In my opinion, the method will violate single responsibility principle if it processes all types of jobs of the queue.

Define specialized job method for job processing

If we want a process method to take care of a single type of job, @Process() decorator accepts name parameter that represents job name.

import { Process, Processor } from '@nestjs/bull';
import { Job } from 'bull';
import { PrimeService } from '../services';

@Processor('prime')
export class PrimeProcessor {
  constructor(private primeService: PrimeService) {}

  @Process('prime-factors')
  findPrimeFactors({ data }: Job<{ input: number }>): void {
    const primeFactors = this.primeService.getPrimeFactors(data.input);
    console.log(`${new Date()} - All prime factors`, primeFactors);
  }

  @Process('distinct-prime-factors')
  findDistinctPrimeFactors({ data }: Job<{ products: number[] }>): void {
    const { primeFactors, count } = this.primeService.getDistinctPrimeFactors(data.products);
    console.log(`${new Date()} - Distinct prime factors`, primeFactors, 'count:', count);
  }
}

findPrimeFactors receives 'prime-factors' job and determines all prime factors of an integer. Similarly, findDistinctPrimeFactors receives 'distinct-prime-factors' job and determines distinct prime factors and the number of them. It is a clean approach than if-then-else to execute different methods of primeService to return the expected results.

Listen to queue events during the lifecycle of job processing

Queue events of Bull offer pre-process and post-process hooks for developers to perform custom actions such as logging and caching. When the events are local, they must implement within the processor such as PrimeProcessor.

The code here implements OnQueueActive and OnQueueCompleted to log start and end time of the job.

@Processor('prime')
export class PrimeProcessor {
  constructor(private primeService: PrimeService) {}

 ...

 @OnQueueActive()
 onJobActive(job: Job) {
    console.log(
      `onJobActive - id: ${job.id}, name: ${job.name}, data: `,
      job.data,
      ` starts at ${new Date(job.timestamp)}`,
    );
 }

 @OnQueueCompleted()
 onJobSuccess(job: Job, result: any) {
    console.log(
      `onJobSuccess - id: ${job.id}, name: ${job.name}, data: `,
      job.data,
      ` completes at ${new Date(job.finishedOn)}`,
      'result',
      result,
    );
 }

Execute cURL request

curl --location --request POST 'http://localhost:3000/queue/prime-factors?input=88'

Console log output:

onJobActive - id: 1, name: prime-factors, data:  { input: 88 }  starts at Sat Sep 17 2022 11:20:54 GMT+0800 (Hong Kong Standard Time)
onJobSuccess - id: 1, name: prime-factors, data:  { input: 88 }  completes at Sat Sep 17 2022 11:20:55 GMT+0800 (Hong Kong Standard Time) result undefined

Final Thoughts

In this post, we show a comprehensive bull queue example in Nestjs. When an application has a task that is expected to run a long time, we have the option to place it in the queue to process somewhere. Then, we do not block the main event loop and ensure UI is responsive for further user actions.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in NestJS and other technologies.

Resources:

  1. Queue: https://docs.nestjs.com/techniques/queues
  2. Bull: https://github.com/OptimalBits/bull
  3. Repo: https://github.com/railsstudent/nestjs-health-terminus

Lint files and dependencies in NestJs with unimported

Reading Time: 5 minutes

 105 total views

Introduction

When developers maintain a NestJS project, they add and delete codes to the project that some dependencies and files become redundant. When unused files and dependencies are left around, they become technical debts and reduce the code quality. Fortunately, we can use unimported to lint files and dependencies before git commit to avoid pushing unused files and dependencies to the code base.

In this blog post, I describe how to execute unimported binary to add .unimportedrc.json to a Nestjs project and create a pre-commit hook to lint files and dependencies of the projects during git commit.

let's go

Create a new Nestjs project

nest generate test-app

Add .unimportedrc.json to lint files and dependencies

First, we execute unimported binary to initialize .unimportedrc.json to the project

npx unimported --init

We will find a barebone .unimportedrc.json after the initialization.

{
  "ignorePatterns": [
    "**/node_modules/**",
    "**/*.tests.{js,jsx,ts,tsx}",
    "**/*.test.{js,jsx,ts,tsx}",
    "**/*.spec.{js,jsx,ts,tsx}",
    "**/tests/**",
    "**/__tests__/**",
    "**/*.d.ts"
  ],
  "ignoreUnimported": [],
  "ignoreUnused": [],
  "ignoreUnresolved": []
}

Most of the Nestjs projects that I work on use TypeOrm to generate migration scripts and interface with PostgreSQL; therefore, I want unimported to ignore **/migrations/** folder in ignorePatterns array.

{
  "ignorePatterns": [
    "**/node_modules/**",
    "**/*.tests.{js,jsx,ts,tsx}",
    "**/*.test.{js,jsx,ts,tsx}",
    "**/*.spec.{js,jsx,ts,tsx}",
    "**/tests/**",
    "**/__tests__/**",
    "**/*.d.ts",
    "**/migrations/**"
  ],
  "ignoreUnimported": [],
  "ignoreUnused": [],
  "ignoreUnresolved": []
}

NestJs is nodejs under the hood and unimported also supports a node preset.

npx unimported --show-preset        // show all available presets
npx unimported --show-preset node   // shows the json file of node preset

The following is the output of the configuration:

{
  preset: 'node',
  entry: [ 'src/main.ts' ],
  extensions: [ '.js', '.jsx', '.ts', '.tsx' ],
  flow: false,
  ignorePatterns: [
    '**/node_modules/**',
    '**/*.tests.{js,jsx,ts,tsx}',
    '**/*.test.{js,jsx,ts,tsx}',
    '**/*.spec.{js,jsx,ts,tsx}',
    '**/tests/**',
    '**/__tests__/**',
    '**/*.d.ts'
  ],
  ignoreUnimported: [],
  ignoreUnresolved: [],
  ignoreUnused: []
}

We copy preset, entry and extensions to the config file in the project and the configuration file finally completes.

{
  "preset": "node",
  "entry": [
    "src/main.ts"
  ],
  "extensions": [
    ".js",
    ".jsx",
    ".ts",
    ".tsx"
  ],
  "ignorePatterns": [
    "**/node_modules/**",
    "**/*.tests.{js,jsx,ts,tsx}",
    "**/*.test.{js,jsx,ts,tsx}",
    "**/*.spec.{js,jsx,ts,tsx}",
    "**/tests/**",
    "**/__tests__/**",
    "**/*.d.ts",
    "**/migrations/**"
  ],
  "ignoreUnimported": [],
  "ignoreUnused": [],
  "ignoreUnresolved": []
}

Create pre-commit hook to perform unimported check

First, we install husky devDependencies and add prepare script in package.json.

npm install husky --save-dev --save-exact
npm set-script prepare "husky install"
npm run prepare

Then, we add a pre-commit husky hook

npx husky add .husky/pre-commit "npx --no unimported"
git add .husky/pre-commit
git commit -m "chore: test unimported check"

If we see the follow output in the terminal, pre-commit hook has successfully performed unimported check.

✓ There don't seem to be any unimported files.
[master f33fe57] chore: test unimported check
 1 file changed, 2 insertions(+), 1 deletion(-)

Up to this point, we focus on setting up code quality tools for the project. Now, we are ready to demonstrate the power of unimported by adding unused file(s) and dependencies. In the demo, husky hook aborts whenever we commit the source codes and unimported prints the report card in the terminal. Then, we systematically modify the codes until we satisfy unimported check and commit the changes.

Demo unimported to lint files and dependencies

The demo is to add configuration and .env validation in Nestjs according to the official documentation https://docs.nestjs.com/techniques/configuration.

First, install dependencies

npm i --save-exact @nestjs/config dotenv joi
git add .
git commit -m "chore(deps): install dependencies"
       summary               unimported v1.21.0 (node)
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       entry file          : src/main.ts

       unresolved imports  : 0
       unused dependencies : 3
       unimported files    : 0

─────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
      │ 3 unused dependencies
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1  │ @nestjs/config
   2  │ dotenv
   3  │ joi
─────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       Inspect the results and run npx unimported -u to update ignore lists
husky - pre-commit hook exited with code 1 (error)

We have 3 unused dependencies but @nestjs/config will go away when we import ConfigModule in AppModule.

import { ConfigModule } from '@nestjs/config';

@Module({
  imports: [
    ConfigModule.forRoot({
      isGlobal: true,
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}
git add .
git commit -m "chore(deps): install dependencies"

       summary               unimported v1.21.0 (node)
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       entry file          : src/main.ts

       unresolved imports  : 0
       unused dependencies : 2
       unimported files    : 0

─────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
      │ 2 unused dependencies
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1  │ dotenv
   2  │ joi
─────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

We have resolved 1 unused dependency and 2 remain. Next, we use joi to validate the schema of .env.

Let’s create a new file to define the schema of .env file

touch src/validation-schema.ts

Then, we copy and paste the schema to the TypeScript file

// validation-schema.ts
import * as Joi from 'joi';

export const validationSchema = Joi.object({
  NODE_ENV: Joi.string()
    .valid('development', 'production', 'test', 'provision')
    .default('development'),
  PORT: Joi.number().default(3000),
});

If we commit the codes now, unimported will detect one unused file and 2 unused dependencies.

       summary               unimported v1.21.0 (node)
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       entry file          : src/main.ts

       unresolved imports  : 0
       unused dependencies : 2
       unimported files    : 1

─────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
      │ 2 unused dependencies
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1  │ dotenv
   2  │ joi
─────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

─────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
      │ 1 unimported files
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1  │ src/validation-schema.ts
─────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

It seems umimported report card is worse than before but it is alright. We forgot to import validation-schema.ts in app.module.ts and pass the validationSchema object to ConfigModule.

Update app.module.ts and commit again

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { ConfigModule } from '@nestjs/config';
import { validationSchema } from './validation-schema';

@Module({
  imports: [
    ConfigModule.forRoot({
      isGlobal: true,
      validationSchema,
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

       summary               unimported v1.21.0 (node)
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       entry file          : src/main.ts

       unresolved imports  : 0
       unused dependencies : 1
       unimported files    : 0

─────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
      │ 1 unused dependencies
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1  │ dotenv
─────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Why is dotenv unused? I referred to package.json of @nestjs/config and found out that dotenv is a dependency of the library. I don’t require to install dotenv in the project and should remove it.

We can make it happen by running npm uninstall dotenv. Let’s commit the codes and not keep the suspense further.

git add .
connie@connie-ThinkPad-T450s ~/Documents/ws_react_graphql_nest/test-app $ git commit -m "chore(deps): install dependencies and add configuration"
✓ There don't seem to be any unimported files.
[redo-demo 44cbf8d] chore(deps): install dependencies and add configuration
 5 files changed, 6 insertions(+), 7 deletions(-)
 rename src/{validationSchema.ts => validation-schema.ts} (100%)

We have finally committed the changes and remove dotenv dependency that the project does not require.

Final Thoughts

In this post, we see how easy it is leave unused files and dependencies in the code base. Fortunately, we can apply unimported command in pre-commit hook to lint files and dependencies before code commit. When Nestjs project has unused files and dependences, code commit fails until we resolve unimported errors. This way, we can keep clean code and avoid introducing technical debts that the successors have to deal with.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in NestJS and other technologies.

Resources:

  1. unimported: https://www.npmjs.com/package/unimported
  2. husky: https://www.npmjs.com/package/husky

Create dynamic module made easy in NestJS 9

Reading Time: 4 minutes

 112 total views,  1 views today

Introduction

Our technology team builds a lot of Nestjs libraries to reuse in Nestjs applications. Our libraries provide forRoot() to create dynamic module only. Therefore, we cannot call forRootAsync() to register asynchronous module by importing modules and injecting services.

Before V9, there are many boilerplate codes to get forRootAsync() to work. Internally, forRootAsync invokes createAsyncProvider and createAsyncOptionsProviders private static methods and both of them accept some sort of ModuleAsyncOptions interface. As the result, the team avoids the implementation unless it is absolutely required.

NestJS 9 introduces ConfiguratableModuleBuilder and it simplifies the steps to create dynamic module by eliminating most of the boilerplate codes.

In this blog post, I describe how to create a nestjs library that provides forRoot and forRootAsync with ConfigurableModuleBuilder. Then, I can install it in a new nestjs 9 app to register the module either synchronously or asynchronously.

let's go

Upgrade NestJS to V9

npm install -g @nestjs/cli npm-check-updates
nest update
npm-check-updates "/@nestjs*/" -u

Create a new nestjs library

nest new nestjs-world-cup-lib

The example is a dummy library to return World Cup finals from 1930 to 2018.

Rename src/ to lib/ and update tsconfig.json accordingly

nestjs-world-cup-lib
├── README.md
├── cspell.json
├── lib
│   ├── enums
│   │   ├── countries.enum.ts
│   │   └── index.ts
│   ├── index.ts
│   ├── interfaces
│   │   ├── index.ts
│   │   ├── world-cup-module-options.interface.ts
│   │   └── world-cup-result.interface.ts
│   ├── services
│   │   ├── index.ts
│   │   └── world-cup.service.ts
│   ├── world-cup-module-definition.ts
│   └── world-cup.module.ts
├── nest-cli.json
├── package-lock.json
├── package.json
├── tsconfig.build.json
└── tsconfig.json
// tsconfig.json
{
  "compilerOptions": {
    "module": "commonjs",
    "declaration": true,
    "removeComments": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "target": "es2017",
    "sourceMap": true,
    "outDir": "./dist",
    "rootDir": "./lib",
    "incremental": true,
    "skipLibCheck": true,
    "strictNullChecks": false,
    "noImplicitAny": false,
    "strictBindCallApply": false,
    "forceConsistentCasingInFileNames": false,
    "noFallthroughCasesInSwitch": false,
    "strict": true,
    "strictPropertyInitialization": false,
    "noImplicitThis": true,
    "noUnusedParameters": true,
    "noUnusedLocals": true,
    "noUncheckedIndexedAccess": true
  },
  "include": ["lib/**/*"],
  "exclude": ["dist", "test", "node_modules"]
}

Update outDir to ./dist and rootDir to ./lib. Include ['lib/**/*'] and excludes ["dist", "test", "node_modules"] from compilation

Steps to create dynamic module

  • Create an interface for the options
  • Instantiate ConfigurableModuleBuilder and export ConfigurableModuleClass and MODULE_OPTIONS_TOKEN
  • Generate providers, services, etc that register and export from the new module
  • Add the configurable module that extends ConfigurableModuleClass
import { ConfigurableModuleBuilder } from '@nestjs/common'
import { WorldCupModuleOptions } from './interfaces'

export const { ConfigurableModuleClass, MODULE_OPTIONS_TOKEN } = new ConfigurableModuleBuilder<WorldCupModuleOptions>()
  .setClassMethodName('forRoot')
  .build()

Step 1: Create an interface for the options

The simple interface has 2 options: the year World Cup is held and your favorite world cup team

// world-cup-module-options.interface.ts

export interface WorldCupModuleOptions {
  year: number
  favoriteCountry: string
}

Step 2: Instantiate ConfigurableModuleBuilder

This step is identical for every dynamic module. We instantiate ConfigurableModuleBuilder and export ConfigurableModuleClass and MODULE_OPTIONS_TOKEN

// world-cup-module-definition.ts

import { ConfigurableModuleBuilder } from '@nestjs/common'
import { WorldCupModuleOptions } from './interfaces'

export const { ConfigurableModuleClass, MODULE_OPTIONS_TOKEN } = new ConfigurableModuleBuilder<WorldCupModuleOptions>()
  .setClassMethodName('forRoot')
  .build()

ConfigurableModuleBuilder generates register and registerAsync by default. In my example, I want to use forRoot and forRootAsync instead. Therefore, I override the configuration by setClassMethodName('forRoot').

Step 3: Generate a service and export from module

// world-cup.service.ts

@Injectable()
export class WorldCupService {
  constructor(@Inject(MODULE_OPTIONS_TOKEN) private options: WorldCupModuleOptions) {}

  getYear(): number {
    return this.options.year
  }
}

The service injects MODULE_OPTIONS_TOKEN in the constructor to access options. We can access the values by this.options.year and this.options.favoriteCountry respectively.

Add the configurable module that extends ConfigurableModuleClass

import { Module } from '@nestjs/common'
import { WorldCupService } from './services'
import { ConfigurableModuleClass } from './world-cup-module-definition'

@Module({
  providers: [WorldCupService],
  exports: [WorldCupService],
})
export class WorldCupModule extends ConfigurableModuleClass {}

The new module extends ConfigurableModuleClass but we can register and export services the same way as before.

The implementation of the library is done, we can publish it to npmjs and try it in a new nestjs V9 application

Register dynamic module and test in nestjs V9 application

npm i nestjs-world-cup-lib
import { WorldCupModule } from 'nestjs-world-cup-lib'

@Module({
  imports: [
    /* Register synchronous dynamic module */
    WorldCupModule.forRoot({
       year: 2018,
       favoriteCountry: 'Brazil'
    })    

    /* Register asynchronous dynamic module
    ConfigModule.forRoot({
      isGlobal: true,
      validationSchema: Joi.object({
        YEAR: Joi.number().optional(),
        FAVORITE_COUNTRY: Joi.string().optional(),
      }),
    }),
    WorldCupModule.forRootAsync({
      imports: [ConfigModule],
      inject: [ConfigService],
      useFactory: (service: ConfigService) => ({
        year: service.get<number>('YEAR'),
        favoriteCountry: service.get<string>('FAVORITE_COUNTRY'),
      })
    })
     */
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Test WorldCupService in AppController

import { WorldCupService } from 'nestjs-world-cup-lib'

@Controller()
export class AppController {
  constructor(private worldCupService: WorldCupService) {}

  @Get('world-cup-year')
  getWorldCupYear(): number {
    return this.worldCupService.getYear()
  }
}

Type http://localhost:3000/world-cup-year in the browser to verify the endpoint.

Final Thoughts

In this post, we see one of the new changes of nestjs v9 that is ConfigurableModuleBuilder. ConfigurableModuleBuilder class simplify the process to create dynamic module. When dynamic module can register without dependency, register, forRoot and forFeature methods are suffice. Alternatively, dynamic module that depends on dependency to initialize configurable options should use registerAsync, forRootAsync and forFeatureAsync counterparts.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in NestJS and other technologies.

Resources:

  1. https://docs.nestjs.com/fundamentals/dynamic-modules#configurable-module-builder
  2. nestjs-world-cup-lib repo: https://github.com/railsstudent/nestjs-world-cup-lib
  3. Source code of ConfigurableModuleBuilder – https://github.com/nestjs/nest/blob/master/packages/common/module-utils/configurable-module.builder.ts
  4. How to update nestjs: https://blog.bitsrc.io/how-to-update-nest-js-ecd0e9466165

Generate i18n pdf invoice in nestjs app with nestjs-i18n

Reading Time: 7 minutes

 163 total views,  1 views today

Introduction

This project is a proof of concept (POC) to generate i18n pdf invoice using nestjs-i18n and html2pdf. In this use case, the i18n language comes from database instead of http request. It is because scheduled job generates and sends out invoices on a monthly basis that does not involve HTTP communication. After retrieving the language, it is stored in Continuation-Local Storage (CLS) namespace that is available in CLS context. Services can get the language from the context to perform translation and date localization to generate i18n pdf invoice.

let's go

Create a new nest application

nest new nest-pdfmake-poc

Download google font files that support English and Traditional Chinese

Navigate to https://fonts.google.com/noto/specimen/Noto+Sans+TC and download Noto Sans Traditional Chinese font files

Navigate to https://fonts.google.com/specimen/Roboto and download Roboto font files

Copy the font files to src/fonts folder

src/fonts
├── NotoSans
│   ├── NotoSansTC-Bold.otf
│   ├── NotoSansTC-Light.otf
│   ├── NotoSansTC-Medium.otf
│   └── NotoSansTC-Regular.otf
└── Roboto
    ├── Roboto-Italic.ttf
    ├── Roboto-Medium.ttf
    ├── Roboto-MediumItalic.ttf
    └── Roboto-Regular.ttf

Add i18n support in nest

npm i --save nestjs-i18n

Create an i18n directory under src/ and create en and hk folders under src/i18n/. The folder names, en and hk, are locales of English and Chinese respectively.

 i18n
├── en
│   └── invoice.json
└── hk
    └── invoice.json

Create JSON translation files

English language, src/i18n/en/invoice.json

{
  "invoice": {
    "bill_to": "Bill To",
    "date_of_issue": "Date of issue",
    "payment_method": "Payment Method",
    "credit_card": "Credit Card",
    "description": "Description",
    "unit_price": "Unit Price",
    "quantity": "Quantity",
    "total": "Total",
    "total_amount": "Total Amount",
    "title": "Invoice",
    "page_number": "Page {currentPage} of {pageCount}"
  }
}

Chinese language, src/i18n/hk/invoice.json

{
  "invoice": {
    "bill_to": "記賬到",
    "date_of_issue": "簽發日期",
    "payment_method": "付款方法",
    "credit_card": "信用卡",
    "description": "描述",
    "unit_price": "單價",
    "quantity": "數量",
    "total": "金額",
    "total_amount": "總金額",
    "title": "發票",
    "page_number": "第 {currentPage} 頁,共 {pageCount} 頁"
  }
}

Copy i18n and fonts folders to dist/ in watch mode

Edit nest-cli.json to copy i18n and fonts folders to dist/ in watch mode. I18n json and font files are refreshed in dist/ folder without server restart

{
  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "assets": [
      {
        "include": "fonts/**/*",
        "watchAssets": true
      },
      {
        "include": "i18n/**/*",
        "watchAssets": true
      }
    ]
  }
}

Inject I18nModule to enable i18n support

imports: [
    I18nModule.forRoot({
      fallbackLanguage: 'en',
      parser: I18nJsonParser,
      parserOptions: {
        path: path.join(__dirname, '/i18n/'),
        watch: true,
      },
    }),
],

fallbackLanguage property indicates that the fallback language is English.

parserOptions: {
    path: path.join(__dirname, '/i18n/'),
    watch: true,
}

watch: true flag enables live translation and monitors i18n folder for changes in translations.

Add dependencies to the project

Install dependencies to generate i18n pdf invoice

npm i html-to-pdfmake pdfmake jsdom date-fns
npm i --save-dev @types/html-to-pdfmake @types/jsdom @types/pdfmake 

html-to-pdfmake converts Html to pdfmake definition to render Pdf documents. pdfmake library call generate pdf documents on the server or client side while date-fns formats i18n date depending on locale.

Install dependencies of CLS hooked and other libraries

npm i cls-hooked cross-env class-transformer class-validator
npm i --save-exact lowdb@1.0.0 
npm i --save-dev @types/cls-hooked @types/lowdb

cls-hooked allows developers to create CLS namespace to store variables in its CLS context. Any method that executes in the scope of the context can access the variables to perform their own logic until it terminates.

lowdb is a local JSON database that stores user profiles to generate pdf invoice based on the user language

Create user database to generate i18n pdf invoice

Create src/db.json and copy the file to dist/ in watch mode

// src/db.json
{
  "users": [
    {
      "id": 1,
      "name": "John Doe Customer",
      "email": "john.doe@email.com",
      "language": "en"
    },
    {
      "id": 2,
      "name": "Jane Doe Customer",
      "email": "jane.doe@email.com",
      "language": "hk"
    }
  ]
}

nest-cli.json

{
  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "assets": [
      ....
      {
        "include": "db.json",
        "watchAssets": true
      }
    ]
  }
}

When user id is 1, the user language is English. When user id is 2, the user language is Chinese.

Use case of applying i18n translation to invoice and date

The use case is to retrieve user language from database and generate a pdf invoice. When language is “en”, we generate an English invoice with “MMM dd, yyyy” date format. Similarly, user receives Chinese invoice when language is “hk” and the invoice date has Chinese characters to represent year, month and day respectively.

English invoice


Chinese invoice

Architecture of the application

The architecture of the application has two modules: core module that is consisted of core services and invoices module that generates pdf invoice

src
├── core
│   ├── core.module.ts
│   ├── helpers
│   │   ├── cls-hook.helper.ts
│   │   ├── font.helper.ts
│   │   ├── i18n-date.helper.ts
│   │   └── translation.helper.ts
│   └── services
│       ├── date-fns.service.ts
│       ├── pdf.service.ts
│       └── translation.service.ts
├── invoice
│   ├── controllers
│   │   └── invoice.controller.ts
│   ├── invoice.module.ts
│   └── services
│       └── invoice.service.ts
└── main.ts

Responsibility of the components

  • PdfService – A service that creates pdf document and streams to the bytes to client
  • DateFnsService – A service that formats date based on locale
  • TranslationService – A service that accesses language from cls context and translates message keys by language
  • ClsHookHelper – A helper function that creates CLS namespace and sets the user language in the CLS context
  • TranslationHelper – A helper that calls TranslationService to translate message keys
  • FontHelper – A helper that determines the google font to use in pdf generation
  • I18nDateHelper – A helper that uses the user language to return the locale and date format
  • InvoiceController – A controller that routes http request to stream i18n Pdf invoice
  • InvoiceService – A service that calls core services to generate pdf invoice with the correct language

Add PDF service to generate i18n pdf invoice

nest g mo core
nest g s core/services/pdf --flat

In order to inject pdfmake library in PdfService, we have to create a custom pdfmake provider in the module.

export const PDF_MAKER_SYMBOL = Symbol('PDF_MAKER')

Format date by locale and format string

nest g s core/services/dateFns --flat

The DateFnsService service has a formatDate method that formats date object/milliseconds by format string and an optional locale

Add CLS hook helper and translation service

First, create a new CLS namespace to set arbitrary language and execute any callback

Second, use nest-cli to generate a TranslationService

nest g s core/services/translation --flat

TranslationService injects I18nService of nestjs-18n to perform i18n translation.

getCurrentLanguage finds the language in CLS context if it exists. Otherwise, the function returns the fallback language that is ‘en’

translate assigns the current language to i18n option, passes it to i18nService to perform translations.

Add other helpers

Since language is available in CLS context, we can define other helpers.

First, we define translation helper functions that return the user language and translate texts.

Then, we add a font helper function to return google font. The font keys of English and Chinese invoices are Roboto and Noto respectively.

Finally, the date helper provides locale and date format by the user language.

After the implementation of core module, we are ready to generate invoice module to generate i18n pdf invoice by user id

Generate invoice module for pdf generation

Similar to core module, we use nest-cli to generate invoice module, controller and service

nest g mo invoice
nest g co invoice/controllers/invoice --flat
nest g s invoice/services/invoice --flat

After scaffolding the boilerplate codes, we add a POST endpoint to generate invoice by user id

Next, we implement generate that does not exist in InvoiceService.

getUser finds user data in the JSON database by user id

export interface User {
  id: number
  name: string
  email: string
  language: string
}

export interface DbSchema {
  users: User[]
}

getInvoiceHtml is the html codes of the dummy invoice

private getInvoiceTranslatedValues() {
    const [
      billTo,
      dateOfIssue,
      paymentMethod,
      creditCard,
      description,
      unitPrice,
      quantity,
      total,
      totalAmount,
      title,
    ] = translates([
      'invoice.invoice.bill_to',
      'invoice.invoice.date_of_issue',
      'invoice.invoice.payment_method',
      'invoice.invoice.credit_card',
      'invoice.invoice.description',
      'invoice.invoice.unit_price',
      'invoice.invoice.quantity',
      'invoice.invoice.total',
      'invoice.invoice.total_amount',
      'invoice.invoice.title',
    ])

    return {
      billTo,
      dateOfIssue,
      paymentMethod,
      creditCard,
      description,
      unitPrice,
      quantity,
      total,
      totalAmount,
      title,
    }
 }

Test i18n pdf invoice generation

Open a terminal and start nest server in development mode

npm run start:dev

Make a HTTP request to preview English invoice

curl --location --request POST 'http://localhost:3000/invoice/1' \
--data-raw ''

Make a HTTP request to preview Chinese invoice

curl --location --request POST 'http://localhost:3000/invoice/2' \
--data-raw ''

The static texts are in Chinese and the date, 2022年4月17日, is shown in Chinese format where

年 means year
月 means month
日 means day 

This’s it. We have done a POC on i18n pdf invoice in Nest and user language controls what we actually see in the invoice. English invoice has English static texts and date format. Chinese invoice has Chinese static texts and date format.

Final thoughts

Another use case of nestjs-i18n is to render pdf invoice according to user preference. The use case retrieves the language from database and saves the value in a CLS namespace such that it is accessible by services and helpers that run in the CLS context. CLS context makes passing language to services and helpers very convenient and we can expand it to store additional variables when required.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in Angular, Nest and other technologies.

Resources:

  1. Github Repository: https://github.com/railsstudent/nest-pdfmake-poc
  2. cls-hooked: https://github.com/jeff-lewis/cls-hooked
  3. nestjs-i18n: https://github.com/ToonvanStrijp/nestjs-i18n
  4. data-fns: https://date-fns.org/v2.28.0/docs/format
  5. lowdb: https://github.com/typicode/lowdb/tree/v1.0.0

Add i18n translation to emails in nestjs app with nestjs-i18n

Reading Time: 6 minutes

 114 total views

Introduction

This project is a proof of concept (POC) to apply i18n translation to emails. Our vendor has a different solution to translate emails at work; therefore, this solution is not picked up. Nonetheless, I want to show my work in this post to demonstrate how to use nestjs-i18n library to translate the content of Mjml template and render the html email. In development mode, we preview emails of different languages on browser. In production mode, we send test emails to Mailtrap to verify the set up of our email service.

let's go

Create a new nest application

nest new nesti18n-mjml-poc

Add i18n support in nest

npm i --save nestjs-i18n

Create an i18n directory under src/ and create en, es and hk folders under src/i18n/. The folder names, en, es and hk, are locales of English, Spanish and Chinese respectively.

 i18n
├── en
│   └── email.json
├── es
│   └── email.json
└── hk
    └── email.json

Create JSON translation files

English language, src/i18n/en/email.json

{
  "welcomeUser": {
    "THANKS_AND_BEST_REGARDS": "Thanks and best regards,",
    "WELCOME": "Welcome to {0.organization}.",
    "MEMBERSHIP_FEE": "Membership fee is {0.total} dollars and starts at {0.startDate}.",
    "TITLE": "Dear {0.name},",
    "ADMIN": "Administrator",
    "SUBJECT": "Registration completed"
  }
}

Spanish language, src/i18n/es/email.json

{
  "welcomeUser": {
    "THANKS_AND_BEST_REGARDS": "Gracias y un saludo,",
    "WELCOME": "Bienvenido a {0.organization}.",
    "MEMBERSHIP_FEE": "La cuota de membresía es {0.total} dolares y comienzas en {0.startDate}.",
    "TITLE": "Quierdo {0.name},",
    "ADMIN": "Administrador",
    "SUBJECT": "Registro completado"
  }
}

Chinese language, src/i18n/hk/email.json

{
  "welcomeUser": {
    "THANKS_AND_BEST_REGARDS": "謝謝,",
    "WELCOME": "歡迎來到{0.organization}。",
    "MEMBERSHIP_FEE": "會員費{0.total}港幣和開始於{0.startDate}。",
    "TITLE": "親愛的{0.name},",
    "ADMIN": "管理員",
    "SUBJECT": "註冊完成"
  }
}

Copy i18n folder and mjml files to dist/ in watch mode

Edit nest-cli.json to copy i18n folder and any mjml file to dist/ in watch mode

{
  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "assets": [
      {
        "include": "**/*.mjml",
        "watchAssets": true
      },
      {
        "include": "i18n/**/*",
        "watchAssets": true
      }
    ]
  }
}

Inject I18nModule to enable i18n support

imports: [
    I18nModule.forRoot({
      fallbackLanguage: 'en',
      parser: I18nJsonParser,
      parserOptions: {
        path: path.join(__dirname, '/i18n/'),
        watch: true,
      },
      resolvers: [new HeaderResolver(['language'])],
    }),
],

fallbackLanguage property indicates that the fallback language is English.

parserOptions: {
    path: path.join(__dirname, '/i18n/'),
    watch: true,
}

watch: true flag enables live translation and monitors i18n folder for changes in translations.

resolvers: [new HeaderResolver(['language'])],

looks up language in HTTP request header. If request header specifies language, the language value is used to translate message and return it back to the client. If request header does not contain language, we use the fallback language for translation.

Install dependencies for apply i18n translation to emails

npm i cross-env lodash mjml nodemailer preview-email
npm i --save-dev @types/lodash @types/mjml @types/nodemailer @types/preview-email
npm i class-validator class-transformer

Use case of apply i18n translation to emails

The use case is to send welcome user email to users according to their language. When language is en, we send English email. Similarly, user receives Spanish email when language is es and Chinese email when language is hk.

English email

Spanish Email

Chinese mail

Architecture of the application

The architecture of the application has two modules: core module that is consisted of core services and user module that sends user emails

Responsibility of the components

  • MjmlService – A service that renders Mjml email template in html format
  • MailserService – A service that previews email on browser in development mode or send email to Mailtrap in production mode
  • AppConfigService – A service that return configuration values of .env file
  • UserController – A controller that routes http request to send welcome user email
  • UserService – A service that calls MjmlService and MailerService to send welcome user email

Add Configurations for the application

First, use nest-cli to generate core module and then AppConfigService service in core/services folder

nest g mo core
nest g s core/services/appConfig --flat

Define configurations of node environment and smtp server in the service

Setup Mjml service to render html email

Next, Use nest cli to generate Mjml Service in the core module

nest g s core/services/mjml --flat

Implement MjmlService

We design the generate template flow as follows:

  1. Load the Mjml template into a string
  2. Use Lodash template to create a compiled template
  3. Compile the template with i18n translations to replace variables in the template
  4. Execute mjml2html to render Mjml template in html format

Create Mailer Service to preview i18n translated emails

Then, use nest cli to generate Mailer Service in the core module

nest g s core/services/mailer --flat

Implement MailerService

In the constructor, we read the from address from the configuration service and create nodemailer smtp transporter.

Define send mail logic

When the application runs in development mode, we preview the email on browser. Otherwise, nodemailer sends the emails with a mail server.

The reason that I choose Mailtrap is the functionality comes out of the box. If I do not use it, I will either create a Docker container with Mailtrap image or configure SMTP server on cloud.

Configure SMTP environment variables

In order to use nodeMailer SMTP transporter to send mail, we have to find a free and safe SMTP server. In this blog post, we choose Mailtrap because it does not require DevOp or networking background to configure a SMTP server.

Finally, we use Mailtrap sandbox to test send email in production mode after development work is done.

First, we navigate to Mailtrap -> Inboxes -> SMTP Settings and select Nodemailer Integrations. Then, we copy SMTP host, SMTP port, SMTP user and SMTP password to .env file

NODE_ENV=development
PORT=3000
MAILER_FROM=no-reply@example.com
SMTP_HOST=smtp.mailtrap.io
SMTP_PORT=2525
SMTP_USER=<mailtrap username>
SMTP_PASSWORD=<mailtrap password>

Implement i18n translation to email in development mode

Step 1 is create a DTO to send welcome user email

import { IsEmail, IsNotEmpty, IsString } from 'class-validator'

export class WelcomeUserDto {
  @IsNotEmpty()
  @IsString()
  organization: string

  @IsNotEmpty()
  @IsString()
  name: string

  @IsNotEmpty()
  @IsEmail()
  email: string
}

Step 2 is create endpoint in user controller to send welcome user email

Inject i18n, Mjml and Mail services in the constructor

Create a helper function that accepts i18n language and arguments, and translates the text

Step 3 is add send welcome user email method in the service

Step 4 is test the endpoint with CURL

Open a terminal and start nest server in development mode

npm run start:dev

Make a HTTP request to preview Spanish email

curl --location --request POST 'http://localhost:3000/user/welcome-useurse ' \
--header 'language: es' \
--header 'Content-Type: application/json' \
--data-raw '{
    "organization": "House of Nest",
    "name": "John Doe",
    "email": "john.doe@example.com"
}'

Repeat i18n translation to emails in production mode

npm run start:production

Since we have set up a running Mailtrap SMTP server, we can mimic the scenario that users receive their mail in inbox from mail provider such as AWS or Google.

As a result, we can call the endpoint to send English, Spanish and Chinese emails to the inbox.

The result is the same as development mode except the destination is mailbox and not the browser.

Up to this point, we have a working solution that is capable of rendering i18n emails in three languages. However, the application can make 2 minor improvements in language extraction and docker.

Improvements

  1. Add a middleware to store language in continuous local storage and retrieve the value in user service.
  2. Create docker-compose.yml to install docker image of mailtrap and test send email locally

Final thoughts

Another use case of nestjs-i18n is to translate email according to the language of http request. We translate the contents by message keys and pass the arguments into Mjml template to replace variables by i18n values. Then, we can preview the emails on browser and send to inbox of email client.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in Angular, Nest and other technologies.

Resources:

  1. Github Repository: https://github.com/railsstudent/nesti18n-mjml-poc
  2. Mjml: https://documentation.mjml.io/#inside-node-js
  3. nestjs-i18n: https://github.com/ToonvanStrijp/nestjs-i18n
  4. Mailtrap: https://mailtrap.io/

Add i18n support in nest app with nestjs-i18n

Reading Time: 7 minutes

 151 total views,  1 views today

Introduction

Many enterprise applications support multiple languages nowadays to cater to customers whose first language is not English. In nest application, we would like to add i18n support to return i18n messages to client-side applications such that users can respond with appropriate actions. Nest framework does not have i18n out of the box but we can install nestjs-i18n to handle translations and return the i18n texts to the client side.

In this post, we are going to create a new nestjs application and install nestjs-i18n to add i18n support. When request is successful, we perform translation by language and return the i18n string in the response. When request throws exception, we implement a global filter that extracts message key from the error object, translates the message and returns a new error object back to the client-side.

let's go

Create a new nest application

nest new nesti18n-poc

Add i18n support in nest

npm i --save nestjs-i18n

Create an i18n directory under src/ and create en and es folders under src/i18n/. The folder names, en and es, are locales of English and Spanish respectively.

src
├─src/i18n
├── en
│   └── error.json
└── es
    └── error.json

Create JSON translation files

English language, src/i18n/en/error.json

{
    "GOOD_MORNING": "Good Morning",
    "SUBSCRIPTION_EXPIRED": "Contact {0.email} for support",
    "SETUP": {
        "WELCOME": "Welcome {0.username}",
        "BYE": "Bye {0.username}", 
        "TOMATO": { 
            "one": "{0.username} buys {0.count} tomato",
            "other": "{0.username} buys {0.count} tomatoes",
            "zero": "{0.username} buys {0.count} tomato"
        }
    },
    "EXCEPTION": "I don't know {0.language}"
}

Spanish language, src/i18n/es/error.json

{
    "GOOD_MORNING": "Buenos Dias",
    "SUBSCRIPTION_EXPIRED": "Contacto {0.email} para soporte",
    "SETUP": {
        "WELCOME": "Bienvenidos {0.username}",
        "BYE": "Adios {0.username}",
        "TOMATO": { 
            "one": "{0.username} compra {0.count} tomate",
            "other": "{0.username} compra {0.count} tomates",
            "zero": "{0.username} compra {0.count} tomate"
        }
    },
    "EXCEPTION": "No se {0.language}"
}

Copy i18n folder to dist/ in watch mode

Edit nest-cli.json to copy i18n folder to dist/ in watch mode

{
  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "assets": [
      {
        "include": "i18n/**/*",
        "watchAssets": true
      }
    ]
  }
}

Inject I18nModule to enable i18n support

imports: [
    I18nModule.forRoot({
      fallbackLanguage: 'en',
      parser: I18nJsonParser,
      parserOptions: {
        path: path.join(__dirname, '/i18n/'),
        watch: true,
      },
      resolvers: [new HeaderResolver(['language'])],
    }),
],

fallbackLanguage property indicates that the fallback language is English.

parserOptions: {
    path: path.join(__dirname, '/i18n/'),
    watch: true,
}

watch: true flag enables live translation and monitors i18n folder for changes in translations.

resolvers: [new HeaderResolver(['language'])],

looks up language in HTTP request header. If request header specifies language, the language value is used to translate message and return it back to the client. If request header does not contain language, we use the fallback language for translation.

Add i18n support in simple cases

We are going to add new endpoints in AppController to show various i18n examples.

Example 1: Display simple i18n message

@I18nLang decorator extracts the language from request header and passes the value to AppService.

First, we inject I18nService in the constructor of AppService. Then, we call this.i18n.translate('error.GOOD_MORNING', { lang }) to obtain the value of 'GOOD_MORNING' key in error.json. The filename of the translation file is error; therefore, the prefix of the key is error. If the filename of the translation file is payment, the message key will change to payment.GOOD_MORNING.

Next, we use CURL to test the endpoint to obtain i18n messages

curl --location --request GET 'http://localhost:3001/good-morning' \
--header 'language: en'

Response displays

Good Morning

To obtain the Spanish message, we modify the header value to es

curl --location --request GET 'http://localhost:3001/good-morning' \
--header 'language: es'

Response displays

Buenos Dias

Example 2: Replace variable in i18n message by JSON object

In translations files, SUBSCRIPTION_EXPIRED is an i18n message that accepts email variable. As the result, we have to replace the variable with an actual email during translation to get back a meaningful text.

Message in i18n/en/error.json

"SUBSCRIPTION_EXPIRED": "Contact {0.email} for support"

Message in i18n/es/error.json

"SUBSCRIPTION_EXPIRED": "Contacto {0.email} para soporte"

To achieve variable replacement, we pass args that is an optional JSON object into the second argument of this.i18n.translate().

Retrieve the English message for error.SUBSCRIPTION_EXPIRED

curl --location --request GET 'http://localhost:3001/translated-message' \
--header 'language: en'

Response should be

Contact abc@example.com for support

Obtain the Spanish counterpart

curl --location --request GET 'http://localhost:3001/translated-message' \
--header 'language: es'

Response should be

Contacto abc@example.com para soporte

Display nested i18n message

Example 3: Display nested i18n message

Displaying nested message is similar to displaying other messages except the message key has multiple parts separated by periods.

"SETUP": {
        "WELCOME": "Bienvenidos {0.username}",
        "BYE": "Adios {0.username}"
},

In this example, we concatenate the message of error.SETUP.WELCOME and error.SETUP.BYE and return the combined text to the client side.

Similarly, when the structure is nested N levels deep, the message key is in the form of error.<level1>.<level2>.....<levelN>.key

The implementation of getNestedTranslationMessage is identical to getTranslatedMessage except the message keys are error.SETUP.WELCOME and error.SETUP.BYE respectively.

Retrieve the English message

curl --location --request GET 'http://localhost:3001/nested-message?username=John Doe' \
--header 'language: en'

Response should be

Welcome John Doe, Bye John Doe

Retrieve the Spanish message

curl --location --request GET 'http://localhost:3001/nested-message?username=John Doe' \
--header 'language: es'

Response should be

Bienvenidos John Doe, Adios John Doe

Support Pluralization in i18n message

Example 4: Pluralization in i18n message

When a message requires to show count, we want to provide the correct plural form. For example, “0 tomato”, “1 tomato” and “2 tomatoes”.

"SETUP": {
    "TOMATO": { 
         "one": "{0.username} buys {0.count} tomato",
         "other": "{0.username} buys {0.count} tomatoes",
         "zero": "{0.username} buys {0.count}| tomato"
     }
}

When count is 0, nestjs-i18n translates the message of error.SETUP.TOMATO.zero. When count is 1, nestjs-i18n translates the message of error.SETUP.TOMATO.one and the rest uses error.SETUP.TOMATO.other. The keywords, “one”, “other” and “zero”, are mandatory in pluralize translation.

Retrieve the English message when the count of tomato is 0

curl --location --request GET 'http://localhost:3001/pluralize-message?username=John&numOfTimes=0' \
--header 'language: en'

Response should be

John buys 0 tomato

Retrieve the English message when the count of tomato is 1

curl --location --request GET 'http://localhost:3001/pluralize-message?username=John&numOfTimes=1' \
--header 'language: en'

Response should be

John buys 1 tomato

Retrieve the English message when count of tomato is 2

curl --location --request GET 'http://localhost:3001/pluralize-message?username=John&numOfTimes=2' \
--header 'language: en'

Response should be

John buys 2 tomatoes

Spanish messages yield similar results when language header switches to es.

Output when count is 0

curl --location --request GET 'http://localhost:3001/pluralize-message?username=John&numOfTimes=0' \
--header 'language: es'
John compra 0 tomate

Output when count is 1

curl --location --request GET 'http://localhost:3001/pluralize-message?username=John&numOfTimes=1' \
--header 'language: es'
John compra 1 tomate

For count that is greater than 1, the output is

curl --location --request GET 'http://localhost:3001/pluralize-message?username=John&numOfTimes=1' \
--header 'language: es'
John compra 2 tomates

We have wrapped up the happy path where HTTP requests execute successfully and return a response with translated message. The next section describes bad HTTP request where endpoint throws an exception and our global filter intercepts the exception to translate the error message.

Translate HttpException message with i18n support

The first step is to define a global filter that catches all HttpException.

We cast errorResponse as { key: string; args?: Record<string, any> } and call this.i18n.translate(error.key, { lang: ctx.getRequest().i18nLang, args: error.args }) to obtain the translated message. Finally, we call this.httpAdapterHost.httpAdapter.reply(ctx.getResponse(), responseBody, statusCode) to return the formatted error response.

The second step is to register the global filter in AppModule

On the third step, we add “EXCEPTION” key and message in translation files

{
    "EXCEPTION": "I don't know {0.language}"
}
{
    "EXCEPTION": "No se {0.language}"
}

Next, we add a new endpoint that always throws BadRequestException. In the constructor of BadRequestException, the argument is an error object consisted of key and args.

Finally, we can verify the translated exception message with CURL

curl --location --request GET 'http://localhost:3001/bad-translated-exception' \
--header 'language: en'

The error response in English becomes

{
    "statusCode": 400,
    "message": "I don't know German",
    "error": "I don't know German"
}
curl --location --request GET 'http://localhost:3001/bad-translated-exception' \
--header 'language: es'

The error response in Spanish becomes

{
    "statusCode": 400,
    "message": "No se German",
    "error": "No se German"
}

Final thoughts

When a Nest application needs to support i18n, we install nestjs-i18n to send i18n messages and exceptions depending on language. The language can be provided in request header, cookie or GraphQL resolver, or a fallback language is used.

nestjs-i18n supports many use cases: plain message, variable replacement in message, nested message and pluralize translation. Moreover, it is a scalable solution since we can split messages by domains and languages to maintain a hierarchy of JSON files.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in Angular, Nest and other technologies.

Resources:

  1. Github Repository: https://github.com/railsstudent/nesti18n-poc
  2. nestjs-i18n: https://github.com/ToonvanStrijp/nestjs-i18n

Swagger In NestJS

Reading Time: 5 minutes

 188 total views

Problem: Frontend developers complained that integration with API is a painful process because there is no documentation on available endpoints, expected payloads and responses. Therefore, our backend team leads elect to enable swagger such that frontend developers can browse all the APIs on dedicated documentation site.

When they showed me how it was done, I was thoroughly impressed and wanted to show NestJS peers the simple steps.

Create a NestJS application

nest new nest-swagger

Install configuration, swagger and other dependencies

npm install --save @nestjs/config joi class-transformer class-validator
npm install --save @nestjs/swagger swagger-ui-express

Store environment variables in environment file

Store Swagger environment variables in .env and we will use configuration service to retrieve the values in main.ts

// main.ts
NODE_ENV=development
PORT=3000
API_VERSION=1.0
SWAGGER_TITLE=Local Swagger Documentation Site
SWAGGER_DESCRIPITON=The task API description

Import Configuration Module and validate schema

// envSchema.ts
import * as Joi from 'joi'

export const validationSchema = Joi.object({
  NODE_ENV: Joi.string().valid('development', 'production', 'test', 'provision').default('development'),
  PORT: Joi.number().default(3000),
  API_VERSION: Joi.string().default('1.0'),
  SWAGGER_TITLE: Joi.string().required(),
  SWAGGER_DESCRIPITON: Joi.string().required(),
})
// app.module.ts
import { Module } from '@nestjs/common'
import { ConfigModule } from '@nestjs/config'
import { AppController } from './app.controller'
import { AppService } from './app.service'
import { validationSchema } from './envSchema'

@Module({
  imports: [
    ConfigModule.forRoot({
      isGlobal: true,
      validationSchema,
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Configure Swagger

Initialize Swagger in main.ts such that API documentation is generated automatically for controllers, entities and data transfer objects (DTOs).

// main.ts
async function bootstrap() {
   ....
   app.useGlobalPipes(
     new ValidationPipe({
       whitelist: true,
     }),
   )

   const configService = app.get(ConfigService)
   const version = configService.get<string>('API_VERSION', '1.0')
   const title = configService.get<string>('SWAGGER_TITLE', '')
   const description = configService.get<string>('SWAGGER_DESCRIPITON', '')

   const config = new DocumentBuilder()
     .setTitle(title)
     .setDescription(description)
     .setVersion(version)
     .addBearerAuth()
     .build()
   const document = SwaggerModule.createDocument(app, config)
   SwaggerModule.setup('api', app, document)

   const port = configService.get<number>('PORT', 0)
   await app.listen(port)
}

Enable Swagger plugin in nest-cli.json

// nest-clis.json
{
  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "plugins": ["@nestjs/swagger"]
  }
}

After Swagger plugin is enabled, Swagger automatically generates documentation for controller and files with either .entity.s or .dto.ts suffix.

Verify Swagger is working by browsing http://localhost:3000/api

The documentation site works because it publishes the GET endpoint of app controller successfully.

Swagger setup is complete and we can add new API and publish the new endpoints in the dedicated documentation site.

Create a new Task Module to generate API documentation

nest g mo Task

Next, I generate task service and task controller and register them in the TaskModule. The following is the declaration of the TaskModule:

import { Module } from '@nestjs/common'
import { TaskController } from './task.controller'
import { TaskService } from './services'

@Module({
  controllers: [TaskController],
  providers: [TaskService],
})
export class TaskModule {}

Define new entities and new DTOs for Task API

// create-task.dto.ts
import { IsNotEmpty, IsString } from 'class-validator'

export class CreateTaskDto {
  @IsNotEmpty()
  @IsString()
  name: string
}

// update-task.dto.ts
import { IsBoolean, IsDefined } from 'class-validator'

export class UpdateTaskDto {
  @IsDefined()
  @IsBoolean()
  completed: boolean
}

Define new entities for the Task API

// task.entity.ts
import { IsBoolean, IsString } from 'class-validator'

export class Task {
  @IsString()
  id: string

  @IsString()
  name: string

  @IsBoolean()
  completed: boolean
}

// delete-result.entity.ts
import { IsNumber, IsOptional, Min, ValidateNested } from 'class-validator'
import { Task } from './task.entity'

export class DeleteResult {
  @IsNumber()
  @Min(0)
  count: number

  @IsOptional()
  @ValidateNested()
  task?: Task
}

Create CRUD methods in Task Service

Create basic methods to add, update, delete and retrieve a task object from a tasks array

// task.service.ts
import { BadRequestException, Injectable } from '@nestjs/common'
import { v4 } from 'uuid'
import { CreateTaskDto, UpdateTaskDto } from '../dtos'
import { DeleteResult, Task } from '../entities'

@Injectable()
export class TaskService {
  tasks: Task[] = [
    ...some tasks objects
  ]

  listAll(): Promise<Task[]> {
    return Promise.resolve(this.tasks)
  }

  findById(id: string): Promise<Task> {
    const task = this.tasks.find((task) => task.id === id)
    if (!task) {
      throw new BadRequestException('Task not found')
    }
    return Promise.resolve(task)
  }

  deleteById(id: string): Promise<DeleteResult> {
    const task = this.tasks.find((task) => task.id === id)
    this.tasks = this.tasks.filter((task) => task.id !== id)
    const deletedResult: DeleteResult = {
      count: task ? 1 : 0,
      task,
    }
    return Promise.resolve(deletedResult)
  }

  updateTask(id: string, dto: UpdateTaskDto): Promise<Task> {
    this.tasks = this.tasks.map((task) => {
      if (task.id !== id) {
        return task
      }
      return {
        ...task,
        completed: dto.completed,
      }
    })

    return this.findById(id)
  }

  createTask(dto: CreateTaskDto): Promise<Task> {
    const newTask: Task = {
      ...dto,
      id: v4(),
      completed: false,
    }
    this.tasks = [...this.tasks, newTask]
    return Promise.resolve(newTask)
  }
}

Create endpoints in Task Controller

task.controller.ts
import { UpdateTaskDto } from './dtos/update-task.dto'
import { Body, Controller, Delete, Get, Param, ParseUUIDPipe, Post, Put } from '@nestjs/common'
import { ApiBadRequestResponse, ApiTags } from '@nestjs/swagger'
import { CreateTaskDto } from './dtos'
import { TaskService } from './services'
import { DeleteResult, Task } from './entities'

@ApiTags('Task')
@Controller('task')
export class TaskController {
  constructor(private service: TaskService) {}

  @Get()
  getAll(): Promise<Task[]> {
    return this.service.listAll()
  }

  @Get(':id')
  @ApiBadRequestResponse({ description: 'Task not found' })
  getTask(@Param('id', ParseUUIDPipe) id: string): Promise<Task> {
    return this.service.findById(id)
  }

  @Delete(':id')
  deleteTask(@Param('id', ParseUUIDPipe) id: string): Promise<DeleteResult> {
    return this.service.deleteById(id)
  }

  @Post()
  createTask(@Body() dto: CreateTaskDto): Promise<Task> {
    return this.service.createTask(dto)
  }

  @Put(':id')
  updateTask(@Param('id', ParseUUIDPipe) id: string, @Body() dto: UpdateTaskDto): Promise<Task> {
    return this.service.updateTask(id, dto)
  }
}

@ApiBadRequestResponse and @ApiBadRequestResponse are Swagger decorators.

DecoratorDescription
ApiTagsProvide name of the API and in this example, the name is Task
ApiBadRequestResponseIndicate the endpoint returns BadRequestException when task is not found

Refresh http://localhost:3000/api and Task API is published.

Task lists all available endpoints (two GETs, POST, DELETE and POST) and Schema section auto-generates its entities (Task and DeleteResult) and DTOs (CreateTaskDto and UpdateTaskDto).

That’s the end of Swagger introduction.

Final thoughts

Swagger provides clear documentation of Restful API that facilitates communication between frontend and backend developers. Rather than constantly asking backend team for sample requests and responses, frontend developers can refer to the Swagger documentation that illustrates all available endpoints and the expected payloads and responses. When teams have no confusion and argument, developers become happy and happy developers are the most productive in application development.

I hope you like this article and share it to people who are interested in NestJS!

Resources:

  1. Repo: https://github.com/railsstudent/nest-swagger
  2. OpenAPI (Swagger): https://docs.nestjs.com/openapi/introduction

Dynamic Task Scheduler In NestJS

Reading Time: 7 minutes

 180 total views,  2 views today

Scenario: Our ESG (Environmental, Social, and Governance) platform requires to run schedule jobs to call endpoints to generate reports for internal teams and send company emails to our customers. Development team was tasked with developing a job scheduler that executes cron jobs at a specified period of time. node-cron is a popular open source library for setting up node-like cron jobs in a typescript program, however, Nest provides Schedule module that does basically the same thing with decorators and flexible API. Since our backend is powered by Nest, we prefer to use existing features of NestJS instead of bringing in a new library and investing time to study a new API.

Problem: We followed Nest documentation and created all cron jobs declaratively. These jobs run on a daily or weekly basis that are inconvenient for QA team to test because they cannot wait for such a long time to verify test results. The approach is to configure jobs to run in different intervals in different environments. For example, a job is configured to run once a day in production environment and run every minute in QA environment to facilitate testing cycle.

To cater to the new QA request, developers converted declarative cron jobs to dynamic counterparts through dynamic schedule module API. However, the codes were no longer DRY and I refactored the codebase to keep future development efforts to the minimum.

Create a NestJS application

nest new nest-dynamic-scheduler

Install configuration, http and schedule dependencies

npm i --save @nestjs/config joi
npm i --save @nestjs/axios
npm install --save @nestjs/schedule
npm install --save-dev @types/cron

Store environment variables in environment file

Store environment variables in .env and we will use configuration service to retrieve the base url in the codes.

Import Configuration Module

Import configuration module and validate the schema of .env file.

Validation part completes. We can move on to add dummy endpoints in AppController and a Task module that calls them in cron jobs.

Create dummy endpoints in App Controller

We define a printMessage function in AppService that will be called by the AppController

AppController has three new endpoints: the first one handles POST method, the second one handles PATCH and the final one handles PUT. It is to demonstrate that Nest cron jobs work with common http request methods.

Create new Task module

nest g mo Task

Next, I generate task service and import HttpModule and ScheduleModule.forRoot() to TaskModule. The following is the declaration of the TaskModule:

import { HttpModule, Module } from '@nestjs/common'
import { ScheduleModule } from '@nestjs/schedule'
import { TaskService } from './task.service'

@Module({
  imports: [HttpModule, ScheduleModule.forRoot()],
  providers: [TaskService],
  controllers: [],
})
export class TaskModule {}

First Attempt: Create declarative cron jobs in TaskService

// task.service.ts

private readonly baseUrl: string

constructor(
    private httpService: HttpService,
    private configService: ConfigService,
  ) {
    this.baseUrl = this.configService.get<string>('BASE_URL', '')
}

@Cron('*/15 * * * * *')
async handlePostJob(): Promise<void> {
    try {
      await this.httpService
        .post(`${this.baseUrl}/post-job`, {
          name: 'connie',
          msg: 'schedule post job every 15 second',
          timestamp: Date.now(),
        })
        .toPromise()
   } catch (err) {
      console.error(err)
   }
}

@Cron('*/20 * * * * *')
async handlePatchJob(): Promise<void> {
  try {
    await this.httpService
       .patch(`${this.baseUrl}/patch-job`, {
          name: 'connie',
          msg: 'schedule patch job every 20 second',
          timestamp: Date.now(),
       })
       .toPromise()
  } catch (err) {
     console.error(err)
  }
}

@Cron('*/30 * * * * *')
async handlePutJob(): Promise<void> {
  try {
    await this.httpService
      .put(`${this.baseUrl}/put-job`, {
         name: 'connie',
         msg: 'schedule put job every 30 second',
         timestamp: Date.now(),
      })
      .toPromise()
  } catch (err) {
    console.error(err)
  }
}

@Cron() decorator tells NestJS when the job starts and terminates and its frequency. The decorator supports all standard cron patterns; therefore, @Cron(‘*/15 * * * * *’) makes the Http POST request every 15 seconds. Similarly, PATCH and PUT requests are triggered every 20 and 30 seconds respectively.

Second Attempt: Convert declarative cron jobs to dynamic cron jobs

If QA team did not make the request, the scheduler was completed and we could go home and rest. The request seemed trivial; we could store the cron patterns as environment variables and applied them in @Cron(), right? Wrong answer, my friends.

// task.service.ts

const defaultInterval = '* * * * * *'    
this.postJobInterval = this.configService.get<string>('POST_JOB_INTERVAL', defaultInterval)

@Cron(this.postJobInterval)
async handlePostJob(): Promise<void> {
    ...
}

I couldn’t use this.postJobInterval inside @Cron() because “this” variable can be null. Fortunately, dynamic scheduler module API exists and we can add cron jobs programmatically with configurable cron patterns.

We define new job intervals in .env (POST_JOB_INTERVAL, PATCH_JOB_INTERVAL, PUT_JOB_INTERVAL)

.env

POST_JOB_INTERVAL='*/15 * * * * *'
PATCH_JOB_INTERVAL='*/20 * * * * *'
PUT_JOB_INTERVAL='*/30 * * * * *'

We comment out the old codes and define addCronJobs function to create dynamic cron jobs and register them in scheduler registry. Finally, we start the jobs such that they are triggered every 15, 20 and 30 seconds. Remember to inject SchedulerRegistry in the constructor.

// task.service.ts

const defaultInterval = '* * * * * *'    

private readonly postJobInterval: string
private readonly patchJobInterval: string
private readonly putJobInterval: string
private readonly baseUrl: string

constructor(
    private httpService: HttpService,
    private schedulerRegistry: SchedulerRegistry,
    private configService: ConfigService,
  ) {
    this.postJobInterval = this.configService.get<string>('POST_JOB_INTERVAL', defaultInterval)
    this.patchJobInterval = this.configService.get<string>('PATCH_JOB_INTERVAL', defaultInterval)
    this.putJobInterval = this.configService.get<string>('PUT_JOB_INTERVAL', defaultInterval)
    this.baseUrl = this.configService.get<string>('BASE_URL', '')
}

addCronJobs(): void {
    const postJob = new CronJob(this.postJobInterval, async () => {
      try {
        await this.httpService
          .post(`${this.baseUrl}/post-job`, {
            name: 'connie',
            msg: 'schedule post job every 15 second',
            timestamp: Date.now(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    })

    const patchJob = new CronJob(this.patchJobInterval, async () => {
      try {
        await this.httpService
          .patch(`${this.baseUrl}/patch-job`, {
            name: 'connie',
            msg: 'schedule patch job every 20 second',
            timestamp: Date.now(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    })

    const putJob = new CronJob(this.putJobInterval, async () => {
      try {
        await this.httpService
          .put(`${this.baseUrl}/put-job`, {
            name: 'connie',
            msg: 'schedule put job every 30 second',
            timestamp: Date.now(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    })

    this.schedulerRegistry.addCronJob('post-job', postJob)
    this.schedulerRegistry.addCronJob('patch-job', patchJob)
    this.schedulerRegistry.addCronJob('put-job', putJob)
    postJob.start()
    patchJob.start()
    putJob.start()
  }

Modify TaskModule to implement OnModuleInit interface and start cron jobs in onModuleInit function.

// task.module.ts

export class TaskModule implements OnModuleInit {  
  constructor(private taskService: TaskService) {}
  async onModuleInit() {    
    await taskService.addCronJobs()    
  }
}

Start the application and observe the messages in the terminal

npm run start:dev

The current implementation meets the requirements but it is not DRY. Why do I say that? It is because I see a common pattern in the three dynamic jobs.

  • Create a callback function that
    • makes a new HTTP request in a try-catch block
    • converts the observable to promise with toPromise()
    • await the result of the HTTP request
  • Create a new cron job with cron pattern and the callback function
  • Add the new cron job to scheduler registry
  • Start the cron job

I can generalize this pattern such that developer can write minimum codes to add new cron job in the future.

Third Attempt: Generalize the dynamic task scheduler

Define a new interface, JobConfiguration, that stores the metadata of the job

// job-configuration.interface.ts
import { Method } from 'axios'

export interface JobConfiguration {
  url: string
  interval: string
  method: Method
  dataFn: () => any
  name: string
}

method is the supported method of Axios and it can be POST, PATCH, PUT or DELETE. dataFn function is used to generate new request data in the callback.

Define metadata of cron jobs in jobConfigurations array.

this.jobConfigurations = [
      {
        url: `${this.baseUrl}/post-job`,
        interval: this.postJobInterval,
        method: 'POST',
        dataFn: () => ({
          name: 'connie',
          msg: 'schedule dynamic post job every 15 second',
          timestamp: Date.now(),
        }),
        name: 'post-job2',
      },
      {
        url: `${this.baseUrl}/patch-job`,
        interval: this.patchJobInterval,
        method: 'PATCH',
        dataFn: () => ({
          name: 'mary',
          msg: 'schedule dynamic patch job every 20 second',
          timestamp: Date.now(),
        }),
        name: 'patch-job2',
      },
      {
        url: `${this.baseUrl}/put-job`,
        interval: this.putJobInterval,
        method: 'PUT',
        dataFn: () => ({
          name: 'job',
          msg: 'schedule dynamic put job every 30 second',
          timestamp: Date.now(),
        }),
        name: 'put-job2',
      },
]

Define a high-order function (HOF), callbackGenerator, that returns a callback function. The callback function is designed to construct Http request to perform a task.

private callbackGenerator(configuration: JobConfiguration): () => Promise<void> {
    const { url, method, dataFn } = configuration
    return async () => {
      try {
        await this.httpService
          .request({
            url,
            method,
            data: dataFn(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    }
 }

Lastly, define addConfigurableCronJobs function to iterate jobConfigurations array and insert the cron jobs to scheduler registry

addConfigurableCronJobs(): void {
    for (const configuration of this.jobConfigurations) {
      const { interval, name } = configuration
      const callback = this.callbackGenerator(configuration)
      const cronjob = new CronJob(interval, callback)
      this.schedulerRegistry.addCronJob(name, cronjob)
      cronjob.start()
    }
}

In task.module.ts,  update onModuleInit function to call addConfigurableCronJobs instead and implement onModuleDestroy to stop cron jobs when application shutdown. 

// task.module.ts
export class TaskModule implements OnModuleInit {  
  constructor(private taskService: TaskService) {}
  async onModuleInit() {    
     // await taskService.addCronJobs()    
     await this.taskService.addConfigurableCronJobs()  
  }
  
  async onModuleDestroy() {    
     await this.taskService.deleteConfigurableCronJobs()  
  }
}
// task.sevice.ts
export class TaskService {  
  ...
  
  deleteConfigurableCronJobs(): void {    
     for (const configuration of this.jobConfigurations) {      
        const { name } = configuration      
        this.schedulerRegistry.deleteCronJob(name)    
     }  
   }
}

deleteConfigurableCronJobs function iterates jobConfigurations array and deletes the jobs one by one.

In main.ts,  add app.enableShutdownHooks() to listen to application shutdown event that ultimately calls onModuleDestroy to stop the cron jobs.

// main.ts
async function bootstrap() {
  ...
  
  // Starts listening for shutdown hooks  
  app.enableShutdownHooks()
  const configService = app.get(ConfigService)
  const port = configService.get<number>('PORT', 0)
  await app.listen(port)
}

Start the application and observe the messages in the terminal

npm run start:dev

Similar to task scheduler v2, cron jobs are triggered every 15, 20 and 35 seconds. We have completed the refactoring and scheduler codes are DRY.

Add New Cron Job

It is made easy after the refactoring. Developers simply add new job configuration in the jobConfigurations array and addConfigurableCronJobs takes care of callback creation, job registration and job launch automatically. Less code and less development time on the part of developers.

Final thoughts

This post shows how our company implements dynamic task scheduler using HttpModule and ScheduleModule of NestJS. We did not settle for the non-DRY version after fulfilling QA request. We identified common pattern and generalized the architecture with high-order function and metadata array. The outcome is elimination of duplicated codes and easy code changes to support new cron job.

I hope you like this article and share it to people who are interested in NestJS!

Resources:

  1. Repo: https://github.com/railsstudent/nest-dynamic-scheduler
  2. Nest Task Scheduling: https://docs.nestjs.com/techniq\ues/task-scheduling
  3. Nest Http Module: https://docs.nestjs.com/techniques/http-module