Build Angular app with Netlify function

 54 total views,  4 views today

Introduction

I created a container component, FoodMenuComponent, to display a menu for users to order food and drink.

There are several ways to create the menu data in JSON format:

  1. Hardcoded data in the component
  2. Create static JSON data in assets folder and use a service to retrieve the data for the component
  3. Design a backend API and use a service to call the endpoint to retrieve the data
  4. Add a serverless function to return the menu data to the component

Option 1 makes component bloated because it contains both business logic and data. Business logic and data should belong to service and external data source respectively.

Option 2 is ideal for Angular beginners such that they don’t have to worry about backend design and implementation.

Option 3 requires developers to have knowledge of one or more backend frameworks, for example, Express and NestJS. Because of the additional skills, it demands more development efforts for a simple side project.

After weighing all alernatives, I chose Netlify function because I can leverage backend capability of cloud provider instead of building my own backend server.

Result

This is the end result of FoodMenuComponent

The menu is consisted of two questions and each question has at least one choice for user to choose.

Let me show you how to create a Netlify function to return menu data that FoodMenuComponent consumes and displays on browser.

Install Netlify CLI

npm install netlify-cli -g

Create Netlify function Configuration

Create netlify.toml that is the configuration file of Netlify. Add the following sections to toml file:

[build]
  publish = "dist/ng-spanish-menu"
  command = "ng build"
  functions = "./src/assets/functions"
[[redirects]]
  from = "/*"
  to = "/index.html"
  status = 200

and the most important line is functions = "./assets/functions".

functions = "./assets/functions" specifies the folder of Netlify functions and all functions should save to it.

Then, start Netlify CLI with Angular in development mode

ntl dev

and open the application at http://localhost:8888

Setting up First Severless Function

Let’s add our first serverless function to return menu data. We can find the function in ./assets/functions/menu.ts.

// menu.ts 

import { Handler } from "@netlify/functions";

const data = {
  "menu": [
    {
      "id": "1",
      "question": "Which appetizer(s) do you wish to order?",
      "choices": [
        {
          "id": "a",
          "name": "Egg salad",
          "description": "Egg salad",
          "currency": "USD",
          "price": 4.99,
          "available": true
        },
        ... omitted for the sake of brevity ...
      ]
    },
    {
      "id": "2",
      "question": "Which dessert(s) do you wish to order?",
      "choices": [
        {
          "id": "a1",
          "name": "Ice cream",
          "description": "Ice cream",
          "currency": "USD",
          "price": 1.99,
          "available": true
        },
        ... omitted for the sake of brevity ...
      ]
    }
  ]
}

const handler: Handler = async () => {
  return {
    statusCode: 200,
    body: JSON.stringify(data),
  };
};

export { handler };

Testing Severless Function

Get Endpoint: http://localhost:8888/.netlify/functions/menu

If the request is successful, the response displays menu data.

Adding endpoint in Angular

Open environment.ts and add a new environment variable, menuUrl, to it. Next, we will create food service to retrieve menu data through this endpoint.

export const environment = {
  production: false,
  menuUrl: 'http://localhost:8888/.netlify/functions/menu',
}

Creating food service

@Injectable({
  providedIn: 'root',
})
export class FoodService {
  constructor(private http: HttpClient) {}

  getFood(url: string): Observable<MenuItem[] | undefined> {
    return this.http.get<Menu>(url).pipe(
      pluck('menu'),
      catchError((err: Error) => {
        console.error(err)
        return of(undefined)
      }),
      share(),
    )
  }
}

getFood function requests menu data from our serverless function and returns the result as Observable<MenuItem[] | undefined>. When request fails, we return undefined since menu does not exist.

Calling service in Angular component with RxJS

Now, FoodMenuComponent can combine RxJS operators and food service to retrieve menu data and assign to Observable<Menu[] | undefined>.

@Component({
  selector: 'app-food-menu',
  templateUrl: './food-menu.component.html',
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class FoodMenuComponent implements OnInit, OnDestroy {
  menuItems$: Observable<MenuItem[] | undefined>

  ngOnInit(): void {
    this.menuItems$ = this.service
       .getFood(environment.menuUrl)
       .pipe(takeUntil(this.unsubscribe$))
  }

  menumItemTrackByFn(index: number, menuItem: MenuItem): string | number {
    return menuItem ? menuItem.id : index
  }

  choiceTrackByFn(index: number, choice: Choice) {
    return choice ? choice.id : index
  }
  ... Omitted for the sake of brevity ....
}

In the template of FoodMenuComponent, async pipe resolves menuItems$ into MenuItem array (menuItems) and Angular provides ngFor directive that can iterate an array to render values in each row.

<div class="container" *ngIf="menuItems$ | async as menuItems; else notAvailable">
  <app-food-menu-card *ngFor="let menuItem of menuItems; index as i; trackBy: menumItemTrackByFn">
    <app-food-question [question]="menuItem.question" head></app-food-question>
    <ng-container *ngFor="let choice of menuItem.choices; index as j; trackBy: choiceTrackByFn" body>
      <app-food-choice [choice]="choice" (foodChoiceAdded)="handleFoodChoice.emit($event)"></app-food-choice>
    </ng-container>
  </app-food-menu-card>
</div>
<ng-template #notAvailable>No menu</ng-template>

The container div iterates menuItems and passes each menuItem to <app-food-menu-card> element. Each menuItem has one question and each question displays some options for user to select. <app-food-question> is responsible for rendering menu item question and <app-food-choice> renders all available and unavailable options of the question.

We have reached the end of the post and FoodMenuComponent is working. You should see the same component that I showed at the beginning of the post.

That’s it! I hope you find this article useful and leverage serverless function in your Angular . Good bye!

Resources:

  1. Repo: https://github.com/railsstudent/ng-spanish-menu
  2. https://www.netlify.com/blog/2021/01/25/getting-started-with-netlify-functions-for-angular/

Angular and Storybook – Component with content projection

 16 total views,  4 views today

In Bonnie’s visual DOM course, I learnt the technique of applying multiple content projections in a component.

I created a FoodMenuCardComponent with two content projections; i.e., a component with two <ng-content> elements. The first <ng-content> has selector ‘head’ that projects FoodQuestion component and the second <ng-content> has selector ‘body’ that projects a list of FoodChoice components.

The same result can be achieved without content projections but I want to practice the new concept and create a reusable card component that has head and body sections.

Create a food menu card component in food module

ng g c foodMenuCard --module=food
// food-menu-card.component.ts 
import { Component } from '@angular/core'

@Component({
  selector: 'app-food-menu-card',
  template: `
  <div>
    <ng-content select="[head]"></ng-content>
    <ng-content select="[body]"></ng-content>
  </div>`,
})
export class FoodMenuCardComponent {}

Create Storybook for Food Menu Card Component

Create food-menu-card.stories.ts under the food folder

// food-menu-card.storeis.ts

import { moduleMetadata } from '@storybook/angular'
// also exported from '@storybook/angular' if you can deal with breaking changes in 6.1
import { action } from '@storybook/addon-actions'
import { Story, Meta } from '@storybook/angular/types-6-0'
import { FoodChoiceComponent, FoodMenuCardComponent, FoodQuestionComponent } from '@/food'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'

export default {
  title: 'Food Menu Card',
  component: FoodMenuCardComponent,
  decorators: [
    moduleMetadata({
      imports: [ReactiveFormsModule, FormsModule],
      declarations: [FoodQuestionComponent, FoodChoiceComponent],
    }),
  ],
} as Meta

const FoodMenuCardTemplate: Story<FoodMenuCardComponent> = (args: FoodMenuCardComponent) => ({
  props: {
    ...args,
    foodChoiceAdded: action('foodChoiceAdded'),
  },
  template: `<app-food-menu-card>
    <app-food-question [question]="menuItem.question" head></app-food-question>
    <ng-container *ngFor="let choice of menuItem.choices; index as j; trackBy: choiceTrackByFn" body>
      <app-food-choice [choice]="choice" (foodChoiceAdded)="foodChoiceAdded($event)"></app-food-choice>
    </ng-container>
  </app-food-menu-card>`,
})

export const Primary = FoodMenuCardTemplate.bind({})
Primary.args = {
  menuItem: {
    id: '1',
    question: 'Do you wish to order dessert?',
    choices: [
      {
        id: 'd',
        name: 'Buffalo Chicken Wings',
        description: 'Spicy chicken wings',
        price: 8.99,
        available: true,
      },
    ],
  },
}

export const Soldout = FoodMenuCardTemplate.bind({})
Soldout.args = {
  menuItem: {
    id: '1',
    question: 'Do you wish to order dessert?',
    choices: [
      {
        id: 'a',
        name: 'Egg salad',
        description: 'Egg salad',
        price: 4.99,
        available: false,
      },
    ],
  },
}

To order to create story for component with content projection, my solution is to create template component for FoodChoiceMenuComponent

template: <app-food-menu-card>
    <app-food-question [question]="menuItem.question" head></app-food-question>
    <ng-container *ngFor="let choice of menuItem.choices; index as j; trackBy: choiceTrackByFn" body>
      <app-food-choice [choice]="choice" (foodChoiceAdded)="foodChoiceAdded($event)"></app-food-choice>
    </ng-container>
</app-food-menu-card>

<app-food-menu-card> is root element containing <app-food-question> and <ng-container> children. <app-root-question> has head attribute and replaces <ng-content select='[head]’> element whereas <ng-container> has body attribute and replaces <ng-content select='[body]’> element.

Two stories are created for FoodMenuCardComponent: Primary and Soldout. Primary renders question and available food choices whereas Soldout renders question and a sold out message.

Unfortunately, Primary story did not log action when Submit button was clicked. After googling, I found the solution in Github issue and Storybook examples repo. I had to pass addon action to props to listen to foodChoiceAdded event emitter of FoodChoiceComponent

props: {
    ...args,
    foodChoiceAdded: action('foodChoiceAdded'),
}

Property name is foodChoiceAdded that is the name of the event emitter. Action name is ‘foodChoiceAdded’ but it can be arbitrary text such as action(‘log’). When button is clicked, Storybook Actions tab logs ‘foodChoiceAdded’ name and its data.

Start storybook application

npm run storybook

Click the title Food Menu Card -> Primary and the component is rendered with a question and once choice

When quantity is updated and submitted, event is emitted and logged on Actions tab

Click the title Food Menu Card -> Soldout and the component is rendered with a question and a sold out message.

This is the end of the blog post and we will keep you posted after progress is made, thanks.

Resources:

  1. Repo: https://github.com/railsstudent/ng-spanish-menu
  2. Storybook: Create a template component: https://storybook.js.org/docs/angular/workflows/stories-for-multiple-components#creating-a-template-component
  3. Angular Storybook Netlify App: https://storybooks-angular.netlify.app/?path=/story/custom-style–default
  4. Angular Storybook Examples: https://github.com/storybookjs/storybook/blob/next/examples/angular-cli/src/stories/core/styles/story-styles.stories.ts
  5. Action Logger does not handle event with template prop: https://github.com/storybookjs/storybook/issues/4820

Angular and Storybook – Simple Component with inputs and actions

 7 total views

Previously, I created a story book for simple component with inputs only (link) . In this post, I want to add new stories for a simple components with both input and action.

The component is FoodChoiceComponent that has two states: choice available to order and choice that has sold out.

If choice is available, user can input quantity and submit the reactive form. Otherwise, the component hides the form and displays a message, “Sorry, this choice is sold out”.

Create a food choice component in food module

ng g c foodChoice --module=food

Food choice component is a simple presentational component that is consisted of name, description, price, quantity and a reactive form.

// food-choice.component.ts 

// for the sakes of brevity, import statements are omitted

@Component({
  selector: 'app-food-choice',
  templateUrl: './food-choice.component.html',
  styleUrls: ['./food-choice.component.scss'],
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class FoodChoiceComponent implements OnInit, OnDestroy {
  @Input()
  choice: Choice

  @Output()
  foodChoiceAdded = new EventEmitter<OrderedFoodChoice>()

  submitChoice$ = new Subject<Event>()
  unsubscribe$ = new Subject<boolean>()
  processing = false

  form = this.fb.group({
    quantity: new FormControl(1, [Validators.required, Validators.min(1)]),
  })

  constructor(private fb: FormBuilder) {}

  ngOnInit(): void {
    this.submitChoice$
      .pipe(
        tap(($event) => {
          $event.preventDefault()
          $event.stopPropagation()
          this.processing = true
        }),
        delay(1000),
        map(() => ({
          ...this.form.value,
          name: this.choice.name,
          description: this.choice.description,
          price: this.choice.price,
        })),
        tap(() => (this.processing = false)),
        takeUntil(this.unsubscribe$),
      )
      .subscribe((value) => this.foodChoiceAdded.emit(value))
  }

  get quantity() {
    return this.form.get('quantity') as FormControl
  }

  ngOnDestroy(): void {
    this.unsubscribe$.next(true)
    this.unsubscribe$.complete()
  }
}
// food-choice.component.html

<div class="container">
  ...
  <ng-container [ngSwitch]="choice.available">
    <form [formGroup]="form" (ngSubmit)="submitChoice$.next($event)" *ngSwitchCase="true">
      <label name="quantity" class="item">
        <span class="field">Quantity: </span>
        <input type="number" formControlName="quantity" />
      </label>
      <div *ngIf="quantity.invalid && (quantity.touched || quantity.dirty)" class="alert alert-danger">
        <div *ngIf="quantity.errors?.min" class="error">Quantity must be at least {{ quantity.errors?.min?.min }}.</div>
      </div>
      <div *ngIf="processing" class="alert">
        <div class="processing">Loading...</div>
      </div>
      <button type="submit" [disabled]="processing || form.value.quantity <= 0">Submit</button>
    </form>
    <ng-container *ngSwitchDefault>
      <ng-container *ngTemplateOutlet="soldout; context: { choice: choice }"> </ng-container>
    </ng-container>
  </ng-container>
  <ng-template #soldout let-choice="choice">
    <span class="item sold-out">Sorry, {{ choice.name }} is sold out</span>
  </ng-template>
</div>

Input is a Choice interface that has several properties

export interface Choice {
  id: string
  name: string
  description: string
  price: number
  available: boolean
}
If available is true, reactive form is rendered.
If available is false, a message is displayed to indicate that the choice is sold out.

The food choice component has an event emitter that emits the user’s choice to container component. When user submits the reactive form, the submit event notifies RxJS subject to process the data and to emit the result to container component.

<form [formGroup]="form" (ngSubmit)="submitChoice$.next($event)" *ngIf="choice.available; else soldout">
....
</form>
submitChoice$ = new Subject<Event>()

@Output()
foodChoiceAdded = new EventEmitter<OrderedFoodChoice>()

form = this.fb.group({
  quantity: new FormControl(1, [Validators.required, Validators.min(1)]),
})

ngOnInit(): void {
    this.submitChoice$
      .pipe(
        tap(($event) => {
          $event.preventDefault()
          $event.stopPropagation()
          this.processing = true
        }),
        delay(1000),
        map(() => ({
          ...this.form.value,
          name: this.choice.name,
          description: this.choice.description,
          price: this.choice.price,
        })),
        tap(() => (this.processing = false)),
        takeUntil(this.unsubscribe$),
      )
      .subscribe((value) => this.foodChoiceAdded.emit(value))
}

Create Storybook for Food Choice Component

Create food-choice.stories.ts under the food folder

// food-choice.storeis.ts
import { moduleMetadata } from '@storybook/angular'
// also exported from '@storybook/angular' if you can deal with breaking changes in 6.1
import { Story, Meta } from '@storybook/angular/types-6-0'
import { FoodChoiceComponent } from '@/food'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'

export default {
  title: 'Food Choice',
  component: FoodChoiceComponent,
  decorators: [
    moduleMetadata({
      imports: [ReactiveFormsModule, FormsModule],
    }),
  ],
  argTypes: { onClick: { action: 'clicked' } },
} as Meta

const Template: Story<FoodChoiceComponent> = (args: FoodChoiceComponent) => ({
  props: args,
})

export const Primary = Template.bind({})
Primary.args = {
  choice: {
    id: '1',
    name: 'Vino tinto',
    description: 'Red wine',
    price: 12.99,
    available: true,
  },
}

export const Soldout = Template.bind({})
Soldout.args = {
  choice: {
    id: '1',
    name: 'Vino tinto',
    description: 'Red wine',
    price: 12.99,
    available: false,
  },
}

Two stories are created for FoodChoiceComponent, Primary and Soldout. Primary renders that state of an available food choice while Soldout renders the state of a sold out food choice.

Start storybook application

npm run storybook

Click the title Food Choice -> Primary and the component is rendered with reactive form

When quantity is updated and submitted, event is emitted and logged on Actions tab

Click the title Food Choice -> Soldout and the component is rendered with a sold out message.

This is the end of the blog post and we will keep you posted after progress is made, thanks.

Resources:

  1. Repo: https://github.com/railsstudent/ng-spanish-menu
  2. Storybook: Build a simple component: https://storybook.js.org/tutorials/intro-to-storybook/angular/en/simple-component/
  3. Actions: https://storybook.js.org/docs/angular/essentials/actions

Angular and Storybook – Simple Component

 6 total views,  2 views today

I recently completed Visual DOM course at Angular Nation and the course covers advanced materials and is beneficial to Angular developers of any level. I personally struggle with the concept of view container ref (vcr) and would like to build a side project about it with my mentor, Nati.

The idea of the application is to displays a restaurant menu in Spanish (I am also learning Spanish), user can order food and beverages from the menu and the selection is added to the page dynamically through vcr. The project is at the early stage and we use storybook to visualize the components as the application comes along the way.

Create an angular application

ng new ng-spanish-menu

Install storybook dependencies

# Add Storybook:
npx sb init

The command adds storybook script in package.json and developer can run npm run storybook to launch storybook site at http://localhost:6006.

Create a food module

ng g module food

This module keeps food-related components and services.

Create a food card component in food module

ng g c foodCard --module=food

Food card component is a simple presentational component that displays name, description, price, quantity and total amount of a food/beverage.

// food-card.component.ts 

import { ChangeDetectionStrategy, Component, Input } from '@angular/core'

@Component({
  selector: 'app-food-card',
  templateUrl: './food-card.component.html',
  styleUrls: ['./food-card.component.scss'],
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class FoodCardComponent {
  @Input()
  name: string

  @Input()
  description: string

  @Input()
  price: number

  @Input()
  quantity: number

  get total() {
    return Math.round(this.price * this.quantity * 100) / 100
  }
}
// food-card.component.html

<div class="container">
  <label name="name" class="item">
    <span class="field">Name:</span>
    <span>{{ name }}</span>
  </label>
  <label class="item" name="description">
    <span class="field">Description:</span>
    <span>{{ description }}</span>
  </label>
  <label class="item" name="price">
    <span class="field">Price:</span>
    <span>{{ price }}</span>
  </label>
  <label class="item" name="quantity">
    <span class="field">Quantity:</span>
    <span>{{ quantity }}</span>
  </label>
  <label class="item" name="total">
    <span class="field">Total:</span>
    <span>USD {{ total }}</span>
  </label>
</div>

Create Storybook for Food Card Component

Create a new food folder under stories.

Create food-card.stories.ts under the food folder

// food-card.storeis.ts
// also exported from '@storybook/angular' if you can deal with breaking changes in 6.1
import { Story, Meta } from '@storybook/angular/types-6-0'
import { FoodCardComponent } from '@/food'

export default {
  title: 'Food Card',
  component: FoodCardComponent,
} as Meta

const Template: Story<FoodCardComponent> = (args: FoodCardComponent) => ({
  props: args,
})

export const Primary = Template.bind({})
Primary.args = {
  name: 'Vino tinto',
  description: 'Red wine',
  price: 12.99,
  quantity: 3,
}

Then, input values are supplied to Primary.args and subsequently passed to FoodCardComponent to be displayed.

Start storybook application

npm run storybook

Click the title Food Card -> Primary and the component is rendered with initial input values.

Input values can be updated and then the component re-renders with new values.

CSS is very simple currently but Nati and I can work on the styling later after functionality is completed.

I am very excited of this project because Nati is very talented Angular developer and we can learn from each other while we work it together and she teaches me Spanish.

This is the end of the blog post and we will keep you posted after progress is made, thanks.

Resources:

  1. Repo: https://github.com/railsstudent/ng-spanish-menu
  2. Install Storybook: https://storybook.js.org/docs/react/get-started/install
  3. Storybook: Build a simple component: https://storybook.js.org/tutorials/intro-to-storybook/angular/en/simple-component/

Swagger In NestJS

 2 total views

Problem: Frontend developers complained that integration with API is a painful process because there is no documentation on available endpoints, expected payloads and responses. Therefore, our backend team leads elect to enable swagger such that frontend developers can browse all the APIs on dedicated documentation site.

When they showed me how it was done, I was thoroughly impressed and wanted to show NestJS peers the simple steps.

Create a NestJS application

nest new nest-swagger

Install configuration, swagger and other dependencies

npm install --save @nestjs/config joi class-transformer class-validator
npm install --save @nestjs/swagger swagger-ui-express

Store environment variables in environment file

Store Swagger environment variables in .env and we will use configuration service to retrieve the values in main.ts

// main.ts
NODE_ENV=development
PORT=3000
API_VERSION=1.0
SWAGGER_TITLE=Local Swagger Documentation Site
SWAGGER_DESCRIPITON=The task API description

Import Configuration Module and validate schema

// envSchema.ts
import * as Joi from 'joi'

export const validationSchema = Joi.object({
  NODE_ENV: Joi.string().valid('development', 'production', 'test', 'provision').default('development'),
  PORT: Joi.number().default(3000),
  API_VERSION: Joi.string().default('1.0'),
  SWAGGER_TITLE: Joi.string().required(),
  SWAGGER_DESCRIPITON: Joi.string().required(),
})
// app.module.ts
import { Module } from '@nestjs/common'
import { ConfigModule } from '@nestjs/config'
import { AppController } from './app.controller'
import { AppService } from './app.service'
import { validationSchema } from './envSchema'

@Module({
  imports: [
    ConfigModule.forRoot({
      isGlobal: true,
      validationSchema,
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Configure Swagger

Initialize Swagger in main.ts such that API documentation is generated automatically for controllers, entities and data transfer objects (DTOs).

// main.ts
async function bootstrap() {
   ....
   app.useGlobalPipes(
     new ValidationPipe({
       whitelist: true,
     }),
   )

   const configService = app.get(ConfigService)
   const version = configService.get<string>('API_VERSION', '1.0')
   const title = configService.get<string>('SWAGGER_TITLE', '')
   const description = configService.get<string>('SWAGGER_DESCRIPITON', '')

   const config = new DocumentBuilder()
     .setTitle(title)
     .setDescription(description)
     .setVersion(version)
     .addBearerAuth()
     .build()
   const document = SwaggerModule.createDocument(app, config)
   SwaggerModule.setup('api', app, document)

   const port = configService.get<number>('PORT', 0)
   await app.listen(port)
}

Enable Swagger plugin in nest-cli.json

// nest-clis.json
{
  "collection": "@nestjs/schematics",
  "sourceRoot": "src",
  "compilerOptions": {
    "plugins": ["@nestjs/swagger"]
  }
}

After Swagger plugin is enabled, Swagger automatically generates documentation for controller and files with either .entity.s or .dto.ts suffix.

Verify Swagger is working by browsing http://localhost:3000/api

The documentation site works because it publishes the GET endpoint of app controller successfully.

Swagger setup is complete and we can add new API and publish the new endpoints in the dedicated documentation site.

Create a new Task Module to generate API documentation

nest g mo Task

Next, I generate task service and task controller and register them in the TaskModule. The following is the declaration of the TaskModule:

import { Module } from '@nestjs/common'
import { TaskController } from './task.controller'
import { TaskService } from './services'

@Module({
  controllers: [TaskController],
  providers: [TaskService],
})
export class TaskModule {}

Define new entities and new DTOs for Task API

// create-task.dto.ts
import { IsNotEmpty, IsString } from 'class-validator'

export class CreateTaskDto {
  @IsNotEmpty()
  @IsString()
  name: string
}

// update-task.dto.ts
import { IsBoolean, IsDefined } from 'class-validator'

export class UpdateTaskDto {
  @IsDefined()
  @IsBoolean()
  completed: boolean
}

Define new entities for the Task API

// task.entity.ts
import { IsBoolean, IsString } from 'class-validator'

export class Task {
  @IsString()
  id: string

  @IsString()
  name: string

  @IsBoolean()
  completed: boolean
}

// delete-result.entity.ts
import { IsNumber, IsOptional, Min, ValidateNested } from 'class-validator'
import { Task } from './task.entity'

export class DeleteResult {
  @IsNumber()
  @Min(0)
  count: number

  @IsOptional()
  @ValidateNested()
  task?: Task
}

Create CRUD methods in Task Service

Create basic methods to add, update, delete and retrieve a task object from a tasks array

// task.service.ts
import { BadRequestException, Injectable } from '@nestjs/common'
import { v4 } from 'uuid'
import { CreateTaskDto, UpdateTaskDto } from '../dtos'
import { DeleteResult, Task } from '../entities'

@Injectable()
export class TaskService {
  tasks: Task[] = [
    ...some tasks objects
  ]

  listAll(): Promise<Task[]> {
    return Promise.resolve(this.tasks)
  }

  findById(id: string): Promise<Task> {
    const task = this.tasks.find((task) => task.id === id)
    if (!task) {
      throw new BadRequestException('Task not found')
    }
    return Promise.resolve(task)
  }

  deleteById(id: string): Promise<DeleteResult> {
    const task = this.tasks.find((task) => task.id === id)
    this.tasks = this.tasks.filter((task) => task.id !== id)
    const deletedResult: DeleteResult = {
      count: task ? 1 : 0,
      task,
    }
    return Promise.resolve(deletedResult)
  }

  updateTask(id: string, dto: UpdateTaskDto): Promise<Task> {
    this.tasks = this.tasks.map((task) => {
      if (task.id !== id) {
        return task
      }
      return {
        ...task,
        completed: dto.completed,
      }
    })

    return this.findById(id)
  }

  createTask(dto: CreateTaskDto): Promise<Task> {
    const newTask: Task = {
      ...dto,
      id: v4(),
      completed: false,
    }
    this.tasks = [...this.tasks, newTask]
    return Promise.resolve(newTask)
  }
}

Create endpoints in Task Controller

task.controller.ts
import { UpdateTaskDto } from './dtos/update-task.dto'
import { Body, Controller, Delete, Get, Param, ParseUUIDPipe, Post, Put } from '@nestjs/common'
import { ApiBadRequestResponse, ApiTags } from '@nestjs/swagger'
import { CreateTaskDto } from './dtos'
import { TaskService } from './services'
import { DeleteResult, Task } from './entities'

@ApiTags('Task')
@Controller('task')
export class TaskController {
  constructor(private service: TaskService) {}

  @Get()
  getAll(): Promise<Task[]> {
    return this.service.listAll()
  }

  @Get(':id')
  @ApiBadRequestResponse({ description: 'Task not found' })
  getTask(@Param('id', ParseUUIDPipe) id: string): Promise<Task> {
    return this.service.findById(id)
  }

  @Delete(':id')
  deleteTask(@Param('id', ParseUUIDPipe) id: string): Promise<DeleteResult> {
    return this.service.deleteById(id)
  }

  @Post()
  createTask(@Body() dto: CreateTaskDto): Promise<Task> {
    return this.service.createTask(dto)
  }

  @Put(':id')
  updateTask(@Param('id', ParseUUIDPipe) id: string, @Body() dto: UpdateTaskDto): Promise<Task> {
    return this.service.updateTask(id, dto)
  }
}

@ApiBadRequestResponse and @ApiBadRequestResponse are Swagger decorators.

DecoratorDescription
ApiTagsProvide name of the API and in this example, the name is Task
ApiBadRequestResponseIndicate the endpoint returns BadRequestException when task is not found

Refresh http://localhost:3000/api and Task API is published.

Task lists all available endpoints (two GETs, POST, DELETE and POST) and Schema section auto-generates its entities (Task and DeleteResult) and DTOs (CreateTaskDto and UpdateTaskDto).

That’s the end of Swagger introduction.

Final thoughts

Swagger provides clear documentation of Restful API that facilitates communication between frontend and backend developers. Rather than constantly asking backend team for sample requests and responses, frontend developers can refer to the Swagger documentation that illustrates all available endpoints and the expected payloads and responses. When teams have no confusion and argument, developers become happy and happy developers are the most productive in application development.

I hope you like this article and share it to people who are interested in NestJS!

Resources:

  1. Repo: https://github.com/railsstudent/nest-swagger
  2. OpenAPI (Swagger): https://docs.nestjs.com/openapi/introduction

Dynamic Task Scheduler In NestJS

 0 total views

Scenario: Our ESG (Environmental, Social, and Governance) platform requires to run schedule jobs to call endpoints to generate reports for internal teams and send company emails to our customers. Development team was tasked with developing a job scheduler that executes cron jobs at a specified period of time. node-cron is a popular open source library for setting up node-like cron jobs in a typescript program, however, Nest provides Schedule module that does basically the same thing with decorators and flexible API. Since our backend is powered by Nest, we prefer to use existing features of NestJS instead of bringing in a new library and investing time to study a new API.

Problem: We followed Nest documentation and created all cron jobs declaratively. These jobs run on a daily or weekly basis that are inconvenient for QA team to test because they cannot wait for such a long time to verify test results. The approach is to configure jobs to run in different intervals in different environments. For example, a job is configured to run once a day in production environment and run every minute in QA environment to facilitate testing cycle.

To cater to the new QA request, developers converted declarative cron jobs to dynamic counterparts through dynamic schedule module API. However, the codes were no longer DRY and I refactored the codebase to keep future development efforts to the minimum.

Create a NestJS application

nest new nest-dynamic-scheduler

Install configuration, http and schedule dependencies

npm i --save @nestjs/config joi
npm i --save @nestjs/axios
npm install --save @nestjs/schedule
npm install --save-dev @types/cron

Store environment variables in environment file

Store environment variables in .env and we will use configuration service to retrieve the base url in the codes.

Import Configuration Module

Import configuration module and validate the schema of .env file.

Validation part completes. We can move on to add dummy endpoints in AppController and a Task module that calls them in cron jobs.

Create dummy endpoints in App Controller

We define a printMessage function in AppService that will be called by the AppController

AppController has three new endpoints: the first one handles POST method, the second one handles PATCH and the final one handles PUT. It is to demonstrate that Nest cron jobs work with common http request methods.

Create new Task module

nest g mo Task

Next, I generate task service and import HttpModule and ScheduleModule.forRoot() to TaskModule. The following is the declaration of the TaskModule:

import { HttpModule, Module } from '@nestjs/common'
import { ScheduleModule } from '@nestjs/schedule'
import { TaskService } from './task.service'

@Module({
  imports: [HttpModule, ScheduleModule.forRoot()],
  providers: [TaskService],
  controllers: [],
})
export class TaskModule {}

First Attempt: Create declarative cron jobs in TaskService

// task.service.ts

private readonly baseUrl: string

constructor(
    private httpService: HttpService,
    private configService: ConfigService,
  ) {
    this.baseUrl = this.configService.get<string>('BASE_URL', '')
}

@Cron('*/15 * * * * *')
async handlePostJob(): Promise<void> {
    try {
      await this.httpService
        .post(`${this.baseUrl}/post-job`, {
          name: 'connie',
          msg: 'schedule post job every 15 second',
          timestamp: Date.now(),
        })
        .toPromise()
   } catch (err) {
      console.error(err)
   }
}

@Cron('*/20 * * * * *')
async handlePatchJob(): Promise<void> {
  try {
    await this.httpService
       .patch(`${this.baseUrl}/patch-job`, {
          name: 'connie',
          msg: 'schedule patch job every 20 second',
          timestamp: Date.now(),
       })
       .toPromise()
  } catch (err) {
     console.error(err)
  }
}

@Cron('*/30 * * * * *')
async handlePutJob(): Promise<void> {
  try {
    await this.httpService
      .put(`${this.baseUrl}/put-job`, {
         name: 'connie',
         msg: 'schedule put job every 30 second',
         timestamp: Date.now(),
      })
      .toPromise()
  } catch (err) {
    console.error(err)
  }
}

@Cron() decorator tells NestJS when the job starts and terminates and its frequency. The decorator supports all standard cron patterns; therefore, @Cron(‘*/15 * * * * *’) makes the Http POST request every 15 seconds. Similarly, PATCH and PUT requests are triggered every 20 and 30 seconds respectively.

Second Attempt: Convert declarative cron jobs to dynamic cron jobs

If QA team did not make the request, the scheduler was completed and we could go home and rest. The request seemed trivial; we could store the cron patterns as environment variables and applied them in @Cron(), right? Wrong answer, my friends.

// task.service.ts

const defaultInterval = '* * * * * *'    
this.postJobInterval = this.configService.get<string>('POST_JOB_INTERVAL', defaultInterval)

@Cron(this.postJobInterval)
async handlePostJob(): Promise<void> {
    ...
}

I couldn’t use this.postJobInterval inside @Cron() because “this” variable can be null. Fortunately, dynamic scheduler module API exists and we can add cron jobs programmatically with configurable cron patterns.

We define new job intervals in .env (POST_JOB_INTERVAL, PATCH_JOB_INTERVAL, PUT_JOB_INTERVAL)

.env

POST_JOB_INTERVAL='*/15 * * * * *'
PATCH_JOB_INTERVAL='*/20 * * * * *'
PUT_JOB_INTERVAL='*/30 * * * * *'

We comment out the old codes and define addCronJobs function to create dynamic cron jobs and register them in scheduler registry. Finally, we start the jobs such that they are triggered every 15, 20 and 30 seconds. Remember to inject SchedulerRegistry in the constructor.

// task.service.ts

const defaultInterval = '* * * * * *'    

private readonly postJobInterval: string
private readonly patchJobInterval: string
private readonly putJobInterval: string
private readonly baseUrl: string

constructor(
    private httpService: HttpService,
    private schedulerRegistry: SchedulerRegistry,
    private configService: ConfigService,
  ) {
    this.postJobInterval = this.configService.get<string>('POST_JOB_INTERVAL', defaultInterval)
    this.patchJobInterval = this.configService.get<string>('PATCH_JOB_INTERVAL', defaultInterval)
    this.putJobInterval = this.configService.get<string>('PUT_JOB_INTERVAL', defaultInterval)
    this.baseUrl = this.configService.get<string>('BASE_URL', '')
}

addCronJobs(): void {
    const postJob = new CronJob(this.postJobInterval, async () => {
      try {
        await this.httpService
          .post(`${this.baseUrl}/post-job`, {
            name: 'connie',
            msg: 'schedule post job every 15 second',
            timestamp: Date.now(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    })

    const patchJob = new CronJob(this.patchJobInterval, async () => {
      try {
        await this.httpService
          .patch(`${this.baseUrl}/patch-job`, {
            name: 'connie',
            msg: 'schedule patch job every 20 second',
            timestamp: Date.now(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    })

    const putJob = new CronJob(this.putJobInterval, async () => {
      try {
        await this.httpService
          .put(`${this.baseUrl}/put-job`, {
            name: 'connie',
            msg: 'schedule put job every 30 second',
            timestamp: Date.now(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    })

    this.schedulerRegistry.addCronJob('post-job', postJob)
    this.schedulerRegistry.addCronJob('patch-job', patchJob)
    this.schedulerRegistry.addCronJob('put-job', putJob)
    postJob.start()
    patchJob.start()
    putJob.start()
  }

Modify TaskModule to implement OnModuleInit interface and start cron jobs in onModuleInit function.

// task.module.ts

export class TaskModule implements OnModuleInit {  
  constructor(private taskService: TaskService) {}
  async onModuleInit() {    
    await taskService.addCronJobs()    
  }
}

Start the application and observe the messages in the terminal

npm run start:dev

The current implementation meets the requirements but it is not DRY. Why do I say that? It is because I see a common pattern in the three dynamic jobs.

  • Create a callback function that
    • makes a new HTTP request in a try-catch block
    • converts the observable to promise with toPromise()
    • await the result of the HTTP request
  • Create a new cron job with cron pattern and the callback function
  • Add the new cron job to scheduler registry
  • Start the cron job

I can generalize this pattern such that developer can write minimum codes to add new cron job in the future.

Third Attempt: Generalize the dynamic task scheduler

Define a new interface, JobConfiguration, that stores the metadata of the job

// job-configuration.interface.ts
import { Method } from 'axios'

export interface JobConfiguration {
  url: string
  interval: string
  method: Method
  dataFn: () => any
  name: string
}

method is the supported method of Axios and it can be POST, PATCH, PUT or DELETE. dataFn function is used to generate new request data in the callback.

Define metadata of cron jobs in jobConfigurations array.

this.jobConfigurations = [
      {
        url: `${this.baseUrl}/post-job`,
        interval: this.postJobInterval,
        method: 'POST',
        dataFn: () => ({
          name: 'connie',
          msg: 'schedule dynamic post job every 15 second',
          timestamp: Date.now(),
        }),
        name: 'post-job2',
      },
      {
        url: `${this.baseUrl}/patch-job`,
        interval: this.patchJobInterval,
        method: 'PATCH',
        dataFn: () => ({
          name: 'mary',
          msg: 'schedule dynamic patch job every 20 second',
          timestamp: Date.now(),
        }),
        name: 'patch-job2',
      },
      {
        url: `${this.baseUrl}/put-job`,
        interval: this.putJobInterval,
        method: 'PUT',
        dataFn: () => ({
          name: 'job',
          msg: 'schedule dynamic put job every 30 second',
          timestamp: Date.now(),
        }),
        name: 'put-job2',
      },
]

Define a high-order function (HOF), callbackGenerator, that returns a callback function. The callback function is designed to construct Http request to perform a task.

private callbackGenerator(configuration: JobConfiguration): () => Promise<void> {
    const { url, method, dataFn } = configuration
    return async () => {
      try {
        await this.httpService
          .request({
            url,
            method,
            data: dataFn(),
          })
          .toPromise()
      } catch (err) {
        console.error(err)
      }
    }
 }

Lastly, define addConfigurableCronJobs function to iterate jobConfigurations array and insert the cron jobs to scheduler registry

addConfigurableCronJobs(): void {
    for (const configuration of this.jobConfigurations) {
      const { interval, name } = configuration
      const callback = this.callbackGenerator(configuration)
      const cronjob = new CronJob(interval, callback)
      this.schedulerRegistry.addCronJob(name, cronjob)
      cronjob.start()
    }
}

In task.module.ts,  update onModuleInit function to call addConfigurableCronJobs instead and implement onModuleDestroy to stop cron jobs when application shutdown. 

// task.module.ts
export class TaskModule implements OnModuleInit {  
  constructor(private taskService: TaskService) {}
  async onModuleInit() {    
     // await taskService.addCronJobs()    
     await this.taskService.addConfigurableCronJobs()  
  }
  
  async onModuleDestroy() {    
     await this.taskService.deleteConfigurableCronJobs()  
  }
}
// task.sevice.ts
export class TaskService {  
  ...
  
  deleteConfigurableCronJobs(): void {    
     for (const configuration of this.jobConfigurations) {      
        const { name } = configuration      
        this.schedulerRegistry.deleteCronJob(name)    
     }  
   }
}

deleteConfigurableCronJobs function iterates jobConfigurations array and deletes the jobs one by one.

In main.ts,  add app.enableShutdownHooks() to listen to application shutdown event that ultimately calls onModuleDestroy to stop the cron jobs.

// main.ts
async function bootstrap() {
  ...
  
  // Starts listening for shutdown hooks  
  app.enableShutdownHooks()
  const configService = app.get(ConfigService)
  const port = configService.get<number>('PORT', 0)
  await app.listen(port)
}

Start the application and observe the messages in the terminal

npm run start:dev

Similar to task scheduler v2, cron jobs are triggered every 15, 20 and 35 seconds. We have completed the refactoring and scheduler codes are DRY.

Add New Cron Job

It is made easy after the refactoring. Developers simply add new job configuration in the jobConfigurations array and addConfigurableCronJobs takes care of callback creation, job registration and job launch automatically. Less code and less development time on the part of developers.

Final thoughts

This post shows how our company implements dynamic task scheduler using HttpModule and ScheduleModule of NestJS. We did not settle for the non-DRY version after fulfilling QA request. We identified common pattern and generalized the architecture with high-order function and metadata array. The outcome is elimination of duplicated codes and easy code changes to support new cron job.

I hope you like this article and share it to people who are interested in NestJS!

Resources:

  1. Repo: https://github.com/railsstudent/nest-dynamic-scheduler
  2. Nest Task Scheduling: https://docs.nestjs.com/techniq\ues/task-scheduling
  3. Nest Http Module: https://docs.nestjs.com/techniques/http-module

Stripe Integration In NestJS

 2 total views,  2 views today

Scenario: Our ESG (Environmental, Social, and Governance) platform offers monthly and annual subscriptions to charge customers for using our ESG reporting service. When we designed our payment module, we chose Stripe as our payment platform because it accepts credit card payment method, provides good documentations and open source node library for the API (stripe-node).

Problem: We have chosen our payment platform but we don’t want the front-end application to call Stripe API directly. The architecture design is to make HTTP requests to NestJS backend and the backend calls node Stripe library to update Stripe accounts and our database. It is a challenge to our development team because we have never done Stripe integration in NestJS before and we intend to implement it by adopting good NestJS philosophy.

Go to Stripe.com to sign up a developer account

Copy the secret key as it will be passed to Stripe API to update Stripe accounts

Create a NestJS application

nest new nest-stripe-integration

Install Stripe and other dependencies

npm i stripe --save
npm i --save @nestjs/config joi

Store Stripe secret key in environment file

Store environment variables in .env and we will use configuration service to retrieve the secret key in the codes.

Import Configuration Module

Import configuration module and validate the schema of .env file.

Validation part completes and we can move on to create a Stripe module that is the core of this post.

Create a Stripe module to encapsulate node Stripe library

nest g mo Stripe

Next, I run nest commands to generate stripe service and stripe controller and the following is the declaration of the StripeModule:

@Module({
  providers: [StripeService],
  controllers: [StripeController],
})
export class StripeModule {}

Add logic to Stripe service to create and retrieve customer

The create customer logic is going to create a new credit card and assign it to the new customer. For brevity’s sake, I won’t show the implementation of createCard here and you can visit my github repo to find it.

// stripe.service.ts

import Stripe from 'stripe'

constructor(private service: ConfigService) {}

async createCustomer(dto: CreateCustomerDto): Promise<Stripe.Response<Stripe.Customer>> {
    const { card = null, email, name, description } = dto || {}

    if (!card) {
      throw new BadRequestException('Card information is not found')
    }
     
    const secretKey = this.service.get<string>('STRIPE_SECRET_KEY') || ''
    const stripe = new Stripe(secretKey, {
        apiVersion: '2020-08-27',
    })    

    const { data } = await stripe.customers.list({
      email,
      limit: 1,
    })

    if (data.length > 0) {
      throw new BadRequestException('Customer email is found')
    }

    const newCustomer = await stripe.customers.create({ email, name, description })
    if (!newCustomer) {
      throw new BadRequestException('Customer is not created')
    }

    await this.createCard(newCustomer.id, card)
    return newCustomer
}

The get customer logic accepts Stripe customer id and returns the Stripe customer object if it exists

async getCustomer(customerId: string): Promise<Stripe.Customer | null> {
    const secretKey = this.service.get<string>('STRIPE_SECRET_KEY') || ''
    const stripe = new Stripe(secretKey, {
        apiVersion: '2020-08-27',
    })

    const customer = await stripe.customers.retrieve(customerId)

    if (customer.deleted) {
      return null
    }

    const { headers, lastResponse, ...rest } = customer
    return rest as Stripe.Customer
  }

We have completed the Stripe service and can proceed to add endpoints to the controller.

Add new endpoints to the controller

@Controller('stripe')
export class StripeController {
  constructor(private service: StripeService) {}

  @Post('customer')
  async createCustomer(@Body() dto: CreateCustomerDto): Promise<Stripe.Response<Stripe.Customer>> {
    return this.service.createCustomer(dto)
  }

  @Get('customer/:customerId')
  async getCustomer(@Param('customerId') customerId: string): Promise<Stripe.Customer | null> {
    return this.service.getCustomer(customerId)
  }
}

The controller is also completed and we can get ourselves new customers.

CURL

// Create new customer
curl --location --request POST 'http://localhost:3000/stripe/customer' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "John Doe",
    "description": "Stripe Customer (john.doe@yopmail.com)",
    "email": "john.doe@yopmail.com",
    "card": {
        "name": "John Doe",
        "number": "4242424242424242",
        "expMonth": "01",
        "expYear": "2026",
        "cvc": "315" 
    }
}

// Response
{
    "id": "cus_Jx6caVHGwhTrAK",
    "object": "customer",
    "address": null,
    "balance": 0,
    "created": 1627715672,
    "currency": null,
    "default_source": null,
    "delinquent": false,
    "description": "Stripe Customer (john.doe@yopmail.com)",
    "discount": null,
    "email": "john.doe@yopmail.com",
    "invoice_prefix": "4D7BFE8B",
    "invoice_settings": {
        "custom_fields": null,
        "default_payment_method": null,
        "footer": null
    },
    "livemode": false,
    "metadata": {},
    "name": "John Doe",
    "next_invoice_sequence": 1,
    "phone": null,
    "preferred_locales": [],
    "shipping": null,
    "tax_exempt": "none"
}
// Retrieve customer
curl --location --request GET 'http://localhost:3000/stripe/customer/cus_Jx6caVHGwhTrAK'

// Response
{
    "id": "cus_Jx6caVHGwhTrAK",
    "object": "customer",
    "address": null,
    "balance": 0,
    "created": 1627715672,
    "currency": null,
    "default_source": "card_1JJCOzBX6xHuypyKQ5nw4PxY",
    "delinquent": false,
    "description": "Stripe Customer (john.doe@yopmail.com)",
    "discount": null,
    "email": "john.doe@yopmail.com",
    "invoice_prefix": "4D7BFE8B",
    "invoice_settings": {
        "custom_fields": null,
        "default_payment_method": null,
        "footer": null
    },
    "livemode": false,
    "metadata": {},
    "name": "John Doe",
    "next_invoice_sequence": 1,
    "phone": null,
    "preferred_locales": [],
    "shipping": null,
    "tax_exempt": "none"
}

At this point, Stripe module has successfully provided Stripe functionality but we can make improvements to Stripe service to make it dry

Improvement #1: Make a reusable function to return Node stripe client

If you observe closely, you will see that every function instantiates a node Stripe client to call the library in order to perform customer-related activity.

We can create a small function that returns the client and reuse it in other functions

initStripeClient() {
    const secretKey = this.service.get<string>('STRIPE_SECRET_KEY') || ''
    const stripe = new Stripe(secretKey, {
        apiVersion: '2020-08-27',
    })
    return stripe
}
// stripe.service.ts

import Stripe from 'stripe'

constructor(private service: ConfigService) {}

async createCustomer(dto: CreateCustomerDto): Promise<Stripe.Response<Stripe.Customer>> {
    const { card = null, email, name, description } = dto || {}

    if (!card) {
      throw new BadRequestException('Card information is not found')
    }
     
    const stripe = this.initStripeClient()

    const { data } = await stripe.customers.list({
      email,
      limit: 1,
    })

    if (data.length > 0) {
      throw new BadRequestException('Customer email is found')
    }

    const newCustomer = await stripe.customers.create({ email, name, description })
    if (!newCustomer) {
      throw new BadRequestException('Customer is not created')
    }

    await this.createCard(newCustomer.id, card)
    return newCustomer
}
async getCustomer(customerId: string): Promise<Stripe.Customer | null> {
    const stripe = this.initStripeClient()

    const customer = await stripe.customers.retrieve(customerId)

    if (customer.deleted) {
      return null
    }

    const { headers, lastResponse, ...rest } = customer
    return rest as Stripe.Customer
 }

We can take one step further by eliminating initStripeClient and injecting the Stripe client in the constructor. How do we do that? The answer is to create a custom provider for the Stripe client in StripeModule.

Improvement #2: Create a custom provider for the Stripe client

// stripe.module.ts

const StripeClientProvider = {
  provide: 'StripeClient',
  inject: [ConfigService],
  useFactory: (service: ConfigService) => {
    const secretKey = service.get<string>('STRIPE_SECRET_KEY', '')
    const stripe = new Stripe(secretKey, {
      apiVersion: '2020-08-27',
    })
    return stripe
  },
}

@Module({
  providers: [StripeService, StripeClientProvider],
  controllers: [StripeController],
})
export class StripeModule {}

Improvement #3: Inject the Stripe client to the constructor of Stripe service

// stripe.service.ts

constructor(@Inject('StripeClient') private stripeClient: Stripe) {}

async createCustomer(dto: CreateCustomerDto): Promise<Stripe.Response<Stripe.Customer>> {
    const { card = null, email, name, description } = dto || {}

    if (!card) {
      throw new BadRequestException('Card information is not found')
    }

    const { data } = await this.stripeClient.customers.list({
      email,
      limit: 1,
    })

    if (data.length > 0) {
      throw new BadRequestException('Customer email is found')
    }

    const newCustomer = await this.stripeClient.customers.create({ email, name, description })
    if (!newCustomer) {
      throw new BadRequestException('Customer is not created')
    }

    await this.createCard(newCustomer.id, card)
    return newCustomer
}

async getCustomer(customerId: string): Promise<Stripe.Customer | null> {
    const customer = await this.stripeClient.customers.retrieve(customerId)

    if (customer.deleted) {
      return null
    }

    const { headers, lastResponse, ...rest } = customer
    return rest as Stripe.Customer
}

This is the final version of stripe service: the functions are DRY and it is possible because of useFactory and injection token provided by NestJs.

Final thoughts

In this post, I have shown the usage of useFactory to create an instance of node-stripe client and make it available in NestJS service through injection. Rather than creating a new stripe client in every method, the service keeps a single reference and uses it to call Stripe APIs to create new Stripe customer. This is an illustration of DRY principle where the methods are lean and without duplicated logic.

I hope you find this blog post helpful and look to add NestJS into your backend technical stack.

Resources:

  1. Repo: https://github.com/railsstudent/nest-stripe-integration
  2. Stripe Token API: https://stripe.com/docs/api/tokens
  3. Stripe Customer API: https://stripe.com/docs/api/customers

How I cache data list in memory using useFactory in NestJS

 2 total views,  2 views today

Scenario: Application at work stores data lists in database tables and they never change after initial population. These data lists are used for dropdown selection in frontend, for example, users select their salutation in user profile page and click to make the change.

Problem: When user opens the page, UI makes a HTTP request to retrieve the salutation list from the server. NestJS API connects the database to query the table and returns the salutation array in the response. Since the list is static, I want to cache the data in memory such that subsequent database calls can be avoided.

Solution #1: When application starts, store data lists in Redis and the API fetches the data from the in-memory cache.

Cons: This solution will require DevOps engineers to provision resources for the Redis instances in the cloud platform and I cannot justify the cost for setting them up for few data lists.

Solution #2: Use useFactory to create a custom provider to store the data lists in memory and use @Inject() to inject the cache in controller to return the cached data

Pros:

  1. The implementation is done within application level and no DevOps help is required.
  2. If data list changes in the future, TypeORM migration script will be deployed to the environment and application restarts to recreate the cache.

Lets start with the naive approach that queries database upon every http request

Naive Approach

Step 1: Create a service that waits 2 seconds before returning a list of salutations

async getSalutations(): Promise<Salutation> {
    const salutations = ['Mr.', 'Mrs.', 'Miss']
    return new Promise((resolve) => setTimeout(resolve, 2000)).then(() => {
      console.log('Wait 2 seconds before returning salutations array')
      const now = Date.now()
      return {
        now,
        salutations,
      }
    })
  }

The promise returns Data.now() to indicate that the result is obtained at real time.

Step 2: Create an endpoint to return the array of salutations

  @Get('salutations-realtime')
  async getRealtimeSalutations(): Promise<Salutation> {
    return this.appService.getSalutations()
  }

Step 3: Curl http://localhost:3000/salutations-realtime to show the timestamp changes every time

Take note that the timestamp is different each time because each call waits 2 seconds before the array is returned in the response.

Next, I will use useFactory to define a custom provider such that getSalutations() is called only once and the 2-second wait is a fixed cost.

Return data as fast as a cheetah

Custom Provider Approach

Step 1: Define interface of the map of data lists

export interface Salutation {
  now: number
  salutations: string[]
}

export interface DataMap {
  salutations: Salutation
}

Step 2: Add a new function in AppService to return the map of data lists

async getDataMap(): Promise<DataMap> {
   return {
      salutations: await this.getSalutations(),
   }
}

I used await this.getSalutations() to wait for the result and to store it in salutations property of the map.

Step 3: Create a custom provider for the map and register it in AppModule

const DataMapFactory = {
  provide: 'DataMap',
  useFactory: (appService: AppService) => {
    return appService.getDataMap()
  },
  inject: [AppService],
}

@Module({
  imports: [],
  controllers: [AppController],
  providers: [AppService, DataMapFactory],
  exports: [DataMapFactory],
})
export class AppModule {}

Whenever I inject the provider name ‘DataMap‘, I have access to the map and use ES6 de-structuring to obtain salutations array

Step 4: Inject ‘DataMap’ in AppController and add a new /salutations endpoint to verify that it actually does what I intend

  constructor(private readonly appService: AppService, @Inject('DataMap') private dataMap: DataMap) {}

  @Get('salutations')
  async getSalutations(): Promise<Salutation> {
    const { salutations } = this.dataMap
    return salutations
  }

Last step: CURL /salutations and verify the timestamp does not change

All CURL responses have the same timestamp and Date.now() can only increase; therefore, /salutations endpoint returns the array from the cached map.

Final thoughts

In this post, I have shown one use case of useFactory of NestJS but useFactory can cache any resource, for example, database connection and instances of third-party libraries. In the future, I can write an article on how to create a custom provider for Node Stripe library in NestJS.

Resources:

  1. Repo: https://github.com/railsstudent/nest-poc-standard-version
  2. Custom Provider: https://docs.nestjs.com/fundamentals/custom-providers

End to end testing (e2e) in NestJS

 4 total views,  2 views today

This is part 3 of Use TypeOrm Custom Repository in NestJS and this post is about end- to-end testing (e2e) of NestJS endpoints.

In part one and part two, controllers, services and custom repositories are tested and they pass all the test cases. The final step is to validate HTTP requests between frontend and backend and the simulated application flows should always return the expected responses. This type of testing is called end-to-end testing because the application flow happens from the beginning to the end.

An analogy of e2e testing is my vacation to Canada. Whenever I visit Canada, my father requests to buy canned salmon, my old sister and I respond by driving to Costco warehouse to buy a pack of canned salmon and a few bags of Lays Ketchup chips. They are in my baggage and import to Hong Kong via cargo.

User API

In this post, I am going to do e2e testing on User API . The api is defined in user controller and consisted of four endpoints.

// user.controller.ts

// return all users
GET /user

// return user and the reports created by the user
GET /user/:id/report

// return user matching user id
GET /user/:id

// create new user
POST /user

Similar to unit tests, NestJS uses Jest and Supertest to write e2e test cases and make HTTP requests respectively, and validate status code and response.

Set up 1: Generate constants and mock data

Install uuid library that provides function to generate unique uuid.

npm install uuid
npm install --save @types/uuid
// constant.mock.ts

import { User } from '@/entities'
import { v4 } from 'uuid'

export const now = new Date()
export const userId = v4()
export const userId2 = v4()
export const reportId = v4()
export const newUserId = v4()
// user.mock.ts

import { CreateUserDto } from '@/user'
import { Report, User } from '@/entities'
import { newUserId, now, reportId, userId, userId2 } from './constant.mock'

// For brevity, delete initialization codes

export const allUsers: User[] = [
  {
    id: userId,
    name: 'John',
    lastname: 'Doe',
    age: 10,
    createdAt: now,
    updatedAt: now,
    version: 1,
    reports: [],
  },
  {
    id: userId2,
    name: 'Jane',
    lastname: 'Doe',
    age: 15,
    createdAt: now,
    updatedAt: now,
    version: 1,
    reports: [],
  },
]

export const expectedUser = {
  id: userId,
  name: 'John',
  lastname: 'Doe',
  age: 10,
  createdAt: now.toISOString(),
  updatedAt: now.toISOString(),
  version: 1,
  reports: [],
}

export const userService = {
  getUsers: () => allUsers,
  getUser: (id: string) => allUsers.find((user) => user.id === id),
  createUser: (dto: CreateUserDto) => {
    const newUser = {
      ...dto,
      id: newUserId,
      createdAt: now,
      updatedAt: now,
      version: 1,
      reports: [],
    }
    return newUser
  },
  getUserWithReports: (id: string) => {
    ...
  },
}

Set up 2: Create Testing Module and Initialize Test App

beforeEach(async () => {
    const moduleFixture: TestingModule = await Test.createTestingModule({
      controllers: [UserController],
      providers: [UserService],
    })
      .overrideProvider(UserService)
      .useValue(userService)
      .compile()

    app = moduleFixture.createNestApplication()
    app.useGlobalPipes(new ValidationPipe())
    await app.init()
})
.overrideProvider(UserService).useValue(userService)

replaces the real user service with mocked service

app.useGlobal(new ValidationPipe())

validates data transfer object (DTO) and throws exception when property fails the condition of class-validator decorators.

Example 1: Test cases to validate GET /user/:id

describe('/user/:id (GET)', () => {
   it('should return user', () => {
      return request(app.getHttpServer())
         .get(`/user/${userId}`)
         .expect(HttpStatus.OK)
         .expect(expectedUser)
    })

    it('should return undefined', () => {
      const id = v4()
      return request(app.getHttpServer())
          .get(`/user/${id}`)
          .expect(HttpStatus.OK)
          .expect({})
    })
}

The first test case validates that request(app.getHttpServer()).get(`/user/${userId}`) returns 200 status and calls getUser() to return user with matching user id.

The second test case validates that request(app.getHttpServer()).get(`/user/${userId}`) returns 200 status and calls getUser() to return undefined user.

Example 2: Negative test cases to validate POST /user

CreateUserDto requires non-empty first name, last name and non-negative age. If invalid DTO is sent in the request, the response should return 400 status. Lets write some test cases to validate the response of the requests

// create-user.dto.ts
import { IsNotEmpty, IsNumber, IsString, Min } from 'class-validator'

export class CreateUserDto {
  @IsNotEmpty()
  @IsString()
  name: string

  @IsNotEmpty()
  @IsString()
  lastname: string

  @IsNotEmpty()
  @Min(0)
  @IsNumber()
  age: number
}
describe('/user (POST)', () => {
    it('should throw error when age is negative', () => {
      const dto = {
        name: 'John',
        lastname: 'Doe',
        age: -1,
      }
      return request(app.getHttpServer())
          .post('/user')
          .send(dto)
          .expect(HttpStatus.BAD_REQUEST)
    })

    it('should throw error when name is blank', () => {
      const dto = {
        name: '',
        lastname: 'Doe',
        age: 62,
      }
      return request(app.getHttpServer())
          .post('/user')
          .send(dto)
          .expect(HttpStatus.BAD_REQUEST)
    })

})

Example 3: Positive test case to validate POST /user

describe('/user (POST)', () => {
    it('should create and return new user', () => {
      const dto = {
        name: 'John',
        lastname: 'Doe',
        age: 10,
      }
      return request(app.getHttpServer())
        .post('/user')
        .send(dto)
        .expect(HttpStatus.CREATED)
        .expect({
          ...dto,
          id: newUserId,
          createdAt: now.toISOString(),
          updatedAt: now.toISOString(),
          version: 1,
          reports: [],
        })
    })
})

When DTO is valid, POST /user is successful and it returns 201 status and a new user object. HTTP converts date time values of response to string; therefore, createAt and updatedAt become now.toISOString().

.expect(HttpStatus.CREATED)

returns 201 status

.expect({
    ...dto,
    id: newUserId,
    createdAt: now.toISOString(),
    updatedAt: now.toISOString(),
    version: 1,
    reports: [],
 })

The test case expects createUser() is called to return a new user object; it has new user id and data of DTO.

Final thoughts

E2e testing in NestJS is feasible when you understand the patterns of mocking and testing different HTTP methods (GET, POST, PATCH, DELETE). After grunt work is laid down, e2e test cases are described in several lines of codes. Happy path expects 200/201 status code and an object in response. Fail path expects 400-level or 500-level status code and an error message.

Test automation takes efforts initially but it can be used continuously to catch errors when API changes; therefore, it is a good investment for any developer and development team.

Resources:

  1. Repo: https://github.com/railsstudent/nest-poc-standard-version
  2. https://docs.nestjs.com/fundamentals/testing#end-to-end-testing

Unit Test Custom Repository in NestJS

 2 total views,  2 views today

This is part 2 of Use TypeOrm Custom Repository in NestJS and this post is about unit testing custom repository.

In part one, I created a NestJS demo with controllers, services and custom repositories. The demo also added unit testing on controllers and services in which I mocked the methods of custom repositories with jest.fn(). The missing piece is unit testing on custom repository and I will fill the hole in this post.

Jest

Jest is an open source JavaScript testing framework that can use with Angular and NestJS. I used Jest to test controllers and services in part 1 and it can also be used to test custom repository.

The functions of the custom repository query database to obtain result set through TypeORM. However, there is no database running in custom repository testing and calling any repository method can lead to error. The solution is to mock the TypeORM functions such that they don’t call the database but the fake implementations to obtain test results. This is where Jest comes in because jest.fn() and jest.Spy().mockImplementation() are designed to do it.

Step 1: Create a spec file for user repository

A spec file is a Javascript/TypeScript file that encapsulates testing codes to validate the functionality of a class. It is important to import the necessary modules and providers to TestingModule before writing the first test case.

// user-repository.spec.ts

describe('UserRepository', () => {
  let service: UserRepository
  let now: Date

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [UserRepository],
    }).compile()

    service = module.get<UserRepository>(UserRepository)
    now = new Date(Date.now())
  })

  it('should be defined', () => {
    expect(service).toBeDefined()
  })
})

Step 2: Write test cases to validate basic CRUD functions

Let remind the readers the implementation of UserRepository.

// user.repository.ts

@EntityRepository(User)
export class UserRepository extends Repository<User> {
  async getUsers(): Promise<User[]> {
    return this.find({
      order: {
        name: 'ASC',
        lastname: 'ASC',
      },
    })
  }

  async getUser(id: string): Promise<User | undefined> {
    return this.findOne(id)
  }

  async createUser(newUser: NewUser): Promise<User> {
    return this.save(this.create(newUser))
  }

  async getUserWithReports(id: string): Promise<User | undefined> {
    return this.createQueryBuilder('user')
      .leftJoinAndSelect('user.reports', 'reports')
      .where('user.id = :id', { id })
      .getOne()
  }
}

I consider getUsers, getUser and createUser basic CRUD functions because they are straightforward; calling TypeOrm functions directly to obtain results. getUserWithReports is complicated because it chains builder functions of TypeORM to compose a SQL query to obtain results from the database.

It also means less effort to mock straightforward TypeORM functions than builder functions.

Let’s write test case for getUsers function

describe('getUsers', () => {
    it('should find and return all users', async () => {
      const users: User[] = [
        { id: v4(), name: 'John', lastname: 'Doe', age: 10, createdAt: now, updatedAt: now, version: 1, reports: [] },
        { id: v4(), name: 'Jane', lastname: 'Doe', age: 22, createdAt: now, updatedAt: now, version: 1, reports: [] },
      ]
      jest.spyOn(service, 'find').mockImplementation(() => Promise.resolve(users))

      const result = await service.getUsers()
      expect(service.find).toBeCalled()
      expect(service.find).toBeCalledWith({
        order: {
          name: 'ASC',
          lastname: 'ASC',
        },
      })
      expect(result).toEqual(users)
    })
  })

Explanation:

  1. jest.spyOn(service, ‘find’).mockImplement() spies this.find() and provides a mock function that returns a promise that returns User array
  2. const result = await service.getUsers() returns the results of getUsers()
  3. expect(service.find).toBeCalled() ensures that this.find() is called within getUsers()
  4. expect(service.find).toBeCalledWith({ orders: { name: ‘ASC’, lastname: ‘ASC’ }}) ensures that this.find accepts expected argument
  5. expect(result).toEqual(users) ensures that the result is the same as the User array

I am not going to cover getUser() because it is similarly to testing getUsers()

Testing createUser() is similar to testing getUsers() but we have to mock two implementations: this.create and this.save

describe('createUser', () => {
    it('should create a new user', async () => {
      const id = v4()
      const newUser = {
        name: 'New',
        lastname: 'User',
        age: 21,
      }

      const user: User = {
        id,
        name: 'John',
        lastname: 'Doe',
        age: 10,
        createdAt: now,
        updatedAt: now,
        version: 1,
        reports: [],
      }
      jest.spyOn(service, 'create').mockImplementation(() => user)
      jest.spyOn(service, 'save').mockImplementation(() => Promise.resolve(user))

      const result = await service.createUser(newUser)
      expect(service.create).toBeCalled()
      expect(service.save).toBeCalled()
      expect(service.create).toBeCalledWith(newUser)
      expect(service.save).toBeCalledWith(service.create(newUser))
      expect(result).toEqual(user)
    })
})

Explanation:

  1. jest.spyOn(service, ‘create’).mockImplement() spies this.create() and provides a mock function that returns a new user
  2. jest.spyOn(service, ‘save’).mockImplement() spies this.save() and provides a mock function that returns a promise that returns the new user
  3. const result = await service.createUser(newUser) returns the result of createUser()
  4. expect(service.create).toBeCalled() ensures that this.create() is called within createUser()
  5. expect(service.save).toBeCalled() ensures that this.save() is called within createUser()
  6. expect(service.create).toBeCalledWith(newUser) ensures that this.create accepts expected argument
  7. expect(service.save).toBeCalledWith(service.create(newUser)) ensures that this.create is passed to this.save()
  8. expect(result).toEqual(user) ensures that the result is the same as the new user

Step 3: Write test case to validate getUserWithReports

// user.repository.ts

async getUserWithReports(id: string): Promise<User | undefined> {
    return this.createQueryBuilder('user')
      .leftJoinAndSelect('user.reports', 'reports')
      .where('user.id = :id', { id })
      .getOne()
}

Testing getUserWithReports is tricky because the builder functions are chained together and finally getOne() returns an entity to service. Unlike this.find, this.createQueryBuilder does not return entity but returning a class/object with a leftJoinAndSelect function. Similarly, leftJoinAndSelect returns a class/object with a where function and where returns a class/object with getOne function. Once you understand what they do, you can easily mock them with jest.fn().

describe('getUserWithReports', () => {
    it('should return a user with reports', async () => {
      // deleted codes that initialize other objects

      const user: User = {
        ...partialUser,
        reports: [report],
      }

      const getOne = jest.fn(() => user)
      const where = jest.fn(() => ({
        getOne,
      }))
      const leftJoinAndSelect = jest.fn(() => ({
        where,
      }))
      const createQueryBuilder = () =>
        ({
          leftJoinAndSelect,
        } as unknown as SelectQueryBuilder<User>)
      jest.spyOn(service, 'createQueryBuilder').mockImplementation(createQueryBuilder)

      const result = await service.getUserWithReports(id)
      expect(service.createQueryBuilder).toBeCalled()
      expect(service.createQueryBuilder).toBeCalledWith('user')
      expect(leftJoinAndSelect).toBeCalled()
      expect(leftJoinAndSelect).toBeCalledWith('user.reports', 'reports')
      expect(where).toBeCalled()
      expect(where).toBeCalledWith('user.id = :id', { id })
      expect(getOne).toBeCalled()
      expect(result).toEqual(user)
    })
  })

Explanation:

  1. const getOne = jest.fn(() => user) is a mock function that returns a user with reports
  2. const where = jest.fn(() => ({ getOne })) is a mock function that returns the getOne function
  3. const leftJoinAndSelect = jest.fn(() => ({ where })) is a mock function that returns the where function
  4. jest.spyOn(service, ‘createQueryBuilder’).mockImplement(createQueryBuilder) spies this.createQueryBuilder() and provides a mock function that returns the leftJoinAndSelect function
  5. const result = await service.getUserWithReports(id) returns the result of getUserWithReports()
  6. expect(service.createQueryBuilder).toBeCalled() ensures that this.createQueryBuilder() is called
  7. expect(service.createQueryBuilder).toBeCalledWith(‘report’) ensures that this.createQueryBuilder(‘report’) is called
  8. expect(leftJoinAndSelect).toBeCalled() ensures that .leftJoinAndSelect() is called
  9. expect(leftJoinAndSelect).toBeCalledWith(‘user.reports’, ‘reports’) ensures that .leftJoinAndSelect(‘user.reports’, ‘reports’) is called
  10. expect(where).toBeCalled() ensures that .where() is called
  11. expect(where).toBeCalledWith(‘user.id = :id’, { id }) ensures that .where(‘user.id = :id’, { id }) is called
  12. expect(getOne).toBeCalled() ensures that .getOne() is called
  13. expect(result).toEqual(user) ensures that result is the same as user

Final thoughts

Testing is tedious because it involves experience and long hours of practices for developers to mock APIs and functions to allow test cases to perform. Test cases should validate that correct results are received, expected functions are invoked and expected arguments are passed to the functions. Otherwise, developers do not have high confidence that the quality of the software is good before it is shipped to quality assurers for quality assurance.

I hope this article helps developers who use NestJS to build their backend applications and need help to unit test resources (controllers, services and custom repositories).

Resources:

  1. Repo: https://github.com/railsstudent/nest-poc-standard-version
  2. Clock-In/Out System Part 9: Back-End Testing — Unit Testing of Services