Introduction
In this blog post, I demonstrated generating replies with prompt chaining. In auction sites such as eBay, buyers can provide ratings and comments on sales transactions. When the feedback is negative, the seller has to provide a response promptly to resolve the dispute. This demo saves time by generating replies in the same language of the buyer according to the the tone (positive, neutral or negative) and topics. Previous prompts obtain answers from the Gemini models and they become the parameters of a new prompt. Similarly, the model receives the new prompt to generate the final reply to keep customers happy.
Generate Gemini API Key
Go to https://aistudio.google.com/app/apikey to generate an API key for a new or an existing Google Cloud project.
Create a new NestJS Project
nest new nestjs-customer-feedback
Install dependencies
npm i --save-exact @nestjs/swagger @nestjs/throttler dotenv compression helmet @google/generative-ai class-validator class-transformer
Generate a Feedback Module
nest g mo advisoryFeedback
nest g co advisoryFeedback/presenters/http/advisoryFeedback --flat
nest g s advisoryFeedback/application/advisoryFeedback --flat
nest g s advisoryFeedback/application/advisoryFeedbackPromptChainingService --flat
Create a AdvisoryFeedbackModule module, a controller, a service for the API and another service to build chained prompts.
Define Gemini environment variables
// .env.example
PORT=3000
GOOGLE_GEMINI_API_KEY=<google gemini api key>
GOOGLE_GEMINI_MODEL=gemini-1.5-pro-latest
Copy .env.example
to .env
, and replace GOOGLE_GEMINI_API_KEY
and GOOGLE_GEMINI_MODEL
with the actual API Key and the Gemini model, respectively.
- PORT – port number of the NestJS application
- GOOGLE_GEMINI_API_KEY – API Key of Gemini
- GOOGLE_GEMINI_MODEL – Google model and I used Gemini 1.5 Pro in this demo
Add .env
to the .gitignore
file to prevent accidentally committing the Gemini API Key to the GitHub repo.
Add configuration files
The project has 3 configuration files. validate.config.ts
validates the payload is valid before any request can route to the controller to execute
// validate.config.ts
import { ValidationPipe } from '@nestjs/common';
export const validateConfig = new ValidationPipe({
whitelist: true,
stopAtFirstError: true,
forbidUnknownValues: false,
});
env.config.ts
extracts the environment variables from process.env
and stores the values in the env
object.
// env.config.ts
import dotenv from 'dotenv';
dotenv.config();
export const env = {
PORT: parseInt(process.env.PORT || '3000'),
GEMINI: {
API_KEY: process.env.GOOGLE_GEMINI_API_KEY || '',
MODEL_NAME: process.env.GOOGLE_GEMINI_MODEL || 'gemini-pro',
},
};
throttler.config.ts
defines the rate limit of the Translation API
// throttler.config.ts
import { ThrottlerModule } from '@nestjs/throttler';
export const throttlerConfig = ThrottlerModule.forRoot([
{
ttl: 60000,
limit: 10,
},
]);
Each route allows ten requests in 60,000 milliseconds or 1 minute.
Bootstrap the application
// bootstrap.ts
export class Bootstrap {
private app: NestExpressApplication;
async initApp() {
this.app = await NestFactory.create(AppModule);
}
enableCors() {
this.app.enableCors();
}
setupMiddleware() {
this.app.use(express.json({ limit: '1000kb' }));
this.app.use(express.urlencoded({ extended: false }));
this.app.use(compression());
this.app.use(helmet());
}
setupGlobalPipe() {
this.app.useGlobalPipes(validateConfig);
}
async startApp() {
await this.app.listen(env.PORT);
}
setupSwagger() {
const config = new DocumentBuilder()
.setTitle('ESG Advisory Feedback with Prompt Chaining ')
.setDescription('Integrate with Gemini to improve ESG advisory feebacking by prompt chaining')
.setVersion('1.0')
.addTag('Gemini API, Gemini 1.5 Pro Model, Prompt Chaining')
.build();
const document = SwaggerModule.createDocument(this.app, config);
SwaggerModule.setup('api', this.app, document);
}
}
Added a Bootstrap
class to setup Swagger, middleware, global validation, CORS, and finally application start.
// main.ts
import { env } from '~configs/env.config';
import { Bootstrap } from '~core/bootstrap';
async function bootstrap() {
const bootstrap = new Bootstrap();
await bootstrap.initApp();
bootstrap.enableCors();
bootstrap.setupMiddleware();
bootstrap.setupGlobalPipe();
bootstrap.setupSwagger();
await bootstrap.startApp();
}
bootstrap()
.then(() => console.log(`The application starts successfully at port ${env.PORT}`))
.catch((error) => console.error(error));
The bootstrap function enabled CORS, registered middleware to the application, set up Swagger documentation, and used a global pipe to validate payloads.
I have laid down the groundwork and the next step is to add an endpoint to receive payload for generating replies with prompt chaining.
Define Feedback DTO
// feedback.dto.ts
import { IsNotEmpty, IsString } from 'class-validator';
export class FeedbackDto {
@IsString()
@IsNotEmpty()
prompt: string;
}
FeedbackDto
accepts a prompt that is the customer feedback.
Construct Gemini Models
// gemini.config.ts
import { GenerationConfig, HarmBlockThreshold, HarmCategory, SafetySetting } from '@google/generative-ai';
export const SAFETY_SETTINGS: SafetySetting[] = [
{
category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
},
{
category: HarmCategory.HARM_CATEGORY_HARASSMENT,
threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
},
{
category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
},
{
category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
},
];
export const GENERATION_CONFIG: GenerationConfig = {
temperature: 0.5,
topK: 10,
topP: 0.5,
maxOutputTokens: 2048,
};
// gemini.constant.ts
export const GEMINI_SENTIMENT_ANALYSIS_MODEL = 'GEMINI_SENTIMENT_ANALYSIS_MODEL';
export const GEMINI_REPLY_MODEL = 'GEMINI_REPLY_MODEL';
export const GEMINI_FIND_LANGUAGE_MODEL = 'GEMINI_FIND_LANGUAGE_MODEL';
// mode.factory.ts
import { GoogleGenerativeAI } from '@google/generative-ai';
import { env } from '~configs/env.config';
import { GENERATION_CONFIG, SAFETY_SETTINGS } from '../configs/genimi.config';
export function modelFactory(systemInstruction: string, toJson = false) {
const genAI = new GoogleGenerativeAI(env.GEMINI.API_KEY);
const generationConfig = toJson ? { ...GENERATION_CONFIG, responseMimeType: 'application/json' } : GENERATION_CONFIG;
return genAI.getGenerativeModel({
model: env.GEMINI.MODEL_NAME,
systemInstruction,
generationConfig,
safetySettings: SAFETY_SETTINGS,
});
}
// gemini-find-language.provider.ts
import { GenerativeModel } from '@google/generative-ai';
import { Provider } from '@nestjs/common';
import { GEMINI_FIND_LANGUAGE_MODEL } from '../constants/gemini.constant';
import { modelFactory } from './model-factory';
const FIND_LANGUAGE_SYSTEM_INSTRUCTION = `You are a multilingual expert that can identify the language used in this piece of text. Give me the language name, and nothing else.
If the text is written in Chinese, please differentiate Traditional Chinese and Simplified Chinese.
`;
export const GeminiFindLanguageProvider: Provider<GenerativeModel> = {
provide: GEMINI_FIND_LANGUAGE_MODEL,
useFactory: () => modelFactory(FIND_LANGUAGE_SYSTEM_INSTRUCTION),
};
Gemini 1.5 Pro model accepts system instruction; therefore, the system instruction gives the context to the model. GeminiFindLanguageProvider
is a model that can detect the written language. When the language is Chinese, it should distinguish Traditional Chinese or Simplified Chinese.
// sentiment-model.provider.ts
const SENTIMENT_ANALYSIS_SYSTEM_INSTRUCTION = `
You are a sentiment analysis assistant who can identify the sentiment and topic of feedback and return the JSON output { "sentiment": string, "topic": string }.
When the sentiment is positive, return 'POSITIVE', is neutral, return 'NEUTRAL', is negative, return 'NEGATIVE'.
`;
export const GeminiSentimentAnalysisProvider: Provider<GenerativeModel> = {
provide: GEMINI_SENTIMENT_ANALYSIS_MODEL,
useFactory: () => modelFactory(SENTIMENT_ANALYSIS_SYSTEM_INSTRUCTION, true),
};
GeminiSentimentAnalysisProvider
is a model that can identify the sentiment and topics of the feedback, and return the result in JSON. The JSON schema of the result is { sentiment: string; topic: string }
.
// advisory-feedback-model.provider.ts
const REPLY_SYSTEM_INSTRUCTION =
"You are a professional ESG advisor, please give a short reply to customer's response and in the same language.";
export const GeminiReplyProvider: Provider<GenerativeModel> = {
provide: GEMINI_REPLY_MODEL,
useFactory: () => modelFactory(REPLY_SYSTEM_INSTRUCTION),
};
GeminiReplyProvider
is a model that writes a short reply in the same language of the feedback.
Implement Reply Service
// sentiment-analysis.type.ts
export type SentimentAnalysis = {
sentiment: 'POSITIVE' | 'NEUTRAL' | 'NEGATIVE';
topic: string;
};
// advisory-feedback-prompt-chaining.service.ts
// Omit the import statements
@Injectable()
export class AdvisoryFeedbackPromptChainingService {
private readonly logger = new Logger(AdvisoryFeedbackPromptChainingService.name);
constructor(
@Inject(GEMINI_SENTIMENT_ANALYSIS_MODEL) private analysisModel: GenerativeModel,
@Inject(GEMINI_REPLY_MODEL) private replyModel: GenerativeModel,
@Inject(GEMINI_FIND_LANGUAGE_MODEL) private findLanguageModel: GenerativeModel,
) {}
async generateReply(prompt: string): Promise<string> {
try {
const [analysis, language] = await Promise.all([this.analyseSentinment(prompt), this.findLanguage(prompt)]);
const { sentiment, topic } = analysis;
const chainedPrompt = `
The customer wrote a ${sentiment} feedback about ${topic} in ${language}. Provided feedback: ${prompt}.
Feedback:
`;
this.logger.log(chainedPrompt);
const result = await this.replyModel.generateContent(chainedPrompt);
const response = await result.response;
const text = response.text();
this.logger.log(text);
return text;
} catch (ex) {
console.error(ex);
throw ex;
}
}
private async analyseSentinment(prompt: string): Promise<SentimentAnalysis> {
try {
const result = await this.analysisModel.generateContent(prompt);
const response = await result.response;
return JSON.parse(response.text()) as SentimentAnalysis;
} catch (ex) {
console.error(ex);
throw ex;
}
}
private async findLanguage(prompt: string): Promise<string> {
try {
const languageResult = await this.findLanguageModel.generateContent(prompt);
const languageResponse = await languageResult.response;
return languageResponse.text();
} catch (ex) {
console.error(ex);
throw ex;
}
}
}
AdvisoryFeedbackPromptChainingService injects three chat models in the constructor.
- findLanguageModel – A chat model for detecting the language of the feedback.
- analysisModel – A chat model declared for determining the sentiment (POSITIVE, NEUTRAL, NEGATIVE) and the topics of the feedback.
- replyModel – A chat model declared for generating replies with prompt chaining.
- findLanguage – a private method that uses the
findLanguageModel
to detect the language of the feedback. - analyseSentinment – a private method that uses the
analysisModel
to determine the sentiment and the topics of the feedback. - generateReply – this method uses the results of
findLanguage
andanalyseSentinment
to construct a chained prompt. Then, it uses thereplyModel
to generate replies in the same language based on sentiment and topics.
The process for generating replies with prompt chaining ended by producing the text output from generateReply
. The method asked questions iteratively and wrote a clear prompt for the LLM to draft a reply that was polite and addressed the need of the customer.
// advisory-feedback.service.ts
// Omit the import statements to save space
@Injectable()
export class AdvisoryFeedbackService {
constructor(private promptChainingService: AdvisoryFeedbackPromptChainingService) {}
generateReply(prompt: string): Promise<string> {
return this.promptChainingService.generateReply(prompt);
}
}
AdvisoryFeedbackService
injects AdvisoryFeedbackPromptChainingService
and derives chained prompts to ask chat models to generate a reply.
Implement Advisory Feedback Controller
// advisory-feedback.controller.ts
// Omit the import statements to save space
@Controller('esg-advisory-feedback')
export class AdvisoryFeedbackController {
constructor(private service: AdvisoryFeedbackService) {}
@Post()
generateReply(@Body() dto: FeedbackDto): Promise<string> {
return this.service.generateReply(dto.prompt);
}
}
The
injects AdvisoryFeedbackController
AdvisoryFeedbackService
using Gemini API and Gemini 1.5 Pro model. The endpoint invokes the method to generate a reply from the prompt.
- /esg-advisory-feedback – generate a reply from a prompt
Module Registration
The AdvisoryFeedbackModule
provides AdvisoryFeedbackPromptChainingService
, AdvisoryFeedbackService
, GeminiSentimentAnalysisProvider
, GeminiAdvisoryFeedbackProvider
and GeminiFindLanguageProvider
. The module has one controller that is AdvisoryFeedbackController
.
// advisory-feedback.module.ts
// Omit the import statements due to brevity reason
@Module({
controllers: [AdvisoryFeedbackController],
providers: [
AdvisoryFeedbackPromptChainingService,
AdvisoryFeedbackService,
GeminiSentimentAnalysisProvider,
GeminiReplyProvider,,
GeminiFindLanguageProvider,
],
})
export class AdvisoryFeedbackModule {}
Import AdvisoryFeedbackModule
into AppModule
.
// app.module.ts
@Module({
imports: [throttlerConfig, AdvisoryFeedbackModule],
controllers: [AppController],
providers: [
{
provide: APP_GUARD,
useClass: ThrottlerGuard,
},
],
})
export class AppModule {}
Test the endpoints
I can test the endpoints with cURL, Postman or Swagger documentation after launching the application.
npm run start:dev
The URL of the Swagger documentation is http://localhost:3000/api.
In cURL
curl --location 'http://localhost:3000/esg-advisory-feedback' \
--header 'Content-Type: application/json' \
--data '{
"prompt": "Looking ahead, the needs of our customers will increasingly be defined by sustainable choices. ESG reporting through diginex has brought us uniformity, transparency and direction. It provides us with a framework to be able to demonstrate to all stakeholders - customers, employees, and investors - what we are doing and to be open and transparent."
}'
Dockerize the application
// .dockerignore
.git
.gitignore
node_modules/
dist/
Dockerfile
.dockerignore
npm-debug.log
Create a .dockerignore
file for Docker to ignore some files and directories.
// Dockerfile
# Use an official Node.js runtime as the base image
FROM node:20-alpine
# Set the working directory in the container
WORKDIR /app
# Copy package.json and package-lock.json to the working directory
COPY package*.json ./
# Install the dependencies
RUN npm install
# Copy the rest of the application code to the working directory
COPY . .
# Expose a port (if your application listens on a specific port)
EXPOSE 3000
# Define the command to run your application
CMD [ "npm", "run", "start:dev"]
I added the Dockerfile
that installs the dependencies, builds the NestJS application, and starts it at port 3000.
// docker-compose.yaml
version: '3.8'
services:
backend:
build:
context: .
dockerfile: Dockerfile
environment:
- PORT=${PORT}
- GOOGLE_GEMINI_API_KEY=${GOOGLE_GEMINI_API_KEY}
- GOOGLE_GEMINI_MODEL=${GOOGLE_GEMINI_MODEL}
ports:
- "${PORT}:${PORT}"
networks:
- ai
restart: unless-stopped
networks:
ai:
I added the docker-compose.yaml in the current folder, which was responsible for creating the NestJS application container.
Launch the Docker application
docker-compose up
Navigate to http://localhost:3000/api to read and execute the API.
This concludes my blog post about using Gemini API and Gemini 1.5 Pro model to tackle generating replies regardless the written languages. Generating replies with prompt chaining reduces the efforts that a writer needs to compose a polite reply to any customer. I only scratched the surface of the capability of Gemini API and Gemini 1.5 Pro model because Gemini 1.5 Pro model can not only understand text inputs but also multimedia such as images and audios. I hope you like the content and continue to follow my learning experience in Angular, NestJS, Generative AI, and other technologies.
Resources:
- Github Repo: https://github.com/railsstudent/fullstack-genai-prompt-chaining-customer-feedback/tree/main/nestjs-customer-feedback
- Build with Gemini API: https://ai.google.dev/gemini-api/docs/get-started/tutorial?lang=node#generate-text-from-text-input
- Story writing with Prompt Chaining – https://github.com/google-gemini/cookbook/blob/main/examples/Story_Writing_with_Prompt_Chaining.ipynb