Orchestrating Cloud Functions by NestJS Events through Pub/Sub Triggers.

In this blog we will see how to trigger cloud functions from google Pub/Sub service within a Nest.js applications.

Introduction:

By combining Pub/Sub, Cloud Functions, and NestJS, you can build highly scalable and modular applications that react to events in an asynchronous and decoupled manner. This architecture enhances the overall performance, maintainability, and scalability of your application.

Embrace the power of Pub/Sub and Cloud Functions in your NestJS applications to unlock the potential of event-driven architecture and build robust, scalable, and responsive systems.

Pub/Sub is a powerful messaging service provided by Google Cloud Platform (GCP) that enables decoupling and asynchronous communication between different components of an application. By leveraging Pub/Sub and Cloud Functions together, we can build scalable and event-driven systems. Let’s dive into the steps required to achieve this integration in a NestJS application. Learn more about it: https://cloud.google.com/pubsub/docs

Cloud Functions is a server-less compute platform provided by Google Cloud Platform (GCP). It allows you to run your code in a fully managed environment without the need to provision or manage servers. Cloud Functions lets you focus on writing the code for your application logic while GCP handles the infrastructure, scaling, and operational aspects. Cloud Functions can be triggered by various events, such as HTTP requests, Pub/Sub messages, Cloud Storage events, or Firestore changes. When a function is triggered, GCP automatically provisions the necessary resources, executes the function, and scales it based on the incoming workload. Learn more about it: https://cloud.google.com/functions/docs

NestJS is a powerful Node.js framework built with TypeScript that combines elements of object-oriented programming, functional programming, and reactive programming. It provides a robust architecture for building scalable and maintainable server-side applications. One of the key features of NestJS is its support for event-driven architecture through the use of events and event handlers. Events allow different components of an application to communicate with each other asynchronously and decoupled. This enables the creation of highly modular and scalable systems.

In NestJS, events are typically implemented using libraries like `@nestjs/event-emitter` or `rxjs`. These libraries provide mechanisms for emitting events and defining event handlers. Events can be used to trigger specific actions or notify other parts of the application when certain events occur, such as user registration, data updates, or system events. By leveraging events in NestJS, you can easily establish communication between different modules, services, and controllers. This promotes loose coupling, improves code maintainability, and allows for better scalability as components can react to events in an independent and asynchronous manner. Documentation URL: https://docs.nestjs.com/ .

Getting Started:

1. Set up a GCP Account:
   i. Go to the GCP Console (https://console.cloud.google.com) and click on “Get started for free” or “Try Free”.
   ii. Follow the instructions to create a new GCP account or sign in with an existing Google account.
2. Create a Pub/Sub Topic:
   i. In the GCP Console, navigate to the Pub/Sub section.
   ii. Click on “Create a topic” and provide a name for your topic.
   iii. Optionally, you can configure settings like message retention duration or encryption.
3. Create a Pub/Sub Subscription:
   i. Within your topic, click on “Create subscription”.
   ii. Specify a name for the subscription and select the topic you created earlier.
   iii. Configure the subscription settings, such as delivery type (pull or push) and acknowledgment deadline.
4. Set up a Cloud Function:
   i. In the GCP Console, navigate to the Cloud Functions section.
   ii. Click on “Create function”.
   iii. Provide a name for your function and choose the runtime environment (Node.js, Python, etc.).
   iv. Specify the Pub/Sub topic and subscription to trigger the function.
   v. Write the code for your Cloud Function that will be executed when a message is published to the topic.
5. Deploy and Test the Cloud Function:
   i. Configure any additional settings like memory allocation or timeouts for your function.
   ii. Click on “Deploy” or “Create” to deploy your Cloud Function.
   iii. Test your function by publishing a message to the Pub/Sub topic and verify that the function is triggered and executed correctly.

Setting up Nest.js :

To call a Cloud Function using Pub/Sub in a NestJS application, you can follow these steps:

1. Set up a Pub/Sub client library in your NestJS project by installing the required dependencies. Run the following command to install the @google-cloud/pubsub package:

npm install @google-cloud/pubsub

2. Create a new Pub/Sub service that will handle the communication with. Pub/Sub. In your NestJS project, create a new file pubsub.service.ts and add the following code:

import { Injectable } from '@nestjs/common';
import { PubSub } from '@google-cloud/pubsub';

@Injectable()
export class PubSubService {
  private pubSubClient: PubSub;

  constructor() {
    this.pubSubClient = new PubSub();
  }

  async publishMessage(topicName: string, data: any): Promise<void> {
    const topic = this.pubSubClient.topic(topicName);
    const dataBuffer = Buffer.from(JSON.stringify(data));

    try {
      await topic.publish(dataBuffer);
      console.log(`Message published to topic ${topicName}`);
    } catch (error) {
      console.error('Error publishing message:', error);
    }
  }
}

This service sets up the Pub/Sub client and provides a method publishMessage that publishes a message to a specific topic.

3. Setup the app.module.ts to subscribe with the trigger-cloud-function.listener.ts which will listen as event will be emitted and also PubSubService also provided to AppService

import { TriggerCloudFunctionService } from './listener/trigger-cloud-function.listener';
import { PubSubService } from 'src/pubsub/pubsub.service';

@Module({
 imports: [],
 controllers: [AppController],
 providers: [
  AppService,
  TriggerCloudFunctionService,
  PubSubService
 ],
 exports: [AppModule],
})
export class AppModule {}

4. An AppService class in NestJS that emits the triggerCloudFunction event using the EventEmitter2 from @nestjs/event-emitter module. When the Operate method is called, it performs some operation on the request and creates a response payload. Then, it emits the triggerCloudFunction event asynchronously with the response payload.

To trigger a Cloud Function when the triggerCloudFunction event is emitted, you need to set up a listener for this event in your NestJS application. Here's an example of how you can do that:

import { EventEmitter2 } from '@nestjs/event-emitter';

@Injectable()
export class AppService {
 constructor(
  private readonly eventEmitter: EventEmitter2,
 ) {}

 async Operate(request): Promise<String> {
  // Do operation on request and create response payload
  response_payload={ message: "Hello from nestjs" }
  this.eventEmitter.emitAsync(
   'triggerCloudFunction',
    response_payload
  );
  return "ok";
 }
}

5. As Event is emitted, it will be listens for the triggerCloudFunction event using the @nestjs/event-emitter module. When this event is emitted, the triggerCloudFunction method is called, and it attempts to trigger a Cloud Function by publishing a message to a Pub/Sub topic using the pubSubService.

import { OnEvent } from "@nestjs/event-emitter";
import { Injectable, Inject } from "@nestjs/common";
import { PubSubService } from "src/pubsub/pubsub.service";

@Injectable()
export class TriggerCloudFunctionService {
    constructor(private readonly pubSubService: PubSubService) { }

    @OnEvent('triggerCloudFunction', { promisify: true, async: true })
    async triggerCloudFunction(event_payload: eventPayload) {
        
        try {
            const topicName = 'your-topic-name';
            const res = this.pubSubService.publishMessage(topicName, event_payload);
            
        } catch (error) {
            console.log(error);
        }
    }
}

Remember to refer to the official documentation for detailed information, examples, and best practices when implementing Pub/Sub and Cloud Functions in your NestJS application.

Happy coding and building event-driven applications with NestJS, Pub/Sub, and Cloud Functions!