How To Log NestJS Applications in a Distributed

System — Part 1: winston

Let us take a look into winston logging, integrate it in a NestJS
application and create centralized logging with Loki and Grafana

How To Log NestJS Applications in a Distributed System

Nowadays it is important to log your applications for better monitoring, debugging and
failure handling (for example, in a Kubernetes cluster), and to ensure centralized logging
for a better overview of a distributed system.

There are numerous applications and frameworks that make this possible. But in this series
I want to focus on NestJS, winston, and the Loki Stack combined with Grafana.
A Distributed System of Microservices
On many platforms we can see how a system of dependent microservices is developed to
satisfy the customer (e.g. Ebay TECH, Facebook). The biggest challenge is to ensure that all
microservices can reached each other and thus to provide fail-safety to keep the customer
happy. If a service is not available (e.g. network problems) or even failed, a responsible
person would have to get notified and the service would have to restart itself if necessary.
For this the applications need to be monitored. To read more about this, check out the guide
on How To Monitor a Distributed System with a NestJS Application

How To Monitor a Distributed System with a NestJS Application

Let us take a look into NestJS’ internal healthcheck module, and combine it
with a Prometheus client
medium.com

Besides application monitoring it is also important to log all information or exceptions of

applications in a centralized logging system to have a maximum overview on what
happens in a distributed system.

A Principle of Centralized Logging (Fluentbit and Loki)

There are many principles for how to create centralized logging in a distributed system. In
this series I want to focus on fluentbit and the Loki-Stack with Grafana. In Kubernetes
fluentbit is deployed as DaemonSet and runs on every node to monitor the log files, which
are stored in /var/log/containers/… . These logs are sent via events to our data storages
like Splunk, Elasticsearch, OpenSearch, Kafka, and more.
 Logging architecture

In this series all fluentbit logs get stored in Loki.

Loki is a horizontally scalable, highly available, multi-tenant log aggregation system

inspired by Prometheus. It is designed to be very cost effective and easy to operate. It does
not index the contents of the logs, but rather a set of labels for each log stream. [Loki
Homepage]
Some advantages of Loki:

It’s really easy to get started

100% persistence to object storage

Builds metrics and generates alerts

No ingestion log formatting requirements

Tails your logs in realtime

Natively integrates with Prometheus, Grafana, and K8s

For me, the main advantage is that it can be easily integrated into a Kubernetes cluster via
helm.

The Software Developer Part: Getting Started With Integration of Winston

Into a NestJS Application

winston is designed to be a simple and universal logging library with support for multiple
transports. A transport is essentially a storage device for your logs. Each winston logger
can have multiple transports (see: Transports) configured at different levels
(see: Logging levels). For example, one may want error logs to be stored in a persistent
remote location (like a database), but all logs output to the console or a local file.

NestJS has a performant logger out of the box.

Documentation | NestJS - A progressive Node.js framework

Nest comes with a built-in text-based logger which is used during
application bootstrapping and several other…
docs.nestjs.com
In addition, winston also logs information, errors, exceptions, and rejections as a file. Such
features are helpful in connection with centralised logging.

We always prefer decoupled and isolated implementations, so in general, we can also build
a simple initialisation function initWinston(…) for the winston logger. There are log levels,
formats, default meta, and different handlers, e.g. for exception or rejections, with storage
paths defined.
As a parameter, we enter a service/api name, which will be visible in the logs. This can then
be used as a Singleton in our application. A Singleton is more of an anti-pattern in
today’s generation. We could also build a NestJS service by creating the instance of
the winston logger in the constructor and making the log function available via
Dependency Injection in each individual NestJS module.

Logs by winston

We can create the instance of the logger at the top in the bootstrap function:

async function bootstrap() {

initWinston(loadApiConfiguration().apiTitle);
...
}

In the end, we can log various things everywhere in our application. And the logs are also
saved as file in the environment.

Here is an example for a controller:

Conclusion
In this series I show how to create a centralized logging system with NestJS, fluentbit and
the Loki Stack. In the first part I described the background of the architecture with fluentbit
and Loki, and introduced the application logging with winston.

In the next part I will show you how to create a local Kubernetes cluster with kind, and how
to deploy the Loki Stack via Helm charts in it. Also, I show the use case for
a Sidecar container to collect custom logs.

How To Log NestJS Applications in a Distributed System with Loki

Stack — Part 2
Collecting Logs with fluenbit and show them in Grafana
medium.com

Thanks for reading! Follow me on Medium, or Twitter, or subscribe here on

Medium to read more about DevOps, Agile & Development Principles, Angular, and other
useful stuff. Happy Coding! :)

Github Project
You can find the practical part of this series on GitHub.

GitHub - Itchimonji/nestjs-logging-tracing: Example project to show logging

and tracing in a…
An example how to log and trace transactions in a microservice environment build
with nestjs applications An example…
github.com