After we discussed the advantages and challenges of microservices in monorepos in Part 1, now we will look at how Nx supports this structure for AWS CDK-based applications.
What is Nx?
Nx is a JavaScript-based build system for monorepos. It enables the efficient execution of all tasks such as build and test across multiple projects in a monorepo, regardless of whether NPM, Yarn, or PNPM is used as the package manager. Nx also offers a fully integrated mode that does not require separate package.json files for each project. This enables deeper integration and is particularly interesting for UI applications with AngularJS or React.
Let’s start with a simple example. If we want to run the build script for service A, we can do this as follows:
npx nx build service-a
Now in our example repository (see Figure 1, Part 1), there are also services B and C. If we want to execute the build script for all projects in the repo, we proceed as follows:
npx nx run-many -t build
It is interesting to see that this works faster than if we were to do it individually. This is because Nx runs them in parallel, if our services are designed to be independent of each other at build time (which they also should be if they are well designed).
The advantage of Nx is particularly evident in its interaction with the version management of the Git repository. It is possible to execute only the target scripts of the projects and their dependencies for which changes were detected in the comparison between head and base. Let’s assume there are only changes in service B in a branch for a pull request to the base branch “dev”. In the following example, the build script would only be executed for service B. The scripts for the other project are not executed.
npx nx affected -t build
Now let’s return to the example repository from Part 1. The package.json of each service contains the corresponding CDK deploy script. For service A, it looks like this in simplified form:
"name": "service-a ",
"version": "0.1.0",
"scripts": {
"deploy": "cdk deploy ServiceAStack …",
},
"dependencies": {
"custom-lib": "*"
}In addition, we have a dependency on a user-defined library that is used by all three services. Suppose that we make a change to this library and then execute the following command:
npx nx affected -t deploy
In this case, the deployment scripts are executed for all three services. This happens because the library has changed and therefore all three services are indirectly affected. Nx therefore considers the dependency graph between the individual projects within the monorepo. Nx offers a useful visualization of all modules contained in a repository and their dependencies to help keep track of everything. The command
npx nx graph
is used to start a webapp locally, which can be utilized to examine the module structures and their dependencies in a browser.
All of this together is very powerful when you consider that corresponding target scripts can be defined in the package.json for practically every task in the projects, such as build, unit tests, code style checks, integration tests, audit, deployment, and much more. Consistent naming of these scripts across projects means that separate automated workflows (e.g. with GitHub Actions) can be provided for each of these tasks. These are universally valid to the extent that they do not even need to be adapted when additional projects, such as services or libraries, are added. This gives us an effective way of dealing with some potential problems of a monolithic CI/CD process.
Nx and AWS CDK: Do they go together?
AWS CDK provides us with a framework for infrastructure as code that enables the entire serverless infrastructure to be defined in TypeScript. When structuring, it is recommended to combine both the infrastructure code and the business code in one application. This means that each service will become a separate CDK application with its own stacks.
Nx enables the simple organization of AWS CDK applications in separate packages. This interaction allows for clear and well-organized development of cloud applications, with AWS CDK efficiently handling the infrastructure aspects and Nx providing the flexibility to manage the different parts of the application in a monorepo.
For our example repository, a greatly reduced version would look like as follows:
monorepo/
├── apps/
│ ├── service-a/
│ │ ├── bin/
│ │ │ └── service-a-app.ts
│ │ ├── lib/
│ │ │ └── service-a-stack.ts
│ │ ├── cdk.json
│ │ └── package.json
│ ├── service-b/
│ │ └── …
│ ├── service-c/
│ │ └── …
│ └── ui/
│ └── …
├── libs/
│ └── custom-lib/
│ ├── index.ts
│ └── package.json
├── nx.json
└── package.jsonIn this structure, there are two different workspaces at the top level. One for all applications (service a, service b, service c, and ui) below the apps folder. Each application follows the recommended structure for an individual CDK application. The second workspace libs contains the shared library custom-lib with its own structure and package.json. The nx.json file serves to configure Nx and contains only the default settings for the entire monorepo.
This structure can be expanded as required with additional services, libraries, and the entire workspaces, by simply adding new packages
CI/CD made easy
In the previous structure, we defined the monorepo for an architecture of multiple cloud-native microservices that exist as separate CDK applications. Nx enables efficient management of these applications.
However, in reality, it is often not enough to execute a deployment script efficiently for selected services only. A common approach is to create the individual CDK stacks of the applications via an AWS CodePipeline and deploy them to the desired target accounts of the various stages. This approach is compatible with the monorepo approach, but will mean that a separate pipeline has to be managed for each service application. This procedure is similar to a multi-repo approach, and the workload increases with the number of services.
An alternative option is to build a single pipeline that creates, tests, and deploys all stacks of all services. However, there is a risk of a monolithic, time-consuming CI/CD process, as described in Part 1. And in addition, the advantages of Nx are lost because AWS CodePipeline does not yet offer any integration for this.
Figure 2: Monorepo CI/CD
So at this point, we would like to consider another option, which is shown in Figure 2. With this approach, we attempt to combine the options outlined above. During the development in particular, we benefit greatly from the monorepo approach in combination with Nx and can automate many of the development steps. As we use GitHub as repository, many tasks for our monorepo can be implemented as GitHub Actions, including the deployment of individual service CDK stacks into an AWS dev account. All of this is based on the Nx affected feature and therefore enables a very automated and efficient development environment.
To deploy the entire application to the other required stages (QA, STG, PROD), we also set up another pipeline project in the monorepo that connects all the necessary stacks and configures the target accounts to which they are to be deployed, depending on the stage. Here, atomic and native provision within the AWS cosmos via an AWS CodePipeline is more important to us than efficiency.
Conclusion
Our analysis has shown that the development process of several microservices in monorepos with Nx can also be very efficient for CDK applications. In particular, individual teams benefit from the clear advantages of simplified dependency management, easier collaboration, and the ability to easily perform extensive refactorings, which makes monorepos an attractive option. The success of cross-team projects depends greatly on how well the teams can work together. Effective coordination on common guidelines and patterns is crucial.
Despite the obvious advantages of monorepos, the design of CI/CD processes remains a challenge. The clever use of suitable tools, however, can create clear, streamlined processes. The monorepo approach in combination with the right tools can offer a promising option for efficient development and provision of cloud-native microservices. It is important to maximize the advantages and address potential challenges in a targeted manner.
This post was written by:
