Building applications in the NPM development life cycle can be very complex. Let’s review the main factors that contribute to this complexity:

• An endless number of variables: Building applications involves a huge number of variables. Not only are there locally installed dependencies in the package.json (which should use absolute versions for each dependency), but there are also environment variables and global system settings.
• Different environments across multiple builds: It can become impossible, especially in larger projects, to maintain the same environment between builds. Even if the same host is used for both builds, local system changes between builds can have a dramatic impact on the published packages. In addition, having a master-slave continuous integration (CI) process further complicates this problem.
• Lack of trust in your binaries: When you pull an internal package in an application, you need to be confident that the binary you are using has been thoroughly tested. Otherwise, it will be the first thing you’ll be blamed for when their project fails.

In this blog post, I will show you the best practices to apply during the npm build process by using an npm registry to ensure that the same binary passes QA and makes it to the production repository.

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

Streamlining Your NPM QA-to-Release Pipeline

You need to have the following set up prior to creating your streamlined pipeline:

• A repository manager recommended for binary management.
• An automated build server, for example, Jenkins.
• Staging and release environments to be configured as site hosts similar to the production environment, where automated load and quality tests can be performed as per your QA team’s requirements.
• An npm project.

Stage 1: Software development

During the development process, run an automated QA cycle to ensure that the binary qualifies this stage.

Stage 2: Source code deployment

In this stage, complete the function and merge it to your Git branch. This triggers the Jenkins npm install to build the package.

Stage 3: CI server pulls dependencies

The CI server pulls the dependencies when you run ‘npm install’ command.

Note: To simplify the project, use a Binary Repository Manager to store the exact binary dependencies used in the project.

Stage 4: Deploy the Binaries to the target binary repository

Running the ‘npm deploy’ command causes the CI server to deploy the produced binaries to a target binary repository manager repository.

Note that at this stage, the binary is untested and is tagged as a “snapshot” build. As the package is untested, it should be moved to a non-production “snapshot” or “staging” repository where developers can use these packages at their own risk.

Stage 5: Run a second cycle of automated QA on the package

I recommend performing a second round of automated QA in addition to the development stage to ensure that package passes this stage.

If the package passes the above tests, it qualifies to enter the staging environment. The same package is “promoted” from snapshot mode to staging. After the app successfully meets the intended basic functionality, it can be deployed to the staging environment.

Stage 6: Deploy the package to a staging environment

This is typically where the development split occurs happens.

A common bad practice would be to rebuild the application after it passes QA, producing a new binary that differs from the original package.
I recommend using the same .tar.gz when promoting to later build steps.

Perform any QA scripts, which may differ depending on the project. The tests running in the staging environment should closely reflect the real environment. This can also include beta and manual testing.

Proceed to move the pulled binary “Staging” repository.

Stage 7: Deploy the package to a release repository

Once again, it is bad practice to rebuild the same package using the same source code but in a different environment during this phase.

The intended developers, who will eventually be using this package, need to be sure it is the same package that has passed the entire round of testing. Otherwise, you will be the one to blame if their application fails because of using this package.

Hurrah! Your npm registry is up and running.

The npm registry DevOps pipeline is now simplified by using the same binaries during the entire process. To conclude, these are the best practices that are recommended to follow when creating your npm registry:

• Ensure the exact same binary that was built makes it through the entire process.
• Instruct project developers to provide meaningful names to repositories. For example: “Snapshot” = Untested.
• Do not re-upload binaries that have been tested. Testing can modify the underlying binary and have unexpected changes.

Try creating your own npm registry using my customized projects as an example framework:

• An NPM project that pulls dependencies and builds a complete .tar.gz package
• A Jenkins CI pipeline script for building the application
• A Jenkins CI script to promote the package into a QA / staging repository

— Patrick Russell