Skip to content

Year: 2021

In-Memory Service Integration Tests

One concept I always use when writing applications or libraries is to test them using fast, reliable and automated integration tests.

The Test Trophy

You have probably heard about the test pyramid before, but if you haven’t heard about the test trophy, you should read this article.

Coming from the .NET world I have come to rely on real time code analyzers such as Roslyn and ReSharper, they help me write cleaner code faster and with little effort. It’s great having continuous feedback on code being written more or less instantly.

For similar reasons I use NCrunch for continuous test execution. I want instant feedback on the functionality I write. This means I need tests that are isolated and stable, run fast and continuously and test business functionality as realistically as possible.

Integration Tests

While unit tests are valuable when writing small isolated pieces of functionality, they are often too isolated to verify the bigger picture, hence the need for integration tests.

In order for integration tests to give near real time feedback they need to work similar to unit tests yet test real business scenarios end-to-end using stable integration points that do not break when refactoring. This is where in-memory, service integration tests come in.

  • In-memory – because it’s fast and can run anywhere.
  • Service – because it exposes stable APIs that don’t break compatibility.
  • Integrations – because they also use stable API’s and are pluggable.

These tests encapsulate an in-memory environment where the service is deployed to a host that acts similar to a real host through which the test can invoke the service. Third party in-memory representations of integrating services are used to mimic dependencies and observe the service’s behavior. The test shortcuts the service’s I/O integrations preferably as close to the network level as possible and redirects traffic back to itself for instrumentation and assertions.

An example of an integration test can be found here. It tests a web server that provides port forwarding functionality to Kubernetes, similar to kubectl port-forward.

Examples of in-memory versions of different components are Kafka.TestFramework, Test.It.With.AMQP, Entity Framework Effort as well as the AspNetCore TestServer.

So how fast do these integration tests run? Let’s churn.

Fast enough to run continuously!

JIT Lag

It is worth mentioning that most test runners do not run each test in a completely isolated application domain because of the overhead caused by the JIT compiler when loading assemblies. Therefor it is important to design services to be instantiable and not have mutating static properties as these would be shared when running tests in parallel. Reusing the same application domain for test executing when running multiple tests simultaneously is a trade-off that increases performance considerably.

Summary

Running integration tests continuously while writing code enables very fast feedback loops. These tests are however not a replacement to post-deployment end-to-end tests which test application functionality on production similar infrastructure.

Its main purpose is to give fast feedback continuously on business functionality during development by mimicking a production deployed instance of the service with minimal mocking while using well known, loosely coupled integration points for refactor stability.

Leave a Comment

Mitigating Infrastructure Drift by using Software Development Principals – Part 2

If you haven’t read the first part on how to mitigate infrastructure drift using software development principals you should! I will refer to parts mentioned in that post during this second, hands-on part.

Work Process

Let’s start by exploring how a simple work process might look like during software development.

 I have included post deployment testing because it is important but I’m not going to talk about how to orchestrate it within deployments, it is out of scope for this article.

Let’s add source control, automation and feedback loops. Traditionally there might be multiple different automation tools responsible for the different parts.

Say hi to trunk based development!

When the team grows, the capability of building functionality in parallel grows. This is where the process complexity also grows. The simplest way of doing parallel work is to duplicate the process and let it run side by side. However, it makes little sense developing functionality within a domain if we cannot use it all together, we need to integrate.

Remember that automation is key for moving fast and the goal is to deploy to production as fast as possible with as high quality as possible. The build and test processes increases quality while automation enables fast feedback within all of the processes pictured.

Integration introduces a risk of breaking functionality, therefor we’d like to integrate as small changes as possible as fast as possible to not loose momentum moving towards production.

Let’s expand trunk based development with a gated check-in strategy using pull requests.

With automation and repeating this process many times a day we now have continuous integration.

A great source control platform that can enable this workflow is for example GitHub.

GitOps

In the previous article I talked about the power of GitOps, where we manage processes and configuration declaratively with source control. By using GitHub Actions we define this process declaratively using YAML and source control it together with the application code. By introducing a simple branching strategy to isolate changes during the parallel and iterative development cycle we also isolate the definition for building and testing.

But why stop there? Git commits are immutable and signed with a checksum and it’s history is a directed acyclic graph of all changes. That means that any thing stored in Git has strong integrity. What if we can treat each commit as a release. Three in one! Source control, quality checks and release management.

This is called continuous delivery.

Infrastructure as Code

Let’s see if a similar automation process can be used for managing declarative cloud infrastructure.

Almost the same. Deployment has been moved into the process enabling continuous deployment. Deployment becomes tightly coupled to a branch, let’s explore this further.

Environments

Traditionally deployments are decoupled from the development process because there is a limitation on the amount of environments to deploy to, causing environments to become a bottle neck for delivery. Deploying to the local machine might also be a completely different process, further complicating and deviating the development process. It would make sense having an environment for every development cycle to further expand on the simplicity of trunk based development. One environment for every branch no matter what the purpose of the environment is.

Using cloud infrastructure we can do that by simply moving the environments to the cloud!

Since each branch represents an environment, it makes working with environments simple.

  • Need a new environment? Create a branch!
  • Switch environment? Switch branch!
  • Delete an environment? Delete the branch!
  • Upgrade an environment? Merge changes from another branch!
  • Downgrade an environment? git reset!

Cost

A common concern with infrastructure and environments is cost. Continuously observing how cloud infrastructure related costs changes over time becomes even more important when all environments are in the cloud since more of the cost becomes related to resource utilization. Most cloud providers have tools available for tracking and alerting on cost fluctuations and since all environments are built the same the tools can be used the same way for all environments. This also enabled observing how cost changes faster and doing something about it even earlier in the development process.

If development environment costs do become too steep, they usually do not need the same amount of resources that exist in a production environment. For performance related development it might still be relevant, but in all other cases lowering cost is quite easy to achieve by lowering the resource tiers used and using auto-scaling as a built in strategy. The latter also lowers cost and increases efficiency for production environments by maximizing resource utilization.

In comparison, how much does building and maintaining local infrastructure for each employee cost? How much does it cost to set up a new on-prem environment, local or shared?

Example with Terraform

There are different tools that can help build cloud infrastructure. We’re going to use Terraform and the HashiCorp Configuration Language as the declarative language.

Let’s start by defining how to build, test and deploy. Here’s a simple GitHub Action workflow that automatically builds infrastructure using the previously mentioned workflow:

name: Build Environment

on: push

jobs:
  build:
    name: Build Environment
    runs-on: ubuntu-latest

    env:
      branch: ${{ github.ref }}

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Creating Environment Variable for Terraform
        run: |
          branch=${{ env.branch }}
          branch_name=${branch#refs/heads/}
          env=${branch_name////-}
          env=${env//_/-}

          cat << EOF > env.auto.tfvars
          env = "$env"
          EOF

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v1

      - name: Terraform Init
        run: terraform init 
        
      - name: Terraform Validate
        run: terraform validate

      - name: Terraform Plan
        id: plan
        run: terraform plan -out=terraform.tfplan -var-file=config.tfvars

      - name: Terraform Plan Status
        if: steps.plan.outcome == 'failure'
        run: exit 1

      - name: Terraform Apply
        run: terraform apply -auto-approve terraform.tfplan

Building could be translated to initiating, validating and planning in Terraform.

When initiating, Terraform initializes the working directory by setting up backend state storage and loading referenced modules. Since many persons might work against the same environment at the same time, it is a good idea to share the Terraform state for the environment module by setting up remote backend state storage that can be locked in order to guarantee consistency. Each environment should have it’s own tracked state which means it needs to be stored and referenced explicitly per environment. This can be done using a partial backend configuration.

To validates that the configuration files are syntactically valid, terraform validate is executed.

Running terraform plan creates an execution plan containing the resources that needs to be created, updated or deleted by comparing the current state with the current configuration files and the objects already created previously. The env.auto.tfvars configuration file created in the second step contains the environment name based on the branch name and can be used to create environment specific resources by naming conventions.

Last step is to apply/deploy the execution plan modifying the targeted resources.

Application Platforms

The infrastructure architecture we have explored so far is quite simple and mostly suited for one-to-one matches between infrastructure and application. This might work well for managed services or serverless but if you need more control you might choose an application platform like Kubernetes.

A service does not only comprise by a binary, it needs a host, security, network, telemetry, certificates, load balancing etc. which quite fast increases overhead and complexity. Even though such platforms would fit single application needs, it does become unnecessary complex operating and orchestrating an application platform per application.

Let’s have a look how Azure Kubernetes Service could be configured to host our applications:

An application platform like Kubernetes works like a second infrastructure layer on top of the cloud infrastructure. It does help simplify operating complex distributed systems and systems architected as microservices, especially when running as a managed service. Kubernetes infrastructure abstractions makes it easier for applications to provision and orchestrate functionality while preserving much of their independency. The applications do also still control provisioning of infrastructure specific for their needs outside Kubernetes.

Provisioning Kubernetes and all it’s cloud specific integrations has become separated from application specific infrastructure provisioning. Application infrastructure on the other hand has taken a dependency on the Kubernetes API alongside cloud resource APIs. The environment has become tiered and partitioned.

I do again want to emphasize the importance of having autonomous, independent and loosely coupled cross-functional teams. Each team should own their own infrastructure for the same reason they own their applications. A Kubernetes cluster should not become a central infrastructure platform that the whole company depends on.

Separated Application Workflow

Since the Kubernetes infrastructure has been decoupled from the application workflow it could make sense moving the applications to separate repositories. As they have become autonomous we need to figure out a simple way to reintegrate the applications downstream. Since branches defines the environment of the application platform infrastructure, we could simply go for a similar branch naming standard, i.e. MyEnvironment/SomethingBeingDeveloped.

Looking at the Kubernetes platform architecture, the GitOps continuous delivery tool ArgoCD is responsible for deploying applications in the cluster. The process for delivering applications becomes similar to the GitOps process described earlier, where deployment becomes a reversed dependency. Instead of deploying to a specific environment after releasing, ArgoCD is instructed to observe for new releases in an application’s repository and if the release matches the strategy, it becomes deployed. This means that many ArgoCD instances can monitor and independently deploy many applications across many k8s clusters without any intervention.

Here is the process again:

We still provision application specific infrastructure, that works the same way as described earlier, except we now have an upstream dependency; the Kubernetes cluster(s) in the targeted environment. To keep the applications separated in Kubernetes, we use separate namespaces. This is also where we share platform defined details, for example where to find the environment’s container registry. We can do this by creating ConfigMaps.

Namespaces can also be used to limit resource access for users and service principals minimizing exposed attack surfaces. Access rights for each application can be defined in the upstream infrastructure repository. Since we use a managed Kubernetes service, which has integration with active directory, we can leverage access to both Kubernetes and cloud infrastructure through managed identities.

Tying it all together with a GitHub Action workflow, it could look something like this:

name: Build and Release Application

on: push

jobs:
  build:
    runs-on: ubuntu-latest

    env:
      branch: ${{ github.ref }}
      version: 1.2.3

    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Extracting Environment From Branch Name
        run: |
          branch=${{ env.branch }}
          branch_name=${branch#refs/heads/}
          env=${branch_name%/*}
          echo "env=$env" >> $GITHUB_ENV

      - name: Login to Azure
        uses: azure/login@v1
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}

      - name: Set AKS Context
        uses: azure/aks-set-context@v1
        with:
          creds: '${{ secrets.AZURE_CREDENTIALS }}'
          cluster-name: my-cluster
          resource-group: rg-${{ env.env }}

      - name: Fetch Environment Metadata
        run: |
          ENVIRONMENT_METADATA=$(kubectl get configmap/metadata -o go-template={{index .data "metadata.yaml"}} | docker run -i --rm mikefarah/yq eval -j)
          ACR_NAME=$(echo "$ENVIRONMENT_METADATA" | jq .acr.name | xargs)
          echo "ACR_NAME=$ACR_NAME" >> $GITHUB_ENV
          ACR_RESOURCE_GROUP_NAME=$(echo "$ENVIRONMENT_METADATA" | jq .acr.resourceGroupName | xargs)
          echo "ACR_RESOURCE_GROUP_NAME=$ACR_RESOURCE_GROUP_NAME" >> $GITHUB_ENV

      # Here could any application specific infrastructure be applied with Terraform
      # ...

      - name: Build and Push Application Container Images
        run : |
          az acr build --registry ${{ env.ACR_NAME }} --resource-group ${{ env.ACR_RESOURCE_GROUP_NAME }} --file path/to/Dockerfile --image my-application:latest --image my-application:${{ env.version }} .

      - name: Update Deployment Strategy (optional)
        run: |
          read -r -d '' helmvalues <<- YAML
          image_tag: ${{ env.version }}
          YAML

          cat application.yaml | \
            docker run -e HELMVALUES="$helmvalue" -i --rm mikefarah/yq eval '(.spec.source.helm.values=strenv(HELMVALUES) | (.spec.source.targetRevision="refs/heads/${{ env.branch }}")' - | \
            tee application.yaml
          
          kubectl apply -f application.yaml -n argocd

What about running and debugging an application during development? Use Bridge to Kubernetes straight from your IDE!

Shared Components

An important strategy when building environments (and applications) is that they need to be autonomous. However some components might need to be shared or at least moved upstream, like the backend state storage and permissions to create and manage components on the cloud platform.

These components should live in separate repositories with similar development strategies.

Permissions

Defining different roles is good practice in order to align with the least privilege principal. Depending on the size of the company, persons might have multiple roles, but remember that autonomous, cross-functional teams are important to move fast, so each team should have all the roles needed to deliver their applications.

Lessons Learned

One important thing about mitigating drift between environments is to continuously integrate between them. In an ideal world, each change is directly integrated into all environments. However, in reality, that will not always happen, which might cause incompatibility issues when introducing changes to environments. Upgrading from A to B to C is not the same thing as upgrading directly from A to C. That is usually what happens when a branch is merged into another branch, and with cloud infrastructure this might lead to unexpected problems. An example is that no minor versions can be skipped when upgrading Azure Kubernetes Service.

Can I skip multiple AKS versions during cluster upgrade?

When you upgrade a supported AKS cluster, Kubernetes minor versions cannot be skipped.

This means that Terraform needs to apply each commit in order. This can be a bit cumbersome to orchestrate.

Another important aspect with infrastructure as code is that it is quite easy to render yourself into an inconsistent state if manually changing configurations, which can be tempting in pressing situations and with the cloud platform tools easily available. Don’t fall in that trap.

Conclusions

Working cross-functional is powerful. It enables teams to work autonomous, end-to-end within a loosely coupled piece of domain. It also enables teams to use development workflows that includes both applications and infrastructure from start simplifying how to make them work efficiently together. By using the same infrastructure for all environments, continuously merging changes downstream, it can help mitigate drift, while simplifying managing infrastructure changes.

Leave a Comment

Mitigating Infrastructure Drift by using Software Development Principals

According to this survey from driftctl, 96% of the teams asked reports manual changes being the main cause for infrastructure drift. Other concerns are not moving from development to production fast enough and introducing many changes at once.

Drift is a big problem in system development. Test environments get broken and halts the whole development process, or they are in an unknown state.

As a software developer, you have probably experienced drift many times due to parallel, isolated development processes and not merging into production fast enough. We mitigate some of this drift by introducing continuous integration and continuous deployment processes that reliably can move software from development to production faster while still guaranteeing quality by test automation and gated check in. We use DevOps to shift knowledge left in order to remove blocking phases and speed up the process, and we use GitOps to source control operational changes in order to gain control over configuration unknown syndromes.

Development Goals

Before we further explore what concepts we have adopted to mitigate drift in application development, and how we can use it to also mitigating infrastructure drift, let’s have a look at some objectives for general product development. In particular, let’s study three core objectives:

  • Fast Deliveries
  • Quality
  • Simplicity

Fast deliveries makes the product reach the market and potential consumers fast, preferably faster than the competition. Quality makes the customer stay with the product, and simplicity makes both of the previous objectives easier to achieve and maintain.

Automation

Key part of moving fast is automation of repetitive tasks. It decreases the risk of drift by continuously moving us forward faster with higher accuracy by replicating and reproducing test scenarios, releases and deployments, continuously reporting back feedback helping us steer in the right direction. The more automation the faster we move, the lower risk of drift, the higher the quality.

Least Privilege / Zero Trust / BeyondCorp

Security is something that should be embraced by software developers during the development cycle, not something that are forced upon from the side or as an after construct. This is maybe even more important when building infrastructure. When security becomes a real problem, it’s not uncommon that it is too late to do something about it. Trust is fragile and so is also the weakest link to our customers precious data.

Applying a least Privilege policy does not only minimize risk of granting god mode to perpetrators, it also minimizes the possibility to introduce manual applied drift.

While least privilege can lower the attack surface, Zero Trust simplifies the way we do security. If there are no hurdles in the way of development progress, there is less risk to succumb to the temptation of disabling security mechanisms in order to make life easier.

Infrastructure as Code / GitOps

Source controlling application code has been common practice for many years. By using manifests to define wanted states and declarative code on how to get there, source control follows naturally. The reasons for why infrastructure as code is powerful are the same as for application source code; to track and visualize how functionality changes while being able to apply it in a reproducible way by automation.

Less risk of drift.

GitOps makes it possible to detect risky configuration changes by introducing reviews and running gated checks. It simplifies moving changes forward (and backwards) in a controllable way by enabling small increments while keeping a breadcrumb trail on how we got where we are.

DevSecInfraOps

Cross functional teams help us bridge knowledge gaps, removing handovers and synchronization problems. It brings needed knowledge closer to the development process, shortening time to production and getting security and operational requirements built in. Infrastructure is very much part of this process.

Local Environments

Using a different architecture for local development increases drift as well as the cost to build and maintain it. Concepts like security, network and scalability are built into cloud infrastructure, and often provided by products that are not available for local development. As for distributed systems, these are hard to use locally since they run elastically over multiple hosts possibly across geographic boundaries.

What if we could minimize both application and infrastructure drift by reusing the same cloud native infrastructure architecture and automation to produce the same type of environment, anywhere, any time, for any purpose, while adhering to the above criteria. Test, staging, it all shifts left into the development stage, shortening the path to production, and enabling using all environments as a production like environment.

In the next part we will deep dive into how we can work with cloud infrastructure similar to how we work with application development while adopting all of the above concepts.

Leave a Comment

Semantic Release Notes with Conventional Commits

Recently I wrote about how to use Conventional Commits to calculate the next release version based on semantic versioning. Today I’m going to show how to turn conventional commit messages into release notes using a simple Github Action called Semantic Release Notes Generator that I’ve written.

release-notes-generator

The action is basically just a wrapper for semantic-release/release-notes-generator which is a plugin for semantic-release, an opinionated version management tool. When trying out semantic-release I found it to be too much functionality entangled in a way I couldn’t get to work the way I wanted it to, but I really liked the release notes generator! Being a fan of the Unix philosophy, I decided to wrap it in a neat little action that I could pipe.

Usage

The generator fetches all commit messages between two git references, and feeds them into the release-notes-generator that formats the messages into a nice looking release log.

The semantic release notes generator actually uses a Github Action workflow to test itself (more about that in another post). It uses gitversion to determine next version and pipe that to the generator to generate release notes based on the commits logged since the last release.

Here’s how that could look like:

...
- name: Determine Version
  id: gitversion
  uses: gittools/actions/gitversion/execute@v0
  with:
    useConfigFile: true
    configFilePath: .github/version_config.yml
- name: Determine Release Info
  id: release
  run: |
    from_tag=$(git tag --points-at ${{ steps.gitversion.outputs.versionSourceSha }} | grep -m 1 ^v[0-9]*\.[0-9]*\.[0-9]* | head -1)
    tag=v${{ steps.gitversion.outputs.majorMinorPatch }}

    echo "::set-output name=tag::$tag"
    echo "::set-output name=from_ref_exclusive::$from_ref_exclusive"
- name: Create Tag
  uses: actions/github-script@v3
  with:
    script: |
      github.git.createRef({
        owner: context.repo.owner,
        repo: context.repo.repo,
        ref: "refs/tags/${{ steps.release.outputs.tag }}",
        sha: "${{ steps.gitversion.outputs.sha }}"
      });
- name: Generate Release Notes
  id: release_notes
  uses: fresa/release-notes-generator@v0
  with:
    version: ${{ steps.release.outputs.tag }}
    from_ref_exclusive: ${{ steps.release.outputs.from_ref_exclusive }}
    to_ref_inclusive: ${{ steps.release.outputs.tag }}
- name: Create Release
  uses: softprops/action-gh-release@v1
  with:
    body: ${{ steps.release_notes.outputs.release_notes }}
    tag_name: ${{ steps.release.outputs.tag }}
...

It calculates the next version and finds the last release’s git reference. Then it creates a tag to mark the next release, which also will be used as to_ref_inclusive when generating the release notes. Lastly it creates the Github release using the release notes and the tag created.

If there’s a need to change the release notes, it’s always possible to edit the release in Github either after the release has been published or by releasing it as a draft and manually publishing it after a review.

Summary

release-notes-generator is a great tool for generating release notes when using conventional commits. With Semantic Release Notes Generator you can in an unopiniated way choose how to use release notes within your automated continuous delivery processes!

Leave a Comment

Simple Release Versioning with Trunkbased Development, SemVer and Conventional Commits

I have scripted versioning strategies for Trunk Based Development many times thoughout my career, including using simple Github Actions like Github Tag Action, but the result was never spot on. So I finally gave in overcoming the complexity of tuning GitVersion!

Trunk Based Development

From trunkbaseddevelopment.com:

A source-control branching model, where developers collaborate on code in a single branch called ‘trunk’ *, resist any pressure to create other long-lived development branches by employing documented techniques. They therefore avoid merge hell, do not break the build, and live happily ever after.

A great fit for Continuous Delivery, and very easy to apply!

SemVer

Versioning does not need to be just about incrementing a number to distinguise releases. It can also be semantic; alas SemVer, or short for Semantic Versioning.

From the site:

Given a version number MAJOR.MINOR.PATCH, increment the:

  1. MAJOR version when you make incompatible API changes,
  2. MINOR version when you add functionality in a backwards compatible manner, and
  3. PATCH version when you make backwards compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

This is all nice and such, but how do you integrate that with the process of continuously delivering code?

Conventional Commits

Conventional Commits is a convention for how to turn human written commit messages into machine readable meaning. It also happen to nicely follow the convention for SemVer.

  • Major / Breaking Change
refactor!: drop support for Node 6

BREAKING CHANGE: refactor to use JavaScript features not available in Node 6.
  • Minor / Feature
feat(lang): add polish language
  • Patch
fix: correct minor typos in code

GitVersion

GitVersion looks simple and promising at first glance.

However, when digging into the documentation, a different picture immerges.

GitVersion comes with three different strategies; Mainline Development, Continuous Delivery and Continuous Deployment, all quite difficult to decipher and grasp. All of them seems at first to be a possible fit for Trunk Based Development, but I ended up only getting Continuous Delivery to follow the simple philipsopy of Trunk Based Development, yet being flexible enough of when to actually bump versions and not bump any of major/minor/patch more then once until merge to main happens.

Configuration

GitVersion uses a config file to configure strategies, and there are quite few configurable aspects of said strategies. Let’s get started!

Branches

Trunk Based Development is all about getting into trunk (main/master in git lingo) fast. We have two branch strategies; either we are on main (where we don’t do any development) or we are on a development branch. GitVersion however comes with a lot of them that needs to be disabled.

  feature:
    # Match nothing
    regex: ^\b$
  develop:
    # Match nothing
    regex: ^\b$
  release:
    # Match nothing
    regex: ^\b$
  pull-request:
    # Match nothing
    regex: ^\b$
  hotfix:
    # Match nothing
    regex: ^\b$
  support:
    # Match nothing
    regex: ^\b$y

We consider all branches except main as development branches. These should incremented with patch by default.

  development:
    increment: Patch
    # Everything except main and master
    regex: ^(?!(main|master)$)

Detecting Conventional Commits

The following regular expressions detect when to bump major/minor/patch, and are based on the Conventional Commit v1.0.0 specification.

  • Major
    major-version-bump-message: "(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test)(\\([a-z]+\\))?(!: .+|: (.+\\n\\n)+BREAKING CHANGE: .+)"
  • Minor
    minor-version-bump-message: "(feat)(\\([a-z]+\\))?: .+"
  • Patch
    patch-version-bump-message: "(build|chore|ci|docs|fix|perf|refactor|revert|style|test)(\\([a-z]+\\))?: .+"

Here’s the complete config:

mode: ContinuousDelivery
# Conventional Commits https://www.conventionalcommits.org/en/v1.0.0/
# https://regex101.com/r/Ms7Vx6/2
major-version-bump-message: "(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test)(\\([a-z]+\\))?(!: .+|: (.+\\n\\n)+BREAKING CHANGE: .+)"
# https://regex101.com/r/Oqhi2m/1
minor-version-bump-message: "(feat)(\\([a-z]+\\))?: .+"
# https://regex101.com/r/f5C4fP/1
patch-version-bump-message: "(build|chore|ci|docs|fix|perf|refactor|revert|style|test)(\\([a-z]+\\))?: .+"
# Match nothing
no-bump-message: ^\b$
continuous-delivery-fallback-tag: ''
branches:
  development:
    increment: Patch
    # Everything except main and master
    regex: ^(?!(main|master)$)
    track-merge-target: true
    source-branches: []
  feature:
    # Match nothing
    regex: ^\b$
  develop:
    # Match nothing
    regex: ^\b$
  main:
    source-branches: []
  release:
    # Match nothing
    regex: ^\b$
  pull-request:
    # Match nothing
    regex: ^\b$
  hotfix:
    # Match nothing
    regex: ^\b$
  support:
    # Match nothing
    regex: ^\b$

Tagging

A common practice when releasing in git is to tag a commit with the release number, i.e. v1.2.3. When practicing continuous delivery, this can become quite verbose, especially when producing pre-releases, as every commit becomes a potential release. GitVersion Continuous Delivery strategy traverses each commit from last version tag when calculating the current version, so pre-releases does not necessary need to be tagged. Instead, the commit sha can be used as the pre-release identifier for back-referencing any pre-release artifacts.

Ofcourse, when using Github or similar platforms, a release is a tag, so in that case you might want to use tagging for pre-releases anyway.

Caveats

Forgetting to merge a merge commit back to the development branch when continuing working on the same branch means the automatic patch increment that happens for commits after branching does not occur. This will skew any pre-release versions. According to the documentation, the parameter track-merge-target should solve that scenario, but it seems it has not been implemented!

Summary

GitVersion turned out to be a really nice versioning tool once overcoming the knowledge curve of understanding all the concepts, features and configurations. Check out the GitVersion Github Action and Azure Pipeline Task for simple integration!

Leave a Comment

Spdy – A .NET client

Protocols, more protocols! I love deep-diving into them, and have wanted to explore the secret mysteries of HTTP/2 and gRPC ever since they were released some years ago, but havn’t had any particular reasons to do so, until now.

Kubernetes and the Streaming APIs

You have probably used them many times, the streaming functions exposed by the Kubernetes API Server; exec, port-forward, attach, logs etc. They all stream data back and forth between your local host and containers running on a Kubernetes node. They more or less support one data transport layer; Spdy (…and partly WebSocket) (…and sooooon HTTP/2).

Spdy

Spdy is a deprecated Chromium project that has been superseeded by HTTP/2. The Spdy protocol specification layed the foundation for the HTTP/2 specification and therefor they naturally share a lot of concepts.

Spdy was born due to the increasing usage of network bandwidth and need for low latency communication by modern web applications where HTTP/1.1 (and WebSocket) become bottle necks.

Web applications today depend on a lot of resources, but HTTP/1.1 only supports one request at a time per connection. This has somewhat been mitigated by browsers creating more connections towards the web servers, which is not very effecient considering the growth rate of consumer ip traffic. At best, this is a work around which at the end leads to socket starvation problems for the web servers.

Other limitations are one-way-communication initiation (client to server), redundant and uncompressed metadata (headers), downstream throttling and head-of-line blocking.

Spdy <3 .NET

Up until now, there hasn’t been a Spdy implementation available for .NET (as far as I can tell), and since I needed a reliable, duplex communication transfer protocol to integrate my upcoming project, Port, with Kubernetes’ port-forward API, I decided to implement Spdy, a high level client / server implementation of the 3.1 protocol specification!

As with the Kafka Test Framework, Spdy also utilizes System.IO.Pipelines to minimize buffer copying resulting in lower latency and CPU usage. It also comes with an ASP.NET Core middleware (soon) enabling upgrades of HTTP/1.1 requests to Spdy similar to how requests can be upgraded to WebSocket.

For a sneak peek of the Kubernetes port-forward integration using Spdy, head over to Port’s repository and the Spdy port forwarder.

Leave a Comment