Detecting Breaking Changes

When integrating with other applications and libraries it’s important to detect when APIs are changing in an incompatible way as that might cause downtime for the downstream application or library. SemVer is a popular versioning strategy that can hint about breaking changes by bumping the major version part but as an upstream application developer it can be difficult to detect that a code change is in fact a breaking change.

ApiCompat

Microsoft.DotNet.ApiCompat is a tool built for .NET that can compare two assemblies for API compatibility. It’s built and maintained by the .NET Core team for usage within the Microsoft .NET development teams, but it’s also open source and available to anyone.

Installation

The tool is provided as a NuGet package on the .NET Core team’s NuGet feed, which is not on nuget.org, which most package managers reference by default, but a custom NuGet server hosted in Azure. The URL to the feed needs to be explicitly specified in the project that like to use it.

In the project file add a reference to the NuGet feed under the Project group:

<PropertyGroup>
  <RestoreAdditionalProjectSources>
    https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json;
  </RestoreAdditionalProjectSources>
</PropertyGroup>

Add a reference to the Microsoft.DotNet.ApiCompat package:

<ItemGroup>
  <PackageReference Include="Microsoft.DotNet.ApiCompat" Version="7.0.0-beta.22115.2" PrivateAssets="All" />
</ItemGroup>

The most recent version of the package can be found by browsing the feed.

Usage

The tool can execute as part of the build process and fail the build if the current source code contains changes that is not compatible with a provided contract, an assembly from the latest major release for example.

While it is possible to commit a contract assembly to Git, another approach is to automatically fetch it from source early in the build process. This example uses a NuGet feed as the release source, but it could be an asset in a GitHub release as well or something else.

<Target Name="DownloadLastMajorVersion" BeforeTargets="PreBuildEvent">
  <DownloadFile SourceUrl="https://www.nuget.org/api/v2/package/MyLibrary/2.0.0" DestinationFolder="LastMajorVersionBinary">
    <Output TaskParameter="DownloadedFile" PropertyName="LastMajorVersionNugetPackage" />
  </DownloadFile>
  <Unzip SourceFiles="$(LastMajorVersionNugetPackage)" DestinationFolder="LastMajorVersionBinary" />
</Target>

This will download a NuGet package to the folder LastMajorVersionBinary in the project directory using the DownloadFile task. If the directory doesn’t exist it will be created. If the file already exist and has not been changed, this becomes a no-op.

The Unzip task will unpack the .nupkg file to the same directory as the next step. Same thing here, if the files already exists this is a no-op.

The last step is to instruct ApiCompat to target the unpacked assembly file as the current contract the source code will be compared with. This is done by setting the ResolvedMatchingContract property, which is the only setting required to run the tool.

<ItemGroup>
  <ResolvedMatchingContract Include="LastMajorVersionBinary/lib/$(TargetFramework)/$(AssemblyName).dll" />
</ItemGroup>

The path points to where the assembly file is located in the unpacked NuGet package directory.

Building the project will now download the contract and execute ApiCompat with the default settings. Remember to use proper access modifiers when writing code. internal is by default not treated as part of a public contract¹ as the referenced member or type cannot be reached by other assemblies, compared to public.

¹ InternalsVisibleToAttribute can make internal member and types accessible by other assemblies and should be used by care.

Handling a Breaking Change

Breaking changes should be avoided as often as possible as they break integrations downstream. This is true for both libraries and applications. An indication of a breaking change should most of the time be identified and mitigated in a non-breaking fashion, like adding another API and deprecating the current one. But at some point the old API needs to be removed, thus a breaking change is introduced which will fail the build.

The property BaselineAllAPICompatError can be used to accept breaking changes. The specification of the breaking changes will be written to a file called ApiCompatBaseline.txt in the root of the project. ApiCompat uses it to ignore these tracked incompatible changes from now on. This property should only be set when a breaking change should be accepted and should result in a new major release.

<PropertyGroup>
  <BaselineAllAPICompatError>true</BaselineAllAPICompatError>
</PropertyGroup>

Once a new build has be executed and the new contract baseline has been established remember to set the property to false or remove it.

The baseline file should be committed to source control where it can be referenced as documentation for the breaking change by for example including it in a BREAKING CHANGE conventional commit message.

Once a new major release has been published, there are two options.

1. Reference the assembly in the new major release as the contract and delete ApiCompatBaseline.txt.
2. Do nothing 🙂

As git can visualize diffs between two references, the content of the ApiCompatBaseline.txt will show all the breaking changes between two version tags, which can be quite useful.

Summary

ApiCompat is a great tool for automating detection of breaking changes during development. It avoids introducing incompatible changes and potential headache to downstream consumers, both for applications and libraries.

A complete example of the mentioned changes are available here.