Browsed by
Tag: nuget

Why you really need a private nuget feed

Why you really need a private nuget feed

So I have to be honest, I’m the kind of person that is always looking for a better way to do something. And it probably speaks volumes to my chosen career that I am that way. One of the reasons I’m so passionate about technology is that it on a foundational level supposed to change and improve our lives.

To that end, I’ve been having a lot of conversations lately around development and managing technical debt, sprawl, and just generally the ability to have more controls around major enterprise applications that are being built.

One of the biggest changes to coding, I would argue since the idea of Object-Oriented programming, has been the advent of the micro-service. This idea that I can build out a swarm of smaller services and leverage all these elements to do much larger tasks, and scale them independently has fundamentally changed how many of us look at Software Architecture and Solution development.

But with this big step forward, all to often I see a big step backward as well.

And one of the ways this regression into bad habits happens is that I all to often see it become the “Wild west” of software development, and everyone runs off building individual micro-services, with no idea to operations or manageability. Which ultimately leads to a myriad of services that are now very difficult to maintain.

For example, all services have several common components:

  1. Configuration Management
  2. Secret Management
  3. Monitoring / Alerting
  4. Exception handling
  5. Logging
  6. Availability Checking
  7. Health probes endpoints

And if these elements are coordinated, then what can happen is that it becomes very difficult to maintain an solution because if you have 100 services in the application, you could potentially have 100 different ways configuration is managed. This is a nightmare to be honest. The good news is that they solved this problem, with npm and nuget!

One of the key benefits of leveraging a custom nuget server is that it allows you to gain the ability to have premade application components that then can be leveraged like lego blocks by developers to increase their productivity, so from their side, you can use these to get a jump on the new services they build.

Now most developers I know, love to reuse things, none of us got into this field to rewrite code for the hell of it.

Now if using the above, you create nuget packages around these types of operations (and more you will identify), and hook them up to their own CI/CD process that ends in an artifacts feed, we gain a couple of benefits:

  1. Developers stop reinventing the wheel:  You gain the productivity gains of being able to plug-and-play with code. You basically get to say “I’m building a new service, and I need X, Y, and Z…”
  2. We get away from “It works on my machine” as all dependencies are bundled, and can get to a container that is more cleanly implemented.
  3. You gain more change control and management:  By versioning changes to nuget packages, and leveraging pre-release packages appropriately we can roll out changes to some services and easily look at all of our services and see which ones are consuming older packages. So ultimately we can manage the updates of each service independently, but take the need to track that off the developer and onto a service.
  4. Make updates to shared code easier and an afterthought:  Stop me if you heard this one before.  You as a developer are performing updates to a service that leverages an older piece of shared code that someone discovered a bug with.  You are working on their bug, and don’t think about the shared code that should be updated at the same time, it fell through the cracks.  And when you finish, the new version of service is deployed with old shared code, and no one is the wiser.  A change happens later to enforce something and it breaks service.”  By leveraging nuget packages, it can become standard practice to say “I’m starting work on this service, select manage nuget packages and it shows me a list of packages with updates, I say update all.”
  5. You gain more ability to catch things earlier ahead of release preventing delays:  It makes it a lot easier to coordinate the changes being made to common code when you use a nuget feed, and make it easier for developers to adjust. For example, If I’m working on Service A, and during our standup I find out that another member of my team has a pre-release of the nuget package I’m using, and both are going to prod around the same time. I can implement the pre-release, and make the changes before I get to a point of needing to integrate this at the end and pushing last minute changes.

There are some great posts that get into the specifics of this, and how to implement it, and those are:

Leveraging Private Nuget Packages to support Micro-services

Leveraging Private Nuget Packages to support Micro-services

So one of the most common elements I see come up as your build out a micro-service architecture, is that there is a fair bit of code-reuse when you do that. And managing that code can be painful, and cause its own configuration drift, that can cause management issues.

Now given the above statement, the first response is probably “But if each Micro-service is an independent unit, and meant to be self-contained, how can their be code re-use?” And my response to that, is that just because ultimately the services and independently deployable does not mean that there isn’t code that will be reused between the different services.

For example, if you are building an OrderProcessingService, that might interface with classes that support things like:

  • Reading from a queue
  • Pushing to a queue
  • Logging
  • Configuration
  • Monitoring
  • Error Handling

And these elements, should not be wildly different from each micro-service, and should have some common elements. For example, if you are leveraging KeyVault for your configuration, odds are you will have the same classes implemented across every possible service.

But this in itself creates its own challenges, and those challenges include but are not limited to:

  • Library Version Management: If each service is its own deployable unit, then as you make changes to common libraries you want to be able to manage versions on going.
  • Creating a central library for developers: Allowing developers to manage the deployment and changes to versions of that code in a centralized way is equally important.
  • Reduce Code Duplication: Personally I have a pathological hatred of code duplication. I’m off the belief that if you are copying and pasting code, you did something wrong. There are plenty of tools to handle this type of situation without doing copy and paste and adding to technical debt.

Now everyone is aware of NuGet, and I would say it would be pretty hard to do software development without it right now. But what you may not know is that Azure DevOps makes it easy to create a private NuGet feed, and then enable the packaging of NuGet packages and publishing via CI/CD.

Creating a NuGet Feed

The process of creating a feed is actually pretty straight forward, it involves going to a section in Azure DevOps called “Artifacts”.

And then select the “Create Feed”, and give the feed a name, and specify who has rights to use this feed:

And from here its pretty easy to setup the publishing of a project as a NuGet package.

Creating a package in Azure Pipelines

A few years ago, this was actually a really painful process, but now its pretty easy. You actually don’t have to do anything in Visual Studio to support this. There are options, but to me, a NuGet package is a deployment activity, so I personally believe it should be handled in the Pipeline.

So the first thing, is you need to specify the details on the configuration. If you go to “Properties” on the project:

These are important as this is the information your developers will see in their NuGet feed.

From here the next step is to configure your pipeline to enable the CI/CD. The good news is this is very easy using the following pieces:

- task: DotNetCoreCLI@2
  inputs:
    command: pack
    versioningScheme: byEnvVar
    versionEnvVar: BUILD_BUILDNUMBER
  displayName: 'dotnet pack $(buildConfiguration)'

- task: NuGetAuthenticate@0
  displayName: 'NuGet Authenticate'

- task: NuGetCommand@2
  inputs:
    command: push
    publishVstsFeed: 'Workshop/WellDocumentedNerd'
    allowPackageConflicts: true
  displayName: 'NuGet push'

Now using the above, I have specified the BUILD_BUILDNUMBER as the identifier for new versions. I do this because I find its easier to ensure that the versions are maintained properly in the NuGet feed.

One thing of note, is that NuGet Authenticate step is absolutely essential to ensure that you are logged in with the appropriate context.

Now after executing that pipeline, I would get the following in my NuGet Feed.

Consuming NuGet in a project

Now, how do we make this available to our developers, and to our build agents. This process is very easy. If you go back to the “Artifacts” section, you will see the following:

The best part is that Azure DevOps will give you the ability to pull the XML required when you select “dotnet”, and it will look something like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />

    <add key="WellDocumentedNerd" value="....index.json" />

  </packageSources>
</configuration>

After this is done, and added to your project, whenever you build pipeline attempts to do a build of your code project it will consider this new data source in the NuGet Restore.

Why go through this?

As I mentioned above, ultimately one of the biggest headaches of a micro-service architecture for all of its benefits, is that it creates a lot of moving parts. And ultimately managing any common code, could make things difficult if you have to replicate code between projects.

So this creates a nice easy solution that allows you to manage a separate code repository with private / domain specific class libraries, and still have versioning to allow for having different versions on each service while enabling them to be independently deployable.