TerraForm Kubernetes Template

Hello All, I wanted to get a quick blog post out here based on something that I worked on, and finally is seeing the light of day.  I’ve been doing a lot of work with TerraForm, and one of the use cases I found was standing up a Kubernetes cluster.  And specifically I’ve been working with Azure Government, which does not have AKS available.  So how can I build a kubernetes cluster and minimize the lift of creating a cluster and then make it easy to add nodes to the cluster.  So the end result of that goal is here.

Below is a description of the project, and if you’d like to contribute please do, I have some ideas for phase 2 of this that I’m going to build out but I’d love to see what others come up with.

Intention:

The purpose of this template is to provide an easy-to-use approach to using an Infrastructure-as-a-service deployment to deploy a kubernetes cluster on Microsoft Azure. The goal being that you can start fresh with a standardized approach and preconfigured master and worker nodes.

How it works?

This template create a master node, and as many worker nodes as you specify, and during creation will automatically execute the scripts required to join those nodes to the cluster. The benefit of this being that once your cluster is created, all that is required to add additional nodes is to increase the count of the “lkwn” vm type, and reapply the template. This will cause the newe VMs to be created and the cluster will start orchestrating them automatically.

This template can also be built into a CI/CD pipeline to automatically provision the kubernetes cluster prior to pushing pods to it.

This guide is designed to help you navigate the use of this template to standup and manage the infrastructure required by a kubernetes cluster on azure. You will find the following documentation to assist:

  • Configure Terraform Development Environment: This page provides details on how to setup your locale machine to leverage this template and do development using Terraform, Packer, and VSCode.
  • Use this template: This document walks you through how to leverage this template to build out your kubernetes environment.
  • Understanding the template: This page describs how to understand the Terraform Template being used and walks you through its structure.

Key Contributors!

A special thanks to the following people who contributed to this template:
Brandon Rohrer: who introduced me to this template structure and how it works, as well as assisted with optimizing the functionality provided by this template.

Distributed Computing and Architecture Patterns

So lately I’ve been doing a lot of work on distributed programming, and specifically looking less at projects that are living on-premise, but need to be moved to cloud, and more with projects that are born in the cloud and how to optimize.  What I’m talking about here is applicable for the “lift-and-shift” type of project.

Ultimately the “cloud” is just like any other development projects, there are considerations that need to be handled as part of leveraging the environment to the best possible outcome.  So there are things you can do to help make your applications perform to their peak in the cloud.

In the traditional “Monolithic” approach to designing applications, we would work ourselves into a corner or more less.  And what I mean by that is we would build out applications to consume servers and predetermined resources, and that meant that if you wanted to take that application and sell it, traditionally you were looking at a large capital expense.  More than that also, if you wanted to increase scale, guess what…another capital expense, and this time with all the time required for a corporate purchase of that size.

Distributed Computing attempts to solve that problem, by enabling us to take that monolithic application and break it into the smallest parts we can.  And then making each of those parts independently scalable, to meet need.  So instead of one big app, we have a “web” of smaller pieces doing different jobs, and the total is more than the sum of its parts.

The value add here, is by leveraging smaller more isolated components, we can really focus on what does the best job.  For example, you might have a dotnet application, but if Python is the best fit for a microservice, why would you handicap yourself and not use the best tool for the job.  Microservices allow you to do that.

Let’s start with things you should keep in mind when building distributed applications within your application:

  • Loosely-Coupled Components:  For a distributed solution to truly work, all the pieces need to be loosely coupled.  And this takes the form of creating “buffers” between these services.  These “buffers” normally take the form of messaging between services, you could use a service bus, or even just a queue to communicate between services.  But the idea being that the services don’t know anything about each other, they just know that one adds an item to a queue, and the other removes the item from the queue.  This allows them to function independently and allows for the value add of creating ability to deploy these components separately.
  • Handle Communication Appropriately:  Given what we just talked about, how your application is a series of interconnected smaller apps, its important take some time and think about how you will pass information back-and-forth between applications.  Given that the application components are subject to change (platform, technology, end points, networking, etc).  You need to remember that you need an abstraction layer between the different micro services to make sure that they can be separated in a way to provide the best overall support for keeping these as separately deploy-able pieces.
  • Build with Monitoring in Mind:  Also, given that your application is really made of all these separate parts, its important to remember that for your application to work, every component must be functioning properly.  Just like an Olympic team can’t play unless every player is operating at the top of their game, your app can’t work if a component is unhealthy.  So when you set out to build micro service applications, make sure that you build and architecture your code with the logic in mind.
  • Build with Scale in Mind:  Given that your application is being built this way to encourage scaling, its important that you build your app in such as way that it can scale to meet the demands users are putting on the system.  Part of this comes down to making sure that your leveraging resources appropriately and not building systems that over (or under) consume the resources that you are using.
  • Build with Errors in mind:  Another item to consider is that your application is now a sum of “moving parts” and that being said, sometimes things can have errors or breakdowns that need to be handled.  These can be unplanned (exception or errors) or planned (upgrade of a service).  So your application should be able to respond to these “transient” faults, and not break down.  For example, one way is to leverage queues.  If component “A” is talking directly to component “B”, and “B” is in the middle of an upgrade, “A” might start throwing errors which will bubble up to the user, during the upgrade.  So now I need to notify users of downtime for the smallest change.  If I put a queue between them, then component “A” can continue to add elements to the queue, and while “B” is updating no errors occur.  When “B” finishes upgrading, it just starts pulling items off the queue as if nothing happened.  This is even less of a problem when you have scaling built into your app.

So the question is how do you do that?  There are a lot of ways to approach this particular problem, and ways to ensure that your app respects these.

I recommend the following steps:

  • Leverage a micro service approach:  There are a lot of articles out there (and some linked to below) that will talk you through how to build a micro service application, this can leverage a lot of technologies including “Service Fabric”, “Kubernetes”, “Docker Swarm”, and others to push your applications out with containers to support this approach.  You don’t have to use containers to do micro services, but they definitely help.
  • Always consider the best tool for the job:  One of the biggest benefits of micro services are the ability that you can leverage different stacks to solve different problems.  Don’t ignore this.
  • Leverage abstraction in communication between services:  As I mentioned above, this is paramount.  You must account for communication using a communication strategy, and a lot of times it helps to be consistent in how you approach this across your apps.  It will make your life simpler in the long run.
  • Make your services backward compatible:  As mentioned above, the benefits of abstraction are that I can push updates to individual components any time.  But to truly take advantage of this, I need to make my services backwards compatible.  Take my example above, imagine that service “A” writes to a queue, and then service “B” reads from that queue to do processing.  Now if Service “B” has scaled out to 10 nodes, and I try to update service “B”, I don’t want to shut them all down at once and take that part of the app offline, instead I want to do a rolling update.  So the idea being, that while service “B.1” is down, B.2-B.10 are still processing messages.  But in order to do that, I have to be careful how I change the signature of the queue and the changes to the database.  If i change the underlying database for service “B”, the database of service “B.2” has to be able to talk to it, even though its running old code.
  • Assume everything can change:  This is the best advice I can give, assume that anything can change around each Micro service that you build and by default you will be able to gracefully handle “transient faults” or “schema changes” without having to debug huge problems.
  • Leverage configuration management:  This is sort of a “1A” to the above entry, if you leverage configuration management, using services like redis, table storage, key vault or other platforms, you can make changes without requiring a redeploy of the application.  This makes your life much easier when you deploy a service and can change the configuration of other services.
  • There is no need to re-invent the wheel from an architectural standpoint, especially when you are learning.  If this is your first distributed project, lean on the known patterns and then take risks when you know more.

I hope this helps, as you start down this road.  What I will tell you, is that despite the changes I’m reviewing here.  Below are some addition links to help with this discussion:

Cloud Design Patterns :  This site provides you with detailed write-ups of some of the most common architecture patterns out there for Cloud.  These are especially helpful if you are currently more accustom to working an on-premise world, as they give a view of some of the considerations you should keep in mind.  I also like this because it pulls in concepts on things you might not be fully used to supporting (High availability vs disaster recovery, for example).

Architecture Center : Another great site, that contains just general architecture guidance for cloud or on-premise.  Its just a helpful site that lays out the pros / cons of common patterns so that you can design solutions appropriately to meet needs.

Architecting Distributed Applications:  A great online course that will walk you through what it means to build a truly distributed application, and this course is technology agnostic which is always a good thing.

Building Distributed Applications with Akka.net:  A great video with an overview of Akka.net which is a technology to help create applications using an Actor pattern.

Distributed Architecture with Microservices / Messaging:  A great video on Microservices which are the corner stone of distributed computing.

Rethinking Distributed Systems for Data Centers:  Another great article on how to build applications in a distributed world to accommodate a varying degree of scale.

Running Entity Framework Migrations in VSTS Release with Azure Functions

Hello All, I hope your having a good week, I wanted to write up a problem I just solved that I can’t be the only one. For me, I’m a big fan of Entity Framework, and specifically Code First Migrations. Now its not perfect for every solution, but I will say that it does provide a method of versioning and controlling database changes in an more elegant method.

Now one of the problems I’ve run into before is that how do you leverage a dev ops pipeline with migrations. The question becomes how do you trigger the migrations to execute as part of an automated release. And this can be a pretty sticky situation if you’ve ever tried to unbox it. The most common answer is this.  Which is in itself, a fine solution.

But one scenario that I’ve run into where this doesn’t always play out, is the scenario where because the above link executes on App_Start, it can cause a slowdown for the first users, of in the scenario of load balancing can cause a performance issue when it hits that method of running the migration.  And to me, from a Dev Ops perspective, this doesn’t feel really that “clean” as I would like to deploy my database changes at the same time as my app, and know that when it says “Finished” everything is done and successful.  By leveraging this approach you run the risk of a “false positive” saying that it was successful even when it wasn’t, as the migrations will fail with the first user.

So an alternative I wrote was to create an azure function to provide an Http endpoint, that allows for me to trigger the migrations to execute.  Under this approach I can make an http call from my release pipeline, and execute the migrations in a controlled state, and if its going to fail, it will happen within my deployment rather than after.

Below is the code I leveraged in the azure function:

[FunctionName("RunMigration")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequestMessage req, TraceWriter log)
{
log.Info("Run Migration");

bool isSuccessful = true;
string resultMessage = string.Empty;

try
{
//Get security key
string key = req.GetQueryNameValuePairs()
.FirstOrDefault(q => string.Compare(q.Key, "key", true) == 0)
.Value;

var keyRecord = ConfigurationManager.AppSettings["MigrationKey"];
if (key != keyRecord)
{
throw new ArgumentException("Key Mismatch, disregarding request");
}

Database.SetInitializer(new MigrateDatabaseToLatestVersion<ApplicationDataContext, Data.Migrations.Configuration>());

var dbContext = new ApplicationDataContext();

dbContext.Database.Initialize(true);

//var list = dbContext.Settings.ToList();
}
catch (Exception ex)
{
isSuccessful = false;
resultMessage = ex.Message;
log.Info("Error: " + ex.Message);
}

return isSuccessful == false
? req.CreateResponse(HttpStatusCode.BadRequest, "Error: " + resultMessage)
: req.CreateResponse(HttpStatusCode.OK, "Migrationed Completed, Database updated");
}
}

Now a couple of quick things to note as you look at the code:
Line 12:  I am extracting a querystring parameter called “key” and then in line 16, I am getting the “MigrationKey” from the Functions App Settings.  The purpose of  this is to quickly secure the azure function.  This function is looking for a querystring value that matches the app settings value to allow the triggering of the migrations.  This prevents just anyone from hitting the endpoint.  I can then secure this value within my release management tool to pass as part of the http request.

Lines 22-26: this is what actually triggers the migration to execute by creating a context and setting the initializer.

Line 37: Allows for handling the response code that is sent back to the client for the http request.  This allows me to throw an error within my release management tool if necessary.

Azure Compute – Searchable and Filterable

Hello All, so a good friend of mine, Brandon Rohrer and I just finished the first iteration of a project we’ve been working on recently.  Being Cloud Solution Architects, we get a lot of questions about the different compute options that are available in Azure.  And it occurred to us that there wasn’t a way to consume this information in a searchable, and filterable format to get the information customers need.

So we created this site:

https://computeinfo.azurewebsites.net
This site scrapes through the documentation provided by Microsoft and extracts the information about the different types of virtual machines you can create in azure and provides it in a way that meets the following criteria:

  • Searchable
  • Filterable
  • Viewable on a mobile device

Hope this helps as you look at your requirements in Azure and build out the appropriate architecture for your solution.

Goals and Grit

Hello All, I wanted to shake things up a little bit and talk about a book I have been working my way through and goals. So its officially January, and a lot of us are looking at the great new year like a blank canvas, waiting to be painted. I have to be honest, I’ve always been a fan of New Years, not the holiday or New Years Eve, although everyone loves a good party night. But every year I enjoy the act of self-reflection and planning that goes into the new year, and the chance to grow and improve.

But the one thing I hate about this process is during the self-reflection, admitting where you came up short. Where did you stumble or fail, what went wrong? Now if I’m being honest I’m a DevOps guy and as a result am big on admitting failure. But if we look at this from a DevOps perspective, teams grow when they fail fast, and on some level this yearly retrospective ritual flies in the face of that.

Lately I’ve been reading a great book call Grit: The Power of Passion and Perseverance by Angela Duckworth. And it really is an amazing book that will change the way you look as success on the whole. Really it promotes this concept that success is not built on talent, but rather on the determination and passion of the person.

In the beginning of the book she calls out West Point. West Point has one of the most rigorous recruiting processes in history, and they only take the best and brightest into their program. But despite that, they were seeing a very high drop out rate, and couldn’t figure out why. The short version is because the people who are most talented are rarely tested, and if you’ve never had to overcome obstacles before, then you are likely to back down when faced with your first wall.

The book also gives an interesting take on goal planning that I had never done before, and its one that to me makes a lot of sense, and I’m giving it a try this year. So I will have to update the blog here with the results. But the one method she talks about was discussed by Warren Buffet, arguably one of the most successful business men of our time. In the book, he describes a planning process he does, which is to write down 25 goals, 25 things you’d like to accomplish this year. This sounds like a lot, but if you start writing goals, you’ll find its not hard. I hit 30 without breaking a sweat. And then pick from that list the top 5, and put those in the “MUST DO” category.

And take the rest…and put them in the “NOT UNDER ANY CIRCUMSTANCES” category. The idea is this, your time is your most valuable resource, and multi-tasking is an illusion. So you should focus your attention on these 5, and the other 20 are a distraction. The focus being that being successful isn’t about saying “Yes”, its about saying “No”.

For me this resonates, as if I pour all my attention and time into 5 specific goals, I am way more likely to accomplish them with greater impact. And this also works well with another planning approach that I’ve leveraged before, which is described by Steven Covey’s 7 Habits of Highly Effective People.

In his book, he describes the idea that if you think of your day as a bucket, and I tell you to fit big rocks, little rocks, and sand into the bucket. What is the most logical way to fill it? Big Rocks, then little, then Sand, and if we are being honest we should approach our goals the same way. But most times we don’t, we avoid the big tasks, and small tasks, and fill our day with emails first.

So he recommends breaking things into the following matrix (called the Eisenhower Decision Matrix):

Important / Urgent Important / Not Urgent
Not important / Urgent Not Important / Not Urgent

In this matrix, the idea is that “Important” means that it lines up with your goals, which I would argue are the five goals provided above. From there we can look at what’s urgent and aligns to our goals as where our time should be spent.

  • Q1 of the above box, is for things that are urgent and related to your goals, like deadlines, crisis, opportunities that are time sensitive.
  • Q2 of the above are items that don’t have a pressing deadline but focus on your goal, this should be next on your priority list.
  • Q3 are items that require immediate attention but don’t move us forward. Which should try to minimize these tasks as much as possible. Things like phone calls, emails, etc.
  • Q4 are items which aren’t urgent or important and are basically time wasters, eliminate at all costs.

So leveraging the above matrix, makes it very easy to keep our focus where it should be on our 5 goals, and avoiding the distractions that undermine our success.

Getting Started

So I thought I would start this new direction for the blog with a post about a topic I get asked about a lot.

“I want to get started in programming, how do I do that?”

And this is a great question, and one that makes a lot of sense to me as the lines between technology and business are blurring.  And more and more people are interacting with developers in their daily life as part of their current jobs, and its leading to people’s eyes being opened to the opportunities in this place.

My next question is normally “Why?” and at first that usually takes people back, but this is an important thing to ask yourself.  I ask this because to be honest, switching fields and taking on something like becoming a developer isn’t an easy journey, and if your motives aren’t clear than your going to set yourself up for failure.  I generally think it’s smart to ask if this is a worthwhile investment of your time.  Because as much as I love this industry, it can be quite brutal at times.

Don’t get me wrong, I’m not making these statements as some grand arbiter who decides if you are worthy of becoming an almighty developer.  I make these statements because the simple truth is that to work as a developer and achieve success you need to be willing to accept the reality, which is less Minority Report and steve jobs, and more the craziness of Silicon Valley.

I would tell you to ask the following questions:

  • Do you like continual education?  Are you willing to read about this stuff in your spare time?
  • Do you like to tinker with things?
  • Do you have “Grit”?

For the final question, specifically I’m referring to the fantastic book by Angela Duckworth, that describes Grit as basically being the intersection of Passion and Perseverance, and that it is the most important part of any equation where someone is hoping for success.  And I would argue, even more so true for developers.

If you look online at the “successful developers” they all have one thing in common…they live for this stuff.  And spend a lot of time doing it, and finding new ways to challenge themselves and push boundaries.  They are constantly looking for ways to change their mindset to find new opportunities and directions.  I don’t claim to be a famous developer, but I can tell you that I’m proud of where I’ve gone in my career and I genuinely love what I do, and much like those “famous developers”, my wife describes me as a “well documented nerd”.

So now the important question is, did I lose you?  If not, I think this is a rewarding career option that can take you in some interesting directions, but you need to know that it will be a slow burn.  This is not something where you will be writing award winning apps by Monday if you start on Friday.

Below are some links to help you out, feel free to reach out with questions, I’ve tried to provide a lot of training material and some notes about each link:

Visual Studio 2017 Community Edition – This is the primary IDE (integrated developer environment) for all things in the Micr0soft stack. And the community edition is free, which is even better.

https://www.visualstudio.com/downloads/

When you go to install it, its going to ask you to customize the install, by selecting different packages and what not.

Great start: C# fundamentals for Absolute Beginners:

https://mva.microsoft.com/en-US/training-courses/16169?l=Lvld4EQIC_2706218949

For training materials I would recommend the following:

Microsoft Virtual Academy – https://mva.microsoft.com/

This is a great site for a lot of training content Microsoft generates to help. I would point you to the absolute beginner classes as well as the learning paths. They also do a good job of categorizing training (100 level, 200 level etc)

C# Courses:

https://mva.microsoft.com/training-topics/c-app-development#!jobf=Developer&lang=1033

Visual Studio Training:

https://mva.microsoft.com/product-training/visual-studio-courses#!jobf=Developer&lang=1033

Getting started with Visual Studio 2017

https://mva.microsoft.com/en-US/training-courses/getting-started-with-visual-studio-2017-17798?l=9oIw0FD6D_3611787171

Learning Paths:

https://mva.microsoft.com/LearningPaths.aspx

Channel 9 – https://channel9.msdn.com/

Great site for general videos, and is updated all the time.

I recommend web as a good place to start, the .net web platform is called ASP.NET and uses HTML, C#, and some javascript to work. (C# and Javascript syntax are pretty close)

https://www.asp.net/get-started

https://www.asp.net/mvc/overview/getting-started

This is probably a good start.

How to punch up your resume?

So I thought given the new direction with this blog, I would focus my attention on some of the questions I get a lot.  And one of the  biggest questions I get asked frequently is “My resume is terrible, how do I make it better?”

To be perfectly honest, most people undervalue their resume, and think of it like some kind of checkbox.  I love hearing people say “I’m not worried, once I go in for the interview the resume is meaningless.”  To which my response is HOW DO YOU THINK YOU GET THE INTERVIEW!

There’s an old adage, that the you never get a second chance to make a first impression, and when applying for jobs, the resume is your first impression.  When I worked for a prior company, part of my job was interviewing new talent and determining if they were a good fit to move the organization forward.  As such, I literally conducted over 100 interviews in a 8 month a period.  I can say I’ve seen a lot of things, and this blog post is really based around the tips that would apply to help get your resume noticed and get you in the interview.

  1. DO NOT stick to one page:  In college they will tell you that your resume must be limited to one page.  That is not realistic for a technical position, because in these positions we are looking for the skills you have.  Don’t go crazy but a good three page resume is a lot better than Times New Roman, size 8 compressed onto a page.  The human eye needs white space more than anything.
  2. Keep it up to date:  This is jumping a little further ahead, but make sure it is 100% current.  I’ve read resumes of people and nothing turns the interviewer off more than to bring you in and here, “Here’s the stuff that I’ve been working on”.
  3. Describe the projects:  Even better than a list of skills is a project description, and acknowledging that you can’t give up all details.  But things like, “Project XYZ was a mobile app built with Xamarin with a Cosmos DB database back end, and I was the lead developer of the mobile side.” tells me a lot about what your skills are.
  4. Be clear about your role:  It helps if you tell me what you did on the project, and be clear about the responsibility not the titles.  I’ll give an example, my first job I was responsible for building software for managing test centers and grading certification exams with the state, but being the state my job title was “LAN Technican”, no even close.  So I found that you should try to change your title, just list what you did on the project.  It gives a clearer picture of what your skills are.
  5. Put in personal projects:  I used to tell people “I can teach someone to code the way I want, but I can’t teach passion”.  So if you’ve contributed to GitHub projects, put it in there, if you have apps in the app store, put them in there.  Talk to me about what you with, that shows perseverance and drive, which I can’t teach.  If you blog list that, if you work with user groups, put that.  I once had a candidate show “my son and I built a cloud enabled race car with a raspberry pi and a cell phone”, that’s fantastic information.  But make sure you limit it to what you’ve done.
  6. Be Honest about how much you’ve worked with something:  It’s a great idea to quantify your technical skills, you can use a 1-10 scale, or some other measure, on my resume I use a 1-5 scale.  This allows them to get a good assessment of your skills and saves everyone time.  And to be honest this is another one where you show “I’m learning Xamarin on my own” is huge.  Expect that during the technical interview you are going to be grilled on all these, and if you aren’t honest, that’s a guaranteed out (next post we talk about the technical interview).

 

Welcome to Mack Bytes

Hello all, consider this the inaugural post of the new Mack Bytes blog.  Welcome!  I’ve chosen to sort of rebrand this blog, and sort of relaunch it for a couple of reasons.  The first and foremost of which is that the other blog has sort of faded.

To be honest I started that blob it feels like a life-time ago, and it was started with the intention of creating a blog to provide articles that were intended to be “how-to” guides, and other targeted posts strictly only on software development.  And there’s nothing wrong with that, but if I’m being honest, I had trouble continually updating it, and it felt like every post started with “Let me start by apologizing for the lack of updates”.  And that’s not a good place to be.  I also feel like my life has changed actually quite a bit recently, and I sort of wanted to refocus my efforts.  So over the past year I’ve been doing a lot of thinking, and planning.  And this has all been geared around life goals and direction.  And I really wanted to relaunch this blog with the intention of being more than it was before.  Don’t get me wrong I’m a big nerd at heart, so there will always be the technical posts, but this newly branded blog is more about changing to being a general blog to help people as I figure this stuff out for myself.

In essence, I’m inviting you all on my journey.  So I’ve found myself at the tender age of 35, and realized that the journey that got me to where I am is very different from the plan when I graduated college, and I’m looking forward to where I can go next.  But really for me to do that, a lot of factors have to be considered.  And a lot will likely happen along the way.  I will be using this blog as a platform to help people and share insights.  And I will also be leveraging other social media platforms as a way of driving those messages.  So if you like what you see here, please pass it along to someone else, if you don’t please talk to me.  I’d love to start a dialog exchange if you disagree with my thoughts, actions, or beliefs.  Dialog is the best way to really grow, and challenging dialog has a way of making you re-examine your beliefs and actions.

So here we go, the first post of the new blog, looking forward to more.