Getting Started with Azure (developer perspective)

So there’s a common question I’ve been getting a lot lately, and that’s “I want to learn Azure, where do I start?” And this is ultimately a very reasonable question, because as much as the cloud has permuted much of the digital world, there are still some organizations who have only recently started to adopt it.

There are many reasons people would choose to adopt the cloud, scalability, cost, flexibility, etc. But for today’s post I’m going to focus on the idea that you have already decided to go to the Azure Cloud and are looking for resources to ramp up. So I wanted to provide those here:

MS Learn: The site provides videos, reading, and walk-through’s that can assist with learning this type of material:

MS Learn for Specific Services: There are several common services out there that many people think of when they think of the cloud, and I wanted to provide some resources here to help with those:

EDX Courses: EDX is a great site with a lot of well made courses, and there are a wealth of options for Azure and Cloud, here are a few I thought relevant, but it is not an exhaustive list.

  • Architecting Distributed Applications: One common mistake, that many make with regard to the cloud is that they think of it as “just another data center”, and that’s just not true. To build effective and scalable applications, they need to be architected to take advantage of distributed compute. This course does a great job of laying out how to make sure you are architected to work in a distributed fashion.
  • Microsoft Azure Storage: A great course on the basics of using Azure Storage.
  • Microsoft Azure Virtual Machines: The virtual machine is the cornerstone of azure, and provides many options to build an scale out effectively. This is a good introduction into the most basic service in Azure.
  • Microsoft Azure App Service: The most popular service in Azure, App Service enables developers to deploy and configure apps without worrying about the machine running under-the-covers. A great overview.
  • Microsoft Azure Virtual Networks: As I mentioned above, Software Based Networking is one of the key pieces required for the cloud and this gives a good introduction into how to leverage it.
  • Databases in Azure: Another key component of the cloud is the Database, and this talks about the options for leveraging platform-as-a-service offerings for databases to eliminate your overhead for maintaining the vms.
  • Azure Security and Compliance: A key component again is security, as the digital threats are constantly evolving, and Azure provides a lot of tools to protect your workload, this is an essential piece of every architecture.
  • Building your azure skills toolkit: A good beginner course for how to get your skills up to speed with Azure.

Additional Tools and Resources, I would recommend the following:

Those are just some of the many resources that can be helpful to starting out with Azure and learning to build applications for the cloud. It is not an exhaustive list, so if you have a resource you’ve found helpful, please post it in the comments below.

Where do I start – Microsoft AI

In the interest of helping to navigate the information available out there, I’ve been putting out there ideas for this “Where Do I start” series on the blog. Right now as I previously mentioned I’ve been studying for the AI-100 exam, and as part of that effort I found a lot of resources online, and I thought I’d share these in the interest of helping others.

There are a wealth of resources out there and I want to make sure I focus your attention on resources related to Microsoft AI and how you can leverage these services as accelerators for your own application development.  I wanted to draw your attention to a lot of the key resources for getting started.

Learning Videos:

 

Now additionally I have done some work on my github implementing the face api, which is available here:

https://github.com/KevinDMack/FacialSearchDemo

Building a Solr Cluster with TerraForm – Part 1

So it’s no surprise that I very much have been talking about how amazing TerraForm is, and recently I’ve been doing a lot of investigation into Solr and how to build a scalable Solr Cluster.

So given the kubernetes template I wanted to try my hand at something similar. The goals of this project were the following:

  1. Build a generic template for creating a Solr cloud cluster with distributed shard.
  2. Build out the ability to scale the cluster for now using TerraForm to manually trigger increases to cluster size.
  3. Make the nodes automatically add themselves to the cluster.

And I could do this just using bash scripts and packer. But instead wanted to try my hand at cloud init.

But that’s going to be the end result, I wanted to walkthrough the various steps I go through to get to the end.  The first real step is to get through the installation of Solr on  linux machines to be implemented. 

So let’s start with “What is Solr?”   The answer is that Solr is an open source software solution that provides a means of creating a search engine.  It works in the same vein as ElasticSearch and other technologies.  Solr has been around for quite a while and is used by some of the largest companies that implement search to handle search requests by their customers.  Some of those names are Netflix and CareerBuilder.  See the following links below:

So I’ve decided to try my hand at this and creating my first Solr cluster, and have reviewed the getting started. 

So I ended up looking into it more, and built out the following script to create a “getting started” solr cluster.

sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
sudo apt-get install -y gnupg-curl
sudo wget https://www.apache.org/dist/lucene/solr/8.0.0/solr-8.0.0.zip.asc | sudo apt-key add

sudo apt-get update -y
sudo apt-get install unzip
sudo wget http://mirror.cogentco.com/pub/apache/lucene/solr/8.0.0/solr-8.0.0.zip

sudo unzip -q solr-8.0.0.zipls
sudo mv solr-8.0.0 /usr/local/bin/solr-8.0.0 -f
sudo rm solr-8.0.0.zip -f

sudo apt-get install -y default-jdk

sudo chmod +x /usr/local/bin/solr-8.0.0/bin/solr
sudo chmod +x /usr/local/bin/solr-8.0.0/example/cloud/node1/solr
sudo chmod +x /usr/local/bin/solr-8.0.0/example/cloud/node2/solr
sudo /usr/local/bin/solr-8.0.0/bin/bin/solr -e cloud -noprompt

The above will configure a “getting started solr cluster” that leverages all the system defaults and is hardly a production implementation. So my next step will be to change this. But for the sake of getting something running, I took the above script and moved it into a packer template using the following json. The above script is the “../scripts/Solr/provision.sh”

{
  "variables": {
    "deployment_code": "",
    "resource_group": "",
    "subscription_id": "",
    "location": "",
    "cloud_environment_name": "Public"
  },
  "builders": [{   
    "type": "azure-arm",
    "cloud_environment_name": "{{user `cloud_environment_name`}}",
    "subscription_id": "{{user `subscription_id`}}",

    "managed_image_resource_group_name": "{{user `resource_group`}}",
    "managed_image_name": "Ubuntu_16.04_{{isotime \"2006_01_02_15_04\"}}",
    "managed_image_storage_account_type": "Premium_LRS",

    "os_type": "Linux",
    "image_publisher": "Canonical",
    "image_offer": "UbuntuServer",
    "image_sku": "16.04-LTS",

    "location": "{{user `location`}}",
    "vm_size": "Standard_F2s"
  }],
  "provisioners": [
    {
      "type": "shell",
      "script": "../scripts/ubuntu/update.sh"
    },
    {
      "type": "shell",
      "script": "../scripts/Solr/provision.sh"
    },
    {
      "execute_command": "chmod +x {{ .Path }}; {{ .Vars }} sudo -E sh '{{ .Path }}'",
      "inline": [
        "/usr/sbin/waagent -force -deprovision+user && export HISTSIZE=0 && sync"
      ],
      "inline_shebang": "/bin/sh -e",
      "type": "shell"
    }]
}

The only other script mentioned is the “update.sh”, which has the following logic in it, to install the cli and update the ubuntu image:

#! /bin/bash

sudo apt-get update -y
sudo apt-get upgrade -y

#Azure-CLI
AZ_REPO=$(sudo lsb_release -cs)
sudo echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure-cli.list
sudo curl -L https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-get install apt-transport-https
sudo apt-get update && sudo apt-get install azure-cli

So the above gets me to a good place for being able to create an image with it configured.

For next steps I will be doing the following:

  • Building a more “production friendly” implementation of Solr into the script.
  • Investigating leveraging cloud init instead of the “golden image” experience with Packer.
  • Building out templates around the use of Zookeeper for managing the nodes.


Configuring Terraform Development Environment

So I’ve been doing a lot of work with a set of open source tools lately, specifically TerraForm and Packer. TerraForm at its core is a method of implementing truly Infrastructure as Code, and does so by providing a simple function style language where you can create basic implementations for the cloud, and then leverage resource providers to deploy. These resource providers allow you to deploy to variety of cloud platforms (the full list can be found here). It also provides robust support for debugging, targeting, and supports a desired state configuration approach that makes it much easier to maintain your environments in the cloud.

Now that being said, like most open source tools, it can require some configuration for your local development environment and I wanted to put this post together to describe it. Below are the steps to configuring your environment.

Step 1: Install Windows SubSystem on your Windows 10 Machine

To start with, you will need to be able to leverage bash as part of the Linux Subsystem. You can enable this on a Windows 10 machine, by following the steps outlined in this guide:

https://docs.microsoft.com/en-us/windows/wsl/install-win10

Once you’ve completed this step, you will be able to move forward with VS Code and the other components required.

Step 2: Install VS Code and Terraform Plugins

For this guide we recommend VS Code as your editor, VS code works on a variety of operating systems, and is a very light-weight code editor.

You can download VS Code from this link:

https://code.visualstudio.com/download

Once you’ve downloaded and installed VS code, we need to install the VS Code Extension for Terraform.

Then click “Install” and “Reload” when completed. This will allow you to have intelli-sense and support for the different terraform file types.

Step 3: Opening Terminal

You can then perform the remaining steps from the VS Code application. Go to the “View” menu and select “integrated terminal”. You will see the terminal appear at the bottom:

By default, the terminal is set to “powershell”, type in “Bash” to switch to Bash Scripting. You can default your shell following this guidance – https://code.visualstudio.com/docs/editor/integrated-terminal#_configuration

Step 4: Install Unzip on Subsystem

Run the following command to install “unzip” on your linux subsystem, this will be required to unzip both terraform and packer.

sudo apt-get install unzip

Step 5: Install TerraForm

You will need to execute the following commands to download and install Terraform, we need to start by getting the latest version of terraform.

Go to this link:

https://www.terraform.io/downloads.html

And copy the link for the appropriate version of the binaries for TerraForm.

Go back to VS Code, and enter the following commands:

wget {url for terraform}
unzip {terraform.zip file name}
sudo mv terraform /usr/local/bin/terraform
rm {terraform.zip file name}
terraform --version

Step 6: Install Packer

To start with, we need to get the most recent version of packer. Go to the following Url, and copy the url of the appropriate version.

https://www.packer.io/downloads.html

Go back to VS Code and execute the following commands:

wget {packer url} 
unzip {packer.zip file name} 
sudo mv packer /usr/local/bin/packer
rm {packer.zip file name}

Step 7: Install Azure CLI 2.0

Go back to VS Code again, and download / install azure CLI. To do so, execute the steps and commands found here:

https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-apt?view=azure-cli-latest

Step 8: Authenticating against Azure

Once this is done you are in a place where you can run terraform projects, but before you do, you need to authenticate against Azure. This can be done by running the following commands in the bash terminal (see link below):

https://docs.microsoft.com/en-us/azure/azure-government/documentation-government-get-started-connect-with-cli

Once that is completed, you will be authenticated against Azure and will be able to update the documentation for the various environments.

NOTE: Your authentication token will expire, should you get a message about an expired token, enter the command, to refresh:

az account get-access-token 

Token lifetimes can be described here – https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-token-and-claims#access-tokens

After that you are ready to use Terraform on your local machine.

Where to I start – Service Fabric?

So containers have become an essential part of modern application development. I would go as far to say that containers and micro services have had a similar impact to software development as “Object Oriented Programming”.

Now that being that I have been talking to a lot of people who use Monolithic applications and are looking for a way to break down their existing applications into a micro service approach and support the idea of using existing infrastructure, and don’t necessarily want to deploy on Linux for a variety of reasons.

Now based on that option, there is an established technology that can leverage your docker containers and orchestrate them in a windows environment. And that is Service Fabric.

I find the learning curve if you are looking at a monolithic application and breaking it into micro services is a lot easier to swallow with Service Fabric, and it does help you to break up your applications to make better use compute on your machines in the cluster and you can still leverage docker.

Below are some links to help you get started with Service Fabric if you are looking for information on this technology:

Concepts and Architecture:

Service Fabric Overview:

Coding Samples:

Videos:

Building out Azure Container Registry in Terraform

So I’ve previously done posts on the TerraForm template that I built to support creating a kubernetes cluster. The intention behind this was to provide a solution for standing up a kubernetes cluster in Azure Government. To see more information on that cluster I have a blog post here.

Now one of the questions I did get with it, is “How do we integrate this with Azure Container Registry?” And for those not familiar, Azure Container Registry is a PaaS offering that Azure provides that allows you to push your container images to a docker registry and not have to manage the underlying VM, patching, updates, and other maintenance. This allows you to just pay for the space to store the container images, which admittedly are very small.

The first part of implementing this logic was to create the Container Registry in TerraForm by using the following.

A key note is that the use of the “count” variable is to enable that this registry will not be created unless you create a “lkma” which is the VM that operates as the master.

resource "azurerm_container_registry" "container-registry" {
  count = "${lookup(var.instance_counts, "lkma", 0) == 0 ? 0 : 1}"
  name                = "containerRegistry1"
  resource_group_name = "${azurerm_resource_group.management.name}"
  location            = "${var.azure_location}"
  admin_enabled       = true
  sku                 = "Standard"

  depends_on = ["azurerm_role_assignment.kub-ad-sp-ra-kv1"]
}

So honestly didn’t require that much in the way of work. For the next part it is literally just adding a few lines of code to enable the connection between the registry and the kubernetes cluster. Those lines are the following :

echo 'Configure ACR registry for Kubernetes cluster'
kubectl create secret docker-registry <SECRET_NAME> --docker-server $5 --docker-email $6 --docker-username=$7 --docker-password $8
echo 'Script Completed'

So really that is about it. I’ve already made these changes to the GitHub template, so please check it out. The above lines of code allow a user principal information that I pass to the script to be used to connect the azure container registry to my cluster. That’s really about it.

Staying Organized with my “digital brain”.

Hello All, so as you probably noticed, I’m trying to do more posts, and trying to cover a wide range of topics. So for this talk, I thought I’d take time to talk about how I stay organized and stay on top of my day.

Productivity methods are a dime a dozen, and honestly everyone has their own flavor of an amalgamation of several methods to keep control of the chaos. For me, I went through a lot of iterations, and then finally settle on the system I describe here to keep myself on top of everything in my life.

Now for all the different variations out there, I know lots of people are still exploring options, which is why I decided to document mine here in hopes that it might help someone else.

So let’s start with tools, for me I use Microsoft To-Do, and its not just cause of where I work, but ultimately I use this tool because I was using Wunderlist, but ended up switching because they ended support of Wunderlist, replacing it with To-Do. So that was the driver, but I also did it, because it supports tags in the text of the items, which helps me to organize them.

So first, I break out my tasks into categories with a tag to start, the categories I use are:

  • Action: These are items that require me to take some small action, like send an email, make a phone call, reply to something, or answer a question. I try to keep these as small items.
  • Investigate: These are items that I need to research or look into, things that require me to do some digging to find an answer.
  • Discuss: These are items that I’ve made a note to get in touch with someone else and discuss a topic.
  • Build: These are my favorite kind of items, this is me taking coding action of some kind, and building something, or working out an idea. Where I am focused on the act of creating something.
  • Learn: These are items that involve my learning goals, to push myself to learn something new and keep it tactical.

Now each day, To-Do has this concept of “My Day” where you take tasks from your task list and indicate that they are going to be part of your day. Now I sort my day alphabetically so that the above items are organized in a way that lines up with how I approach them.

For me I usually tackle as many actions as I can right away and get them out of the way for the first hour of my day, and then spend the next 6 hours as a mix of new actions, and build / investigate actions.  Finally I have a set section of my week that is spent of learning activities.  The idea being to quote Bobby Axelrod, “The successful figure out how to both, handle the immediate while securing the future.”

Finally I maintain a separate list called #Waiting(…). When I am awaiting a response from someone, I change the category (like #Action) to #Waiting(name of person) and move it to the waiting list and take it off “My Day”. This let’s me put it out of my mind without losing track of the item.

After the category, I add the group, these are customer names for work, or a designation to describe the sub category of the work.  Like for example this is a monthly recurring task:

#Action – #Financial – Pay Monthly Bill’s

This allows me to quickly group the category or all “Financial” tasks if I need a big picture.  

I have been using this system for the past year and it’s done a lot to help me stay organized and measure my impact not activity.

I’ve talked previously about how import impact is over activity. And one of the downsides of many of these kinds of systems is that people tend to focus their energy on the “checking off items” and not on the overall impact of those items. I find by using this kind of grouping on the front I am able to focus energy on tasks that are high impact not low impact.

At the end of the day productivity itself is a lie and I believe that completely the idea is not to produce more, but to make every action have a return on investment.

Another book, Essentialism by Greg McKeown calls out this difference in basically saying that the key is to make the distinction of saying “what can I go big on?” or its either a “Hell Yes” or an “Absolute No”. So I find this system assists me by allowing me to make sure that I am focusing on tasks that will return dividends and not on topics that are smaller activity just to drive “checked” items.

Starting out with Data Science, where to go from here

Data Science, let’s get started!

You can’t turn around anymore without hearing people talk about data. It’s everywhere, and honestly its only growing more. Here are a couple of statistics I found interesting (all from Forbes). The big one being for me, 79% of enterprise executives agree that companies that do not embrace Big Data will lose their competitive position and could face extinction.

Let’s stop and think about that a minute, AI is all over the news, and people are constantly figuring out new ways to leverage data to encourage new innovations. It really is quite the time to be alive!

So that being said, I’ve decided to start to shift my focus and learn more about this field and how to leverage it for the future. And I know there are a lot of other people out there in the same boat as me, so I figured I’d start documenting it all here and help those looking for resources with some of the ones I’ve found.

So where to start, I started with a link to this course track:

Data Science Track from Microsoft Academy:
This takes a bunch of courses on EdX and puts them into a nice track and I find that this is a good 100 level entry to Data Science, AI and Analytics. I’m halfway through now, and working through the content and have found it helpful.

All Posts

Getting Started

So I thought I would start this new direction for the blog with a post about a topic I get asked about a lot.

“I want to get started in programming, how do I do that?”

And this is a great question, and one that makes a lot of sense to me as the lines between technology and business are blurring.  And more and more people are interacting with developers in their daily life as part of their current jobs, and its leading to people’s eyes being opened to the opportunities in this place.

My next question is normally “Why?” and at first that usually takes people back, but this is an important thing to ask yourself.  I ask this because to be honest, switching fields and taking on something like becoming a developer isn’t an easy journey, and if your motives aren’t clear than your going to set yourself up for failure.  I generally think it’s smart to ask if this is a worthwhile investment of your time.  Because as much as I love this industry, it can be quite brutal at times.

Don’t get me wrong, I’m not making these statements as some grand arbiter who decides if you are worthy of becoming an almighty developer.  I make these statements because the simple truth is that to work as a developer and achieve success you need to be willing to accept the reality, which is less Minority Report and steve jobs, and more the craziness of Silicon Valley.

I would tell you to ask the following questions:

  • Do you like continual education?  Are you willing to read about this stuff in your spare time?
  • Do you like to tinker with things?
  • Do you have “Grit”?

For the final question, specifically I’m referring to the fantastic book by Angela Duckworth, that describes Grit as basically being the intersection of Passion and Perseverance, and that it is the most important part of any equation where someone is hoping for success.  And I would argue, even more so true for developers.

If you look online at the “successful developers” they all have one thing in common…they live for this stuff.  And spend a lot of time doing it, and finding new ways to challenge themselves and push boundaries.  They are constantly looking for ways to change their mindset to find new opportunities and directions.  I don’t claim to be a famous developer, but I can tell you that I’m proud of where I’ve gone in my career and I genuinely love what I do, and much like those “famous developers”, my wife describes me as a “well documented nerd”.

So now the important question is, did I lose you?  If not, I think this is a rewarding career option that can take you in some interesting directions, but you need to know that it will be a slow burn.  This is not something where you will be writing award winning apps by Monday if you start on Friday.

Below are some links to help you out, feel free to reach out with questions, I’ve tried to provide a lot of training material and some notes about each link:

Visual Studio 2017 Community Edition – This is the primary IDE (integrated developer environment) for all things in the Micr0soft stack. And the community edition is free, which is even better.

https://www.visualstudio.com/downloads/

When you go to install it, its going to ask you to customize the install, by selecting different packages and what not.

Great start: C# fundamentals for Absolute Beginners:

https://mva.microsoft.com/en-US/training-courses/16169?l=Lvld4EQIC_2706218949

For training materials I would recommend the following:

Microsoft Virtual Academy – https://mva.microsoft.com/

This is a great site for a lot of training content Microsoft generates to help. I would point you to the absolute beginner classes as well as the learning paths. They also do a good job of categorizing training (100 level, 200 level etc)

C# Courses:

https://mva.microsoft.com/training-topics/c-app-development#!jobf=Developer&lang=1033

Visual Studio Training:

https://mva.microsoft.com/product-training/visual-studio-courses#!jobf=Developer&lang=1033

Getting started with Visual Studio 2017

https://mva.microsoft.com/en-US/training-courses/getting-started-with-visual-studio-2017-17798?l=9oIw0FD6D_3611787171

Learning Paths:

https://mva.microsoft.com/LearningPaths.aspx

Channel 9 – https://channel9.msdn.com/

Great site for general videos, and is updated all the time.

I recommend web as a good place to start, the .net web platform is called ASP.NET and uses HTML, C#, and some javascript to work. (C# and Javascript syntax are pretty close)

https://www.asp.net/get-started

https://www.asp.net/mvc/overview/getting-started

This is probably a good start.

How to punch up your resume?

So I thought given the new direction with this blog, I would focus my attention on some of the questions I get a lot.  And one of the  biggest questions I get asked frequently is “My resume is terrible, how do I make it better?”

To be perfectly honest, most people undervalue their resume, and think of it like some kind of checkbox.  I love hearing people say “I’m not worried, once I go in for the interview the resume is meaningless.”  To which my response is HOW DO YOU THINK YOU GET THE INTERVIEW!

There’s an old adage, that the you never get a second chance to make a first impression, and when applying for jobs, the resume is your first impression.  When I worked for a prior company, part of my job was interviewing new talent and determining if they were a good fit to move the organization forward.  As such, I literally conducted over 100 interviews in a 8 month a period.  I can say I’ve seen a lot of things, and this blog post is really based around the tips that would apply to help get your resume noticed and get you in the interview.

  1. DO NOT stick to one page:  In college they will tell you that your resume must be limited to one page.  That is not realistic for a technical position, because in these positions we are looking for the skills you have.  Don’t go crazy but a good three page resume is a lot better than Times New Roman, size 8 compressed onto a page.  The human eye needs white space more than anything.
  2. Keep it up to date:  This is jumping a little further ahead, but make sure it is 100% current.  I’ve read resumes of people and nothing turns the interviewer off more than to bring you in and here, “Here’s the stuff that I’ve been working on”.
  3. Describe the projects:  Even better than a list of skills is a project description, and acknowledging that you can’t give up all details.  But things like, “Project XYZ was a mobile app built with Xamarin with a Cosmos DB database back end, and I was the lead developer of the mobile side.” tells me a lot about what your skills are.
  4. Be clear about your role:  It helps if you tell me what you did on the project, and be clear about the responsibility not the titles.  I’ll give an example, my first job I was responsible for building software for managing test centers and grading certification exams with the state, but being the state my job title was “LAN Technican”, no even close.  So I found that you should try to change your title, just list what you did on the project.  It gives a clearer picture of what your skills are.
  5. Put in personal projects:  I used to tell people “I can teach someone to code the way I want, but I can’t teach passion”.  So if you’ve contributed to GitHub projects, put it in there, if you have apps in the app store, put them in there.  Talk to me about what you with, that shows perseverance and drive, which I can’t teach.  If you blog list that, if you work with user groups, put that.  I once had a candidate show “my son and I built a cloud enabled race car with a raspberry pi and a cell phone”, that’s fantastic information.  But make sure you limit it to what you’ve done.
  6. Be Honest about how much you’ve worked with something:  It’s a great idea to quantify your technical skills, you can use a 1-10 scale, or some other measure, on my resume I use a 1-5 scale.  This allows them to get a good assessment of your skills and saves everyone time.  And to be honest this is another one where you show “I’m learning Xamarin on my own” is huge.  Expect that during the technical interview you are going to be grilled on all these, and if you aren’t honest, that’s a guaranteed out (next post we talk about the technical interview).