Building a facial search in less than 2 hours

So, I’ve been pretty up front that I’ve been expanding my current skills in the data science and AI space, and its been a pretty fun process, and I wanted to point everyone to a demo I was able to build out very quickly.

Facial Searching, is a pretty common use case, so much so that we see it everywhere, Google Photos allows you to tag people in photos and indexes them for you automatically. Facebook makes suggestions for tagging people when you upload new pictures.

So I wanted to build a service that would take a set of selected images or 3 members of my family, and build a solution that would allow me to run any family photo through and it would search for the known members of my family to apply. Seems like a pretty basic use-case, but I’ve been wanting to get some hands on experience with the Azure Cognitive Services.

So I researched and read through our documentation, and decided before I started I was going to set aside 2 hours and see how far I could get to do the following:

  • Build a console app to read in images of 3 family members.
  • Build logic to upload an image and read back attributes about that person, things like age, gender, hair color, glasses, etc.
  • Build logic to search and match faces of people in a photo with the library that was previously uploaded.

The cool news is that I was able to implement all that logic. The full solution can be found here.

So to start, I focused on the first use case, and Azure Cognitive services has this concept of “PersonGroups” that can be leveraged with the SDK. For starters you need to install the sdk from nuget, and this is the required package.

Microsoft.Azure.CognitiveServices.Vision.Face

the first part is the key part is the client, which I configured in a parent class as follows:

public class BaseFaceApi
    {
        protected string _apiKey = ConfigurationManager.AppSettings["FaceApiKey"];
        protected string _apiUrl = ConfigurationManager.AppSettings["FaceApiUrl"];
        protected string _apiEndpoint = ConfigurationManager.AppSettings["FaceApiEndpoint"];
        protected FaceClient _client;

        protected void InitializeClient()
        {
            _client = new FaceClient(new ApiKeyServiceClientCredentials(_apiKey));
            _client.Endpoint = _apiEndpoint;
        }
    }

This allows for configuration to be the app.config, and this face client will be leveraged for all operations to hit the API.

The Face API leverages this concept of “PersonGroups” and “Persons” to handle the library of faces you are going to compare against. The process is broken into 4 parts.

  • Create the group
  • Create the person
  • Register Images for that person
  • Train the Model

If you review the source code you will find that I have broken these out to separate methods. The benefit of creating the groups is that you can limit your searching to specific groups, and have your application recognize the differences between groups.

Once you completed loading these images and “Persons” into the service you are ready to search through this repository by uploading an image. This is done with the following code:

public async Task<Dictionary<Guid,FacePerson>> IdentifyFaces(string filePath,string groupID)
        {
            InitializeClient();

            Dictionary<Guid,FacePerson> ret = new Dictionary<Guid, FacePerson>();

            using (Stream s = File.OpenRead(filePath))
            {
                // The list of Face attributes to return.
                IList<FaceAttributeType> faceAttributes =
                    new FaceAttributeType[]
                    {
            FaceAttributeType.Gender, FaceAttributeType.Age,
            FaceAttributeType.Smile, FaceAttributeType.Emotion,
            FaceAttributeType.Glasses, FaceAttributeType.Hair
                    };

                var facesTask = await _client.Face.DetectWithStreamWithHttpMessagesAsync(s,true,true,faceAttributes);
                var faceIds = facesTask.Body.Select(face => face.FaceId.Value).ToList();

                var identifyTask = await _client.Face.IdentifyWithHttpMessagesAsync(faceIds,groupID);
                foreach (var identifyResult in identifyTask.Body)
                {
                    Console.WriteLine("Result of face: {0}", identifyResult.FaceId);
                    if (identifyResult.Candidates.Count > 0)
                    { 
                        // Get top 1 among all candidates returned
                        var candidateId = identifyResult.Candidates[0].PersonId;
                        var person = await _client.PersonGroupPerson.GetWithHttpMessagesAsync(groupID, candidateId);

                        var fp = new FacePerson();
                        fp.PersonID = person.Body.PersonId;
                        fp.Name = person.Body.Name;
                        fp.FaceIds = person.Body.PersistedFaceIds.ToList();

                        var faceInstance = facesTask.Body.Where(f => f.FaceId.Value == identifyResult.FaceId).SingleOrDefault();
                        fp.Age = faceInstance.FaceAttributes.Age.ToString();
                        fp.EmotionAnger = faceInstance.FaceAttributes.Emotion.Anger.ToString();
                        fp.EmotionContempt = faceInstance.FaceAttributes.Emotion.Contempt.ToString();
                        fp.EmotionDisgust = faceInstance.FaceAttributes.Emotion.Disgust.ToString();
                        fp.EmotionFear = faceInstance.FaceAttributes.Emotion.Fear.ToString();
                        fp.EmotionHappiness = faceInstance.FaceAttributes.Emotion.Happiness.ToString();
                        fp.EmotionNeutral = faceInstance.FaceAttributes.Emotion.Neutral.ToString();
                        fp.EmotionSadness = faceInstance.FaceAttributes.Emotion.Sadness.ToString();
                        fp.EmotionSurprise = faceInstance.FaceAttributes.Emotion.Surprise.ToString();
                        fp.Gender = faceInstance.FaceAttributes.Gender.ToString();

                        ret.Add(person.Body.PersonId, fp);
                    }
                }
            }

            return ret;
        }

One key note above is the face attributes, this identifies the attributes you would like the service to review and discover. You can limit this list as you like.

Please feel free to review the sample code and I hope you find a great use-case. For me, a very cool project that is on my list next is to build a camera with a raspberry pi that captures people who come to the door and compares them against a known database of people.

It’s also worth mentioning that this service is fully available in Azure Government for customers that have requirements to be deployed in a sovereign cloud.

TerraForm Kubernetes Template

Hello All, I wanted to get a quick blog post out here based on something that I worked on, and finally is seeing the light of day.  I’ve been doing a lot of work with TerraForm, and one of the use cases I found was standing up a Kubernetes cluster.  And specifically I’ve been working with Azure Government, which does not have AKS available.  So how can I build a kubernetes cluster and minimize the lift of creating a cluster and then make it easy to add nodes to the cluster.  So the end result of that goal is here.

Below is a description of the project, and if you’d like to contribute please do, I have some ideas for phase 2 of this that I’m going to build out but I’d love to see what others come up with.

Intention:

The purpose of this template is to provide an easy-to-use approach to using an Infrastructure-as-a-service deployment to deploy a kubernetes cluster on Microsoft Azure. The goal being that you can start fresh with a standardized approach and preconfigured master and worker nodes.

How it works?

This template create a master node, and as many worker nodes as you specify, and during creation will automatically execute the scripts required to join those nodes to the cluster. The benefit of this being that once your cluster is created, all that is required to add additional nodes is to increase the count of the “lkwn” vm type, and reapply the template. This will cause the newe VMs to be created and the cluster will start orchestrating them automatically.

This template can also be built into a CI/CD pipeline to automatically provision the kubernetes cluster prior to pushing pods to it.

This guide is designed to help you navigate the use of this template to standup and manage the infrastructure required by a kubernetes cluster on azure. You will find the following documentation to assist:

  • Configure Terraform Development Environment: This page provides details on how to setup your locale machine to leverage this template and do development using Terraform, Packer, and VSCode.
  • Use this template: This document walks you through how to leverage this template to build out your kubernetes environment.
  • Understanding the template: This page describs how to understand the Terraform Template being used and walks you through its structure.

Key Contributors!

A special thanks to the following people who contributed to this template:
Brandon Rohrer: who introduced me to this template structure and how it works, as well as assisted with optimizing the functionality provided by this template.

Getting Started

So I thought I would start this new direction for the blog with a post about a topic I get asked about a lot.

“I want to get started in programming, how do I do that?”

And this is a great question, and one that makes a lot of sense to me as the lines between technology and business are blurring.  And more and more people are interacting with developers in their daily life as part of their current jobs, and its leading to people’s eyes being opened to the opportunities in this place.

My next question is normally “Why?” and at first that usually takes people back, but this is an important thing to ask yourself.  I ask this because to be honest, switching fields and taking on something like becoming a developer isn’t an easy journey, and if your motives aren’t clear than your going to set yourself up for failure.  I generally think it’s smart to ask if this is a worthwhile investment of your time.  Because as much as I love this industry, it can be quite brutal at times.

Don’t get me wrong, I’m not making these statements as some grand arbiter who decides if you are worthy of becoming an almighty developer.  I make these statements because the simple truth is that to work as a developer and achieve success you need to be willing to accept the reality, which is less Minority Report and steve jobs, and more the craziness of Silicon Valley.

I would tell you to ask the following questions:

  • Do you like continual education?  Are you willing to read about this stuff in your spare time?
  • Do you like to tinker with things?
  • Do you have “Grit”?

For the final question, specifically I’m referring to the fantastic book by Angela Duckworth, that describes Grit as basically being the intersection of Passion and Perseverance, and that it is the most important part of any equation where someone is hoping for success.  And I would argue, even more so true for developers.

If you look online at the “successful developers” they all have one thing in common…they live for this stuff.  And spend a lot of time doing it, and finding new ways to challenge themselves and push boundaries.  They are constantly looking for ways to change their mindset to find new opportunities and directions.  I don’t claim to be a famous developer, but I can tell you that I’m proud of where I’ve gone in my career and I genuinely love what I do, and much like those “famous developers”, my wife describes me as a “well documented nerd”.

So now the important question is, did I lose you?  If not, I think this is a rewarding career option that can take you in some interesting directions, but you need to know that it will be a slow burn.  This is not something where you will be writing award winning apps by Monday if you start on Friday.

Below are some links to help you out, feel free to reach out with questions, I’ve tried to provide a lot of training material and some notes about each link:

Visual Studio 2017 Community Edition – This is the primary IDE (integrated developer environment) for all things in the Micr0soft stack. And the community edition is free, which is even better.

https://www.visualstudio.com/downloads/

When you go to install it, its going to ask you to customize the install, by selecting different packages and what not.

Great start: C# fundamentals for Absolute Beginners:

https://mva.microsoft.com/en-US/training-courses/16169?l=Lvld4EQIC_2706218949

For training materials I would recommend the following:

Microsoft Virtual Academy – https://mva.microsoft.com/

This is a great site for a lot of training content Microsoft generates to help. I would point you to the absolute beginner classes as well as the learning paths. They also do a good job of categorizing training (100 level, 200 level etc)

C# Courses:

https://mva.microsoft.com/training-topics/c-app-development#!jobf=Developer&lang=1033

Visual Studio Training:

https://mva.microsoft.com/product-training/visual-studio-courses#!jobf=Developer&lang=1033

Getting started with Visual Studio 2017

https://mva.microsoft.com/en-US/training-courses/getting-started-with-visual-studio-2017-17798?l=9oIw0FD6D_3611787171

Learning Paths:

https://mva.microsoft.com/LearningPaths.aspx

Channel 9 – https://channel9.msdn.com/

Great site for general videos, and is updated all the time.

I recommend web as a good place to start, the .net web platform is called ASP.NET and uses HTML, C#, and some javascript to work. (C# and Javascript syntax are pretty close)

https://www.asp.net/get-started

https://www.asp.net/mvc/overview/getting-started

This is probably a good start.