Weekly Links – 9/16

Hello all, its another week. Been really busy as we are approaching the end of a quarter. Been doing a lot of code work this week on a project that has been delayed for too long. So really enjoying that.

See the source image

Now here we go with the business:

Development:

Cloud:

Audio / Video:

Fun Stuff:

So as I said last time I’m a bit of a gamer, and as this comic points to, I’m a well documented nerd. Lately I’ve found myself getting pulled back into tabletop gaming, specifically Dungeons and Dragons, and have a good game going (we play monthly, right now we’ve been done about 8 sessions). So as I get deeper into this, new things are announced all the time, and the new one is Unearthed Arcana, which is basically “Beta” content for the game for players to use.

The latest are two new sub classes Aberrant Mind sorcer, and Lurker of the Deep Warlock.

Weekly Links – 9/9

Welcome back everyone, for another weekly links post. The important note here is its fall, which means kids are in school, leaves are going to start turning and ….

See the source image

So down to business:

Development:

Cloud:

Audio / Video:

Fun Stuff:

I’m a bit of a gamer, as he said to the surprise of no one. And its official that on September 5th, the new Gears of War 5 was available for Early Access period. The world wide release is September 10th. Very awesome. I always enjoyed Gears of War because it is one seriously intense game.

Here’s the article. Warning, Mature audiences.

Weekly Links – 9/3

Hello All, I’m a little late getting this out the door, but a fully sick family, complete with a side-helping of pneumonia for me didn’t help this weekend. But here it is.

See the source image

Development:

Cloud:

Video and Audio:

Weekly Links – 8/26

End of summer, school is back in session. I’m just going to leave this here:

See the source image

But here are the latest for this week:

Development:

Cloud:

Videos / Audio:

Leveraging Azure Search with Python

So lately I’ve been working on a side project, to showcase some of the capabilities in Azure with regard to PaaS services, and the one I’ve become the most engaged with is Azure Search.

So let’s start with the obvious question, what is Azure Search? Azure Search is a Platform-as-a-Service offering that allows for implementing search as part of your cloud solution in a scalable manner.

Below are some links on the basics of “What is Azure Search?”

The first part is how to create a search service, and really I find the easiest way is to create it via CLI:

az search service create --name {name} --resource-group {group} --location {location}

So after you create an Azure Search Service, the next part is to create all the pieces needed. For this, I’ve been doing work with the REST API via Python to manage these elements, so you will see that code here.

  • Create the data source
  • Create an index
  • Create an indexer
  • Run the Indexer
  • Get the indexer status
  • Run the Search

Project Description:

For this post, I’m building a search index that crawls through the data compiled from the Chicago Data Portal, which makes statistics and public information available via their API. This solution is pulling in data from that API into cosmos db to make that information searchable. I am using only publicly consumable information as part of this. The information on the portal can be found here.

Create the Data Source

So, the first part of any search discussion, is that you need to have a data source that you can search. Can’t get far without that. So the question becomes, what do you want to search. Azure Search supports a wide variety of data sources, and for the purposes of this discussion, I am pointing it at Cosmos Db. The intention is to search the contents of a cosmos db to ensure that I can pull back relevant entries.

Below is the code that I used to create the data source for the search:

import json
import requests
from pprint import pprint

#The url of your search service
url = 'https://[Url of the search service]/datasources?api-version=2017-11-11'
print(url)

#The API Key for your search service
api_key = '[api key for the search service]'


headers = {
    'Content-Type': 'application/json',
    'api-key': api_key
}

data = {
    'name': 'cosmos-crime',
    'type': 'documentdb',
    'credentials': {'connectionString': '[connection string for cosmos db]'},
    'container': {'name': '[collection name]'}
}

data = json.dumps(data)
print(type(data))

response = requests.post(url, data=data, headers=headers)
pprint(response.status_code)

To get the API key, you need the management key which can be found with the following command:

az search admin-key show --service-name [name of the service] -g [name of the resource group]

After running the above you will have created a data source to connect to for searching.

Create an Index

Once you have the above datasource, the next step is to create an index. This index is what Azure Search will map your data to, and how it will actually perform searches in the future. So ultimately think of this as the format your search will be in after completion. To create the index, use the following code:

import json
import requests
from pprint import pprint

url = 'https://[Url of the search service]/indexes?api-version=2017-11-11'
print(url)

api_key = '[api key for the search service]'

headers = {
    'Content-Type': 'application/json',
    'api-key': api_key
}

data = {
     "name": "crimes",  
     "fields": [
       {"name": "id", "type": "Edm.String", "key":"true", "searchable": "false"},
       {"name": "iucr","type": "Edm.String", "searchable":"true", "filterable":"true", "facetable":"true"},
       {"name": "location_description","type":"Edm.String", "searchable":"true", "filterable":"true"},
       {"name": "primary_description","type":"Edm.String","searchable":"true","filterable":"true"},
       {"name": "secondary_description","type":"Edm.String","searchable":"true","filterable":"true"},
       {"name": "arrest","type":"Edm.String","facetable":"true","filterable":"true"},
       {"name": "beat","type":"Edm.Double","filterable":"true","facetable":"true"},
       {"name": "block", "type":"Edm.String","filterable":"true","searchable":"true","facetable":"true"},
       {"name": "case","type":"Edm.String","searchable":"true"},
       {"name": "date_occurrence","type":"Edm.DateTimeOffset","filterable":"true"},
       {"name": "domestic","type":"Edm.String","filterable":"true","facetable":"true"},
       {"name": "fbi_cd", "type":"Edm.String","filterable":"true"},
       {"name": "ward","type":"Edm.Double", "filterable":"true","facetable":"true"},
       {"name": "location","type":"Edm.GeographyPoint"}
      ]
     }

data = json.dumps(data)
print(type(data))

response = requests.post(url, data=data, headers=headers)
pprint(response.status_code)

Using the above code, I’ve identified the different data types of the final product, and these all map to the data types identified for azure search. The supported data types can be found here.

Its worth mentioning, that there are other key attributes above to consider:

  • facetable: This denotes if this data is able to be faceted. For example, in Yelp if I bring back a search for cost, all restuarants have a “$” to “$$$$$” rating, and I want to be able to group results based on this facet.
  • filterable: This denotes if the dataset can be filtered based on those values.
  • searchable: This denotes whether or not the field is having a full-text search performed on it, and is limited in the different types of data that can used to perform the search.

Creating an indexer

So the next step is to create the indexer. The purpose of the indexer is that this does the real work. The indexer is responsible for performing the following operations:

  • Connect to the data source
  • Pull in the data and put it into the appropriate format for the index
  • Perform any data transformations
  • Manage pulling in no data ongoing
import json
import requests
from pprint import pprint

url = 'https://[Url of the search service]/indexers?api-version=2017-11-11'
print(url)

api_key = '[api key for the search service]'

headers = {
    'Content-Type': 'application/json',
    'api-key': api_key
}

data = {
    "name": "cosmos-crime-indexer",
    "dataSourceName": "cosmos-crime",
    "targetIndexName": "crimes",
    "schedule": {"interval": "PT2H"},
    "fieldMappings": [
        {"sourceFieldName": "iucr", "targetFieldName": "iucr"},
        {"sourceFieldName": "location_description", "targetFieldName": "location_description"},
        {"sourceFieldName": "primary_decsription", "targetFieldName": "primary_description"},
        {"sourceFieldName": "secondary_description", "targetFieldName": "secondary_description"},
        {"sourceFieldName": "arrest", "targetFieldName": "arrest"},
        {"sourceFieldName": "beat", "targetFieldName": "beat"},
        {"sourceFieldName": "block", "targetFieldName": "block"},
        {"sourceFieldName": "casenumber", "targetFieldName": "case"},
        {"sourceFieldName": "date_of_occurrence", "targetFieldName": "date_occurrence"},
        {"sourceFieldName": "domestic", "targetFieldName": "domestic"},
        {"sourceFieldName": "fbi_cd", "targetFieldName": "fbi_cd"},
        {"sourceFieldName": "ward", "targetFieldName": "ward"},
        {"sourceFieldName": "location", "targetFieldName":"location"}
    ]
}

data = json.dumps(data)
print(type(data))

response = requests.post(url, data=data, headers=headers)
pprint(response.status_code)

What you will notice is that for each field, two attributes are assigned:

  • targetFieldName: This is the field in the index that you are targeting.
  • sourceFieldName: This is the field name according to the data source.

Run the indexer

Once you’ve created the indexer, the next step is to run it. This will cause indexer to pull data into the index:

import json
import requests
from pprint import pprint

url = 'https://[Url of the search service]/indexers/cosmos-crime-indexer/run/?api-version=2017-11-11'
print(url)

api_key = '[api key for the search service]'

headers = {
    'Content-Type': 'application/json',
    'api-key': api_key
}

reseturl = 'https://[Url of the search service]/indexers/cosmos-crime-indexer/reset/?api-version=2017-11-11'

resetResponse = requests.post(reseturl, headers=headers)

response = requests.post(url, headers=headers)
pprint(response.status_code)

By triggering the “running” the indexer which will load the index.

Getting the indexer status

Now, depending the size of your data source, this indexing process could take time, so I wanted to provide a rest call that will let you get the status of the indexer.

import json
import requests
from pprint import pprint

url = 'https://[Url of the search service]/indexers/cosmos-crime-indexer/status/?api-version=2017-11-11'
print(url)

api_key = '[api key for the search service]'

headers = {
    'Content-Type': 'application/json',
    'api-key': api_key
}

response = requests.get(url, headers=headers)
index_list = response.json()
pprint(index_list)

This will provide you with the status of the indexer, so that you can find out when it completes.

Run the search

Finally if you want to confirm the search is working afterward, you can do the following:

import json
import requests
from pprint import pprint

url = 'https://[Url of the search service]/indexes/crimes/docs?api-version=2017-11-11'
print(url)

api_key = '[api key for the search service]'

headers = {
    'Content-Type': 'application/json',
    'api-key': api_key
}

response = requests.get(url, headers=headers)
index_list = response.json()
pprint(index_list)

This will bring back the results of the search. This will bring back everything as it is an empty string search.

I hope this helps with your configuring of Azure Search, Happy searching :)!

Weekly Links – 8/19

Another week of links, right now I’m running around disney world with the family but managed to still get this out the door.

Development

Cloud

Audio / Video:

See you all next week!

Weekly Links – 8/13

Hello all, I’m trying to resurrect this post type, and below are some links I found interesting.

Cloud:

DevOps / Development Tools:

Mobile:

Podcasts / Videos:

Book Review – Multipliers

So I have another book review, and honestly I’ve found myself traveling so there will be probably quite a few of these as I continue to have time to kill on a plane. The latest book I just finished was “Multipliers” by Liz Weisman and Greg McKeown.

Now I’ve read a book by Greg McKeown before, Essentialism, and found it to be really an excellent and thought provoking read. It caused me to re-examine a lot of the ways I’d approached things in my life. So I was very excited to finally get around to reading the book he wrote before with Lizi Weisman.

This book takes the position that their are two types of leaders in this world, Multipliers and Diminishers. The earlier being the type of leader that causes their teams to aspire to great heights, and to rise to meet an impossible challenge. The later being the type of leader who crushes the spirit of the people they lead, causing them to deliver less and less.

I found this book to be rather insightful and interesting, as it made me question the type of leader that I want to be. Now that being said, I do feel like to say that all leaders fall into one of two buckets, is a bit of a falsehood. I believe all leaders have elements of both diminishers and multipliers in their approach as no one is perfect.

But what I found in this book is that the examples are pretty dramatic, and in that regard its easy to say “I’m not that bad”. But I found it eye opening in that it made me re-evaluate how I approach leadership. I think the focus of the book is on a “binary” nature of these two types of leaders, and to be honest I don’t find that I totally agree with that assessment.

After reading this, I’m convinced, that in a way similar to the nature of introverted vs extroverted aren’t binary but a sliding scale, I believe the same can be said for multipliers and diminishers. Most of the leaders I’ve worked with are somewhere on that scale, but no one is perfect by any stretch of the imagination. But I don’t feel that comes across in the author’s description, and that could because of the reliance on dramatic examples.

The other element I found in this book, is that it does focus on what I call “Official leadership”, which is having an official title or position that puts you in a position of leadership. But in my experience, leadership includes people who are not in a position of authority but who act as leaders. In the beginning of the book it seems to exclusively focus on the earlier, and it is easy to say “this doesn’t apply to me” but I find that is not true.

Overall I found this to be pretty insightful book. Below is a talk from Liz Wiseman at a CEO summit about the content of the book.

Capturing Web Cam Pictures with .net

So I recently was working on a project where we needed to be able to have a web cam on a laptop take pictures, and then send those images against a web api endpoint.

Basically the use-cases behind this are plenty, but this was around work done to support using Microsoft Cognitive services. The project itself being a slimmed down version of the intelligent kiosk from Microsoft.

So I have to be honest, I expected this problem to be a lot harder than it actually is. There is a great library that made this work called AForge.Video, that I was able to install from nuget, and from there this is the code required:

static void Main(string[] args)
        {
            // enumerate video devices
            var videoDevices = new FilterInfoCollection(
                    FilterCategory.VideoInputDevice);
            // create video source
            VideoCaptureDevice videoSource = new VideoCaptureDevice(
                    videoDevices[0].MonikerString);
            // set NewFrame event handler
            videoSource.NewFrame += new NewFrameEventHandler(video_NewFrame);

            videoSource.ProvideSnapshots = true;

            // start the video source
            videoSource.Start();
            
            //videoSource.SignalToStop();


            Console.ReadLine();
        }

The above code is just identifying the video / photo capture devices available on this machine, and leveraging the first. And then connecting an event to handle new frame capture.

From there, once I turn on “videoSource.Start();” the application starts executing the NewFrameEventHandler to process it.

private static void video_NewFrame(object sender,
                    NewFrameEventArgs eventArgs)
        {
            // get new frame
            Bitmap bitmap = eventArgs.Frame;

            var fileName = @"C:\temp\camera\File_Frame.jpg";

            //bitmap.Save(string.Format(fileName));

            EncoderParameters encoderParameters = new EncoderParameters(1);
            encoderParameters.Param[0] = new EncoderParameter(System.Drawing.Imaging.Encoder.Quality, 100L);
            bitmap.Save(fileName, GetEncoder(ImageFormat.Jpeg), encoderParameters);

            var bytes = bitmap.ToByteArray(ImageFormat.Bmp);

            Thread.Sleep(1000);
        }

public static ImageCodecInfo GetEncoder(ImageFormat format)
        {
            ImageCodecInfo[] codecs = ImageCodecInfo.GetImageDecoders();

            foreach (ImageCodecInfo codec in codecs)
            {
                if (codec.FormatID == format.Guid)
                {
                    return codec;
                }
            }

            return null;
        }

Now the event handler above will take the code, and extract the bitmp, convert it to a Jpg and save the file. But additionally I’ve added the logic at the end to convert it to a byte array. This would allow you to push this up to an HTTP endpoint for processing by any services you need. Pretty simple for 74 line of code.

How to run a meeting that actually gets something done.

So for this post I wanted to do something more around soft skills, and I’ve actually had a couple of people ask me to write something up about this. Running a meeting is not the easiest thing in the world, but there always seems to be this perception that everyone should know how to without any guidance or instruction.

As part of my day job, I’m a pre-sales resource, so that means I run and coordinate a lot of meetings with a wide variety of people, everything from tech talks for developers, to regular cadence check-ins, and business strategy sessions with executives. And over the years I’ve come up with some tips and tricks to ensure that those meetings are productive, efficient, and don’t waste anyone’s time. Here are some tips to help if you find yourself in a position of having to run meetings and want to make sure they are productive.

Tip #1 – Time is valuable

This is more of a guiding principle than a tip, and one that you should take to heart immediately, and it really is the foundation of everything else in this blog. Everyone is busy, all the time…we live in a connected world where multi-tasking is the new normal. If someone is having a meeting with you, they are giving you the two most important things they have, time and attention. You need to treat these as valuable resources to be utilized appropriately, and not something to waste. This means do the following:

  • Be on time – This is common sense, you will never recover from arriving late, it already convinces the people your meeting with that you don’t see their time as valuable.
  • End on time or EARLY – I know, blasphemy, but if I can end early, I always do. Your customers will thank you for this. I don’t rush meetings, but if we accomplish what we need to, just wrap up, no need to draw things out just because of the time block.
  • Make sure you have enough to justify the meeting: Not everything needs to be a meeting, sometime a phone call will do. Always better to do things via a phone call than email, but if one of these can replace a meeting, take that option.

Tip #2 – Begin with the end in mind

This goes to the points above about making sure you have enough to justify the meeting, and looking at how valuable both the people you are meeting with’s time is, and how valuable your time is. The first question you should be clear on is…”What do I hope to accomplish here?”

This goes to Steven Covey’s principle, “Begin with the end in mind.” If you can’t answer this question, don’t waste anyone’s time. But if you can, great, use that to structure the rest of the meeting and work backwards.

For example, if the goal is to get approval to embrace a new technology, start with the problems it solves. Give them a reason to care and then work backwards into what it takes to implement.

If the goal of the meeting is to understand the ramifications of an old technology start with the downsides of the status quo and work towards the solution.

Make sure you know what your goal is because this provides a key metric for success and you can then objectively measure when the meeting is over.

Tip #3 – Have an agenda out ahead of time

This is another facet of the above, never go into a meeting without an agenda, even if its informal. You need to know in your mind how the meeting will run, and keep things focused on the goals you identified above.

Whenever possible, send out that agenda to let the attendees know exactly what will be covered. This is important not just because they know how it will flow, but it can help your attendees to identify people who they should include to make the meeting productive.

Tip #4 – Don’t skip small talk

This is the most common mistake I’ve seen with people, they are too focused on the immediate. Small talk before the meeting begins is important, this is how you build a relationship and re pore with your customer. If you don’t take time to build the relationship and help them see you as a person, it will hurt your credibility in the long run.

Now its important to know when to cut this off, and keep things light, butt having small talk before a meeting helps to make people comfortable. The more you can get to know people and reference things they’ve said in future meetings drives home that you respect them and care about them as a person.

Tip #5 – Be respectful of their time

Start on time. Period. This is not hard people, do not start late if you can avoid it. This shows that you have no respect for their time which as I previously said is the most important thing they have to give.

Also, if it looks like you might run over, make sure to give them an out, something like “I want to be respectful of your time, and we have 3 minutes…” and start to wrap it up. If they want to go long, they will allow you too. But this gives them an out and shows you care.

Tip #6 – Do introductions

If its a larger meeting, make sure you encourage introductions, and not just you and your team, but make sure everyone on the call or at the table introduces themselves. This shows each person in the room you see them and care about them, and want to hear their voice. This helps to be inclusive in making sure everyone feels comfortable.

Also resist the urge to introduce other people, let them introduce themselves, and what I mean by this is say something like “and given this topic, I wanted to bring Claire to this conversation…Claire, can you introduce yourself.”

Tip #7 – Never leave without confirming actions

Always make sure at the end of the meeting that you summarize the action items, take 5 minutes at the end to say that “These are the items I heard that have follow-up involved…” and make sure you say a name of a person with each item to drive home who is responsible. Also ask the customer for confirmation. This makes sure each person is aware of actions and expectations before they leave. This will make it easier to engage after the meeting.

Tip #8 – Your agenda should not be iron clad…be flexible

Another common mistake I see a lot, is people get too attached to their agenda. They say well we are supposed to cover that last, so “you customer have to wait”. This is a mistake, as I said before its their time, so if they want to restructure things, you should allow it. Now I say this with a couple of rules:

  • It has to be on topic with the intent of the meeting.
  • There needs to be agreement from the team for the change of direction.
  • And all appropriate people need to be at the table.
  • If the order matters and you can address it very soon.

This is a fine line, but ultimately it goes back to tip 1, which is remember this is their time, and your agenda is not important than their time.

Tip #9 – Don’t get derailed

During any meeting, some times you get someone who will try to derail the meeting to meet their own needs, never dismiss these concerns but if you have to push them off as out of scope, but give them the validation around when you will address the topic. Something like “That’s a little out of scope, but see me after and we can address those concerns.”