Penny Pinching in the Cloud: Running and Managing LOTS of Web Apps on a single Azure App Service

I’ve blogged before about “penny pinching in the cloud.” I’ll update that series for 2017 soon, but the underlying concepts still apply. Many if you are still using bigger virtual machines than are needed when doing IaaS (Infrastructure as a Service) or when doing PaaS (Platform as a Service) folks are doing “one website per App Service.” That’s super expensive.

Remember that you can fit as many web applications as memory and CPU will into an Azure App Service Plan. An “App Service Plan” in Azure is effectively the Virtual Machine under your Web Apps. You don’t need to think about it as it’s totally managed and hidden – but – if you choose think about it you’ll be able to squeeze more out of it and you’ll pay less.

For example, I have 20 web applications running in a plan I named “DefaultServerFarm.” It’s a Small Standard Plan (S1) and I pay about $70 a month. Some folks use a Basic (B1) plan if they don’t need to scale out and that’s about $50 a month. Both B1 and S1 support “unlimited” web apps within them, to the limits of memory. That’s what allows me to run 20 modest (but real) sites on the one plan and that’s what makes it a good deal from a pricing perspective for me.

I logged in to the Azure Portal recently and noticed the CPU percentage on my plan was higher than usual and higher than I’d like.

Why is that web app using so much CPU?

That’s the CPU of the machine “under” my 20 sites. I can click here on my App Service Plan’s “blade” to see the underlying sites, or just click “Apps” in the blade menu.

Running 20 apps in a Single Azure App Service

However, when I’m looking at an app that lives within my plan, there’s two super powerful menu items to check out. One is  called “Metrics per instance (Apps)” and one is “Metrics per instance (App Service).” Click the latter option. For many of you it’s going to become your favorite area in the Azure Portal. It was a game changer for me as it gave me the internal insight I needed to make sure I can get maximum density in my plan (thereby saving the most money).

Metrics per Instance - App Service Plan

I click here and see “Sites in App Service Plan.”

20 sites in a single plan

I can see that over the last few days my CPU has been going up and up…

The CPU is going up and up over a few days

I can see by site:

A graph showing ALL 20 sites and their CPU

So now I can filter by site and I see that it’s ONE site that’s going nuts.

One site is using all the CPU

I can then dig in, go to the main CPU charge and see exactly when it started:

The site is using 2.12 days of CPU

I can change the scale

It started on Feb 11th

I had a Web Job stuck in a loop. I restarted and will be monitoring but for now, I’m in a much better place for this one app.

Now it's calming down

Now if I check the App Service Plan itself, I can see everything has calmed down.

Things have calmed down after the one rogue site was restarted

The point here is that even though it’s “Platform as a Service” and we want a layer of abstraction, at no point are things HIDDEN from us. If you want to see the hardware, you can. If you want to see the process tree, you can. A good reminder.


Sponsor: Excited about the future in ASP.NET? The folks at Progress held an awesome webinar which gives a 360⁰ view of the new ASP.NET Core and how it compares to WebForms and MVC. Watch it now on demand!


© 2016 Scott Hanselman. All rights reserved.
     

I've blogged before about "penny pinching in the cloud." I'll update that series for 2017 soon, but the underlying concepts still apply. Many if you are still using bigger virtual machines than are needed when doing IaaS (Infrastructure as a Service) or when doing PaaS (Platform as a Service) folks are doing "one website per App Service." That's super expensive.

Remember that you can fit as many web applications as memory and CPU will into an Azure App Service Plan. An "App Service Plan" in Azure is effectively the Virtual Machine under your Web Apps. You don't need to think about it as it's totally managed and hidden - but - if you choose think about it you'll be able to squeeze more out of it and you'll pay less.

For example, I have 20 web applications running in a plan I named "DefaultServerFarm." It's a Small Standard Plan (S1) and I pay about $70 a month. Some folks use a Basic (B1) plan if they don't need to scale out and that's about $50 a month. Both B1 and S1 support "unlimited" web apps within them, to the limits of memory. That's what allows me to run 20 modest (but real) sites on the one plan and that's what makes it a good deal from a pricing perspective for me.

I logged in to the Azure Portal recently and noticed the CPU percentage on my plan was higher than usual and higher than I'd like.

Why is that web app using so much CPU?

That's the CPU of the machine "under" my 20 sites. I can click here on my App Service Plan's "blade" to see the underlying sites, or just click "Apps" in the blade menu.

Running 20 apps in a Single Azure App Service

However, when I'm looking at an app that lives within my plan, there's two super powerful menu items to check out. One is  called "Metrics per instance (Apps)" and one is "Metrics per instance (App Service)." Click the latter option. For many of you it's going to become your favorite area in the Azure Portal. It was a game changer for me as it gave me the internal insight I needed to make sure I can get maximum density in my plan (thereby saving the most money).

Metrics per Instance - App Service Plan

I click here and see "Sites in App Service Plan."

20 sites in a single plan

I can see that over the last few days my CPU has been going up and up...

The CPU is going up and up over a few days

I can see by site:

A graph showing ALL 20 sites and their CPU

So now I can filter by site and I see that it's ONE site that's going nuts.

One site is using all the CPU

I can then dig in, go to the main CPU charge and see exactly when it started:

The site is using 2.12 days of CPU

I can change the scale

It started on Feb 11th

I had a Web Job stuck in a loop. I restarted and will be monitoring but for now, I'm in a much better place for this one app.

Now it's calming down

Now if I check the App Service Plan itself, I can see everything has calmed down.

Things have calmed down after the one rogue site was restarted

The point here is that even though it's "Platform as a Service" and we want a layer of abstraction, at no point are things HIDDEN from us. If you want to see the hardware, you can. If you want to see the process tree, you can. A good reminder.


Sponsor: Excited about the future in ASP.NET? The folks at Progress held an awesome webinar which gives a 360⁰ view of the new ASP.NET Core and how it compares to WebForms and MVC. Watch it now on demand!



© 2016 Scott Hanselman. All rights reserved.
     

What is Serverless Computing? Exploring Azure Functions

There’s a lot of confusing terms in the Cloud space. And that’s not counting the term “Cloud.” 😉

  • IaaS (Infrastructure as a Services) – Virtual Machines and stuff on demand.
  • PaaS (Platform as a Service) – You deploy your apps but try not to think about the Virtual Machines underneath. They exist, but we pretend they don’t until forced.
  • SaaS (Software as a Service) – Stuff like Office 365 and Gmail. You pay a subscription and you get email/whatever as a service. It Just Works.

“Serverless Computing” doesn’t really mean there’s no server. Serverless means there’s no server you need to worry about. That might sound like PaaS, but it’s higher level that than.

Serverless Computing is like this – Your code, a slider bar, and your credit card. You just have your function out there and it will scale as long as you can pay for it. It’s as close to “cloudy” as The Cloud can get.

Serverless Computing is like this. Your code, a slider bar, and your credit card.

With Platform as a Service, you might made a Node or C# app, check it into Git, deploy it to a Web Site/Application, and then you’ve got an endpoint. You might scale it up (get more CPU/Memory/Disk) or out (have 1, 2, n instances of the Web App) but it’s not seamless. It’s totally cool, to be clear, but you’re always aware of the servers.

New cloud systems like Amazon Lambda and Azure Functions have you upload some code and it’s running seconds later. You can have continuous jobs, functions that run on a triggered event, or make Web APIs or Webhooks that are just a function with a URL.

I’m going to see how quickly I can make a Web API with Serverless Computing.

I’ll go to http://functions.azure.com and make a new function. If you don’t have an account you can sign up free.

Getting started with Azure Functions

You can make a function in JavaScript or C#.

Getting started with Azure Functions - Create This Function

Once you’re into the Azure Function Editor, click “New Function” and you’ve got dozens of templates and code examples for things like:

  • Find a face in an image and store the rectangle of where the face is.
  • Run a function and comment on a GitHub issue when a GitHub webhook is triggered
  • Update a storage blob when an HTTP Request comes in
  • Load entities from a database or storage table

I figured I’d change the first example. It is a trigger that sees an image in storage, calls a cognitive services API to get the location of the face, then stores the data. I wanted to change it to:

  • Take an image as input from an HTTP Post
  • Draw a rectangle around the face
  • Return the new image

You can do this work from Git/GitHub but for easy stuff I’m literally doing it all in the browser. Here’s what it looks like.

Azure Functions can be done in the browser

I code and iterate and save and fail fast, fail often. Here’s the starter code I based it on. Remember, that this is a starter function that runs on a triggered event, so note its Run()…I’m going to change this.

#r "Microsoft.WindowsAzure.Storage"
#r "Newtonsoft.Json"
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json;
using Microsoft.WindowsAzure.Storage.Table;
using System.IO; 
public static async Task Run(Stream image, string name, IAsyncCollector<FaceRectangle> outTable, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    
    string result = await CallVisionAPI(image); //STREAM
    log.Info(result); 
    if (String.IsNullOrEmpty(result))
    {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    foreach (Face face in imageData.Faces)
    {
        var faceRectangle = face.FaceRectangle;
        faceRectangle.RowKey = Guid.NewGuid().ToString();
        faceRectangle.PartitionKey = "Functions";
        faceRectangle.ImageFile = name + ".jpg";
        await outTable.AddAsync(faceRectangle); 
    }
    return req.CreateResponse(HttpStatusCode.OK, "Nice Job");  
}
static async Task<string> CallVisionAPI(Stream image)
{
    using (var client = new HttpClient())
    {
        var content = new StreamContent(image);
        var url = "https://api.projectoxford.ai/vision/v1.0/analyze?visualFeatures=Faces";
        client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", Environment.GetEnvironmentVariable("Vision_API_Subscription_Key"));
        content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
        var httpResponse = await client.PostAsync(url, content);
        if (httpResponse.StatusCode == HttpStatusCode.OK){
            return await httpResponse.Content.ReadAsStringAsync();
        }
    }
    return null;
}
public class ImageData {
    public List<Face> Faces { get; set; }
}
public class Face {
    public int Age { get; set; }
    public string Gender { get; set; }
    public FaceRectangle FaceRectangle { get; set; }
}
public class FaceRectangle : TableEntity {
    public string ImageFile { get; set; }
    public int Left { get; set; }
    public int Top { get; set; }
    public int Width { get; set; }
    public int Height { get; set; }
}

GOAL: I’ll change this Run() and make this listen for an HTTP request that contains an image, read the image that’s POSTed in (ya, I know, no validation), draw rectangle around detected faces, then return a new image.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log) {
var image = await req.Content.ReadAsStreamAsync();

As for the body of this function, I’m 20% sure I’m using too many MemoryStreams but they are getting disposed so take this code as a initial proof of concept. However, I DO need at least the two I have. Regardless, happy to chat with those who know more, but it’s more subtle than even I thought. That said, basically call out to the API, get back some face data that looks like this:

2016-08-26T23:59:26.741 {"requestId":"8be222ff-98cc-4019-8038-c22eeffa63ed","metadata":{"width":2808,"height":1872,"format":"Jpeg"},"faces":[{"age":41,"gender":"Male","faceRectangle":{"left":1059,"top":671,"width":466,"height":466}},{"age":41,"gender":"Male","faceRectangle":{"left":1916,"top":702,"width":448,"height":448}}]}

Then take that data and DRAW a Rectangle over the faces detected.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    MemoryStream mem = new MemoryStream();
    image.CopyTo(mem); //make a copy since one gets destroy in the other API. Lame, I know.
    image.Position = 0;
    mem.Position = 0;
    
    string result = await CallVisionAPI(image); 
    log.Info(result); 
    if (String.IsNullOrEmpty(result)) {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    MemoryStream outputStream = new MemoryStream();
    using(Image maybeFace = Image.FromStream(mem, true))
    {
        using (Graphics g = Graphics.FromImage(maybeFace))
        {
            Pen yellowPen = new Pen(Color.Yellow, 4);
            foreach (Face face in imageData.Faces)
            {
                var faceRectangle = face.FaceRectangle;
                g.DrawRectangle(yellowPen, 
                    faceRectangle.Left, faceRectangle.Top, 
                    faceRectangle.Width, faceRectangle.Height);
            }
        }
        maybeFace.Save(outputStream, ImageFormat.Jpeg);
    }
    
    var response = new HttpResponseMessage()
    {
        Content = new ByteArrayContent(outputStream.ToArray()),
        StatusCode = HttpStatusCode.OK,
    };
    response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
    return response;
}

 

Now I go into Postman and POST an image to my new Azure Function endpoint. Here I uploaded a flattering picture of me and unflattering picture of The Oatmeal. He’s pretty in real life just NOT HERE. 😉

Image Recognition with Azure Functions

So in just about 15 min with no idea and armed with just my browser, Postman (also my browser), Google/StackOverflow, and Azure Functions I’ve got a backend proof of concept.

Azure Functions supports Node.js, C#, F#, Python, PHP *and* Batch, Bash, and PowerShell, which really opens it up to basically anyone. You can use them for anything when you just want a function (or more) out there on the web. Send stuff to Slack, automate your house, update GitHub issues, act as a Webhook, etc. There’s some great 3d party Azure Functions sample code in this GitHub repo as well. Inputs can be from basically anywhere and outputs can be basically anywhere. If those anywheres are also cloud services like Tables or Storage, you’ve got a “serverless backed” that is easy to scale.

I’m still learning, but I can see when I’d want a VM (total control) vs a Web App (near total control) vs a “Serverless” Azure Function (less control but I didn’t need it anyway, just wanted a function in the cloud.)


Sponsor: Aspose makes programming APIs for working with files, like: DOC, XLS, PPT, PDF and countless more.  Developers can use their products to create, convert, modify, or manage files in almost any way.  Aspose is a good company and they offer solid products.  Check them out, and download a free evaluation.


© 2016 Scott Hanselman. All rights reserved.
     

There's a lot of confusing terms in the Cloud space. And that's not counting the term "Cloud." ;)

  • IaaS (Infrastructure as a Services) - Virtual Machines and stuff on demand.
  • PaaS (Platform as a Service) - You deploy your apps but try not to think about the Virtual Machines underneath. They exist, but we pretend they don't until forced.
  • SaaS (Software as a Service) - Stuff like Office 365 and Gmail. You pay a subscription and you get email/whatever as a service. It Just Works.

"Serverless Computing" doesn't really mean there's no server. Serverless means there's no server you need to worry about. That might sound like PaaS, but it's higher level that than.

Serverless Computing is like this - Your code, a slider bar, and your credit card. You just have your function out there and it will scale as long as you can pay for it. It's as close to "cloudy" as The Cloud can get.

Serverless Computing is like this. Your code, a slider bar, and your credit card.

With Platform as a Service, you might made a Node or C# app, check it into Git, deploy it to a Web Site/Application, and then you've got an endpoint. You might scale it up (get more CPU/Memory/Disk) or out (have 1, 2, n instances of the Web App) but it's not seamless. It's totally cool, to be clear, but you're always aware of the servers.

New cloud systems like Amazon Lambda and Azure Functions have you upload some code and it's running seconds later. You can have continuous jobs, functions that run on a triggered event, or make Web APIs or Webhooks that are just a function with a URL.

I'm going to see how quickly I can make a Web API with Serverless Computing.

I'll go to http://functions.azure.com and make a new function. If you don't have an account you can sign up free.

Getting started with Azure Functions

You can make a function in JavaScript or C#.

Getting started with Azure Functions - Create This Function

Once you're into the Azure Function Editor, click "New Function" and you've got dozens of templates and code examples for things like:

  • Find a face in an image and store the rectangle of where the face is.
  • Run a function and comment on a GitHub issue when a GitHub webhook is triggered
  • Update a storage blob when an HTTP Request comes in
  • Load entities from a database or storage table

I figured I'd change the first example. It is a trigger that sees an image in storage, calls a cognitive services API to get the location of the face, then stores the data. I wanted to change it to:

  • Take an image as input from an HTTP Post
  • Draw a rectangle around the face
  • Return the new image

You can do this work from Git/GitHub but for easy stuff I'm literally doing it all in the browser. Here's what it looks like.

Azure Functions can be done in the browser

I code and iterate and save and fail fast, fail often. Here's the starter code I based it on. Remember, that this is a starter function that runs on a triggered event, so note its Run()...I'm going to change this.

#r "Microsoft.WindowsAzure.Storage"
#r "Newtonsoft.Json"
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json;
using Microsoft.WindowsAzure.Storage.Table;
using System.IO; 
public static async Task Run(Stream image, string name, IAsyncCollector<FaceRectangle> outTable, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    
    string result = await CallVisionAPI(image); //STREAM
    log.Info(result); 
    if (String.IsNullOrEmpty(result))
    {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    foreach (Face face in imageData.Faces)
    {
        var faceRectangle = face.FaceRectangle;
        faceRectangle.RowKey = Guid.NewGuid().ToString();
        faceRectangle.PartitionKey = "Functions";
        faceRectangle.ImageFile = name + ".jpg";
        await outTable.AddAsync(faceRectangle); 
    }
    return req.CreateResponse(HttpStatusCode.OK, "Nice Job");  
}
static async Task<string> CallVisionAPI(Stream image)
{
    using (var client = new HttpClient())
    {
        var content = new StreamContent(image);
        var url = "https://api.projectoxford.ai/vision/v1.0/analyze?visualFeatures=Faces";
        client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", Environment.GetEnvironmentVariable("Vision_API_Subscription_Key"));
        content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
        var httpResponse = await client.PostAsync(url, content);
        if (httpResponse.StatusCode == HttpStatusCode.OK){
            return await httpResponse.Content.ReadAsStringAsync();
        }
    }
    return null;
}
public class ImageData {
    public List<Face> Faces { get; set; }
}
public class Face {
    public int Age { get; set; }
    public string Gender { get; set; }
    public FaceRectangle FaceRectangle { get; set; }
}
public class FaceRectangle : TableEntity {
    public string ImageFile { get; set; }
    public int Left { get; set; }
    public int Top { get; set; }
    public int Width { get; set; }
    public int Height { get; set; }
}

GOAL: I'll change this Run() and make this listen for an HTTP request that contains an image, read the image that's POSTed in (ya, I know, no validation), draw rectangle around detected faces, then return a new image.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log) {

var image = await req.Content.ReadAsStreamAsync();

As for the body of this function, I'm 20% sure I'm using too many MemoryStreams but they are getting disposed so take this code as a initial proof of concept. However, I DO need at least the two I have. Regardless, happy to chat with those who know more, but it's more subtle than even I thought. That said, basically call out to the API, get back some face data that looks like this:

2016-08-26T23:59:26.741 {"requestId":"8be222ff-98cc-4019-8038-c22eeffa63ed","metadata":{"width":2808,"height":1872,"format":"Jpeg"},"faces":[{"age":41,"gender":"Male","faceRectangle":{"left":1059,"top":671,"width":466,"height":466}},{"age":41,"gender":"Male","faceRectangle":{"left":1916,"top":702,"width":448,"height":448}}]}

Then take that data and DRAW a Rectangle over the faces detected.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    MemoryStream mem = new MemoryStream();
    image.CopyTo(mem); //make a copy since one gets destroy in the other API. Lame, I know.
    image.Position = 0;
    mem.Position = 0;
    
    string result = await CallVisionAPI(image); 
    log.Info(result); 
    if (String.IsNullOrEmpty(result)) {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    MemoryStream outputStream = new MemoryStream();
    using(Image maybeFace = Image.FromStream(mem, true))
    {
        using (Graphics g = Graphics.FromImage(maybeFace))
        {
            Pen yellowPen = new Pen(Color.Yellow, 4);
            foreach (Face face in imageData.Faces)
            {
                var faceRectangle = face.FaceRectangle;
                g.DrawRectangle(yellowPen, 
                    faceRectangle.Left, faceRectangle.Top, 
                    faceRectangle.Width, faceRectangle.Height);
            }
        }
        maybeFace.Save(outputStream, ImageFormat.Jpeg);
    }
    
    var response = new HttpResponseMessage()
    {
        Content = new ByteArrayContent(outputStream.ToArray()),
        StatusCode = HttpStatusCode.OK,
    };
    response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
    return response;
}

 

Now I go into Postman and POST an image to my new Azure Function endpoint. Here I uploaded a flattering picture of me and unflattering picture of The Oatmeal. He's pretty in real life just NOT HERE. ;)

Image Recognition with Azure Functions

So in just about 15 min with no idea and armed with just my browser, Postman (also my browser), Google/StackOverflow, and Azure Functions I've got a backend proof of concept.

Azure Functions supports Node.js, C#, F#, Python, PHP *and* Batch, Bash, and PowerShell, which really opens it up to basically anyone. You can use them for anything when you just want a function (or more) out there on the web. Send stuff to Slack, automate your house, update GitHub issues, act as a Webhook, etc. There's some great 3d party Azure Functions sample code in this GitHub repo as well. Inputs can be from basically anywhere and outputs can be basically anywhere. If those anywheres are also cloud services like Tables or Storage, you've got a "serverless backed" that is easy to scale.

I'm still learning, but I can see when I'd want a VM (total control) vs a Web App (near total control) vs a "Serverless" Azure Function (less control but I didn't need it anyway, just wanted a function in the cloud.)


Sponsor: Aspose makes programming APIs for working with files, like: DOC, XLS, PPT, PDF and countless more.  Developers can use their products to create, convert, modify, or manage files in almost any way.  Aspose is a good company and they offer solid products.  Check them out, and download a free evaluation.


© 2016 Scott Hanselman. All rights reserved.
     

What is Serverless Computing? Exploring Azure Functions

There’s a lot of confusing terms in the Cloud space. And that’s not counting the term “Cloud.” 😉

  • IaaS (Infrastructure as a Services) – Virtual Machines and stuff on demand.
  • PaaS (Platform as a Service) – You deploy your apps but try not to think about the Virtual Machines underneath. They exist, but we pretend they don’t until forced.
  • SaaS (Software as a Service) – Stuff like Office 365 and Gmail. You pay a subscription and you get email/whatever as a service. It Just Works.

“Serverless Computing” doesn’t really mean there’s no server. Serverless means there’s no server you need to worry about. That might sound like PaaS, but it’s higher level that than.

Serverless Computing is like this – Your code, a slider bar, and your credit card. You just have your function out there and it will scale as long as you can pay for it. It’s as close to “cloudy” as The Cloud can get.

Serverless Computing is like this. Your code, a slider bar, and your credit card.

With Platform as a Service, you might made a Node or C# app, check it into Git, deploy it to a Web Site/Application, and then you’ve got an endpoint. You might scale it up (get more CPU/Memory/Disk) or out (have 1, 2, n instances of the Web App) but it’s not seamless. It’s totally cool, to be clear, but you’re always aware of the servers.

New cloud systems like Amazon Lambda and Azure Functions have you upload some code and it’s running seconds later. You can have continuous jobs, functions that run on a triggered event, or make Web APIs or Webhooks that are just a function with a URL.

I’m going to see how quickly I can make a Web API with Serverless Computing.

I’ll go to http://functions.azure.com and make a new function. If you don’t have an account you can sign up free.

Getting started with Azure Functions

You can make a function in JavaScript or C#.

Getting started with Azure Functions - Create This Function

Once you’re into the Azure Function Editor, click “New Function” and you’ve got dozens of templates and code examples for things like:

  • Find a face in an image and store the rectangle of where the face is.
  • Run a function and comment on a GitHub issue when a GitHub webhook is triggered
  • Update a storage blob when an HTTP Request comes in
  • Load entities from a database or storage table

I figured I’d change the first example. It is a trigger that sees an image in storage, calls a cognitive services API to get the location of the face, then stores the data. I wanted to change it to:

  • Take an image as input from an HTTP Post
  • Draw a rectangle around the face
  • Return the new image

You can do this work from Git/GitHub but for easy stuff I’m literally doing it all in the browser. Here’s what it looks like.

Azure Functions can be done in the browser

I code and iterate and save and fail fast, fail often. Here’s the starter code I based it on. Remember, that this is a starter function that runs on a triggered event, so note its Run()…I’m going to change this.

#r "Microsoft.WindowsAzure.Storage"
#r "Newtonsoft.Json"
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json;
using Microsoft.WindowsAzure.Storage.Table;
using System.IO; 
public static async Task Run(Stream image, string name, IAsyncCollector<FaceRectangle> outTable, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    
    string result = await CallVisionAPI(image); //STREAM
    log.Info(result); 
    if (String.IsNullOrEmpty(result))
    {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    foreach (Face face in imageData.Faces)
    {
        var faceRectangle = face.FaceRectangle;
        faceRectangle.RowKey = Guid.NewGuid().ToString();
        faceRectangle.PartitionKey = "Functions";
        faceRectangle.ImageFile = name + ".jpg";
        await outTable.AddAsync(faceRectangle); 
    }
    return req.CreateResponse(HttpStatusCode.OK, "Nice Job");  
}
static async Task<string> CallVisionAPI(Stream image)
{
    using (var client = new HttpClient())
    {
        var content = new StreamContent(image);
        var url = "https://api.projectoxford.ai/vision/v1.0/analyze?visualFeatures=Faces";
        client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", Environment.GetEnvironmentVariable("Vision_API_Subscription_Key"));
        content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
        var httpResponse = await client.PostAsync(url, content);
        if (httpResponse.StatusCode == HttpStatusCode.OK){
            return await httpResponse.Content.ReadAsStringAsync();
        }
    }
    return null;
}
public class ImageData {
    public List<Face> Faces { get; set; }
}
public class Face {
    public int Age { get; set; }
    public string Gender { get; set; }
    public FaceRectangle FaceRectangle { get; set; }
}
public class FaceRectangle : TableEntity {
    public string ImageFile { get; set; }
    public int Left { get; set; }
    public int Top { get; set; }
    public int Width { get; set; }
    public int Height { get; set; }
}

GOAL: I’ll change this Run() and make this listen for an HTTP request that contains an image, read the image that’s POSTed in (ya, I know, no validation), draw rectangle around detected faces, then return a new image.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log) {
var image = await req.Content.ReadAsStreamAsync();

As for the body of this function, I’m 20% sure I’m using too many MemoryStreams but they are getting disposed so take this code as a initial proof of concept. However, I DO need at least the two I have. Regardless, happy to chat with those who know more, but it’s more subtle than even I thought. That said, basically call out to the API, get back some face data that looks like this:

2016-08-26T23:59:26.741 {"requestId":"8be222ff-98cc-4019-8038-c22eeffa63ed","metadata":{"width":2808,"height":1872,"format":"Jpeg"},"faces":[{"age":41,"gender":"Male","faceRectangle":{"left":1059,"top":671,"width":466,"height":466}},{"age":41,"gender":"Male","faceRectangle":{"left":1916,"top":702,"width":448,"height":448}}]}

Then take that data and DRAW a Rectangle over the faces detected.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    MemoryStream mem = new MemoryStream();
    image.CopyTo(mem); //make a copy since one gets destroy in the other API. Lame, I know.
    image.Position = 0;
    mem.Position = 0;
    
    string result = await CallVisionAPI(image); 
    log.Info(result); 
    if (String.IsNullOrEmpty(result)) {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    MemoryStream outputStream = new MemoryStream();
    using(Image maybeFace = Image.FromStream(mem, true))
    {
        using (Graphics g = Graphics.FromImage(maybeFace))
        {
            Pen yellowPen = new Pen(Color.Yellow, 4);
            foreach (Face face in imageData.Faces)
            {
                var faceRectangle = face.FaceRectangle;
                g.DrawRectangle(yellowPen, 
                    faceRectangle.Left, faceRectangle.Top, 
                    faceRectangle.Width, faceRectangle.Height);
            }
        }
        maybeFace.Save(outputStream, ImageFormat.Jpeg);
    }
    
    var response = new HttpResponseMessage()
    {
        Content = new ByteArrayContent(outputStream.ToArray()),
        StatusCode = HttpStatusCode.OK,
    };
    response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
    return response;
}

 

Now I go into Postman and POST an image to my new Azure Function endpoint. Here I uploaded a flattering picture of me and unflattering picture of The Oatmeal. He’s pretty in real life just NOT HERE. 😉

Image Recognition with Azure Functions

So in just about 15 min with no idea and armed with just my browser, Postman (also my browser), Google/StackOverflow, and Azure Functions I’ve got a backend proof of concept.

Azure Functions supports Node.js, C#, F#, Python, PHP *and* Batch, Bash, and PowerShell, which really opens it up to basically anyone. You can use them for anything when you just want a function (or more) out there on the web. Send stuff to Slack, automate your house, update GitHub issues, act as a Webhook, etc. There’s some great 3d party Azure Functions sample code in this GitHub repo as well. Inputs can be from basically anywhere and outputs can be basically anywhere. If those anywheres are also cloud services like Tables or Storage, you’ve got a “serverless backed” that is easy to scale.

I’m still learning, but I can see when I’d want a VM (total control) vs a Web App (near total control) vs a “Serverless” Azure Function (less control but I didn’t need it anyway, just wanted a function in the cloud.)


Sponsor: Aspose makes programming APIs for working with files, like: DOC, XLS, PPT, PDF and countless more.  Developers can use their products to create, convert, modify, or manage files in almost any way.  Aspose is a good company and they offer solid products.  Check them out, and download a free evaluation.


© 2016 Scott Hanselman. All rights reserved.
     

There's a lot of confusing terms in the Cloud space. And that's not counting the term "Cloud." ;)

  • IaaS (Infrastructure as a Services) - Virtual Machines and stuff on demand.
  • PaaS (Platform as a Service) - You deploy your apps but try not to think about the Virtual Machines underneath. They exist, but we pretend they don't until forced.
  • SaaS (Software as a Service) - Stuff like Office 365 and Gmail. You pay a subscription and you get email/whatever as a service. It Just Works.

"Serverless Computing" doesn't really mean there's no server. Serverless means there's no server you need to worry about. That might sound like PaaS, but it's higher level that than.

Serverless Computing is like this - Your code, a slider bar, and your credit card. You just have your function out there and it will scale as long as you can pay for it. It's as close to "cloudy" as The Cloud can get.

Serverless Computing is like this. Your code, a slider bar, and your credit card.

With Platform as a Service, you might made a Node or C# app, check it into Git, deploy it to a Web Site/Application, and then you've got an endpoint. You might scale it up (get more CPU/Memory/Disk) or out (have 1, 2, n instances of the Web App) but it's not seamless. It's totally cool, to be clear, but you're always aware of the servers.

New cloud systems like Amazon Lambda and Azure Functions have you upload some code and it's running seconds later. You can have continuous jobs, functions that run on a triggered event, or make Web APIs or Webhooks that are just a function with a URL.

I'm going to see how quickly I can make a Web API with Serverless Computing.

I'll go to http://functions.azure.com and make a new function. If you don't have an account you can sign up free.

Getting started with Azure Functions

You can make a function in JavaScript or C#.

Getting started with Azure Functions - Create This Function

Once you're into the Azure Function Editor, click "New Function" and you've got dozens of templates and code examples for things like:

  • Find a face in an image and store the rectangle of where the face is.
  • Run a function and comment on a GitHub issue when a GitHub webhook is triggered
  • Update a storage blob when an HTTP Request comes in
  • Load entities from a database or storage table

I figured I'd change the first example. It is a trigger that sees an image in storage, calls a cognitive services API to get the location of the face, then stores the data. I wanted to change it to:

  • Take an image as input from an HTTP Post
  • Draw a rectangle around the face
  • Return the new image

You can do this work from Git/GitHub but for easy stuff I'm literally doing it all in the browser. Here's what it looks like.

Azure Functions can be done in the browser

I code and iterate and save and fail fast, fail often. Here's the starter code I based it on. Remember, that this is a starter function that runs on a triggered event, so note its Run()...I'm going to change this.

#r "Microsoft.WindowsAzure.Storage"
#r "Newtonsoft.Json"
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json;
using Microsoft.WindowsAzure.Storage.Table;
using System.IO; 
public static async Task Run(Stream image, string name, IAsyncCollector<FaceRectangle> outTable, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    
    string result = await CallVisionAPI(image); //STREAM
    log.Info(result); 
    if (String.IsNullOrEmpty(result))
    {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    foreach (Face face in imageData.Faces)
    {
        var faceRectangle = face.FaceRectangle;
        faceRectangle.RowKey = Guid.NewGuid().ToString();
        faceRectangle.PartitionKey = "Functions";
        faceRectangle.ImageFile = name + ".jpg";
        await outTable.AddAsync(faceRectangle); 
    }
    return req.CreateResponse(HttpStatusCode.OK, "Nice Job");  
}
static async Task<string> CallVisionAPI(Stream image)
{
    using (var client = new HttpClient())
    {
        var content = new StreamContent(image);
        var url = "https://api.projectoxford.ai/vision/v1.0/analyze?visualFeatures=Faces";
        client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", Environment.GetEnvironmentVariable("Vision_API_Subscription_Key"));
        content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
        var httpResponse = await client.PostAsync(url, content);
        if (httpResponse.StatusCode == HttpStatusCode.OK){
            return await httpResponse.Content.ReadAsStringAsync();
        }
    }
    return null;
}
public class ImageData {
    public List<Face> Faces { get; set; }
}
public class Face {
    public int Age { get; set; }
    public string Gender { get; set; }
    public FaceRectangle FaceRectangle { get; set; }
}
public class FaceRectangle : TableEntity {
    public string ImageFile { get; set; }
    public int Left { get; set; }
    public int Top { get; set; }
    public int Width { get; set; }
    public int Height { get; set; }
}

GOAL: I'll change this Run() and make this listen for an HTTP request that contains an image, read the image that's POSTed in (ya, I know, no validation), draw rectangle around detected faces, then return a new image.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log) {

var image = await req.Content.ReadAsStreamAsync();

As for the body of this function, I'm 20% sure I'm using too many MemoryStreams but they are getting disposed so take this code as a initial proof of concept. However, I DO need at least the two I have. Regardless, happy to chat with those who know more, but it's more subtle than even I thought. That said, basically call out to the API, get back some face data that looks like this:

2016-08-26T23:59:26.741 {"requestId":"8be222ff-98cc-4019-8038-c22eeffa63ed","metadata":{"width":2808,"height":1872,"format":"Jpeg"},"faces":[{"age":41,"gender":"Male","faceRectangle":{"left":1059,"top":671,"width":466,"height":466}},{"age":41,"gender":"Male","faceRectangle":{"left":1916,"top":702,"width":448,"height":448}}]}

Then take that data and DRAW a Rectangle over the faces detected.

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
    var image = await req.Content.ReadAsStreamAsync();
    MemoryStream mem = new MemoryStream();
    image.CopyTo(mem); //make a copy since one gets destroy in the other API. Lame, I know.
    image.Position = 0;
    mem.Position = 0;
    
    string result = await CallVisionAPI(image); 
    log.Info(result); 
    if (String.IsNullOrEmpty(result)) {
        return req.CreateResponse(HttpStatusCode.BadRequest);
    }
    
    ImageData imageData = JsonConvert.DeserializeObject<ImageData>(result);
    MemoryStream outputStream = new MemoryStream();
    using(Image maybeFace = Image.FromStream(mem, true))
    {
        using (Graphics g = Graphics.FromImage(maybeFace))
        {
            Pen yellowPen = new Pen(Color.Yellow, 4);
            foreach (Face face in imageData.Faces)
            {
                var faceRectangle = face.FaceRectangle;
                g.DrawRectangle(yellowPen, 
                    faceRectangle.Left, faceRectangle.Top, 
                    faceRectangle.Width, faceRectangle.Height);
            }
        }
        maybeFace.Save(outputStream, ImageFormat.Jpeg);
    }
    
    var response = new HttpResponseMessage()
    {
        Content = new ByteArrayContent(outputStream.ToArray()),
        StatusCode = HttpStatusCode.OK,
    };
    response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
    return response;
}

 

Now I go into Postman and POST an image to my new Azure Function endpoint. Here I uploaded a flattering picture of me and unflattering picture of The Oatmeal. He's pretty in real life just NOT HERE. ;)

Image Recognition with Azure Functions

So in just about 15 min with no idea and armed with just my browser, Postman (also my browser), Google/StackOverflow, and Azure Functions I've got a backend proof of concept.

Azure Functions supports Node.js, C#, F#, Python, PHP *and* Batch, Bash, and PowerShell, which really opens it up to basically anyone. You can use them for anything when you just want a function (or more) out there on the web. Send stuff to Slack, automate your house, update GitHub issues, act as a Webhook, etc. There's some great 3d party Azure Functions sample code in this GitHub repo as well. Inputs can be from basically anywhere and outputs can be basically anywhere. If those anywheres are also cloud services like Tables or Storage, you've got a "serverless backed" that is easy to scale.

I'm still learning, but I can see when I'd want a VM (total control) vs a Web App (near total control) vs a "Serverless" Azure Function (less control but I didn't need it anyway, just wanted a function in the cloud.)


Sponsor: Aspose makes programming APIs for working with files, like: DOC, XLS, PPT, PDF and countless more.  Developers can use their products to create, convert, modify, or manage files in almost any way.  Aspose is a good company and they offer solid products.  Check them out, and download a free evaluation.


© 2016 Scott Hanselman. All rights reserved.
     

Visual Studio 2015 – Fixing “Dependencies npm not installed” from fsevents with node on Windows

Maria and I were doing some work together on Thursday and I did a clone of one of her ASP.NET Core repositories and opened it in Visual Studio 2015 Community Edition on my local machine. Some of the JavaScript tool libraries didn’t load so I went back into Solution Explorer to see if there was a problem and I saw this weird error that said (or at least, I read it as) “npm not installed.”

image

That’s weird, since I have npm installed. I dropped out to the command line and ran:

C:\Users\scott>npm --version
3.10.5

I also ran “where” to see where (and how many) npm was installed. Side note: WHERE is super useful and not used as often as it should be by y’all.

C:\Users\scott>where npm
C:\Program Files\nodejs\npm
C:\Program Files\nodejs\npm.cmd
C:\Users\scott\AppData\Roaming\npm\npm
C:\Users\scott\AppData\Roaming\npm\npm.cmd

This looks OK as two of those npms are shell scripts and not run on Windows. Just to make sure you have the version of npm you want, Felix made a VERY useful npm-windows-upgrade utility that you run like this, ironically with npm.

npm install --global --production npm-windows-upgrade
npm-windows-upgrade

I’ve used this tool many times with success. It also lets you choose the exact version you want and it’s very smart.

However, then I realized that Visual Studio wasn’t saying that npm wasn’t installed, it was saying a dependency in the npm tree below wasn’t installed. That hyphen – is intended to mean something. For the record, I think this is not intuitive and is a poor UX. Perhaps it should say “package(s) not installed” but you get the idea now.

I started manually opening the tree up one item at a time looking for the bad package – while quietly swearing to myself – until I realized that the Solution Explorer tree nodes are searchable.

image

This is UX issue number two, for me. I think the “broken” library should also include the BANG (!) or warning icon over it. Regardless, now I know I can quickly do a string search.

So what’s up with the fsevents library?

I can check the Visual Studio Output Window and I see this:

npm WARN install Couldn't install optional dependency: Unsupported
npm WARN EPACKAGEJSON [email protected] No description

I dropped out to the command prompt into to the project’s folder and run “npm ls” to see an ASCII tree that effectively is the same tree you see in Visual Studio. Note the screenshot with the ASCII tree below.

The output of "npm ls" is a nice ASCII tree

What’s that “UNMET OPTIONAL DEPENDENCY?” for fsevents?

Looks like fsevents is a package that is only used on OSX, so it’s an optional dependency that in the case of Windows is “unmet.” It’s a WARNING, of sorts. Perhaps it’s more an “INFO” or perhaps, one could argue, it doesn’t need to be shown at all on Windows.

A quick google shows that, well, the entire world is asking the same thing. So much so that it got pretty heated in this GitHub Issue asking a similar question.

Regardless, it seems to me that something inside Visual Studio really doesn’t appreciate that warning/info/notOKness and says, “hey it’s not installed.” The part that is missing is that, well, it doesn’t need to be installed, so Visual Studio should relax.

A Fix

Here’s where it gets super interesting. Visual Studio (consider it from their point of view) needs to keep things consistent so it tests with, and ships with, a version of npm.

I can check that like this:

C:\>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\npm.cmd" --version
3.3.4

Again, this makes sense. That way when working in a large group we can all be sure we’re on the same version of stuff. When you get a new version of the Web Tools for Visual Studio, presumably this version of npm gets updated. This also means that unless you tell Visual Studio otherwise, the npm and node you get when running a build in VS is gonna be different than if you do it outside – if you stick with the default settings.

Fear not. It’s easily updated. I went over to Tools | Option in Visual Studio:

Tools | Options | External Web Tools has its own list of node copies, and it doesn't include mine

See the Web\External there? I added my node installation like this and made sure it was at the top.

Tools | Options | External Web Tools has its own list of node copies, and I have ADDED my path now.

Then I right click on npm and Restore Packages and all is right with the world as now I’m using npm 3.10.5 rather than 3.3.4.

There are no npm errors anymore

The Output Window for 3.10.5 shows this different output, which apparently stresses Visual Studio out less than npm 3.3.4. (Actually I think VS is calling an npm library, rather than shelling out and parsing the “npm ls –json” payload, but you get the idea.)

npm WARN optional Skipping failed optional dependency /chokidar/fsevents:
npm WARN notsup Not compatible with your operating system or architecture: [email protected]

Adding my global npm/node folder fixes this issue for me and I can move on.

The Weird

BUT. In my fix I’m using npm globally – it’s in my %PATH%. The change I made affects all Visual Studio projects on my machine, forever.

Maybe I need to be smarter? Maybe Visual Studio is already being smart. Note the second option there in the list? It’s pointing to .\node_modules\bin. That’s a LOCAL node-modules folder, right? Ah, I can just add specific version of npm to packages.json there if need be on a project by project basis without affecting my entire system OR changing my Visual Studio settings, right?

However, when I run my build, it’s ignoring my project’s locally installed npm!

PATH=.\node_modules\.bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External;%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\git
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\npm.CMD" install
npm WARN install Couldn't install optional dependency: Unsupported
npm WARN EPACKAGEJSON [email protected] No description
npm WARN EPACKAGEJSON [email protected] No repository field.
npm WARN EPACKAGEJSON [email protected] No license field.

How can I be sure it’s ignoring that relative path? I can temporarily hardcode my local node_modules, like this. Note the PATH *and* the newer output. And it works.

PATH=D:\github\sample-NerdDinner\NerdDinner.Web\node_modules\.bin;node_modules\.bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External;%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\git
"D:\github\sample-NerdDinner\NerdDinner.Web\node_modules\.bin\npm.CMD" install
npm WARN optional Skipping failed optional dependency /chokidar/fsevents:
npm WARN notsup Not compatible with your operating system or architecture: [email protected]
npm WARN [email protected] No description
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.

In this screenshot below I have npm 3.10.6 installed in my project, locally, but it’s not being used by Visual Studio if the path remains relative.

I am 99% sure that the relative path “.\node_modules\.bin” that’s prepended to the PATH above either isn’t being used OR is interfering in some way.

Am I misunderstanding how it should work? Perhaps I don’t understand npm enough?

image

I’m going to continue to dig into this and I’ll update this post when I have an answer. For now, I have a fix given my globally installed npm.

I’ve also passed my feedback/bug report onto the Visual Studio team in the form of an email with a link to this blog post. I hope this helps someone!


Sponsor: Big thanks to Redgate for sponsoring the feed this week. Could you deploy 1,000 databases?Imagine working in a 70-strong IT team, with 91 applications and 1,000+ databases. Now imagine deployment time. It’s not fiction, it’s fact. Read FlexiGroup’s story.


© 2016 Scott Hanselman. All rights reserved.
     

Maria and I were doing some work together on Thursday and I did a clone of one of her ASP.NET Core repositories and opened it in Visual Studio 2015 Community Edition on my local machine. Some of the JavaScript tool libraries didn't load so I went back into Solution Explorer to see if there was a problem and I saw this weird error that said (or at least, I read it as) "npm not installed."

image

That's weird, since I have npm installed. I dropped out to the command line and ran:

C:\Users\scott>npm --version

3.10.5

I also ran "where" to see where (and how many) npm was installed. Side note: WHERE is super useful and not used as often as it should be by y'all.

C:\Users\scott>where npm

C:\Program Files\nodejs\npm
C:\Program Files\nodejs\npm.cmd
C:\Users\scott\AppData\Roaming\npm\npm
C:\Users\scott\AppData\Roaming\npm\npm.cmd

This looks OK as two of those npms are shell scripts and not run on Windows. Just to make sure you have the version of npm you want, Felix made a VERY useful npm-windows-upgrade utility that you run like this, ironically with npm.

npm install --global --production npm-windows-upgrade

npm-windows-upgrade

I've used this tool many times with success. It also lets you choose the exact version you want and it's very smart.

However, then I realized that Visual Studio wasn't saying that npm wasn't installed, it was saying a dependency in the npm tree below wasn't installed. That hyphen - is intended to mean something. For the record, I think this is not intuitive and is a poor UX. Perhaps it should say "package(s) not installed" but you get the idea now.

I started manually opening the tree up one item at a time looking for the bad package - while quietly swearing to myself - until I realized that the Solution Explorer tree nodes are searchable.

image

This is UX issue number two, for me. I think the "broken" library should also include the BANG (!) or warning icon over it. Regardless, now I know I can quickly do a string search.

So what's up with the fsevents library?

I can check the Visual Studio Output Window and I see this:

npm WARN install Couldn't install optional dependency: Unsupported

npm WARN EPACKAGEJSON [email protected] No description

I dropped out to the command prompt into to the project's folder and run "npm ls" to see an ASCII tree that effectively is the same tree you see in Visual Studio. Note the screenshot with the ASCII tree below.

The output of "npm ls" is a nice ASCII tree

What's that "UNMET OPTIONAL DEPENDENCY?" for fsevents?

Looks like fsevents is a package that is only used on OSX, so it's an optional dependency that in the case of Windows is "unmet." It's a WARNING, of sorts. Perhaps it's more an "INFO" or perhaps, one could argue, it doesn't need to be shown at all on Windows.

A quick google shows that, well, the entire world is asking the same thing. So much so that it got pretty heated in this GitHub Issue asking a similar question.

Regardless, it seems to me that something inside Visual Studio really doesn't appreciate that warning/info/notOKness and says, "hey it's not installed." The part that is missing is that, well, it doesn't need to be installed, so Visual Studio should relax.

A Fix

Here's where it gets super interesting. Visual Studio (consider it from their point of view) needs to keep things consistent so it tests with, and ships with, a version of npm.

I can check that like this:

C:\>"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\npm.cmd" --version

3.3.4

Again, this makes sense. That way when working in a large group we can all be sure we're on the same version of stuff. When you get a new version of the Web Tools for Visual Studio, presumably this version of npm gets updated. This also means that unless you tell Visual Studio otherwise, the npm and node you get when running a build in VS is gonna be different than if you do it outside - if you stick with the default settings.

Fear not. It's easily updated. I went over to Tools | Option in Visual Studio:

Tools | Options | External Web Tools has its own list of node copies, and it doesn't include mine

See the Web\External there? I added my node installation like this and made sure it was at the top.

Tools | Options | External Web Tools has its own list of node copies, and I have ADDED my path now.

Then I right click on npm and Restore Packages and all is right with the world as now I'm using npm 3.10.5 rather than 3.3.4.

There are no npm errors anymore

The Output Window for 3.10.5 shows this different output, which apparently stresses Visual Studio out less than npm 3.3.4. (Actually I think VS is calling an npm library, rather than shelling out and parsing the "npm ls --json" payload, but you get the idea.)

npm WARN optional Skipping failed optional dependency /chokidar/fsevents:

npm WARN notsup Not compatible with your operating system or architecture: [email protected]

Adding my global npm/node folder fixes this issue for me and I can move on.

The Weird

BUT. In my fix I'm using npm globally - it's in my %PATH%. The change I made affects all Visual Studio projects on my machine, forever.

Maybe I need to be smarter? Maybe Visual Studio is already being smart. Note the second option there in the list? It's pointing to .\node_modules\bin. That's a LOCAL node-modules folder, right? Ah, I can just add specific version of npm to packages.json there if need be on a project by project basis without affecting my entire system OR changing my Visual Studio settings, right?

However, when I run my build, it's ignoring my project's locally installed npm!

PATH=.\node_modules\.bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External;%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\git

"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\npm.CMD" install
npm WARN install Couldn't install optional dependency: Unsupported
npm WARN EPACKAGEJSON [email protected] No description
npm WARN EPACKAGEJSON [email protected] No repository field.
npm WARN EPACKAGEJSON [email protected] No license field.

How can I be sure it's ignoring that relative path? I can temporarily hardcode my local node_modules, like this. Note the PATH *and* the newer output. And it works.

PATH=D:\github\sample-NerdDinner\NerdDinner.Web\node_modules\.bin;node_modules\.bin;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External;%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 14.0\Web\External\git

"D:\github\sample-NerdDinner\NerdDinner.Web\node_modules\.bin\npm.CMD" install
npm WARN optional Skipping failed optional dependency /chokidar/fsevents:
npm WARN notsup Not compatible with your operating system or architecture: [email protected]
npm WARN [email protected] No description
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.

In this screenshot below I have npm 3.10.6 installed in my project, locally, but it's not being used by Visual Studio if the path remains relative.

I am 99% sure that the relative path ".\node_modules\.bin" that's prepended to the PATH above either isn't being used OR is interfering in some way.

Am I misunderstanding how it should work? Perhaps I don't understand npm enough?

image

I'm going to continue to dig into this and I'll update this post when I have an answer. For now, I have a fix given my globally installed npm.

I've also passed my feedback/bug report onto the Visual Studio team in the form of an email with a link to this blog post. I hope this helps someone!


Sponsor: Big thanks to Redgate for sponsoring the feed this week. Could you deploy 1,000 databases?Imagine working in a 70-strong IT team, with 91 applications and 1,000+ databases. Now imagine deployment time. It’s not fiction, it’s fact. Read FlexiGroup's story.



© 2016 Scott Hanselman. All rights reserved.
     

UPDATED for 2015: How to install the nodejs Ghost blog software on Azure Web Apps (and the Deploy to Azure Button)

What? Didn’t I blog about this before? Yes, in fact, in 2013 I did an article showing how to get Ghost – a lovely nodejs-based blogging app – running on Azure.  The instructions involved making some changes to Ghost itself to make it friendlier to Azure and doing a git deploy. Since this post both Ghost AND Azure have become friendlier. 😉

Fast forward two years and the instructions have changed! In fact, they’ve gotten WAY easier. You can still theoretically follow most of my old instructions, but these new ones are WAY better, so just do it this way.

STEP 1

There’s a “Deploy to Azure” button up at http://deploy.azure.com that you can get an put in YOUR GitHub project’s Readme.md so folks can easily and quickly deploy your app as well!

Here’s what you do. Head over to the GitHub repository. Click Deploy to Azure. That will automatically install Ghost for you in Azure.

The Deploy to Azure Button

The Deploy.Azure.com website will look at the repository from the HTTP Referer header. You’ll end up with a nice form like this.

DO make sure to double check your settings, the Site Location is alphabetical right now, and you may not want your blog in Brazil. 😉

image

Hit Next, then check the summary that will warn you what’s getting created, then hit Deploy. Boom. Azure will actually run through the template and setup Ghost (or whatever app you wanted) automatically.

  • Note it’s setting it up from Source Control as well, although you can certainly change this. For example, you might want to Fork it yourself, and then Deploy.
  • However, as this is set up today, you won’t get updates until you go to Deployments within the Azure Portal and click Sync. You decide if you want the app to update when new code is committed.

After it’s created, you can manage your site in the Azure Portal. I made a little free one for this example, as can you if you like.

Ghost in the Azure Portal

And it works just great!

Ghost running on Azure

It’s not super obvious what to do next. You’ll need click the little chevron there, or visit something like /admin, and you’ll get redirected to the Ghost setup process online.

image

Now you can start your first post!

My first post in Ghost

What’s going on here?

Felix Rieseberg added a few files to Ghost and has the fork up here on GitHub. The most interesting one is the AzureDeploy.json. This is an Azure Resource Manager template. Here’s another simple example from Elliot Hamai for an ASP.NET MVP app. This file tells Azure (and the Deploy to Azure button) what kinds of things it needs to create and actually gives the system enough information to build a whole form for you!

Maybe this is the perfect time for you to start your own blog! Perhaps you’ve been putting it off. Go check out my FREE two-hour documentary movie with Rob Conery called Get Involved in Tech! We will get you ready to jump into the world of Social Software Development.

Here’s a video of Elliot and I talking about the Deploy to Azure button on Azure Friday. Here’s Elliot’s blog announcing Deploy to Azure and explaining more.

Remember that Ghost is open source and you can learn more at https://ghost.org!

DON’T WANT TO SIGN UP FOR AZURE? You can try Azure out for an hour without signing up for anything. Check out http://try.azurewebsites.net. You can make a PHP, Java, nodejs, Python, ASP.NET web app, or even setup Ghost itself. You can also try out Visual Studio online, which is basically a complete IDE in your browser written entirely in JavaScript.


Sponsor: Big thanks to our friends at Raygun for sponsoring the feed this week. I use Raygun myself and I encourage you to explore their stuff, it’s really something special. Get full stack error reporting with Raygun! Detect and diagnose problems across your entire application. Raygun supports every major programming language and platform – Try Raygun FREE!


© 2015 Scott Hanselman. All rights reserved.
     

What? Didn't I blog about this before? Yes, in fact, in 2013 I did an article showing how to get Ghost - a lovely nodejs-based blogging app - running on Azure.  The instructions involved making some changes to Ghost itself to make it friendlier to Azure and doing a git deploy. Since this post both Ghost AND Azure have become friendlier. ;)

Fast forward two years and the instructions have changed! In fact, they've gotten WAY easier. You can still theoretically follow most of my old instructions, but these new ones are WAY better, so just do it this way.

STEP 1

There's a "Deploy to Azure" button up at http://deploy.azure.com that you can get an put in YOUR GitHub project's Readme.md so folks can easily and quickly deploy your app as well!

Here's what you do. Head over to the GitHub repository. Click Deploy to Azure. That will automatically install Ghost for you in Azure.

The Deploy to Azure Button

The Deploy.Azure.com website will look at the repository from the HTTP Referer header. You'll end up with a nice form like this.

DO make sure to double check your settings, the Site Location is alphabetical right now, and you may not want your blog in Brazil. ;)

image

Hit Next, then check the summary that will warn you what's getting created, then hit Deploy. Boom. Azure will actually run through the template and setup Ghost (or whatever app you wanted) automatically.

  • Note it's setting it up from Source Control as well, although you can certainly change this. For example, you might want to Fork it yourself, and then Deploy.
  • However, as this is set up today, you won't get updates until you go to Deployments within the Azure Portal and click Sync. You decide if you want the app to update when new code is committed.

After it's created, you can manage your site in the Azure Portal. I made a little free one for this example, as can you if you like.

Ghost in the Azure Portal

And it works just great!

Ghost running on Azure

It's not super obvious what to do next. You'll need click the little chevron there, or visit something like /admin, and you'll get redirected to the Ghost setup process online.

image

Now you can start your first post!

My first post in Ghost

What's going on here?

Felix Rieseberg added a few files to Ghost and has the fork up here on GitHub. The most interesting one is the AzureDeploy.json. This is an Azure Resource Manager template. Here's another simple example from Elliot Hamai for an ASP.NET MVP app. This file tells Azure (and the Deploy to Azure button) what kinds of things it needs to create and actually gives the system enough information to build a whole form for you!

Maybe this is the perfect time for you to start your own blog! Perhaps you've been putting it off. Go check out my FREE two-hour documentary movie with Rob Conery called Get Involved in Tech! We will get you ready to jump into the world of Social Software Development.

Here's a video of Elliot and I talking about the Deploy to Azure button on Azure Friday. Here's Elliot's blog announcing Deploy to Azure and explaining more.

Remember that Ghost is open source and you can learn more at https://ghost.org!

DON'T WANT TO SIGN UP FOR AZURE? You can try Azure out for an hour without signing up for anything. Check out http://try.azurewebsites.net. You can make a PHP, Java, nodejs, Python, ASP.NET web app, or even setup Ghost itself. You can also try out Visual Studio online, which is basically a complete IDE in your browser written entirely in JavaScript.


Sponsor: Big thanks to our friends at Raygun for sponsoring the feed this week. I use Raygun myself and I encourage you to explore their stuff, it's really something special. Get full stack error reporting with Raygun! Detect and diagnose problems across your entire application. Raygun supports every major programming language and platform - Try Raygun FREE!



© 2015 Scott Hanselman. All rights reserved.