Referencing .NET Standard Assemblies from both .NET Core and .NET Framework

Lots of .NET Projects sharing a .NET Standard LibraryI like getting great questions in email but I LOVE getting great questions in email with a complete and clear code repro (reproduction) that’s in a git somewhere. Then I can just clone, build (many many bonus points for a clean build) and check out the bug.

I got a great .NET Core question and repro here https://github.com/ScarlettCode/Example. I forked it, fixed it, and submitted a PR. Here’s the question and issue and today’s fix.

The project has a C# library project (an assembly) that is written to the .NET Standard 2.0. You’ll recall that the .NET Standard isn’t a runtime or a library in itself, but rather an interface. They are saying that this library will work anywhere that the .NET Standard is supported, like Linux, Mac, and Windows.

Here’s that main .NET Standard Library called “Example.Data” written in C#.

Then he had:

  • Windows Forms (WinForms) application in VB.NET using .NET “full” Framework 4.6
  • Console Application also using .NET Framework 4.6
  • Console Application using .NET Core 2.0

Each of these apps is referring to the Example.Data library. The Example.Data library then pulls in a database access library in the form of Microsoft.EntityFrameworkCore.InMemory via NuGet.

WinForms app -> Data Access library -> Some other library. A->B->C where B and C are packages from NuGet.

The .NET Core console builds and runs great. However, when the other projects are run you get this error:

Can't load
Could not load file or assembly
'Microsoft.EntityFrameworkCore, Version=2.0.0.0,
Culture=neutral, PublicKeyToken=adb9793829ddae60'
or one of its dependencies. The system cannot find
the file specified.

Pretty low level error, right? First thing is to check the bin folder (the results of the compile) for a project that doesn’t run. Looks like there’s no Microsoft.EntityFrameworkCore there. Why not? It’s assembly “C” downstream of “A” and “B”. EntityFramework’s assembly is referred to by the Example.Data assembly…but why didn’t it get copied in?

The “full” Framework projects are using the older .csproj format and by default, they use package.config to manage dependencies. The newer projects can reference Packages as first-class references. So we need to tell ALL projects in this solution to manage and restore their packages as “PackageReferences.”

I can open up the .csproj file for the Framework projects and add this line within the first <PropertyGroup> like this to change the restore style:

 <RestoreProjectStyle>PackageReference</RestoreProjectStyle>

As Oren wisely says:

“Using .NET Standard requires you to use PackageReference to eliminate the pain of “lots of packagesâ€� as well as properly handle transitive dependencies. While you may be able to use .NET Standard without PackageReference, I wouldn’t recommend it.”

I can also change the default within VS’s Package Management options here in this dialog.

 <RestoreProjectStyle>PackageReference</RestoreProjectStyle> Default Package management format

Hope this helps.


© 2017 Scott Hanselman. All rights reserved.
     

Lots of .NET Projects sharing a .NET Standard LibraryI like getting great questions in email but I LOVE getting great questions in email with a complete and clear code repro (reproduction) that's in a git somewhere. Then I can just clone, build (many many bonus points for a clean build) and check out the bug.

I got a great .NET Core question and repro here https://github.com/ScarlettCode/Example. I forked it, fixed it, and submitted a PR. Here's the question and issue and today's fix.

The project has a C# library project (an assembly) that is written to the .NET Standard 2.0. You'll recall that the .NET Standard isn't a runtime or a library in itself, but rather an interface. They are saying that this library will work anywhere that the .NET Standard is supported, like Linux, Mac, and Windows.

Here's that main .NET Standard Library called "Example.Data" written in C#.

Then he had:

  • Windows Forms (WinForms) application in VB.NET using .NET "full" Framework 4.6
  • Console Application also using .NET Framework 4.6
  • Console Application using .NET Core 2.0

Each of these apps is referring to the Example.Data library. The Example.Data library then pulls in a database access library in the form of Microsoft.EntityFrameworkCore.InMemory via NuGet.

WinForms app -> Data Access library -> Some other library. A->B->C where B and C are packages from NuGet.

The .NET Core console builds and runs great. However, when the other projects are run you get this error:

Can't load

Could not load file or assembly
'Microsoft.EntityFrameworkCore, Version=2.0.0.0,
Culture=neutral, PublicKeyToken=adb9793829ddae60'
or one of its dependencies. The system cannot find
the file specified.

Pretty low level error, right? First thing is to check the bin folder (the results of the compile) for a project that doesn't run. Looks like there's no Microsoft.EntityFrameworkCore there. Why not? It's assembly "C" downstream of "A" and "B". EntityFramework's assembly is referred to by the Example.Data assembly...but why didn't it get copied in?

The "full" Framework projects are using the older .csproj format and by default, they use package.config to manage dependencies. The newer projects can reference Packages as first-class references. So we need to tell ALL projects in this solution to manage and restore their packages as "PackageReferences."

I can open up the .csproj file for the Framework projects and add this line within the first <PropertyGroup> like this to change the restore style:

 <RestoreProjectStyle>PackageReference</RestoreProjectStyle>

As Oren wisely says:

"Using .NET Standard requires you to use PackageReference to eliminate the pain of “lots of packages� as well as properly handle transitive dependencies. While you may be able to use .NET Standard without PackageReference, I wouldn’t recommend it."

I can also change the default within VS's Package Management options here in this dialog.

 <RestoreProjectStyle>PackageReference</RestoreProjectStyle> Default Package management format

Hope this helps.



© 2017 Scott Hanselman. All rights reserved.
     

Draft – .NET Glossary Diagram

I’m working on this slide as support for this excellent .NET Glossary. It’s not done yet, but I’m curious for your thoughts. Every system has terms and concepts that are initially unfamiliar but make sense once you grok them.

image

Here are these concepts used in an example sentence, for context:

  • Application Framework – “Are you using the ASP.NET Core web framework for that microservice?â€�
  • Metapackage – “I want to install the ASP.NET Core framework; it’s a package of packagesâ€�
  • Package/NuGet – “I know there’s a NuGet package for decoding JSON.â€�
  • Library/Assembly – “Now, you’ll compile your source into an assemblyâ€�
  • .NET Standard – “Which version of the .NET Standard specification does your assembly target?”
    • “My Apple Watch supports .NET Standard 1.6 but my Windows 10 laptop supports 2.0 with more APIs.â€�
  • C#, F#, VB, etc – “Which language did you use?â€�
  • .NET SDK – “Did you get the developer tools?â€�
  • CLR/CoreCLR – “Which runtime is your app using?â€�
  • An implementation of .NET is a runtime along with libraries that implement a version of the .NET Standard
    • “Are you using .NET Core, .NET Framework, or Mono for this project?â€�
  • Platform – An operating system and some hardware (ARM, x64, etc.)
    • “Is that an ASP.NET Core app running in Docker on a Raspberry Pi?â€�

Constructive feedback, please. This is a draft.


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2017 Scott Hanselman. All rights reserved.
     

I'm working on this slide as support for this excellent .NET Glossary. It's not done yet, but I'm curious for your thoughts. Every system has terms and concepts that are initially unfamiliar but make sense once you grok them.

image

Here are these concepts used in an example sentence, for context:

  • Application Framework - “Are you using the ASP.NET Core web framework for that microservice?â€�
  • Metapackage - “I want to install the ASP.NET Core framework; it’s a package of packagesâ€�
  • Package/NuGet - “I know there’s a NuGet package for decoding JSON.â€�
  • Library/Assembly - “Now, you’ll compile your source into an assemblyâ€�
  • .NET Standard – “Which version of the .NET Standard specification does your assembly target?"
    • "My Apple Watch supports .NET Standard 1.6 but my Windows 10 laptop supports 2.0 with more APIs.â€�
  • C#, F#, VB, etc – “Which language did you use?â€�
  • .NET SDK - “Did you get the developer tools?â€�
  • CLR/CoreCLR – “Which runtime is your app using?â€�
  • An implementation of .NET is a runtime along with libraries that implement a version of the .NET Standard
    • “Are you using .NET Core, .NET Framework, or Mono for this project?â€�
  • Platform - An operating system and some hardware (ARM, x64, etc.)
    • “Is that an ASP.NET Core app running in Docker on a Raspberry Pi?â€�

Constructive feedback, please. This is a draft.


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!



© 2017 Scott Hanselman. All rights reserved.
     

Exploring refit, an automatic type-safe REST library for .NET Standard

I dig everything that Paul Betts does. He’s a lovely person and a prolific coder. One of his recent joints is called Refit. It’s a REST library for .NET that is inspired by Square’s Retrofit library. It turns your REST API into a live interface:

public interface IGitHubApi
{
[Get("/users/{user}")]
Task<User> GetUser(string user);
}

That’s an interface that describes a REST API that’s elsewhere. Then later you just make a RestService.For<YourInterface> and you go to town.

var gitHubApi = RestService.For<IGitHubApi>("https://api.github.com");

var octocat = await gitHubApi.GetUser("octocat");

imageThat’s lovely! It is a .NET Standard 1.4 library which means you can use it darn near everywhere. Remember that .NET Standard isn’t a runtime, it’s a version interface – a list of methods you can use under many different “.NETs.” You can use Refit on UWP, Xamarin.*, .NET “full” Frameowrk, and .NET Core, which runs basically everywhere.

Sure, you can make your own HttpClient calls, but that’s a little low level and somewhat irritating. Sure, you can look for a .NET SDK for your favorite REST interface but what if it doesn’t have one? It strikes a nice balance between the low-level and the high-level.

I’ll give an example and use it as a tiny exercise for Refit. I have a service that hosts a realtime feed of my blood sugar, as I’m a Type 1 Diabetic. Since I have a Continuous Glucose Meter that is attached to me and sending my sugar details to a web service called Nightscout running in Azure, I figured it’d be cool to use Refit to pull my sugar info back down with .NET.

The REST API for Nightscout is simple, but doe have a lot of options, query strings, and multiple endpoints. I can start by making a simple interface for the little bits I want now, and perhaps expand the interface later to get more.

For example, if I want my sugars, I would go

https://MYWEBSITE/api/v1/entries.json?count=10

And get back some JSON data like this:

[
{
_id: "5993c4aa8d60c09b63ba1c",
sgv: 162,
date: 1502856279000,
dateString: "2017-08-16T04:04:39.000Z",
trend: 4,
direction: "Flat",
device: "share2",
type: "sgv"
},
{
_id: "5993c37d8d60c09b93ba0b",
sgv: 162,
date: 1502855979000,
dateString: "2017-08-16T03:59:39.000Z",
trend: 4,
direction: "Flat",
device: "share2",
type: "sgv"
}
]

Where “sgv” is serum glucose value, or blood sugar.

Starting with .NET Core 2.0 and the SDK that I installed from http://dot.net, I’ll first make a console app from the command line and add refit like this:

C:\users\scott\desktop\refitsugars> dotnet new console
C:\users\scott\desktop\refitsugars> dotnet add package refit

Here’s my little bit of code.

  • I made an object shaped like each recorded. Added aliases for weirdly named stuff like “sgv”
  • COOL SIDE NOTE: I added <LangVersion>7.1</LangVersion> to my project so I could have my public static Main entry point be async. That’s new as many folks have wanted to have a “public static async void Main()” equivalent.

After that it’s REALLY lovely and super easy to make a quick strongly-typed REST Client in C# for pretty much anything. I could see myself easily extending this to include the whole NightScout diabetes management API without a lot of effort.

using Newtonsoft.Json;
using Refit;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace refitsugars
{
public interface INightScoutApi
{
[Get("/api/v1/entries.json?count={count}")]
Task<List<Sugar>> GetSugars(int count);
}

public class Sugar
{
[JsonProperty(PropertyName = "_id")]
public string id { get; set; }

[JsonProperty(PropertyName = "sgv")]
public int glucose { get; set; }

[JsonProperty(PropertyName = "dateString")]
public DateTime itemDate { get; set; }
public int trend { get; set; }
}

class Program
{
public static async Task Main(string[] args)
{
var nsAPI = RestService.For<INightScoutApi>("https://MYURL.azurewebsites.net");
var sugars = await nsAPI.GetSugars(3);
sugars.ForEach(x => { Console.WriteLine($"{x.itemDate.ToLocalTime()} {x.glucose} mg/dl"); });
}
}
}

And here’s the result of the run.

PS C:\Users\scott\Desktop\refitsugars> dotnet run
8/15/2017 10:29:39 PM 110 mg/dl
8/15/2017 10:24:39 PM 108 mg/dl
8/15/2017 10:19:40 PM 109 mg/dl

You should definitely check out Refit. It’s very easy and quite fun. The fact that it targets .NET Standard 1.4 means you can use it in nearly all your .NET projects, and it already has creative people thinking of cool ideas.


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2017 Scott Hanselman. All rights reserved.
     

I dig everything that Paul Betts does. He's a lovely person and a prolific coder. One of his recent joints is called Refit. It's a REST library for .NET that is inspired by Square's Retrofit library. It turns your REST API into a live interface:

public interface IGitHubApi

{
[Get("/users/{user}")]
Task<User> GetUser(string user);
}

That's an interface that describes a REST API that's elsewhere. Then later you just make a RestService.For<YourInterface> and you go to town.

var gitHubApi = RestService.For<IGitHubApi>("https://api.github.com");


var octocat = await gitHubApi.GetUser("octocat");

imageThat's lovely! It is a .NET Standard 1.4 library which means you can use it darn near everywhere. Remember that .NET Standard isn't a runtime, it's a version interface - a list of methods you can use under many different ".NETs." You can use Refit on UWP, Xamarin.*, .NET "full" Frameowrk, and .NET Core, which runs basically everywhere.

Sure, you can make your own HttpClient calls, but that's a little low level and somewhat irritating. Sure, you can look for a .NET SDK for your favorite REST interface but what if it doesn't have one? It strikes a nice balance between the low-level and the high-level.

I'll give an example and use it as a tiny exercise for Refit. I have a service that hosts a realtime feed of my blood sugar, as I'm a Type 1 Diabetic. Since I have a Continuous Glucose Meter that is attached to me and sending my sugar details to a web service called Nightscout running in Azure, I figured it'd be cool to use Refit to pull my sugar info back down with .NET.

The REST API for Nightscout is simple, but doe have a lot of options, query strings, and multiple endpoints. I can start by making a simple interface for the little bits I want now, and perhaps expand the interface later to get more.

For example, if I want my sugars, I would go

https://MYWEBSITE/api/v1/entries.json?count=10

And get back some JSON data like this:

[

{
_id: "5993c4aa8d60c09b63ba1c",
sgv: 162,
date: 1502856279000,
dateString: "2017-08-16T04:04:39.000Z",
trend: 4,
direction: "Flat",
device: "share2",
type: "sgv"
},
{
_id: "5993c37d8d60c09b93ba0b",
sgv: 162,
date: 1502855979000,
dateString: "2017-08-16T03:59:39.000Z",
trend: 4,
direction: "Flat",
device: "share2",
type: "sgv"
}
]

Where "sgv" is serum glucose value, or blood sugar.

Starting with .NET Core 2.0 and the SDK that I installed from http://dot.net, I'll first make a console app from the command line and add refit like this:

C:\users\scott\desktop\refitsugars> dotnet new console

C:\users\scott\desktop\refitsugars> dotnet add package refit

Here's my little bit of code.

  • I made an object shaped like each recorded. Added aliases for weirdly named stuff like "sgv"
  • COOL SIDE NOTE: I added <LangVersion>7.1</LangVersion> to my project so I could have my public static Main entry point be async. That's new as many folks have wanted to have a "public static async void Main()" equivalent.

After that it's REALLY lovely and super easy to make a quick strongly-typed REST Client in C# for pretty much anything. I could see myself easily extending this to include the whole NightScout diabetes management API without a lot of effort.

using Newtonsoft.Json;

using Refit;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace refitsugars
{
public interface INightScoutApi
{
[Get("/api/v1/entries.json?count={count}")]
Task<List<Sugar>> GetSugars(int count);
}

public class Sugar
{
[JsonProperty(PropertyName = "_id")]
public string id { get; set; }

[JsonProperty(PropertyName = "sgv")]
public int glucose { get; set; }

[JsonProperty(PropertyName = "dateString")]
public DateTime itemDate { get; set; }
public int trend { get; set; }
}

class Program
{
public static async Task Main(string[] args)
{
var nsAPI = RestService.For<INightScoutApi>("https://MYURL.azurewebsites.net");
var sugars = await nsAPI.GetSugars(3);
sugars.ForEach(x => { Console.WriteLine($"{x.itemDate.ToLocalTime()} {x.glucose} mg/dl"); });
}
}
}

And here's the result of the run.

PS C:\Users\scott\Desktop\refitsugars> dotnet run

8/15/2017 10:29:39 PM 110 mg/dl
8/15/2017 10:24:39 PM 108 mg/dl
8/15/2017 10:19:40 PM 109 mg/dl

You should definitely check out Refit. It's very easy and quite fun. The fact that it targets .NET Standard 1.4 means you can use it in nearly all your .NET projects, and it already has creative people thinking of cool ideas.


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2017 Scott Hanselman. All rights reserved.
     

.NET and WebAssembly – Is this the future of the front-end?

6 years ago Erik Meijer and I were talking about how JavaScript is/was an assembly language. It turned into an interesting discussion/argument (some people really didn’t buy it) but it still kept happening. Currently WebAssembly world is marching forward and is supported in Chrome, Firefox, and in Development in Edge, Opera, and Safari.

“The avalanche has begun, it’s too late for the pebbles to vote.” – Ambassador Kosh

Today in 2017, WebAssembly is absolutely a thing and you can learn about it at http://webassembly.org. I even did a podcast on WebAssembly with Mozilla Fellow David Bryant (you really should check out my podcast, I’m very proud of it. It’s good.)

The classic JavaScript TODO app, written with C# and .NET and Blazor

The image above is from Steve Sanderson’s NDC presentation. He’s writing the classic client-side JavaScript ToDo application…except he’s writing the code in C#.

What is WebAssembly?

“WebAssembly or wasm is a low-level bytecode format for in-browser client-side scripting, evolved from JavaScript.” You can easily compile to WebAssembly from C and C++ today…and more languages are jumping in to include WebAssembly as a target every day.

Since I work in open source .NET and since .NET Core 2.0 is cross-platform with an imminent release, it’s worth exploring where WebAssembly fits into a .NET world.

Here’s some projects I have identified that help bridge the .NET world and the WebAssembly world. I think that this is going to be THE hot space in the next 18 months.

WebAssembly for .NET

Despite its overarching name, this OSS project is meant to consume WASM binary files and execute them from within .NET assemblies. To be clear, this isn’t compiling .NET languages’ (C#, VB.NET, F#) into WebAssembly, this is for using WebAssembly as if it’s any other piece of resuable compiled code. Got an existing WASM file you REALLY want to call from .NET? This is for that.

Interestingly, this project doesn’t spin up a V8 or Chakra JavaScript engine to run WASM, instead it reads in the bytecode and converts them to .NET via System.Reflection.Emit. Interesting stuff!

Mono and WebAssembly

One of the great things happening in the larger .NET Ecosystem is that there is more than one “.NET” today. In the past, .NET was a thing that you installed on Windows and generally feared. Today, there’s .NET 4.x+ on basically every Windows machine out there, there’s .NET Core that runs in Docker, on Mac, Windows, and a dozen Linuxes…even Raspberry Pi, and Mono is another instance of .NET that allows you to run code in dozens of other platforms. There’s multiple “instances of .NET” out there in active development.

The Mono Project has two prototypes using Mono and WebAssembly.

The first one uses the traditional full static compilation mode of Mono, this compiled both the Mono C runtime and the Mono class libraries along with the user code into WebAssembly code. It produces one large statically compiled application. You can try this fully statically compiled Hello World here. The full static compilation currently lives here.

So that’s a totally statically compiled Hello World…it’s all of Mono and your app into Web Assembly. They have another prototype with a difference perspective:

The second prototype compiles the Mono C runtime into web assembly, and then uses Mono’s IL interpreter to run managed code. This one is a smaller download, but comes at the expense of performance. The mixed mode execution prototype currently lives here.

Here they’ve got much of Mono running in Web Assembly, but your IL code is interpreted. One of the wonderful things about Computer Science – There is more than one way to do something, and they are often each awesome in their own way!

“Blazor” – Experimental UI Framework running .NET in the browser

With a similar idea as the Mono Project’s second prototype, Steve Sanderson took yet another “instance of .NET,” the six year old open source DotNetAnywhere (DNA) project and compiled it into Web Assembly. DNA was an interpreted .NET runtime written in portable C. It takes standard IL or CIL (Common Intermediate Language) and runs it “on resource-constrained devices where it is not possible to run a full .NET runtime (e.g. Mono).” Clever, huh? What “resource-constrained device do we have here six years later?” Why, it’s the little virtual machine that could – the JavaScript VM that your browser already has, now powered by a standard bytecode format called WebAssembly.

To prove the concept, Steve compiles DotNetAnywhere to WASM but then takes it further. He’s combined standard programming models that we see on the web with things like Angular, Knockoutjs, or Ember, except rather than writing your web applications’ UI in JavaScript, you write in C# – a .NET language.

Here in the middle of some Razor (basically HTML with C# inline) pages, he does what looks like a call to a backend. This is C# code, but it’ll run as WASM on the client side within a Blazor app.

@functions {
WeatherForecast[] forecasts;

override protected async Task InitAsync()
{
using (var client = new HttpClient())
{
var json = await client.GetStringAsync(AbsoluteUrl("/api/SampleData/WeatherForecasts"));
forecasts = JsonUtil.Deserialize<WeatherForecast[]>(json);
}
}
}

This would allow a .NET programmer to use the same data models on the client and the server – much like well-factored JavaScript should today – as well as using other .NET libraries they might be familiar or comfortable with.

Why do this insane thing? “To see how well such a framework might work, and how much anyone would care.” How far could/should this go? David Fowler already has debugging working (again this is ALL prototypes) in Visual Studio Code. Don’t take my word for it, watch the video as Steve presents the concept at the NDC Conference.

Blazor as a prototype has a number of people excited, and there was a Blazor Hackthon recently that produced some interesting samples including a full-blown app.

Other possibilities?

There’s lots of other projects that are compiling or transpiling things to JavaScript. Could they be modified to support WebAssembly? You can take F# and compile it to JavaScript with F#’s Fable project, and some folks have asked about WebAssembly.

At this point it’s clear that everyone is prototyping and hacking and enjoying themselves.

What do YOU think about WebAssembly?


© 2017 Scott Hanselman. All rights reserved.
     

6 years ago Erik Meijer and I were talking about how JavaScript is/was an assembly language. It turned into an interesting discussion/argument (some people really didn't buy it) but it still kept happening. Currently WebAssembly world is marching forward and is supported in Chrome, Firefox, and in Development in Edge, Opera, and Safari.

"The avalanche has begun, it's too late for the pebbles to vote." - Ambassador Kosh

Today in 2017, WebAssembly is absolutely a thing and you can learn about it at http://webassembly.org. I even did a podcast on WebAssembly with Mozilla Fellow David Bryant (you really should check out my podcast, I'm very proud of it. It's good.)

The classic JavaScript TODO app, written with C# and .NET and Blazor

The image above is from Steve Sanderson's NDC presentation. He's writing the classic client-side JavaScript ToDo application...except he's writing the code in C#.

What is WebAssembly?

"WebAssembly or wasm is a low-level bytecode format for in-browser client-side scripting, evolved from JavaScript." You can easily compile to WebAssembly from C and C++ today...and more languages are jumping in to include WebAssembly as a target every day.

Since I work in open source .NET and since .NET Core 2.0 is cross-platform with an imminent release, it's worth exploring where WebAssembly fits into a .NET world.

Here's some projects I have identified that help bridge the .NET world and the WebAssembly world. I think that this is going to be THE hot space in the next 18 months.

WebAssembly for .NET

Despite its overarching name, this OSS project is meant to consume WASM binary files and execute them from within .NET assemblies. To be clear, this isn't compiling .NET languages' (C#, VB.NET, F#) into WebAssembly, this is for using WebAssembly as if it's any other piece of resuable compiled code. Got an existing WASM file you REALLY want to call from .NET? This is for that.

Interestingly, this project doesn't spin up a V8 or Chakra JavaScript engine to run WASM, instead it reads in the bytecode and converts them to .NET via System.Reflection.Emit. Interesting stuff!

Mono and WebAssembly

One of the great things happening in the larger .NET Ecosystem is that there is more than one ".NET" today. In the past, .NET was a thing that you installed on Windows and generally feared. Today, there's .NET 4.x+ on basically every Windows machine out there, there's .NET Core that runs in Docker, on Mac, Windows, and a dozen Linuxes...even Raspberry Pi, and Mono is another instance of .NET that allows you to run code in dozens of other platforms. There's multiple "instances of .NET" out there in active development.

The Mono Project has two prototypes using Mono and WebAssembly.

The first one uses the traditional full static compilation mode of Mono, this compiled both the Mono C runtime and the Mono class libraries along with the user code into WebAssembly code. It produces one large statically compiled application. You can try this fully statically compiled Hello World here. The full static compilation currently lives here.

So that's a totally statically compiled Hello World...it's all of Mono and your app into Web Assembly. They have another prototype with a difference perspective:

The second prototype compiles the Mono C runtime into web assembly, and then uses Mono’s IL interpreter to run managed code. This one is a smaller download, but comes at the expense of performance. The mixed mode execution prototype currently lives here.

Here they've got much of Mono running in Web Assembly, but your IL code is interpreted. One of the wonderful things about Computer Science - There is more than one way to do something, and they are often each awesome in their own way!

"Blazor" - Experimental UI Framework running .NET in the browser

With a similar idea as the Mono Project's second prototype, Steve Sanderson took yet another "instance of .NET," the six year old open source DotNetAnywhere (DNA) project and compiled it into Web Assembly. DNA was an interpreted .NET runtime written in portable C. It takes standard IL or CIL (Common Intermediate Language) and runs it "on resource-constrained devices where it is not possible to run a full .NET runtime (e.g. Mono)." Clever, huh? What "resource-constrained device do we have here six years later?" Why, it's the little virtual machine that could - the JavaScript VM that your browser already has, now powered by a standard bytecode format called WebAssembly.

To prove the concept, Steve compiles DotNetAnywhere to WASM but then takes it further. He's combined standard programming models that we see on the web with things like Angular, Knockoutjs, or Ember, except rather than writing your web applications' UI in JavaScript, you write in C# - a .NET language.

Here in the middle of some Razor (basically HTML with C# inline) pages, he does what looks like a call to a backend. This is C# code, but it'll run as WASM on the client side within a Blazor app.

@functions {

WeatherForecast[] forecasts;

override protected async Task InitAsync()
{
using (var client = new HttpClient())
{
var json = await client.GetStringAsync(AbsoluteUrl("/api/SampleData/WeatherForecasts"));
forecasts = JsonUtil.Deserialize<WeatherForecast[]>(json);
}
}
}

This would allow a .NET programmer to use the same data models on the client and the server - much like well-factored JavaScript should today - as well as using other .NET libraries they might be familiar or comfortable with.

Why do this insane thing? "To see how well such a framework might work, and how much anyone would care." How far could/should this go? David Fowler already has debugging working (again this is ALL prototypes) in Visual Studio Code. Don't take my word for it, watch the video as Steve presents the concept at the NDC Conference.

Blazor as a prototype has a number of people excited, and there was a Blazor Hackthon recently that produced some interesting samples including a full-blown app.

Other possibilities?

There's lots of other projects that are compiling or transpiling things to JavaScript. Could they be modified to support WebAssembly? You can take F# and compile it to JavaScript with F#'s Fable project, and some folks have asked about WebAssembly.

At this point it's clear that everyone is prototyping and hacking and enjoying themselves.

What do YOU think about WebAssembly?



© 2017 Scott Hanselman. All rights reserved.
     

Peachpie – Open Source PHP Compiler to .NET and WordPress under ASP.NET Core

The Peachpie PHP compiler project joined the .NET Foundation this week and I’m trying to get my head around it. PHP in .NET? PHP on .NET? Under .NET? What compiles to what? Why would I want this? How does it work? Does it feel awesome or does it feel gross?

image

Just drink this in.

C:\Users\scott\Desktop\peachcon> type program.php
<?php

function main()
{
echo "Hello .NET World!";
}

main();

C:\Users\scott\Desktop\peachcon> dotnet run
Hello .NET World!

Just like that. Starting from a .NET SDK (They say 1.1, although I used a 2.0 preview) you just add their templates

dotnet new -i Peachpie.Templates::*

Then dotnet new now shows a bunch of php options.

C:\Users\scott\Desktop\peachcon> dotnet new | find /i "php"
Peachpie console application peachpie-console PHP Console
Peachpie Class library peachpie-classlibrary PHP Library
Peachpie web application peachpie-web PHP Web/Empty

dotnet new peachpie-console for example, then dotnet restore and dotnet run. Boom.

NOTE: I did have to comment out his one line “<Import Project=”$(CSharpDesignTimeTargetsPath)” />” in their project file that doesn’t work at the command line. It’s some hack they did to make things work in Visual Studio but I’m using VS Code. I’m sure it’s an alpha-point-in-time thing.

It’s really compiling PHP into .NET Intermediate Language!

PHP to .NET

You can see my string here:

Hello .NET World inside a PHP app inside the CLR

But…why? Here’s what they say, and much of it makes sense to me.

  1. Performance: compiled code is fast and also optimized by the .NET Just-in-Time Compiler for your actual system. Additionally, the .NET performance profiler may be used to resolve bottlenecks.
  2. C# Extensibility: plugin functionality can be implemented in a separate C# project and/or PHP plugins may use .NET libraries.
  3. Sourceless distribution: after the compilation, most of the source files are not needed.
  4. Power of .NET: Peachpie allows the compiled WordPress clone to run in a .NET JIT’ted, secure and manageable environment, updated through windows update.
  5. No need to install PHP: Peachpie is a modern compiler platform and runtime distributed as a dependency to your .NET project. It is downloaded automatically on demand as a NuGet package or it can be even deployed standalone together with the compiled application as its library dependency.

PHP does have other VMs/Runtimes that are used (beyond just PHP.exe) but the idea that I could reuse code between PHP and C# is attractive, not to mention the “PHP as dependency” part. Imagine if I have an existing .NET shop or project and now I want to integrate something like WordPress?

PHP under ASP.NET Core

Their Web Sample is even MORE interesting, as they’ve implemented PHP as ASP.NET Middleware. Check this out. See where they pass in the PHP app as an assembly they compiled?

using Peachpie.Web;

namespace peachweb.Server
{
class Program
{
static void Main(string[] args)
{
var host = new WebHostBuilder()
.UseKestrel()
.UseUrls("http://*:5004/")
.UseStartup<Startup>()
.Build();

host.Run();
}
}

class Startup
{
public void ConfigureServices(IServiceCollection services)
{
// Adds a default in-memory implementation of IDistributedCache.
services.AddDistributedMemoryCache();

services.AddSession(options =>
{
options.IdleTimeout = TimeSpan.FromMinutes(30);
options.CookieHttpOnly = true;
});
}

public void Configure(IApplicationBuilder app)
{
app.UseSession();

app.UsePhp(new PhpRequestOptions(scriptAssemblyName: "peachweb"));
app.UseDefaultFiles();
app.UseStaticFiles();
}
}
}

Interesting, but it’s still Hello World. Let’s run WordPress under PeachPie (and hence, under .NET). I’ll run MySQL in a local Docker container for simplicity:

docker run -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=wordpress -p 3306:3306 -d mysql

I downloaded WordPress from here (note they have the “app” bootstrapper” that hosts .NET and then runs WordPress) restore and run.

WordPress under .NET Core

It’s early and it’s alpha – so set your expectations appropriately – but it’s surprisingly useful and appears to be under active development.

What do you think?

Be sure to explore their resources at http://www.peachpie.io/resources and watch their video of WordPress running on .NET. It’s all Open Source, in the .NET Foundation, and the code is up at https://github.com/iolevel/ and you can get started here: http://www.peachpie.io/getstarted


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2017 Scott Hanselman. All rights reserved.
     

The Peachpie PHP compiler project joined the .NET Foundation this week and I'm trying to get my head around it. PHP in .NET? PHP on .NET? Under .NET? What compiles to what? Why would I want this? How does it work? Does it feel awesome or does it feel gross?

image

Just drink this in.

C:\Users\scott\Desktop\peachcon> type program.php

<?php

function main()
{
echo "Hello .NET World!";
}

main();

C:\Users\scott\Desktop\peachcon> dotnet run
Hello .NET World!

Just like that. Starting from a .NET SDK (They say 1.1, although I used a 2.0 preview) you just add their templates

dotnet new -i Peachpie.Templates::*

Then dotnet new now shows a bunch of php options.

C:\Users\scott\Desktop\peachcon> dotnet new | find /i "php"

Peachpie console application peachpie-console PHP Console
Peachpie Class library peachpie-classlibrary PHP Library
Peachpie web application peachpie-web PHP Web/Empty

dotnet new peachpie-console for example, then dotnet restore and dotnet run. Boom.

NOTE: I did have to comment out his one line "<Import Project="$(CSharpDesignTimeTargetsPath)" />" in their project file that doesn't work at the command line. It's some hack they did to make things work in Visual Studio but I'm using VS Code. I'm sure it's an alpha-point-in-time thing.

It's really compiling PHP into .NET Intermediate Language!

PHP to .NET

You can see my string here:

Hello .NET World inside a PHP app inside the CLR

But...why? Here's what they say, and much of it makes sense to me.

  1. Performance: compiled code is fast and also optimized by the .NET Just-in-Time Compiler for your actual system. Additionally, the .NET performance profiler may be used to resolve bottlenecks.
  2. C# Extensibility: plugin functionality can be implemented in a separate C# project and/or PHP plugins may use .NET libraries.
  3. Sourceless distribution: after the compilation, most of the source files are not needed.
  4. Power of .NET: Peachpie allows the compiled WordPress clone to run in a .NET JIT'ted, secure and manageable environment, updated through windows update.
  5. No need to install PHP: Peachpie is a modern compiler platform and runtime distributed as a dependency to your .NET project. It is downloaded automatically on demand as a NuGet package or it can be even deployed standalone together with the compiled application as its library dependency.

PHP does have other VMs/Runtimes that are used (beyond just PHP.exe) but the idea that I could reuse code between PHP and C# is attractive, not to mention the "PHP as dependency" part. Imagine if I have an existing .NET shop or project and now I want to integrate something like WordPress?

PHP under ASP.NET Core

Their Web Sample is even MORE interesting, as they've implemented PHP as ASP.NET Middleware. Check this out. See where they pass in the PHP app as an assembly they compiled?

using Peachpie.Web;


namespace peachweb.Server
{
class Program
{
static void Main(string[] args)
{
var host = new WebHostBuilder()
.UseKestrel()
.UseUrls("http://*:5004/")
.UseStartup<Startup>()
.Build();

host.Run();
}
}

class Startup
{
public void ConfigureServices(IServiceCollection services)
{
// Adds a default in-memory implementation of IDistributedCache.
services.AddDistributedMemoryCache();

services.AddSession(options =>
{
options.IdleTimeout = TimeSpan.FromMinutes(30);
options.CookieHttpOnly = true;
});
}

public void Configure(IApplicationBuilder app)
{
app.UseSession();

app.UsePhp(new PhpRequestOptions(scriptAssemblyName: "peachweb"));
app.UseDefaultFiles();
app.UseStaticFiles();
}
}
}

Interesting, but it's still Hello World. Let's run WordPress under PeachPie (and hence, under .NET). I'll run MySQL in a local Docker container for simplicity:

docker run -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=wordpress -p 3306:3306 -d mysql

I downloaded WordPress from here (note they have the "app" bootstrapper" that hosts .NET and then runs WordPress) restore and run.

WordPress under .NET Core

It's early and it's alpha - so set your expectations appropriately - but it's surprisingly useful and appears to be under active development.

What do you think?

Be sure to explore their resources at http://www.peachpie.io/resources and watch their video of WordPress running on .NET. It's all Open Source, in the .NET Foundation, and the code is up at https://github.com/iolevel/ and you can get started here: http://www.peachpie.io/getstarted


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2017 Scott Hanselman. All rights reserved.
     

dotnet sdk list and dotnet sdk latest

dotnet sdk listCan someone make .NET Core better with a simple global command? Fanie Reynders did and he did it in a simple and elegant way. I’m envious, in fact, because I spec’ed this exact thing out in a meeting a few months ago but I could have just done it like he did and I would have used fewer keystrokes!

Last year when .NET Core was just getting started, there was a “DNVM” helper command that you could use to simplify dealing with multiple versions of the .NET SDK on one machine. Later, rather than ‘switching global SDK versions,’ switching was simplified to be handled on a folder by folder basis. That meant that if you had a project in a folder with no global.json that pinned the SDK version, your project would use the latest installed version. If you liked, you could create a global.json file and pin your project’s folder to a specific version. Great, but I would constantly have to google to remember the format for the global.json file, and I’d constantly go into c:\Program Files\dotnet in order to get a list of the currently installed SDKs. I proposed that Microsoft make a “dotnet sdk list” command and the ability to pin down versions like “dotnet sdk 1.0.4” and even maybe install new ones with “dotnet sdk install 2.1.0” or something.

Fanie did all this for us except the installation part, and his implementation is clean and simple. It’s so simple that I just dropped his commands into my Dropbox’s Utils folder that I have in my PATH on all my machines. Now every machine I dev on has this extension.

Note that if I type “dotnet foo” the dotnet.exe driver will look in the path for an executable command called dotnet-foo.* and run it.

C:\Users\scott\Desktop>dotnet foo
No executable found matching command "dotnet-foo"

C:\Users\scott\Desktop>dotnet sdk
No executable found matching command "dotnet-sdk"

He created a dotnet-sdk.cmd you can get on his GitHub. Download his repo and put his command somewhere in your path. Now I can do this:

C:\Users\scott\Desktop>dotnet sdk list
The installed .NET Core SDKs are:
1.0.0
1.0.0-preview2-003131
1.0.0-rc3-004530
1.0.2
1.0.4

Which is lovely, but the real use case is this:

C:\Users\scott\Desktop\fancypants>dotnet --version
1.0.4

C:\Users\scott\Desktop\fancypants>dotnet sdk 1.0.0
Switching .NET Core SDK version to 1.0.0

C:\Users\scott\Desktop\fancypants>dotnet --version
1.0.0

C:\Users\scott\Desktop\fancypants>dir
Volume in drive C is Windows
Directory of C:\Users\scott\Desktop\fancypants

07/26/2017 04:53 PM 47 global.json
1 File(s) 47 bytes

Then if I go “dotnet sdk latest” it just deletes the global.json. Perhaps in a perfect world it should just remove the sdk JSON node in case global.json has been modified, but for now it’s great. Without the global.json the dotnet.exe will just use your latest installed SDK.

This works with .NET Core 2.0 as well. This should be built-in, but for now it’s a very nice example of a clean extension to dotnet.exe.

Oh, and by the way, he also made a “.net.cmd” so you can do this with all your dotnet.exe commands.

.NET run

Give these commands a try!


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2017 Scott Hanselman. All rights reserved.
     

dotnet sdk listCan someone make .NET Core better with a simple global command? Fanie Reynders did and he did it in a simple and elegant way. I'm envious, in fact, because I spec'ed this exact thing out in a meeting a few months ago but I could have just done it like he did and I would have used fewer keystrokes!

Last year when .NET Core was just getting started, there was a "DNVM" helper command that you could use to simplify dealing with multiple versions of the .NET SDK on one machine. Later, rather than 'switching global SDK versions,' switching was simplified to be handled on a folder by folder basis. That meant that if you had a project in a folder with no global.json that pinned the SDK version, your project would use the latest installed version. If you liked, you could create a global.json file and pin your project's folder to a specific version. Great, but I would constantly have to google to remember the format for the global.json file, and I'd constantly go into c:\Program Files\dotnet in order to get a list of the currently installed SDKs. I proposed that Microsoft make a "dotnet sdk list" command and the ability to pin down versions like "dotnet sdk 1.0.4" and even maybe install new ones with "dotnet sdk install 2.1.0" or something.

Fanie did all this for us except the installation part, and his implementation is clean and simple. It's so simple that I just dropped his commands into my Dropbox's Utils folder that I have in my PATH on all my machines. Now every machine I dev on has this extension.

Note that if I type "dotnet foo" the dotnet.exe driver will look in the path for an executable command called dotnet-foo.* and run it.

C:\Users\scott\Desktop>dotnet foo

No executable found matching command "dotnet-foo"

C:\Users\scott\Desktop>dotnet sdk
No executable found matching command "dotnet-sdk"

He created a dotnet-sdk.cmd you can get on his GitHub. Download his repo and put his command somewhere in your path. Now I can do this:

C:\Users\scott\Desktop>dotnet sdk list

The installed .NET Core SDKs are:
1.0.0
1.0.0-preview2-003131
1.0.0-rc3-004530
1.0.2
1.0.4

Which is lovely, but the real use case is this:

C:\Users\scott\Desktop\fancypants>dotnet --version

1.0.4

C:\Users\scott\Desktop\fancypants>dotnet sdk 1.0.0
Switching .NET Core SDK version to 1.0.0

C:\Users\scott\Desktop\fancypants>dotnet --version
1.0.0

C:\Users\scott\Desktop\fancypants>dir
Volume in drive C is Windows
Directory of C:\Users\scott\Desktop\fancypants

07/26/2017 04:53 PM 47 global.json
1 File(s) 47 bytes

Then if I go "dotnet sdk latest" it just deletes the global.json. Perhaps in a perfect world it should just remove the sdk JSON node in case global.json has been modified, but for now it's great. Without the global.json the dotnet.exe will just use your latest installed SDK.

This works with .NET Core 2.0 as well. This should be built-in, but for now it's a very nice example of a clean extension to dotnet.exe.

Oh, and by the way, he also made a ".net.cmd" so you can do this with all your dotnet.exe commands.

.NET run

Give these commands a try!


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!



© 2017 Scott Hanselman. All rights reserved.
     

13 hours debugging a segmentation fault in .NET Core on Raspberry Pi and the solution was…

Debugging is a satisfying and special kind of hell. You really have to live it to understand it. When you’re deep into it you never know when it’ll be done. When you do finally escape it’s almost always a DOH! moment.

I spent an entire day debugging an issue and the solution ended up being a checkbox.

NOTE: If you get a third of the way through this blog post and already figured it out, well, poop on you. Where were you after lunch WHEN I NEEDED YOU?

I wanted to use a Raspberry Pi in a tech talk I’m doing tomorrow at a conference. I was going to show .NET Core 2.0 and ASP.NET running on a Raspberry Pi so I figured I’d start with Hello World. How hard could it be?

You’ll write and build a .NET app on Windows or Mac, then publish it to the Raspberry Pi. I’m using a preview build of the .NET Core 2.0 command line and SDK (CLI) I got from here.

C:\raspberrypi> dotnet new console
C:\raspberrypi> dotnet run
Hello World!
C:\raspberrypi> dotnet publish -r linux-arm
Microsoft Build Engine version for .NET Core

raspberrypi1 -> C:\raspberrypi\bin\Debug\netcoreapp2.0\linux-arm\raspberrypi.dll
raspberrypi1 -> C:\raspberrypi\bin\Debug\netcoreapp2.0\linux-arm\publish\

Notice the simplified publish. You’ll get a folder for linux-arm in this example, but could also publish osx-x64, etc. You’ll want to take the files from the publish folder (not the folder above it) and move them to the Raspberry Pi. This is a self-contained application that targets ARM on Linux so after the prerequisites that’s all you need.

I grabbed a mini-SD card, headed over to https://www.raspberrypi.org/downloads/ and downloaded the latest Raspbian image. I used etcher.io – a lovely image burner for Windows, Mac, or Linux – and wrote the image to the SD Card. I booted up and got ready to install some prereqs. I’m only 15 min in at this point. Setting up a Raspberry Pi 2 or Raspberry Pi 3 is VERY smooth these days.

Here’s the prereqs for .NET Core 2 on Ubuntu or Debian/Raspbian. Install them from the terminal, natch.

sudo apt-get install libc6 libcurl3 libgcc1 libgssapi-krb5-2 libicu-dev liblttng-ust0 libssl-dev libstdc++6 libunwind8 libuuid1 zlib1g

I also added an FTP server and ran vncserver, so I’d have a few ways to talk to the Raspberry Pi. Yes, I could also SSH in but I have a spare monitor, and with that monitor plus VNC I didn’t see a need.

sudo apt-get pure-ftpd
vncserver

Then I fire up Filezilla – my preferred FTP client – and FTP the publish output folder from my dotnet publish above. I put the files in a folder off my ~\Desktop.

FTPing files

Then from a terminal I

[email protected]:~/Desktop/helloworld $ chmod +x raspberrypi

(or whatever the name of your published “exe” is. It’ll be the name of your source folder/project with no extension. As this is a self-contained published app, again, all the .NET Core runtime stuff is in the same folder with the app.

[email protected]:~/Desktop/helloworld $ ./raspberrypi 
Segmentation fault

The crash was instant…not a pause and a crash, but it showed up as soon as I pressed enter. Shoot.

I ran “strace ./raspberrypi” and got this output. I figured maybe I missed one of the prerequisite libraries, and I just needed to see which one and apt-get it. I can see the ld.so.nohwcap error, but that’s a historical Debian-ism and more of a warning than a fatal.

strace on a bad exe in Linux

I used to be able to read straces 20 years ago but much like my Spanish, my skills are only good at Chipotle. I can see it just getting started loading libraries, seeking around in them, checking file status,  mapping files to memory, setting memory protection, then it all falls apart. Perhaps we tried to do something inappropriate with some memory that just got protected? We are dereferencing a null pointer.

Maybe you can read this and you already know what is going to happen! I did not.

I run it under gdb:

[email protected]:~/Desktop/WTFISTHISCRAP $ gdb ./raspberrypi 
GNU gdb (Raspbian 7.7.1+dfsg-5+rpi1) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
This GDB was configured as "arm-linux-gnueabihf".
"/home/pi/Desktop/helloworldWRONG/./raspberrypi1": not in executable format: File truncated
(gdb)

Ok, sick files?

I called Peter Marcu from the .NET team and we chatted about how he got it working and compared notes.

I was using a Raspberry Pi 2, he a Pi 3. Ok, I’ll try a 3. 30 minutes later, new SD card, new burn, new boot, pre-reqs, build, FTP, run, SAME RESULT – segfault.

Weird.

Maybe corruption? Here’s a thread about Corrupted Files on Raspbian Jesse 2017-07-05! That’s the version I have. OK, I’ll try the build of Raspbian from a week before.

30 minutes later, burn another SD card, new boot, pre-reqs, build, FTP, run, SAME RESULT – segfault.

BUT IT WORKS ON PETER’S MACHINE.

Weird.

Maybe a bad nuget.config? No.

Bad daily .NET build? No.

BUT IT WORKS ON PETER’S MACHINE.

Ok, I’ll try Ubuntu Mate for Raspberry Pi. TOTALLY different OS.

30 minutes later, burn another SD card, new boot, pre-reqs, build, FTP, run, SAME RESULT – segfault.

What’s the common thread here? Ok, I’ll try from another Windows machine.

SAME RESULT – segfault.

I call Peter back and we figure it’s gotta be prereqs…but the strace doesn’t show we’re even trying to load any interesting libraries. We fail FAST.

Ok, let’s get serious.

We both have Raspberry Pi 3s. Check.

What kind of SD card does he have? Sandisk? Ok,  I’ll use Sandisk. But disk corruption makes no sense at that level…because the OS booted!

What did he burn with? He used Win32diskimager and I used Etcher. Fine, I’ll bite.

30 minutes later, burn another SD card, new boot, pre-reqs, build, FTP, run, SAME RESULT – segfault.

He sends me HIS build of a HelloWorld and I FTP it over to the Pi. SAME RESULT – segfault.

Peter is freaking out. I’m deeply unhappy and considering quitting my job. My kids are going to sleep because it’s late.

I ask him what he’s FTPing with, and he says WinSCP. I use FileZilla, ok, I’ll try WinSCP.

WinSCP’s New Session dialog starts here:

SFTP is Default

I say, WAIT. Are you using SFTP or FTP? Peter says he’s using SFTP so I turn on SSH on the Raspberry Pi and SFTP into it with WinSCP and copy over my Hello World.

IT FREAKING WORKS. IMMEDIATELY.

Hello World on a Raspberry Pi

BUT WHY.

I make a folder called Good and a folder called BAD. I copy with FileZilla to BAD and with WinSCP to GOOD. Then I run a compare. Maybe some part of .NET Core got corrupted? Maybe a supporting native library?

[email protected]:~/Desktop $ diff --brief -r helloworld/ helloworldWRONG/
Files helloworld/raspberrypi1 and helloworldWRONG/raspberrypi1 differ

Wait, WHAT? The executable are different? One is 67,684 bytes and the bad one is 69,632 bytes.

Time for a  visual compare.

All the ODs are gone

At this point I saw it IMMEDIATELY.

0D is CR (13) and 0A is LF (10). I know this because I’m old and I’ve written printer drivers for printers that had both carriages and lines to feed. Why do YOU know this? Likely because you’ve transferred files between Unix and Windows once or thrice, perhaps with FTP or Git.

All the CRs are gone. From my binary file.

Why?

I went straight to settings in FileZilla:

Treat files without extensions as ASCII files

See it?

Treat files without extensions as ASCII files

That’s the default in FileZilla. To change files that are just chilling, minding their own business, as ASCII, and then just randomly strip out carriage returns. What could go wrong? And it doesn’t even look for CR LF pairs! No, it just looks for CRs and strips them. Classy.

In retrospect I should have used known this, but it wasn’t even the switch to SFTP, it was the switch to an FTP program with different defaults.

This bug/issue whatever burned my whole Monday. But, it’ll never burn another Monday, Dear Reader, because I’ve seen it before now.

FAIL FAST FAIL OFTEN my friends!

Why does experience matter? It means I’ve failed a lot in the past and it’s super useful if I remember those bugs because then next time this happens it’ll only burn a few minutes rather than a day.

Go forth and fail a lot, my loves.

Oh, and FTP sucks.


Sponsor: Thanks to Redgate! A third of teams don’t version control their database. Connect your database to your version control system with SQL Source Control and find out who made changes, what they did, and why. Learn more


© 2017 Scott Hanselman. All rights reserved.
     

Debugging is a satisfying and special kind of hell. You really have to live it to understand it. When you're deep into it you never know when it'll be done. When you do finally escape it's almost always a DOH! moment.

I spent an entire day debugging an issue and the solution ended up being a checkbox.

NOTE: If you get a third of the way through this blog post and already figured it out, well, poop on you. Where were you after lunch WHEN I NEEDED YOU?

I wanted to use a Raspberry Pi in a tech talk I'm doing tomorrow at a conference. I was going to show .NET Core 2.0 and ASP.NET running on a Raspberry Pi so I figured I'd start with Hello World. How hard could it be?

You'll write and build a .NET app on Windows or Mac, then publish it to the Raspberry Pi. I'm using a preview build of the .NET Core 2.0 command line and SDK (CLI) I got from here.

C:\raspberrypi> dotnet new console

C:\raspberrypi> dotnet run
Hello World!
C:\raspberrypi> dotnet publish -r linux-arm
Microsoft Build Engine version for .NET Core

raspberrypi1 -> C:\raspberrypi\bin\Debug\netcoreapp2.0\linux-arm\raspberrypi.dll
raspberrypi1 -> C:\raspberrypi\bin\Debug\netcoreapp2.0\linux-arm\publish\

Notice the simplified publish. You'll get a folder for linux-arm in this example, but could also publish osx-x64, etc. You'll want to take the files from the publish folder (not the folder above it) and move them to the Raspberry Pi. This is a self-contained application that targets ARM on Linux so after the prerequisites that's all you need.

I grabbed a mini-SD card, headed over to https://www.raspberrypi.org/downloads/ and downloaded the latest Raspbian image. I used etcher.io - a lovely image burner for Windows, Mac, or Linux - and wrote the image to the SD Card. I booted up and got ready to install some prereqs. I'm only 15 min in at this point. Setting up a Raspberry Pi 2 or Raspberry Pi 3 is VERY smooth these days.

Here's the prereqs for .NET Core 2 on Ubuntu or Debian/Raspbian. Install them from the terminal, natch.

sudo apt-get install libc6 libcurl3 libgcc1 libgssapi-krb5-2 libicu-dev liblttng-ust0 libssl-dev libstdc++6 libunwind8 libuuid1 zlib1g

I also added an FTP server and ran vncserver, so I'd have a few ways to talk to the Raspberry Pi. Yes, I could also SSH in but I have a spare monitor, and with that monitor plus VNC I didn't see a need.

sudo apt-get pure-ftpd

vncserver

Then I fire up Filezilla - my preferred FTP client - and FTP the publish output folder from my dotnet publish above. I put the files in a folder off my ~\Desktop.

FTPing files

Then from a terminal I

[email protected]:~/Desktop/helloworld $ chmod +x raspberrypi

(or whatever the name of your published "exe" is. It'll be the name of your source folder/project with no extension. As this is a self-contained published app, again, all the .NET Core runtime stuff is in the same folder with the app.

[email protected]:~/Desktop/helloworld $ ./raspberrypi 

Segmentation fault

The crash was instant...not a pause and a crash, but it showed up as soon as I pressed enter. Shoot.

I ran "strace ./raspberrypi" and got this output. I figured maybe I missed one of the prerequisite libraries, and I just needed to see which one and apt-get it. I can see the ld.so.nohwcap error, but that's a historical Debian-ism and more of a warning than a fatal.

strace on a bad exe in Linux

I used to be able to read straces 20 years ago but much like my Spanish, my skills are only good at Chipotle. I can see it just getting started loading libraries, seeking around in them, checking file status,  mapping files to memory, setting memory protection, then it all falls apart. Perhaps we tried to do something inappropriate with some memory that just got protected? We are dereferencing a null pointer.

Maybe you can read this and you already know what is going to happen! I did not.

I run it under gdb:

[email protected]:~/Desktop/WTFISTHISCRAP $ gdb ./raspberrypi 

GNU gdb (Raspbian 7.7.1+dfsg-5+rpi1) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
This GDB was configured as "arm-linux-gnueabihf".
"/home/pi/Desktop/helloworldWRONG/./raspberrypi1": not in executable format: File truncated
(gdb)

Ok, sick files?

I called Peter Marcu from the .NET team and we chatted about how he got it working and compared notes.

I was using a Raspberry Pi 2, he a Pi 3. Ok, I'll try a 3. 30 minutes later, new SD card, new burn, new boot, pre-reqs, build, FTP, run, SAME RESULT - segfault.

Weird.

Maybe corruption? Here's a thread about Corrupted Files on Raspbian Jesse 2017-07-05! That's the version I have. OK, I'll try the build of Raspbian from a week before.

30 minutes later, burn another SD card, new boot, pre-reqs, build, FTP, run, SAME RESULT - segfault.

BUT IT WORKS ON PETER'S MACHINE.

Weird.

Maybe a bad nuget.config? No.

Bad daily .NET build? No.

BUT IT WORKS ON PETER'S MACHINE.

Ok, I'll try Ubuntu Mate for Raspberry Pi. TOTALLY different OS.

30 minutes later, burn another SD card, new boot, pre-reqs, build, FTP, run, SAME RESULT - segfault.

What's the common thread here? Ok, I'll try from another Windows machine.

SAME RESULT - segfault.

I call Peter back and we figure it's gotta be prereqs...but the strace doesn't show we're even trying to load any interesting libraries. We fail FAST.

Ok, let's get serious.

We both have Raspberry Pi 3s. Check.

What kind of SD card does he have? Sandisk? Ok,  I'll use Sandisk. But disk corruption makes no sense at that level...because the OS booted!

What did he burn with? He used Win32diskimager and I used Etcher. Fine, I'll bite.

30 minutes later, burn another SD card, new boot, pre-reqs, build, FTP, run, SAME RESULT - segfault.

He sends me HIS build of a HelloWorld and I FTP it over to the Pi. SAME RESULT - segfault.

Peter is freaking out. I'm deeply unhappy and considering quitting my job. My kids are going to sleep because it's late.

I ask him what he's FTPing with, and he says WinSCP. I use FileZilla, ok, I'll try WinSCP.

WinSCP's New Session dialog starts here:

SFTP is Default

I say, WAIT. Are you using SFTP or FTP? Peter says he's using SFTP so I turn on SSH on the Raspberry Pi and SFTP into it with WinSCP and copy over my Hello World.

IT FREAKING WORKS. IMMEDIATELY.

Hello World on a Raspberry Pi

BUT WHY.

I make a folder called Good and a folder called BAD. I copy with FileZilla to BAD and with WinSCP to GOOD. Then I run a compare. Maybe some part of .NET Core got corrupted? Maybe a supporting native library?

[email protected]:~/Desktop $ diff --brief -r helloworld/ helloworldWRONG/

Files helloworld/raspberrypi1 and helloworldWRONG/raspberrypi1 differ

Wait, WHAT? The executable are different? One is 67,684 bytes and the bad one is 69,632 bytes.

Time for a  visual compare.

All the ODs are gone

At this point I saw it IMMEDIATELY.

0D is CR (13) and 0A is LF (10). I know this because I'm old and I've written printer drivers for printers that had both carriages and lines to feed. Why do YOU know this? Likely because you've transferred files between Unix and Windows once or thrice, perhaps with FTP or Git.

All the CRs are gone. From my binary file.

Why?

I went straight to settings in FileZilla:

Treat files without extensions as ASCII files

See it?

Treat files without extensions as ASCII files

That's the default in FileZilla. To change files that are just chilling, minding their own business, as ASCII, and then just randomly strip out carriage returns. What could go wrong? And it doesn't even look for CR LF pairs! No, it just looks for CRs and strips them. Classy.

In retrospect I should have used known this, but it wasn't even the switch to SFTP, it was the switch to an FTP program with different defaults.

This bug/issue whatever burned my whole Monday. But, it'll never burn another Monday, Dear Reader, because I've seen it before now.

FAIL FAST FAIL OFTEN my friends!

Why does experience matter? It means I've failed a lot in the past and it's super useful if I remember those bugs because then next time this happens it'll only burn a few minutes rather than a day.

Go forth and fail a lot, my loves.

Oh, and FTP sucks.


Sponsor: Thanks to Redgate! A third of teams don’t version control their database. Connect your database to your version control system with SQL Source Control and find out who made changes, what they did, and why. Learn more



© 2017 Scott Hanselman. All rights reserved.
     

Porting a 15 year old .NET 1.1 Virtual CPU Tiny Operating System school project to .NET Core 2.0

The 2002 TinyOS in C# is now on .NET Core in 2017 running on UbuntuI’ve had a number of great guests on the podcast lately. One topic that has come up a number of times is the “toy project.” I’ve usually kept mine private – never putting them on GitHub – Somewhat concerned that people would judge me and my code. However, hypocrite that am (aren’t we all?) I have advocated that others put their “Garage Sale Code” online. So here’s some crappy code. 😉

The Preamble

While I’ve been working as an engineer for 25 years this year, I didn’t graduate from school with a 4 year degree until 2003 – I just needed to get it done, for myself. I was poking around recently and found my project from OIT’s CST352 “Operating Systems” class. One of the projects was to create a “Virtual CPU and OS.” This is kind of a thought exercise. It’s not really a parser/lexer – although there is both – and it’s not a real OS. But it needs to be able to take in a made-up quasi-Assembly Language instruction set and execute them on a virtual CPU while managing virtual memory of arbitrary size. Again, a thought exercise made real to confirm that the student understands the responsibilities of a CPU.

Here’s an example “application.” Confused yet? Here’s the original spec I was given in 2002 that includes the 36 instructions the “CPU” should understand. It has 10 general-purpose 32bit registers address as 1 through 10. Register 10 is the stack pointer. There are two bit flag registers – sign flag and zero flag.

Instructions are “opcode arg1 arg2” with constants prefixed with “$.”

11 r8        ;Print r8
6 r1 $10 ;Move 10 into r1
6 r2 $6 ;Move 6 into r2
6 r3 $25 ;Move 25 into r3
23 r1 ;Acquire lock in r1 (currently 10)
11 r3 ;Print r3 (currently 25)
24 r1 ;Release r4 (currently 10)
25 r3 ;Sleep r3 (currently 25)
11 r3 ;Print r3 (currently 25)
27 ;Exit

I write my homework assignment in 2002 in the idiomatic C# of the time on .NET 1.1. That means no Generics<T> – I had to make my own strongly typed collections. That means C# has dozens of (if not a hundred) language and syntax improvements. I didn’t use a Unit Testing Framework as TDD was just starting around 1999 during the XP (eXtreme Programming) days and NUnit was just getting start. It also uses “unsafe” to pin down memory in a few places. I’m sure there are WAY WAY WAY better and more sophisticated ways to do this today in idiomatic C# of 2017. Those are excuses, the real reasons are my own ignorance, ability, combined with some night-school laziness.

One of the more fun parts of this exercise was moving from physical memory (a byte array as I recall) to a full-on Memory Manager where each Process thought it could address a whole bunch of Virtual Memory while actual Physical Memory was arbitrarily sized. Then – as a joke – I would swap out memory pages as XML! 😉 Yes, to be clear, it was a joke and I still love it.

You can run an “app” by passing in the total physical memory along with the text file containing the program, but you can also run an arbitrary number of programs by passing in an arbitrary number  of text files! The “TinyOS” will handle each process thinking it has its own memory and will time

If you are more of a visual learner, perhaps you’d prefer this 20-slide PowerPoint on this Tiny CPU that I presented in Malaysia later that year. You dig those early 2000-era slides? I KNOW YOU DO.

Tiny OS Memory SlidesTiny OS Memory SlidesTiny OS Memory Slides 

Updating a .NET 1.1 app to cross-platform .NET Core 2.0

Step 1 was to download the original code from my own blog. 😉 This is also Reason #4134 why you should have a blog.

I decided to use Visual Studio 2017 to upgrade it, and even worse I decided to use .NET Core 2.0 which is currently in Preview. I wanted to use .NET Core 2.0 not just because it’s cross-platform but also because it promises to have a pretty large API surface area and I want this to “just work.” The part about getting my old application running on Linux is going to be awesome, though.

Visual Studio then pops a scary dialog about upgrading files. NOTE that another totally valid way to do this (that I will end up doing later in this blog post) is to just make a new project and move the source files into it. Natch.

image

Visual Studio says it’s targeting .NET 2.0 Full Framework, but I ratchet it up to 4.6 to see what happens. It builds but with a bunch of errors about Obsolete methods, the most interesting one being this one:

Warning CS0618    
'ConfigurationSettings.AppSettings' is obsolete:
'This method is obsolete, it has been replaced by
System.Configuration!System.Configuration.ConfigurationManager.AppSettings'
C:\Users\scott\Downloads\TinyOSOLDOLD\OS Project\CPU.cs 72

That’s telling me that my .NET 1/2 API will work but has been replaced in .NET 4.x, but I’m more interested in .NET Core 2.0. I could make my EXE a LIB and target .NET Standard 2.0 or I could make a .NET Core 2.0 app and perhaps get a few more APIs. I didn’t do a formal analysis with the .NET Portability Analyzer but I will add that to the list of Things To Do. I may be able to make a library that works on an iPhone – a product that didn’t exist when I started this assignment. That would be Just Cool(tm).

I decided to just make a new empty .NET Core 2.0 app and copy the source .cs files into it. A few interesting things.

  • My app also used “unsafe” code (it pins memory down and accesses it directly).
  • It has extensive inline documentation in comments that I used to use NDoc to make a CHM Help file. I’d like that doc to turn into HTML at some point.
  • It also has an appsettings.json file that needs to get copied to the output folder when it compiles.
  • While I could publish it to a self-contained .NET Core exe, for now I’m running it like this in my test batch files – example:
    • dotnet netcoreapp2.0/TinyOSCore.dll 512 scott13.txt

Here’s the resulting csproj file.

<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>

<PropertyGroup>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>

<ItemGroup>
<None Remove="appsettings.json" />
</ItemGroup>

<ItemGroup>
<Content Include="appsettings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>

<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Configuration" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0-preview2-final" />
</ItemGroup>

</Project>

Other than the obsolete configuration warning and a few malformed XML comments, the app compiled and ran! You can actually “watch” the nightmare process here https://github.com/shanselman/TinyOS/commits/Core2Port in the form of GitHub commits. I also moved the docs from a 2002 Word Doc to Markdown so be sure to explore the fairly extensive spec https://github.com/shanselman/TinyOS.

The only significant change was loading the config. Configuration is even more different on .NET Core 2.0 than Full Framework. It’s FAR more, ahem, configurable. I could have used “Options,” I could have written my own config provider if it was important to keep the file format.

This little TinyOS has a bunch of config options that come in from a .exe.config file in XML like this (truncated):

<configuration>
<appSettings>
<!--
Must be a factor of 4
This is the total Physical Memory in bytes that the CPU can address.
This should not be confused with the amount of total or addressable memory
that is passed in on the command line.
-->
<add key="PhysicalMemory" value="128" />
<!--
Must be a factor of 4
This is the ammount of memory in bytes each process is allocated
Therefore, if this is 256 and you want to load 4 processes into the OS,
you'll need to pass a number > 1024 as the total ammount of addressable memory
on the command line.
-->
<add key="ProcessMemory" value="384" />
<add key="DumpPhysicalMemory" value="true" />
<add key="DumpInstruction" value="true" />
<add key="DumpRegisters" value="true" />
<add key="DumpProgram" value="true" />
<add key="DumpContextSwitch" value="true" />
<add key="PauseOnExit" value="false" />

I have a few choices. I could make a Configuration Provider and reach .NET Core to read this format (there’s an XML adapter, in fact) or make the code porting easier by moving these “name/value” pairs to a JSON file like this:

{
"PhysicalMemory": "128",
"ProcessMemory": "384",
"DumpPhysicalMemory": "true",
"DumpInstruction": "true",
"DumpRegisters": "true",
"DumpProgram": "true",
"DumpContextSwitch": "true",
"PauseOnExit": "false",
"SharedMemoryRegionSize": "16",
"NumOfSharedMemoryRegions": "4",
"MemoryPageSize": "16",
"StackSize": "16",
"DataSize": "16"
}

This was just a few minutes of search and replace to change the XML to JSON. I could have also written a little app or shell script. By changing the config (rather than writing an adapter) I could then keep the code 99% the same.

My code was doing things like this (all over…there was no DI container yet):

bytesOfPhysicalMemory = uint.Parse(ConfigurationSettings.AppSettings["PhysicalMemory"]);

And I’d like to avoid major refactoring – yet. I added this bit of .NET Core configuration at the top of the EntryPoint and saved away an IConfigurationHost:

var builder = new ConfigurationBuilder()
.AddJsonFile("appsettings.json");
Configuration = builder.Build();

I’ve got a Dictionary in the format of the IConfiguration host called “Configuration.” So now I just do this in a dozen places and the app compiles again:

bytesOfPhysicalMemory = uint.Parse(Configuration["PhysicalMemory"]);

This brings up that feeling we all have when we look at old code – especially our own old code. I should have abstracted that away! Why didn’t I use an interface? Why so many statics? What was I thinking?

We can beat ourselves up or we can feel good about ourselves and remember this. The app worked. It still works. There is value in it. I learned a lot. I’m a better programmer now. I don’t know how far I’ll take this old code but I had a lovely afternoon porting it to .NET Core 2.0 and I may refactor the heck out if it or I may not.

TinyOS on Ubuntu

For now I did update the smoke tests to run on both Windows and Linux and I’m happy with the experiment.

Related Links

Have YOU done a project like this, either in school or on your own?


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test, build and debug ASP.NET, .NET Framework, .NET Core, or Unity applications. Learn more and get access to early builds!


© 2017 Scott Hanselman. All rights reserved.
     

The 2002 TinyOS in C# is now on .NET Core in 2017 running on UbuntuI've had a number of great guests on the podcast lately. One topic that has come up a number of times is the "toy project." I've usually kept mine private - never putting them on GitHub - Somewhat concerned that people would judge me and my code. However, hypocrite that am (aren't we all?) I have advocated that others put their "Garage Sale Code" online. So here's some crappy code. ;)

The Preamble

While I've been working as an engineer for 25 years this year, I didn't graduate from school with a 4 year degree until 2003 - I just needed to get it done, for myself. I was poking around recently and found my project from OIT's CST352 "Operating Systems" class. One of the projects was to create a "Virtual CPU and OS." This is kind of a thought exercise. It's not really a parser/lexer - although there is both - and it's not a real OS. But it needs to be able to take in a made-up quasi-Assembly Language instruction set and execute them on a virtual CPU while managing virtual memory of arbitrary size. Again, a thought exercise made real to confirm that the student understands the responsibilities of a CPU.

Here's an example "application." Confused yet? Here's the original spec I was given in 2002 that includes the 36 instructions the "CPU" should understand. It has 10 general-purpose 32bit registers address as 1 through 10. Register 10 is the stack pointer. There are two bit flag registers - sign flag and zero flag.

Instructions are "opcode arg1 arg2" with constants prefixed with "$."

11 r8        ;Print r8

6 r1 $10 ;Move 10 into r1
6 r2 $6 ;Move 6 into r2
6 r3 $25 ;Move 25 into r3
23 r1 ;Acquire lock in r1 (currently 10)
11 r3 ;Print r3 (currently 25)
24 r1 ;Release r4 (currently 10)
25 r3 ;Sleep r3 (currently 25)
11 r3 ;Print r3 (currently 25)
27 ;Exit

I write my homework assignment in 2002 in the idiomatic C# of the time on .NET 1.1. That means no Generics<T> - I had to make my own strongly typed collections. That means C# has dozens of (if not a hundred) language and syntax improvements. I didn't use a Unit Testing Framework as TDD was just starting around 1999 during the XP (eXtreme Programming) days and NUnit was just getting start. It also uses "unsafe" to pin down memory in a few places. I'm sure there are WAY WAY WAY better and more sophisticated ways to do this today in idiomatic C# of 2017. Those are excuses, the real reasons are my own ignorance, ability, combined with some night-school laziness.

One of the more fun parts of this exercise was moving from physical memory (a byte array as I recall) to a full-on Memory Manager where each Process thought it could address a whole bunch of Virtual Memory while actual Physical Memory was arbitrarily sized. Then - as a joke - I would swap out memory pages as XML! ;) Yes, to be clear, it was a joke and I still love it.

You can run an "app" by passing in the total physical memory along with the text file containing the program, but you can also run an arbitrary number of programs by passing in an arbitrary number  of text files! The "TinyOS" will handle each process thinking it has its own memory and will time

If you are more of a visual learner, perhaps you'd prefer this 20-slide PowerPoint on this Tiny CPU that I presented in Malaysia later that year. You dig those early 2000-era slides? I KNOW YOU DO.

Tiny OS Memory SlidesTiny OS Memory SlidesTiny OS Memory Slides 

Updating a .NET 1.1 app to cross-platform .NET Core 2.0

Step 1 was to download the original code from my own blog. ;) This is also Reason #4134 why you should have a blog.

I decided to use Visual Studio 2017 to upgrade it, and even worse I decided to use .NET Core 2.0 which is currently in Preview. I wanted to use .NET Core 2.0 not just because it's cross-platform but also because it promises to have a pretty large API surface area and I want this to "just work." The part about getting my old application running on Linux is going to be awesome, though.

Visual Studio then pops a scary dialog about upgrading files. NOTE that another totally valid way to do this (that I will end up doing later in this blog post) is to just make a new project and move the source files into it. Natch.

image

Visual Studio says it's targeting .NET 2.0 Full Framework, but I ratchet it up to 4.6 to see what happens. It builds but with a bunch of errors about Obsolete methods, the most interesting one being this one:

Warning CS0618    

'ConfigurationSettings.AppSettings' is obsolete:
'This method is obsolete, it has been replaced by
System.Configuration!System.Configuration.ConfigurationManager.AppSettings'
C:\Users\scott\Downloads\TinyOSOLDOLD\OS Project\CPU.cs 72

That's telling me that my .NET 1/2 API will work but has been replaced in .NET 4.x, but I'm more interested in .NET Core 2.0. I could make my EXE a LIB and target .NET Standard 2.0 or I could make a .NET Core 2.0 app and perhaps get a few more APIs. I didn't do a formal analysis with the .NET Portability Analyzer but I will add that to the list of Things To Do. I may be able to make a library that works on an iPhone - a product that didn't exist when I started this assignment. That would be Just Cool(tm).

I decided to just make a new empty .NET Core 2.0 app and copy the source .cs files into it. A few interesting things.

  • My app also used "unsafe" code (it pins memory down and accesses it directly).
  • It has extensive inline documentation in comments that I used to use NDoc to make a CHM Help file. I'd like that doc to turn into HTML at some point.
  • It also has an appsettings.json file that needs to get copied to the output folder when it compiles.
  • While I could publish it to a self-contained .NET Core exe, for now I'm running it like this in my test batch files - example:
    • dotnet netcoreapp2.0/TinyOSCore.dll 512 scott13.txt

Here's the resulting csproj file.

<Project Sdk="Microsoft.NET.Sdk">


<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>

<PropertyGroup>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>

<ItemGroup>
<None Remove="appsettings.json" />
</ItemGroup>

<ItemGroup>
<Content Include="appsettings.json">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>

<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Configuration" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="2.0.0-preview2-final" />
<PackageReference Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="2.0.0-preview2-final" />
</ItemGroup>

</Project>

Other than the obsolete configuration warning and a few malformed XML comments, the app compiled and ran! You can actually "watch" the nightmare process here https://github.com/shanselman/TinyOS/commits/Core2Port in the form of GitHub commits. I also moved the docs from a 2002 Word Doc to Markdown so be sure to explore the fairly extensive spec https://github.com/shanselman/TinyOS.

The only significant change was loading the config. Configuration is even more different on .NET Core 2.0 than Full Framework. It's FAR more, ahem, configurable. I could have used "Options," I could have written my own config provider if it was important to keep the file format.

This little TinyOS has a bunch of config options that come in from a .exe.config file in XML like this (truncated):

<configuration>

<appSettings>
<!--
Must be a factor of 4
This is the total Physical Memory in bytes that the CPU can address.
This should not be confused with the amount of total or addressable memory
that is passed in on the command line.
-->
<add key="PhysicalMemory" value="128" />
<!--
Must be a factor of 4
This is the ammount of memory in bytes each process is allocated
Therefore, if this is 256 and you want to load 4 processes into the OS,
you'll need to pass a number > 1024 as the total ammount of addressable memory
on the command line.
-->
<add key="ProcessMemory" value="384" />
<add key="DumpPhysicalMemory" value="true" />
<add key="DumpInstruction" value="true" />
<add key="DumpRegisters" value="true" />
<add key="DumpProgram" value="true" />
<add key="DumpContextSwitch" value="true" />
<add key="PauseOnExit" value="false" />

I have a few choices. I could make a Configuration Provider and reach .NET Core to read this format (there's an XML adapter, in fact) or make the code porting easier by moving these "name/value" pairs to a JSON file like this:

{

"PhysicalMemory": "128",
"ProcessMemory": "384",
"DumpPhysicalMemory": "true",
"DumpInstruction": "true",
"DumpRegisters": "true",
"DumpProgram": "true",
"DumpContextSwitch": "true",
"PauseOnExit": "false",
"SharedMemoryRegionSize": "16",
"NumOfSharedMemoryRegions": "4",
"MemoryPageSize": "16",
"StackSize": "16",
"DataSize": "16"
}

This was just a few minutes of search and replace to change the XML to JSON. I could have also written a little app or shell script. By changing the config (rather than writing an adapter) I could then keep the code 99% the same.

My code was doing things like this (all over...there was no DI container yet):

bytesOfPhysicalMemory = uint.Parse(ConfigurationSettings.AppSettings["PhysicalMemory"]);

And I'd like to avoid major refactoring - yet. I added this bit of .NET Core configuration at the top of the EntryPoint and saved away an IConfigurationHost:

var builder = new ConfigurationBuilder()

.AddJsonFile("appsettings.json");
Configuration = builder.Build();

I've got a Dictionary in the format of the IConfiguration host called "Configuration." So now I just do this in a dozen places and the app compiles again:

bytesOfPhysicalMemory = uint.Parse(Configuration["PhysicalMemory"]);

This brings up that feeling we all have when we look at old code - especially our own old code. I should have abstracted that away! Why didn't I use an interface? Why so many statics? What was I thinking?

We can beat ourselves up or we can feel good about ourselves and remember this. The app worked. It still works. There is value in it. I learned a lot. I'm a better programmer now. I don't know how far I'll take this old code but I had a lovely afternoon porting it to .NET Core 2.0 and I may refactor the heck out if it or I may not.

TinyOS on Ubuntu

For now I did update the smoke tests to run on both Windows and Linux and I'm happy with the experiment.

Related Links

Have YOU done a project like this, either in school or on your own?


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test, build and debug ASP.NET, .NET Framework, .NET Core, or Unity applications. Learn more and get access to early builds!


© 2017 Scott Hanselman. All rights reserved.
     

Speed of dotnet run vs the speed of dotnet for published apps (plus self-contained .NET Core apps)

The .NET Core team really prides themselves on performance. However, it’s not immediately obvious (as with all systems) if you just do Hello World as a develop. Just today I was doing a Ruby on Rails app in Development Mode with mruby – but that’s not what you’d go to production with.

Let’s look at a great question I got today on Twitter.

@shanselman @davidfowl Is it normal for dotnet run on the latest .net core 2.0 preview 2 bits to take 4 seconds to run? pic.twitter.com/wvD2aqtfi0

— Jerome Terry (@jeromeleoterry) June 28, 2017

Dotnet Run – Builds and Runs Source Code in Development

That’s a great question. If you install .NET Core 2.0 Preview – this person is on a Mac, but you can use Linux or Windows as well – then do just this:

$ dotnet new console
$ dotnet run

It’ll be about 3-4 seconds. dotnet is the SDK and dotnet run will build and run your source code. Here’s a short bit from the docs:

The dotnet run command provides a convenient option to run your application from the source code with one command. It’s useful for fast iterative development from the command line. The command depends on the dotnet build command to build the code. Any requirements for the build, such as that the project must be restored first, apply to dotnet run as well.

While this is super convenient, it’s not totally obvious that dotnet run isn’t something you’d go to production with (especially Hello World Production, which is quite demanding! 😉 ).

Dotnet Publish then Dotnet YOUR.DLL for Production

Instead, do a dotnet publish, note the compiled DLL created, then run “dotnet tst.dll.”

For example:

C:\Users\scott\Desktop\tst> dotnet publish
Microsoft (R) Build Engine version 15.3 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\tst.dll
tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\publish\
C:\Users\scott\Desktop\tst> dotnet run .\bin\Debug\netcoreapp2.0\tst.dll
Hello World!

On my machine, dotnet run is 2.7s, but dotnet tst.dll is 0.04s.

.NET Core is fast

Dotnet publish –self-contained

I could then publish a complete self-contained app – I’m using Windows, so I’ll publish for Windows but you could even build on a Windows machine but target a Mac runtime, etc and that will make a \publish folder.

C:\Users\scott\Desktop\tst> dotnet publish  --self-contained -r win10-x64
Microsoft (R) Build Engine version 15.3 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\win10-x64\tst.dll
tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\win10-x64\publish\
C:\Users\scott\Desktop\tst> .\bin\Debug\netcoreapp2.0\win10-x64\publish\tst.exe
Hello World!

Note in this case I have a “Self-Contained” app, so all of .NET Core is in that folder and below. Here I run tst.exe, not dotnet.exe because now I’m an end-user.

The results of a published .NET Core App

I hope this helps clear things up.


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test, build and debug ASP.NET, .NET Framework, .NET Core, or Unity applications. Learn more and get access to early builds!


© 2017 Scott Hanselman. All rights reserved.
     

The .NET Core team really prides themselves on performance. However, it's not immediately obvious (as with all systems) if you just do Hello World as a develop. Just today I was doing a Ruby on Rails app in Development Mode with mruby - but that's not what you'd go to production with.

Let's look at a great question I got today on Twitter.

Dotnet Run - Builds and Runs Source Code in Development

That's a great question. If you install .NET Core 2.0 Preview - this person is on a Mac, but you can use Linux or Windows as well - then do just this:

$ dotnet new console

$ dotnet run

It'll be about 3-4 seconds. dotnet is the SDK and dotnet run will build and run your source code. Here's a short bit from the docs:

The dotnet run command provides a convenient option to run your application from the source code with one command. It's useful for fast iterative development from the command line. The command depends on the dotnet build command to build the code. Any requirements for the build, such as that the project must be restored first, apply to dotnet run as well.

While this is super convenient, it's not totally obvious that dotnet run isn't something you'd go to production with (especially Hello World Production, which is quite demanding! ;) ).

Dotnet Publish then Dotnet YOUR.DLL for Production

Instead, do a dotnet publish, note the compiled DLL created, then run "dotnet tst.dll."

For example:

C:\Users\scott\Desktop\tst> dotnet publish

Microsoft (R) Build Engine version 15.3 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\tst.dll
tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\publish\
C:\Users\scott\Desktop\tst> dotnet run .\bin\Debug\netcoreapp2.0\tst.dll
Hello World!

On my machine, dotnet run is 2.7s, but dotnet tst.dll is 0.04s.

.NET Core is fast

Dotnet publish --self-contained

I could then publish a complete self-contained app - I'm using Windows, so I'll publish for Windows but you could even build on a Windows machine but target a Mac runtime, etc and that will make a \publish folder.

C:\Users\scott\Desktop\tst> dotnet publish  --self-contained -r win10-x64

Microsoft (R) Build Engine version 15.3 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\win10-x64\tst.dll
tst -> C:\Users\scott\Desktop\tst\bin\Debug\netcoreapp2.0\win10-x64\publish\
C:\Users\scott\Desktop\tst> .\bin\Debug\netcoreapp2.0\win10-x64\publish\tst.exe
Hello World!

Note in this case I have a "Self-Contained" app, so all of .NET Core is in that folder and below. Here I run tst.exe, not dotnet.exe because now I'm an end-user.

The results of a published .NET Core App

I hope this helps clear things up.


Sponsor: Check out JetBrains Rider: a new cross-platform .NET IDE. Edit, refactor, test, build and debug ASP.NET, .NET Framework, .NET Core, or Unity applications. Learn more and get access to early builds!



© 2017 Scott Hanselman. All rights reserved.
     

Exploring CQRS within the Brighter .NET open source project

The logo for the "Brighter" Open Source project is a little cannon. Fire and Forget?There’s a ton of cool new .NET Core open source projects lately, and I’ve very much enjoyed exploring this rapidly growing space. Today at lunch I was checking out a project called “Brighter.” It’s actually been around in the .NET space for many years and is in the process of moving to .NET Core for greater portability and performance.

Brighter is a “.NET Command Dispatcher, with Command Processor features for QoS (like Timeout, Retry, and Circuit Breaker), and support for Task Queues”

Whoa, that’s a lot of cool and fancy words. What’s it mean? The Brighter project is up on GitHub incudes a bunch of libraries and examples that you can pull in to support CQRS architectural styles in .NET. CQRS stands for Command Query Responsibility Segregation. As Martin Fowler says, “At its heart is the notion that you can use a different model to update information than the model you use to read information.” The Query Model reads and the Command Model updates/validates. Greg Young gives the first example of CQRS here. If you are a visual learner, there’s a video from late 2015 where Ian Cooper explains a lot of this a the London .NET User Group or an interview with Ian Cooper on Channel 9.

Brighter also supports “Distributed Task Queues” which you can use to improve performance when you’re using a query or integrating with microservices.

When building distributed systems, Hello World is NOT the use case. BUT, it is a valid example in that it strips aside any business logic and shows you the basic structure and concepts.

Let’s say there’s a command you want to send. The GreetingCommand. A command can be any write or “do this” type command.

internal class GreetingCommand : Command
{
public GreetingCommand(string name)
:base(new Guid())
{
Name = name;
}

public string Name { get; private set; }
}

Now let’s say that something else will “handle” these commands. This is the DoIt() method. No where do we call Handle() ourselves. Similar to dependency injection, we won’t be in the business of calling Handle() ourselves; the underlying framework will abstract that away.

internal class GreetingCommandHandler : RequestHandler<GreetingCommand>
{
[RequestLogging(step: 1, timing: HandlerTiming.Before)]
public override GreetingCommand Handle(GreetingCommand command)
{
Console.WriteLine("Hello {0}", command.Name);
return base.Handle(command);
}
}

We then register a factory that takes types and returns handlers. In a real system you’d use IoC (Inversion of Control) dependency injection for this mapping as well.

Our Main() has a registry that we pass into a larger pipeline where we can set policy for processing commands. This pattern may feel familiar with “Builders” and “Handlers.”

private static void Main(string[] args)
{
var registry = new SubscriberRegistry();
registry.Register<GreetingCommand, GreetingCommandHandler>();


var builder = CommandProcessorBuilder.With()
.Handlers(new HandlerConfiguration(
subscriberRegistry: registry,
handlerFactory: new SimpleHandlerFactory()
))
.DefaultPolicy()
.NoTaskQueues()
.RequestContextFactory(new InMemoryRequestContextFactory());

var commandProcessor = builder.Build();

...
}

Once we have a commandProcessor, we can Send commands to it easier and the work will get done. Again, how you ultimately make the commands is up to you.

commandProcessor.Send(new GreetingCommand("HanselCQRS"));

Methods within RequestHandlers can also have other behaviors associated with them, as in the case of “[RequestLogging] on the Handle() method above. You can add other stuff like Validation, Retries, or Circuit Breakers. The idea is that Brighter offers a pipeline of handlers that can all operate on a Command. The Celery Project is a similar project except written in Python. The Brighter project has stated they have lofty goals, intending to one day handle fault tolerance like Netflix’s Hystrix project.

One of the nicest aspects to Brighter is that it’s prescriptive but not heavy-handed. They say:

Brighter is intended to be a library not a framework, so it is consciously lightweight and divided into packages that allow you to consume only those facilities that you need in your project.

Moving beyond Hello World, there are more fleshed out examples like a TaskList with a UI, back end Http API, a Mailer service, and core library.

Be sure to explore Brighter’s excellent documentation and examples, but be aware, this is a project under active development. Perhaps if you’re new to OSS, if you find a broken link or two or a misspelling, you can do Your First Pull Request with a small fix?

Do be aware, again, that CQRS is not for every project. It’s non-trivial and it’s a “mental leap” as Martin Fowler puts it. If you buy in, you’re adding complexity…for a reason. Keep your eyes open and do your research. It’s a great pattern if you have a high performance/volume application that struggles with write concurrency or a flaky backend.

In fact there are quite a few mature CQRS libraries in the .NET open source space. I’ll explore a few – which are your favorites?


Sponsor: Seq is simple centralized logging, on your infrastructure, with great support for ASP.NET Core and Serilog. Version 4 adds integrated dashboards and alerts – check it out!


© 2017 Scott Hanselman. All rights reserved.
     

The logo for the "Brighter" Open Source project is a little cannon. Fire and Forget?There's a ton of cool new .NET Core open source projects lately, and I've very much enjoyed exploring this rapidly growing space. Today at lunch I was checking out a project called "Brighter." It's actually been around in the .NET space for many years and is in the process of moving to .NET Core for greater portability and performance.

Brighter is a ".NET Command Dispatcher, with Command Processor features for QoS (like Timeout, Retry, and Circuit Breaker), and support for Task Queues"

Whoa, that's a lot of cool and fancy words. What's it mean? The Brighter project is up on GitHub incudes a bunch of libraries and examples that you can pull in to support CQRS architectural styles in .NET. CQRS stands for Command Query Responsibility Segregation. As Martin Fowler says, "At its heart is the notion that you can use a different model to update information than the model you use to read information." The Query Model reads and the Command Model updates/validates. Greg Young gives the first example of CQRS here. If you are a visual learner, there's a video from late 2015 where Ian Cooper explains a lot of this a the London .NET User Group or an interview with Ian Cooper on Channel 9.

Brighter also supports "Distributed Task Queues" which you can use to improve performance when you're using a query or integrating with microservices.

When building distributed systems, Hello World is NOT the use case. BUT, it is a valid example in that it strips aside any business logic and shows you the basic structure and concepts.

Let's say there's a command you want to send. The GreetingCommand. A command can be any write or "do this" type command.

internal class GreetingCommand : Command

{
public GreetingCommand(string name)
:base(new Guid())
{
Name = name;
}

public string Name { get; private set; }
}

Now let's say that something else will "handle" these commands. This is the DoIt() method. No where do we call Handle() ourselves. Similar to dependency injection, we won't be in the business of calling Handle() ourselves; the underlying framework will abstract that away.

internal class GreetingCommandHandler : RequestHandler<GreetingCommand>

{
[RequestLogging(step: 1, timing: HandlerTiming.Before)]
public override GreetingCommand Handle(GreetingCommand command)
{
Console.WriteLine("Hello {0}", command.Name);
return base.Handle(command);
}
}

We then register a factory that takes types and returns handlers. In a real system you'd use IoC (Inversion of Control) dependency injection for this mapping as well.

Our Main() has a registry that we pass into a larger pipeline where we can set policy for processing commands. This pattern may feel familiar with "Builders" and "Handlers."

private static void Main(string[] args)

{
var registry = new SubscriberRegistry();
registry.Register<GreetingCommand, GreetingCommandHandler>();


var builder = CommandProcessorBuilder.With()
.Handlers(new HandlerConfiguration(
subscriberRegistry: registry,
handlerFactory: new SimpleHandlerFactory()
))
.DefaultPolicy()
.NoTaskQueues()
.RequestContextFactory(new InMemoryRequestContextFactory());

var commandProcessor = builder.Build();

...
}

Once we have a commandProcessor, we can Send commands to it easier and the work will get done. Again, how you ultimately make the commands is up to you.

commandProcessor.Send(new GreetingCommand("HanselCQRS"));

Methods within RequestHandlers can also have other behaviors associated with them, as in the case of "[RequestLogging] on the Handle() method above. You can add other stuff like Validation, Retries, or Circuit Breakers. The idea is that Brighter offers a pipeline of handlers that can all operate on a Command. The Celery Project is a similar project except written in Python. The Brighter project has stated they have lofty goals, intending to one day handle fault tolerance like Netflix's Hystrix project.

One of the nicest aspects to Brighter is that it's prescriptive but not heavy-handed. They say:

Brighter is intended to be a library not a framework, so it is consciously lightweight and divided into packages that allow you to consume only those facilities that you need in your project.

Moving beyond Hello World, there are more fleshed out examples like a TaskList with a UI, back end Http API, a Mailer service, and core library.

Be sure to explore Brighter's excellent documentation and examples, but be aware, this is a project under active development. Perhaps if you're new to OSS, if you find a broken link or two or a misspelling, you can do Your First Pull Request with a small fix?

Do be aware, again, that CQRS is not for every project. It's non-trivial and it's a "mental leap" as Martin Fowler puts it. If you buy in, you're adding complexity...for a reason. Keep your eyes open and do your research. It's a great pattern if you have a high performance/volume application that struggles with write concurrency or a flaky backend.

In fact there are quite a few mature CQRS libraries in the .NET open source space. I'll explore a few - which are your favorites?


Sponsor: Seq is simple centralized logging, on your infrastructure, with great support for ASP.NET Core and Serilog. Version 4 adds integrated dashboards and alerts - check it out!



© 2017 Scott Hanselman. All rights reserved.