How to setup Signed Git Commits with a YubiKey NEO and GPG and Keybase on Windows

This week in obscure blog titles, I bring you the nightmare that is setting up Signed Git Commits with a YubiKey NEO and GPG and Keybase on Windows. This is one of those “it’s good for you” things like diet and exercise and setting up 2 Factor Authenti…

This commit was signed with a verified signature.This week in obscure blog titles, I bring you the nightmare that is setting up Signed Git Commits with a YubiKey NEO and GPG and Keybase on Windows. This is one of those "it's good for you" things like diet and exercise and setting up 2 Factor Authentication. I just want to be able to sign my code commits to GitHub so I might avoid people impersonating my Git Commits (happens more than you'd think and has happened recently.) However, I also was hoping to make it more security buy using a YubiKey NEO security key. They're happy to tell you that it supports a BUNCH of stuff that you have never heard of like Yubico OTP, OATH-TOTP, OATH-HOTP, FIDO U2F, OpenPGP, Challenge-Response. I am most concerned with it acting like a Smart Card that holds a PGP (Pretty Good Privacy) key since the YubiKey can look like a "PIV (Personal Identity Verification) Smart Card."

NOTE: I am not a security expert. Let me know if something here is wrong (be nice) and I'll update it. Note also that there are a LOT of guides out there. Some are complete and encyclopedic, some include recommendations and details that are "too much," but this one was my experience. This isn't The Bible On The Topic but rather  what happened with me and what I ran into and how I got past it. Until this is Super Easy (TM) on Windows, there's gonna be guides like this.

As with all things security, there is a balance between Capital-S Secure with offline air-gapped what-nots, and Ease Of Use with tools like Keybase. It depends on your tolerance, patience, technical ability, and if you trust any online services. I like Keybase and trust them so I'm starting there with a Private Key. You can feel free to get/generate your key from wherever makes you happy and secure.

Welcome to Keybase.io

I use Windows and I like it, so if you want to use a Mac or Linux this blog post likely isn't for you. I love and support you and your choice though. ;)

Make sure you have a private PGP key that has your Git Commit Email Address associated with it

I download and installed (and optionally donated) a copy of Gpg4Win here.

Take your private key - either the one you got from Keybase or one you generated locally - and make sure that your UUID (your email address that you use on GitHub) is a part of it. Here you can see mine is not, yet. That could be the main email or might be an alias or "uuid" that you'll add.

Certs in Kleopatra

If not, as in my case since I'm using a key from keybase, you'll need to add a new uuid to your private key. You will know you got it right when you run this command and see your email address inside it.

> gpg --list-secret-keys --keyid-format LONG


------------------------------------------------
sec# rsa4096/MAINKEY 2015-02-09 [SCEA]

uid [ultimate] keybase.io/shanselman <[email protected]>

You can adduuid in the gpg command line or you can add it in the Kleopatra GUI.

image

If not, as in my case since I'm using a key from keybase, you'll need to add a new uuid to your private key. You will know you got it right when you run this command and see your email address inside it.

> gpg --list-secret-keys --keyid-format LONG


------------------------------------------------
sec# rsa4096/MAINKEY 2015-02-09 [SCEA]
uid [ultimate] keybase.io/shanselman <[email protected]>
uid [ unknown] Scott Hanselman <[email protected]>

Then, when you make changes like this, you can export your public key and update it in Keybase.io (again, if you're using Keybase).

image

Plugin your YubiKey

I installed the YubiKey Smart card mini-driver from here.  Some people have said this driver is optional but I needed it on my main machine. Can anyone confirm?

When you plug your YubiKey in (assuming it's newer than 2015) it should get auto-detected and show up like this "Yubikey NEO OTP+U2F+CCID." You want it so show up as this kind of "combo" or composite device. If it's older or not in this combo mode, you may need to download the YubiKey NEO Manager and switch modes.

Setting up a YubiKey on Windows

Test that your YubiKey can be seen as a Smart Card

Go to the command line and run this to confirm that your Yubikey can be see as a smart card by the GPG command line.

> gpg --card-status

Reader ...........: Yubico Yubikey NEO OTP U2F CCID 0
Version ..........: 2.0
....

IMPORTANT: Sometimes Windows machines and Corporate Laptops have multiple smart card readers, especially if they have Windows Hello installed like my SurfaceBook2! If you hit this, you'll want to create a text file at %appdata%\Roaming\gnupg\scdaemon.conf and include a reader-port that points to your YubiKey. Mine is a NEO, yours might be a 4, etc, so be aware. You may need to reboot or at least restart/kill the GPG services/background apps for it to notice you made a change.
If you want to know what string should go in that file, go to Device Manager, then View | Show Hidden Devices and look under Software Devices. THAT is the string you want. Put this in scdaemon.conf:

reader-port "Yubico Yubikey NEO OTP+U2F+CCID 0"

Yubico Yubikey NEO OTP+U2F+CCID 0

Yubikey NEO can hold keys up to 2048 bits and the Yubikey 4 can hold up to 4096 bits - that's MOAR bits! However, you might find yourself with a 4096 bit key that is too big for the Yubikey NEO. Lots of folks believe this is a limitation of the NEO that sucks and is unacceptable. Since I'm using Keybase and starting with a 4096 bit key, one solution is to make separate 2048 bit subkeys for Authentication and Signing, etc.

From the command line, edit your keys then "addkey"

> gpg --edit-key <[email protected]>

You'll make a 2048 bit Signing key and you'll want to decide if it ever expires. If it never does, also make a revocation certificate so you can revoke it at some future point.

gpg> addkey

Please select what kind of key you want:
(3) DSA (sign only)
(4) RSA (sign only)
(5) Elgamal (encrypt only)
(6) RSA (encrypt only)
Your selection? 4
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048)
Requested keysize is 2048 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all

Save your changes, and then export the keys. You can do that with Kleopatra or with the command line:

--export-secret-keys --armor KEYID

Here's a GUI view. I have my main 4096 bit key and some 2048 bit subkeys for Signing or Encryption, etc. Make as many as you like

image

LEVEL SET - It will be the public version of the 2048 bit Signing Key that we'll tell GitHub about and we'll put the private part on the YubiKey, acting as a Smart Card.

Move the signing subkey over to the YubiKey

Now I'm going to take my keychain here, select the signing one (note the ASTERISK after I type "key 1" then "keytocard" to move/store it on the YubyKey's SmartCard Signature slot. I'm using my email as a way to get to my key, but if your email is used in multiple keys you'll want to use the unique Key Id/Signature.

> gpg --edit-key [email protected]


gpg (GnuPG) 2.2.6; Copyright (C) 2018 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

sec rsa4096/MAINKEY
created: 2015-02-09 expires: never usage: SCEA
trust: ultimate validity: ultimate
ssb rsa2048/THEKEYIDFORTHE2048BITSIGNINGKEY
created: 2015-02-09 expires: 2023-02-07 usage: S
card-no: 0006
ssb rsa2048/KEY2
created: 2015-02-09 expires: 2023-02-07 usage: E
[ultimate] (1). keybase.io/shanselman <[email protected]>
[ultimate] (2) Scott Hanselman <[email protected]>
gpg> toggle
gpg> key 1

sec rsa4096/MAINKEY
created: 2015-02-09 expires: never usage: SCEA
trust: ultimate validity: ultimate
ssb* rsa2048/THEKEYIDFORTHE2048BITSIGNINGKEY
created: 2015-02-09 expires: 2023-02-07 usage: S
card-no: 0006
ssb rsa2048/KEY2
created: 2015-02-09 expires: 2023-02-07 usage: E
[ultimate] (1). keybase.io/shanselman <[email protected]>
[ultimate] (2) Scott Hanselman <[email protected]>

gpg> keytocard
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1

If you're storing thing on your Smart Card, it should have a pin to protect it. Also, make sure you have a backup of your primary key (if you like) because keytocard is a destructive action.

Have you set up PIN numbers for your Smart Card?

There's a PIN and an Admin PIN. The Admin PIN is the longer one. The default admin PIN is usually ‘12345678’ and the default PIN is usually ‘123456’. You'll want to set these up with either the Kleopatra GUI "Tools | Manage Smart Cards" or the gpg command line:

>gpg --card-edit

gpg/card> admin
Admin commands are allowed
gpg/card> passwd
*FOLLOW THE PROMPTS TO SET PINS, BOTH ADMIN AND STANDARD*

Tell Git about your Signing Key Globally

Be sure to tell Git on your machine some important configuration info like your signing key, but also WHERE the gpg.exe is. This is important because git ships its own older local copy of gpg.exe and you installed a newer one!

git config --global gpg.program "c:\Program Files (x86)\GnuPG\bin\gpg.exe"

git config --global commit.gpgsign true
git config --global user.signingkey THEKEYIDFORTHE2048BITSIGNINGKEY

If you don't want to set ALL commits to signed, you can skip the commit.gpgsign=true and just include -S as you commit your code:

git commit -S -m your commit message

Test that you can sign things

if you are running Kleopatra (the noob Windows GUI) when you run gpg --card-status you'll notice the cert will turn boldface and get marked as certified.

The goal here is for you to make sure GPG for Windows knows that there's a private key on the smart card, and associates a signing Key ID with that private key so when Git wants to sign a commit, you'll get a Smart Card PIN Prompt.

Advanced: If you make SubKeys for individual things so that they might also be later revoked without torching your main private key. Using the Kleopatra tool from GPG for Windows you can explore the keys and get their IDs. You'll use those Subkey IDs in your git config to remove to your signingkey.

At this point things should look kinda like this in the Kleopatra GUI:

Multiple PGP Sub keys

Make sure to prove you can sign something by making a text file and signing it. If you get a Smart Card prompt (assuming a YubiKey) and a larger .gpg file appears, you're cool.

> gpg --sign .\quicktest.txt

> dir quic*

Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 4/18/2018 3:29 PM 9 quicktest.txt
-a---- 4/18/2018 3:38 PM 360 quicktest.txt.gpg

Now, go up into GitHub to https://github.com/settings/keys at the bottom. Remember that's GPG Keys, not SSH Keys. Make a new one and paste in your public signing key or subkey.

Note the KeyID (or the SubKey ID) and remember that one of them (either the signing one or the primary one) should be the ID you used when you set up user.signingkey in git above.

GPG Keys in GitHub

The most important thing is that:

  • the email address associated with the GPG Key
  • is the same as the email address GitHub has verified for you
  • is the same as the email in the Git Commit

If not, double check your email addresses and make sure they are the same everywhere.

Try a signed commit

If pressing enter pops a PIN Dialog then you're getting somewhere!

Please unlock the card

Commit and push and go over to GitHub and see if your commit is Verified or Unverified. Unverified means that the commit was signed but either had an email GitHub had never seen OR that you forgot to tell GitHub about your signing public key.

Signed Verified Git Commits

Yay!

Setting up to a second (or third) machine

Once you've told Git about your signing key and you've got your signing key stored in your YubiKey, you'll likely want to set up on another machine.

  • Install the Yubikey SmartCard Mini Driver (may be optional)
  • Install GPG for Windows
    • gpg --card-status
    • Import your public key. If I'm setting up signing on another machine, I'll can import my PUBLIC certificates like this or graphically in Kleopatra.
      >gpg --import "keybase public key.asc"
      
      gpg: key *KEYID*: "keybase.io/shanselman <[email protected]>" not changed
      gpg: Total number processed: 1
      gpg: unchanged: 1

      You may also want to run gpg --expert --edit-key *KEYID* and type "trust" to certify your key as someone (yourself) that you trust.

  • Install Git (I assume you did this) and configure GPG
    • git config --global gpg.program "c:\Program Files (x86)\GnuPG\bin\gpg.exe"
    • git config --global commit.gpgsign true
    • git config --global user.signingkey THEKEYIDFORTHE2048BITSIGNINGKEY
  • Sign something with "gpg --sign" to test
  • Do a test commit.

Finally, feel superior for 8 minutes, then realize you're really just lucky because you just followed the blog post of someone who ALSO has no clue, then go help a co-worker because this is TOO HARD.


Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!



© 2018 Scott Hanselman. All rights reserved.
     

Automatic Unit Testing in .NET Core plus Code Coverage in Visual Studio Code

I was talking to Toni Edward Solarin on Skype yesterday about his open source spike (early days) of Code Coverage for .NET Core called “coverlet.” There’s a few options out there for cobbling together .NET Core Code Coverage but I wanted to see if I co…

I was talking to Toni Edward Solarin on Skype yesterday about his open source spike (early days) of Code Coverage for .NET Core called "coverlet." There's a few options out there for cobbling together .NET Core Code Coverage but I wanted to see if I could use the lightest tools I could find and make a "complete" solution for Visual Studio Code that would work for .NET Core cross platform. I put my own living spike of a project up on GitHub.

Now, keeping in mine that Toni's project is just getting started and (as of the time of this writing) currently supports line and method coverage, and branch coverage is in progress, this is still a VERY compelling developer experience.

Using VS Code, Coverlet, xUnit, plus these Visual Studio Code extensions

Here's what we came up with.

Auto testing, code coverage, line coloring, test explorers, all in VS Code

There's a lot going on here but take a moment and absorb the screenshot of VS Code above.

  • Our test project is using xunit and the xunit runner that integrates with .NET Core as expected.
    • That means we can just "dotnet test" and it'll build and run tests.
  • Added coverlet, which integrates with MSBuild and automatically runs when you "dotnet test" if you "dotnet test /p:CollectCoverage=true"
    • (I think this should command line switch should be more like --coverage" but there may be an MSBuild limitation here.)

I'm interested in "The Developer's Inner Loop." . That means I want to have my tests open, my code open, and as I'm typing I want the solution to build, run tests, and update code coverage automatically the way Visual Studio proper does auto-testing, but in a more Rube Goldbergian way. We're close with this setup, although it's a little slow.

Coverlet can product opencover, lcov, or json files as a resulting output file. You can then generate detailed reports from this. There is a language agnostic VS Code Extension called Coverage Gutters that can read in lcov files and others and highlight line gutters with red, yellow, green to show test coverage. Those lcov files look like this, showing file names, file numbers, coverage, and number of exceptions.

SF:C:\github\hanselminutes-core\hanselminutes.core\Constants.cs
DA:3,0
end_of_record
SF:C:\github\hanselminutes-core\hanselminutes.core\MarkdownTagHelper.cs
DA:21,5
DA:23,5
DA:49,5

I should be able to pick the coverage file manually with the extension, but due to a small bug, it's easier to just tell Coverlet to generate a specific file name in a specific format.

dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov.info .\my.tests

The lcov.info files then watched by the VSCode Coverage Gutters extension and updates as the file changes if you click watch in the VS Code Status Bar.

You can take it even further if you add "dotnet watch test" which will compile and re-run tests if code changes:

dotnet watch --project .\my.tests test /p:CollectCoverage=true /p:CoverletOutputFormat=lcov /p:CoverletOutput=./lcov.info 

I can run "WatchTests.cmd" in another terminal, or within the VS Code integrated terminal.

tests automatically running as code changes

NOTE: If you're doing code coverage you'll want to ensure your tests and tested assembly are NOT the same file. You might be able to get it to work but it's easier to keep things separate.

Next, add in the totally under appreciated .NET Core Test Explorer extension (this should have hundreds of thousands of downloads - it's criminal) to get this nice Test Explorer pane:

A Test Explorer tree view in VS Code for NET Core projects

Even better, .NET Test Explorer lights up some "code lens" style interfaces over each test as well as a green checkmark for passing tests. Having "debug test" available for .NET Core is an absolute joy.

Check out "run test" and "debug test"

Finally we make some specific improvements to the .vscode/tasks.json file that drives much of VS Code's experience with our app. The "BUILD" label is standard but note both the custom "test" and "testwithcoverage" labels, as well as the added group with kind: "test."

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "build",
            "command": "dotnet",
            "type": "process",
            "args": [
                "build",
                "${workspaceFolder}/hanselminutes.core.tests/hanselminutes.core.tests.csproj"
            ],
            "problemMatcher": "$msCompile",
            "group": {
                "kind": "build",
                "isDefault": true
            }
        },
        {
            "label": "test",
            "command": "dotnet",
            "type": "process",
            "args": [
                "test",
                "${workspaceFolder}/hanselminutes.core.tests/hanselminutes.core.tests.csproj"
            ],
            "problemMatcher": "$msCompile",
            "group": {
                "kind": "test",
                "isDefault": true
            }
        },
        {
            "label": "test with coverage",
            "command": "dotnet",
            "type": "process",
            "args": [
                "test",
                "/p:CollectCoverage=true",
                "/p:CoverletOutputFormat=lcov",
                "/p:CoverletOutput=./lcov.info",
                "${workspaceFolder}/hanselminutes.core.tests/hanselminutes.core.tests.csproj"
            ],
            "problemMatcher": "$msCompile",
            "group": {
                "kind": "test",
                "isDefault": true
            }
        },
    ]
}

This lets VS Code know what's for building and what's for testing, so if I use the Command Palette to "Run Test" then I'll get this dropdown that lets me run tests and/or update coverage manually if I don't want the autowatch stuff going.

Test or Test with Coverage

Again, all this is just getting started but I've applied it to my Podcast Site that I'm currently rewriting and the experience is very smooth!

Here's a call to action for you! Toni is just getting started on Coverlet and I'm sure he'd love some help. Head over to the Coverlet github and don't just file issues and complain! This is an opportunity for you to get to know the deep internals of .NET and create something cool for the larger community.

What are your thoughts?


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.



© 2018 Scott Hanselman. All rights reserved.
     

Turn your Raspberry Pi into a portable Touchscreen Tablet with SunFounder’s RasPad

I was very fortunate to get a preview version of the “RasPad” from SunFounder. Check it out at https://raspad.sunfounder.com/ and at the time of these writing they have a Kickstarter I’m backing! I’ve written a lot about Raspberry Pis and the cool pro…

RasPadI was very fortunate to get a preview version of the "RasPad" from SunFounder. Check it out at https://raspad.sunfounder.com/ and at the time of these writing they have a Kickstarter I'm backing!

I've written a lot about Raspberry Pis and the cool projects you can do with them. My now-10 and 12 year olds love making stuff with Raspberry Pis and we have at least a dozen of them around the house. A few are portable arcades (some quite tiny PiArcades), one runs PiMusicBox and is a streaming radio, and I have a few myself in a Kubernetes Cluster.

I've built Raspberry Pi Cars with SunFounder parts, so they sent me an early evaluation version of their "RasPad." I was familiar with the general idea as I'd tried (and failed) to make something like it with their 10" Touchscreen LCD for Raspberry Pi.

At its heart, the RasPad is quiet elegant and simple. It's a housing for your Raspberry Pi that includes a battery for portable use along with an integrated touchscreen LCD. However, it's the little details where it shines.

RasPad - Raspberry Pi Touchscreen

It's not meant to be an iPad. It's not trying. It's thick on one end, and beveled to an angle. You put your RaspberryPi inside the back corner and it sits nicely on the plastic posts without screws. Power and HDMI and are inside with cables, then it's one button to turn it on. There's an included power supply as well as batteries to run the Pi and screen for a few hours while portable.

RasPad ports are extensive

I've found with my 10 year old that this neat, organized little tablet mode makes the Pi more accessible and interesting to him - as opposed to the usual mess of wires and bare circuit boards we usually have on my workbench. I could see a fleet of RasPads in a classroom environment being far more engaging than just "raw" Pis on a table.

The back of the RasPad has a slot where a GPIO Ribbon cable can come out to a breakout  board:

GPIO slot is convenient

At this point you can do all the same cool hardware projects you can do with a Raspberry Pi, with all the wires, power, touchscreen, ports, and everything nice and sanitary.

The inside hatch is flexible enough for other boards as well:

Raspberry Pi or TinkerBoard

I asked my 10 year old what he wanted to make with the RasPad/Raspberry Pi and he said he wanted to make a "burglar alarm" for his bedroom. Pretty sure he just wants to keep the 12 year old out of his room.

We started with a Logitech 930e USB Webcam we had laying around. The Raspberry PI can use lots of off-the-shelf high-quality web cams without drivers, and the RasPad keeps all the USB ports exposed.

Then we installed the "Motion" Project. It's on GitHub at https://github.com/Motion-Project/motion with:

sudo apt-get install motion

Then edited /etc/motion/motion.conf with the nano editor (easier for kids then vim). You'll want to confirm the height and width. Smaller is easier on the Pi, but you can go big with 1280x720 if you like! We also set the target_dir to /tmp since motion's daemon doesn't have access to ~/.

There's a number of events you can take action on, like "on_motion_detected." We just added a little Python script to let people know WE SEE YOU"

It's also cool to set location_motion_style to "redbox" so you can see WHERE motion was detected in a frame, and be sure to set stream_localhost to "off" so you can hit http://yourraspberrypiname:8081 to see the stream remotely!

When motion is detected, the 10 year old's little Python script launches:

GET OUT OF MY ROOM

And as a bonus, here is the 10 year old trying to sneak into the room. Can you spot him? (The camera did)

IMG_3389

What would you build with a RaspberryPi Tablet?

BTW, there's a Community Build of the .NET Core SDK for Raspberry Pi!


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.



© 2018 Scott Hanselman. All rights reserved.
     

Cross-platform GUIs with open source .NET using Eto.Forms

This is one of those “Did you know you could do THAT?” Many folks have figured out that C#/F#/.NET is cross-platform and open0source and runs on basically any operating system. People are using it to create micro services, web sites, and webAPI’s all o…

Amazing Cross Platform ANSI art editorThis is one of those "Did you know you could do THAT?" Many folks have figured out that C#/F#/.NET is cross-platform and open0source and runs on basically any operating system. People are using it to create micro services, web sites, and webAPI's all over. Not to mention iPhone/Android apps with Xamarin and video games with Unity and MonoGame.

But what about cross platform UIs?

While not officially supported by Microsoft - you can do some awesome stuff...as is how Open Source is supposed to work! Remember that there's a family of .NET Runtimes now, there's the .NET Framework on Windows, there's xplat .NET Core, and there's xplat Mono.

Eto.Forms has been in development since 2012 and is a cross-platform framework for creating GUI (Graphical User Interface, natch) applications with .NET that run across multiple platforms using their native toolkit. Not like Java in the 90s with custom painted buttons on canvas.

It's being used for real stuff! In fact, PabloDraw is an Ansi/Ascii text editor that you didn't know you needed in your life. But you do. It runs on Windows, Mac, and Linux and was written using Eto.Forms but has a native UI on each platform. Be sure to check out Curtis Wensley's Twitter account for some cool examples of what PabloDraw and Eto.Forms can do!

  • OS X: MonoMac or Xamarin.Mac (and also iOS via Xamarin)
  • Linux: GTK# 2 or 3
  • Windows: Windows Forms (using GDI or Direct2D) or WPF

Here's an example Hello World. Note that it's not just Code First, you can also use Xaml, or even Json (.jeto) to layout your forms!

using Eto.Forms;

using Eto.Drawing;

public class MyForm : Form
{
public MyForm ()
{
Title = "My Cross-Platform App";
ClientSize = new Size(200, 200);
Content = new Label { Text = "Hello World!" };
}

[STAThread]
static void Main()
{
new Application().Run(new MyForm());
}
}

Or I can just File | New Project with their Visual Studio Extension. You should definitely give it a try.

image

Even on the same platform (Windows in the below example) amazingly Eto.Forms can use whatever Native Controls you prefer. Here's a great example zip that has precompiled test apps.

WinForms, WPF, and Direct2D apps

Once you've installed a new version of Mono on Ubuntu, you can run the same sample as Gtk3, as I'm doing here in a VM. AMAZING.

image

Here's some example applications that are in the wild, using Eto.Forms:

There's so much cool stuff happening in open source .NET right now, and Eto.Forms is actively looking for help. Go check out their excellent Wiki, read the Tutorials, and maybe get involved!


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.



© 2017 Scott Hanselman. All rights reserved.
     

Major build speed improvements – Try .NET Core 2.1 Preview 1 today

Head over to the main .NET Core download page and pick up .NET Core 2.1 – Preview 1. The SDK contains the tools you need to build and run apps with .NET Core and supports Mac, Windows, Ubuntu, Debian, Fedora, CentOS/Oracle, openSUSE, and we even have …

Head over to the main .NET Core download page and pick up .NET Core 2.1 - Preview 1.

The SDK contains the tools you need to build and run apps with .NET Core and supports Mac, Windows, Ubuntu, Debian, Fedora, CentOS/Oracle, openSUSE, and we even have Docker images for Stretch, Alpine, and more. It's not your grandmother's Microsoft. ;)

Once you've installed it, from a prompt type "dotnet" and note a few new built-in switches:

C:\Users\scott> dotnet
Usage: dotnet [options]
Usage: dotnet [path-to-application]
Options:
  -h|--help         Display help.
  --version         Display the current SDK version.
  --list-sdks       Display the installed SDKs.
  --list-runtimes   Display the installed runtimes.
path-to-application:
  The path to an application .dll file to execute.

I'll run it again twice with --list-sdks and --list-runtimes:

C:\Users\scott> dotnet --list-sdks
2.1.300-preview1-008174 [C:\Program Files\dotnet\sdk]
2.1.4 [C:\Program Files\dotnet\sdk]

C:\Users\scott> dotnet --list-runtimes Microsoft.AspNetCore.All 2.1.0-preview1-final [C:\Program Files\dotnet\shared] Microsoft.AspNetCore.App 2.1.0-preview1-final [C:\Program Files\dotnet\shared] Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared] Microsoft.NETCore.App 2.1.0-preview1-26216-03 [C:\Program Files\dotnet\shared]

There's a few interesting things happening here. Youc an see before I had the runtime for .NET Core 2.0.5, and now I also have the 2.1.0 Preview.

It can also be a little confusing that the SDK and Runtime sometimes have different versions, similar to JREs and JDKs. Simply stated - the thing that builds sometimes updates while then thing that runs doesn't. So the .NET Core SDK and compilers might get fixes but the runtime doesn't. I'm told they're going to line things up better. You can read deeply on versioning if you like.

You'll also notice AspNetCore.App, which is a metapackage (package of packages) that's got less than All and helps you make smaller apps.

If you install a beta or preview you might be worried it'll mess stuff up. It won't.

You can type "dotnet new globaljson" and make a file that looks like this! Then "pin" the SDK version you want to use:

{
  "sdk": {
    "version": "2.1.300-preview1-008174"
  }
}

I'll change this to .NET Core's older SDK and try building the .NET Core based Gameboy Emulator in my last post. It's amazing.

Let's see how fast it builds today on .NET 2.0:

C:\github\Retro.Net> Measure-Command { dotnet build }
Milliseconds      : 586
Ticks             : 65864065
TotalSeconds      : 6.5864065
TotalMilliseconds : 6586.4065

Ok, about 6.5 seconds on my Surface.

Let's make the SDK version the new .NET Core 2.1 Preview 1 - it has a bunch of build speed improvements.

All I have to do is change the global.json file. Update the sdk version in the global.json and type "dotnet --version" to see that it took.

I can have as many .NET Core SDKs as I like on my machine and I can control what SDK versions are being used on a tree by tree basis. That means you CAN download .NET Core 2.1 and not mess things up if you're paying attention.

C:\github\Retro.Net> Measure-Command { dotnet build }
Milliseconds      : 727
Ticks             : 27270864
TotalSeconds      : 2.7270864
TotalMilliseconds : 2727.0864

Hey it's less than 3 seconds. 2.7 in fact! More than twice as fast.

Build times as much as 10x faster

The bigger the app, the faster incremental builds should be. In some cases we will see (by release) 10x improvements.

It's quick to install (and quick to uninstall) and you can control the SDK version (list them with "dotnet --list-sdks") with the global.json.

Please go download the preview and let me know either on Twitter or in the comments what your before and after build times are!


Sponsor: Unleash a faster Python! Supercharge your applications performance on future forward Intel platforms with The Intel Distribution for Python. Available for Windows, Linux, and macOS. Get the Intel® Distribution for Python Now!



© 2017 Scott Hanselman. All rights reserved.
     

A multi-player server-side GameBoy Emulator written in .NET Core and Angular

One of the great joys of sharing and discovering code online is when you stumble upon something so truly epic, so amazing, that you have to dig in. Head over to https://github.com/axle-h/Retro.Net and ask yourself why this GitHub project has only 20 st…

Server-side GameBoyOne of the great joys of sharing and discovering code online is when you stumble upon something so truly epic, so amazing, that you have to dig in. Head over to https://github.com/axle-h/Retro.Net and ask yourself why this GitHub project has only 20 stars?

Alex Haslehurst has created some retro hardware libraries in open source .NET Core with an Angular Front End!

Translation?

A multiplayer server-side Game Boy emulator. Epic.

You can run it in minutes with

docker run -p 2500:2500 alexhaslehurst/server-side-gameboy

Then just browse to http://localhost:2500 and play Tetris on the original GameBoy!

I love this for a number of reasons.

First, I love his perspective:

Please check out my GameBoy emulator written in .NET Core; Retro.Net. Yes, a GameBoy emulator written in .NET Core. Why? Why not. I plan to do a few write-ups about my experience with this project. Firstly: why it was a bad idea.

  1. Emulation on .NET
  2. Emulating the GameBoy CPU on .NET

The biggest issue one has trying to emulate a CPU with a platform like .NET is the lack of reliable high-precision timing. However, he manages a nice from-scratch emulation of the Z80 processor, modeling low level things like registers in very high level C#. I love that public class GameBoyFlagsRegister is a thing. ;) I did similar things when I ported a 15 year old "Tiny CPU" to .NET Core/C#.

Address space diagram from https://ax-h.com/software/development/emulation/2017/12/03/emulating-the-gameboy-cpu-on-dot-net.html

Be sure to check out Alex's extremely detailed explanation on how he modeled the Z80 microprocessor.

Luckily the GameBoy CPU, a Sharp LR35902, is derived from the popular and very well documented Zilog Z80 - A microprocessor that is unbelievably still in production today, over 40 years after it’s introduction.

The Z80 is an 8-bit microprocessor, meaning that each operation is natively performed on a single byte. The instruction set does have some 16-bit operations but these are just executed as multiple cycles of 8-bit logic. The Z80 has a 16-bit wide address bus, which logically represents a 64K memory map. Data is transferred to the CPU over an 8-bit wide data bus but this is irrelevant to simulating the system at state machine level. The Z80 and the Intel 8080 that it derives from have 256 I/O ports for accessing external peripherals but the GameBoy CPU has none - favouring memory mapped I/O instead

He didn't just create an emulator - there's lots of those - but uniquely he runs it on the server-side while allowing shared controls in a browser. "In between each unique frame, all connected clients can vote on what the next control input should be. The server will choose the one with the most votes… most of the time." Massively multi-player online GameBoy! Then he streams out the next frame! "GPU rendering is completed on the server once per unique frame, compressed with LZ4 and streamed out to all connected clients over websockets."

This is a great learning repository because:

  • it has complex business logic on the server-side but the front end uses Angular and web-sockets and open web technologies.
  • It's also nice that he has a complete multi-stage Dockerfile that is itself a great example of how to build both .NET Core and Angular apps in Docker.
  • Extensive (thousands) of Unit Tests with the Shouldly Assertion Framework and Moq Mocking Framework.
  • Great example usages of Reactive Programming
  • Unit Testing on both server AND client, using Karma Unit Testing for Angular

Here's a few favorite elegant code snippets in this huge repository.

The Reactive Button Presses:

_joyPadSubscription = _joyPadSubject
    .Buffer(FrameLength)
    .Where(x => x.Any())
    .Subscribe(presses =>
                {
                    var (button, name) = presses
                        .Where(x => !string.IsNullOrEmpty(x.name))
                        .GroupBy(x => x.button)
                        .OrderByDescending(grp => grp.Count())
                        .Select(grp => (button: grp.Key, name: grp.Select(x => x.name).First()))
                        .FirstOrDefault();
                    joyPad.PressOne(button);
                    Publish(name, $"Pressed {button}");
                    Thread.Sleep(ButtonPressLength);
                    joyPad.ReleaseAll();
                });

The GPU Renderer:

private void Paint()
{
    var renderSettings = new RenderSettings(_gpuRegisters);
    var backgroundTileMap = _tileRam.ReadBytes(renderSettings.BackgroundTileMapAddress, 0x400);
    var tileSet = _tileRam.ReadBytes(renderSettings.TileSetAddress, 0x1000);
    var windowTileMap = renderSettings.WindowEnabled ? _tileRam.ReadBytes(renderSettings.WindowTileMapAddress, 0x400) : new byte[0];
    byte[] spriteOam, spriteTileSet;
    if (renderSettings.SpritesEnabled) {
        // If the background tiles are read from the sprite pattern table then we can reuse the bytes.
        spriteTileSet = renderSettings.SpriteAndBackgroundTileSetShared ? tileSet : _tileRam.ReadBytes(0x0, 0x1000);
        spriteOam = _spriteRam.ReadBytes(0x0, 0xa0);
    }
    else {
        spriteOam = spriteTileSet = new byte[0];
    }
    var renderState = new RenderState(renderSettings, tileSet, backgroundTileMap, windowTileMap, spriteOam, spriteTileSet);
    var renderStateChange = renderState.GetRenderStateChange(_lastRenderState);
    if (renderStateChange == RenderStateChange.None) {
        // No need to render the same frame twice.
        _frameSkip = 0;
        _framesRendered++;
        return;
    }
    _lastRenderState = renderState;
    _tileMapPointer = _tileMapPointer == null ? new TileMapPointer(renderState) : _tileMapPointer.Reset(renderState, renderStateChange);
    var bitmapPalette = _gpuRegisters.LcdMonochromePaletteRegister.Pallette;
    for (var y = 0; y < LcdHeight; y++) {
        for (var x = 0; x < LcdWidth; x++) {
            _lcdBuffer.SetPixel(x, y, (byte) bitmapPalette[_tileMapPointer.Pixel]);
            if (x + 1 < LcdWidth) {
                _tileMapPointer.NextColumn();
            }
        }
        if (y + 1 < LcdHeight){
            _tileMapPointer.NextRow();
        }
    }
    
    _renderer.Paint(_lcdBuffer);
    _frameSkip = 0;
    _framesRendered++;
}

The GameBoy Frames are composed on the server side then compressed and sent to the client over WebSockets. He's got backgrounds and sprites working, and there's still work to be done.

The Raw LCD is an HTML5 canvas:

<canvas #rawLcd [width]="lcdWidth" [height]="lcdHeight" class="d-none"></canvas>
<canvas #lcd
        [style.max-width]="maxWidth + 'px'"
        [style.max-height]="maxHeight + 'px'"
        [style.min-width]="minWidth + 'px'"
        [style.min-height]="minHeight + 'px'"
        class="lcd"></canvas>

I love this whole project because it has everything. TypeScript, 2D JavaScript Canvas, retro-gaming, and so much more!

const raw: HTMLCanvasElement = this.rawLcdCanvas.nativeElement;
const rawContext: CanvasRenderingContext2D = raw.getContext("2d");
const img = rawContext.createImageData(this.lcdWidth, this.lcdHeight);
for (let y = 0; y < this.lcdHeight; y++) {
  for (let x = 0; x < this.lcdWidth; x++) {
    const index = y * this.lcdWidth + x;
    const imgIndex = index * 4;
    const colourIndex = this.service.frame[index];
    if (colourIndex < 0 || colourIndex >= colours.length) {
      throw new Error("Unknown colour: " + colourIndex);
    }
    const colour = colours[colourIndex];
    img.data[imgIndex] = colour.red;
    img.data[imgIndex + 1] = colour.green;
    img.data[imgIndex + 2] = colour.blue;
    img.data[imgIndex + 3] = 255;
  }
}
rawContext.putImageData(img, 0, 0);
context.drawImage(raw, lcdX, lcdY, lcdW, lcdH);

I would encourage you to go STAR and CLONE https://github.com/axle-h/Retro.Net and give it a run with Docker! You can then use Visual Studio Code and .NET Core to compile and run it locally. He's looking for help with GameBoy sound and a Debugger.


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.



© 2017 Scott Hanselman. All rights reserved.
     

Running ASP.NET Core on GoDaddy’s cheapest shared Linux Hosting – Don’t Try This At Home

First, a disclaimer. Don’t do this. I did this to test a theory and to prove a point. ASP.NET Core and the .NET Core that it runs on are open source and run pretty much anywhere. I wanted to see if I could run an ASP.NET Core site on GoDaddy’s cheapest…

First, a disclaimer. Don't do this. I did this to test a theory and to prove a point. ASP.NET Core and the .NET Core that it runs on are open source and run pretty much anywhere. I wanted to see if I could run an ASP.NET Core site on GoDaddy's cheapest hosting ($3, although it scales to $8) that basically supports only PHP. It's not a full Linux VM. It's locked-down and limited. You don't have root. You are missing most tools you'd expect you'd have.

BUT.

I wanted to see if I could get ASP.NET Core running on it anyway. Maybe if I do, they (and other inexpensive hosts) will talk to the .NET team, learn that ASP.NET Core is open source and could easily run on their existing infrastructure.

AGAIN: Don't do this. It's hacky. It's silly. But it's hella cool. IMHO. Also, big thanks to Tomas Weinfurt for his help!

First, I went to GoDaddy and signed up for their cheap hosting. Again, not a VM, but their shared one. I also registered supercheapaspnetsite.com as well. They use a cPanel-based web management system that doesn't really let you do anything. You can turn on SSH, do some PHP stuff, and generally poke around, but it's not exactly low-level.

First I ssh (shoosh!) in and see what I'm working with. I'm shooshing with Ubuntu on Windows 10 feature, that every developer should turn on. It's makes it really easy to work with Linux hosts if you're starting from Linux on Windows 10.

[email protected] [/proc]$ cat version
Linux version 2.6.32-773.26.1.lve1.4.46.el6.x86_64 ([email protected]) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-18) (GCC) ) #1 SMP Tue Dec 5 18:55:41 EST 2017
[email protected] [/proc]$

OK, looks like Red Hat, so CentOS 6 should be compatible.

I'm going to use .NET Core 2.1 (which is in preview now!) and get the SDK at https://www.microsoft.com/net/download/all and install it on my Windows machine where I will develop and build the app. I don't NEED to use Windows to do this, but it's the laptop I have and it's also nice to know I can build on Windows but target CentOS/RHEL6.

Next I'll make a new ASP.NET site with

dotnet new razor

and then I'll publish a self-contained version like this:

dotnet publish -r rhel.6-x64

And those files will end up in a folder like \supercheapaspnetsite\bin\Debug\netcoreapp2.1\rhel.6-x64\publish\

NOTE: You may need to add the NuGet feed for the dailies for this .NET Core preview in order to get the RHEL6 runtime downloaded during this local publish.

Then I used WinSCP (or whatever FTP/SCP client you like, rsync, etc) to get the files over to the ~/www folder on your GoDaddy shared site. Then I

chmod +x ./supercheapasnetsite

to make it executable. Now, from my ssh session at GoDaddy, let's try to run my app!

[email protected] [~/www]$ ./supercheapaspnetsite
Failed to load hb, error: libunwind.so.8: cannot open shared object file: No such file or directory
Failed to bind to CoreCLR at '/home/secretname/public_html/libcoreclr.so'

Of course it couldn't be that easy, right? .NET Core wants the unwind library (shared object) and it doesn't exist on this locked down system.

AND I don't have yum/apt/rpm or a way to install it right?

I could go looking for tar.gz file somewhere like this http://download.savannah.nongnu.org/releases/libunwind/ but I need to think about versions and make sure things line up. Given that I'm targeting CentOS6, I should start here https://centos.pkgs.org/6/epel-x86_64/libunwind-1.1-3.el6.x86_64.rpm.html and download libunwind-1.1-3.el6.x86_64.rpm.

I need to crack open that rpm file and get the library. RPM packages are just headers on top of a CPIO archive, so I can apt-get install rpm2cpio from my local Ubuntu instances (on Windows 10). Then from /mnt/c/users/scott/Downloads (where I downloaded the file) I will extract it.

rpm2cpio ./libunwind-1.1-3.el6.x86_64.rpm | cpio -idmv

There they are.

image

This part is cool. Even though I have these files, I don't have root or any way to "install" them. However I could either export/use the LD_LIBRARY_PATH environment variable to control how libraries get loaded OR I could put these files in $ORIGIN/netcoredeps. You can read more about Self Contained Linux Applications on .NET Core here.

The main executable of published .NET Core applications (which is the .NET Core host) has an RPATH property set to $ORIGIN/netcoredeps. That means that when Linux shared library loader is looking for shared libraries, it looks to this location before looking to default shared library locations. It is worth noting that the paths specified by the LD_LIBRARY_PATHenvironment variable or libraries specified by the LD_PRELOAD environment variable are still used before the RPATH property. So, in order to use local copies of the third-party libraries, developers need to create a directory named netcoredeps next to the main application executable and copy all the necessary dependencies into it.

At this point I've added a "netcoredeps" folder to my public folder, and then copied it (scp) over to GoDaddy. Let's run it again.

[email protected] [~/www]$ ./supercheapaspnetsite
FailFast: Couldn't find a valid ICU package installed on the system. Set the configuration flag System.Globalization.Invariant to true if you want to run with no globalization support.
   at System.Environment.FailFast(System.String)
   at System.Globalization.GlobalizationMode.GetGlobalizationInvariantMode()
   at System.Globalization.GlobalizationMode..cctor()
   at System.Globalization.CultureData.CreateCultureWithInvariantData()
   at System.Globalization.CultureData.get_Invariant()
   at System.Globalization.CultureInfo..cctor()
   at System.StringComparer..cctor()
   at System.AppDomain.InitializeCompatibilityFlags()
   at System.AppDomain.Setup(System.Object)
Aborted

Ok, now it's complaining about ICU packages. These are for globalization. That is also mentioned in the self-contained-linux apps docs and there's a precompiled binary I could download. But there's options.

If your app doesn't explicitly opt out of using globalization, you also need to add libicuuc.so.{version}, libicui18n.so.{version}, and libicudata.so.{version}

I like "opt-out" so I don't have to go dig these ups (although I could) so I can either set the CORECLR_GLOBAL_INVARIANT env var to 1, or I can add System.Globalization.Invariant = true to supercheapaspnetsite.runtimeconfig.json, which I'll do with just to be obnoxious. ;)

When I run it again I get another complained about libuv. Yet another shared library that isn't installed on this instance. I could  go get it and put it in netcoredeps OR since I'm using the .NET Core 2.1, I could try something new. There were some improvements made in .NET Core 2.1 around sockets and http performance. On the client side, these new managed libraries are written from the ground up in managed code using the new high-performance Span<T> and on the server-side I could use Kestrel's (Kestrel is the .NET Core webserver) experimental UseSockets() as they are starting to move that over.

In other words, I can bypass libuv usage entirely by changing my Program.cs to use the use UseSockets() like this.

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
     WebHost.CreateDefaultBuilder(args)
     .UseSockets()
     .UseStartup<Startup>();

Let's run it again. I'll add the ASPNETCORE_URLS environment variable and set it to a high port like 8080. Remember, I'm not admin so I can't use any port under 1024.

[email protected] [~/www]$ export ASPNETCORE_URLS="http://*:8080"
[email protected] [~/www]$ ./supercheapaspnetsite
Hosting environment: Production
Content root path: /home/secretname/public_html
Now listening on: http://0.0.0.0:8080
Application started. Press Ctrl+C to shut down.

Holy crap it actually started.

Ok, but I can't access it from supercheapaspnetsite.com:8080 because this is GoDaddy's locked down managed shared hosting. I can't just open a port or forward a port in their control panel.

But. They use Apache, and that has the .htaccess file!

Could I use mod_proxy and try this?

ProxyPassReverse / http://127.0.0.1:8080/

Looks like no, they haven't turned this on. Likely they don't want to proxy off to external domains, but it'd be nice if they allowed localhost. Bummer. So close.

Fine, I'll proxy the traffic myself. (Not perfect, but this is all a spike)

RewriteRule ^(.*)$  "show.php" [L]

Cool, now a cheesy proxy goes in show.php.

<?php
$site = 'http://127.0.0.1:8080';
$request = $_SERVER['REQUEST_URI'];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $site . $request);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
$f = fopen("headers.txt", "a");
    curl_setopt($ch, CURLOPT_VERBOSE, 0);
    curl_setopt($ch, CURLOPT_STDERR, $f);
    #don't output curl response, I need to strip the headers.
    #yes I know I can just CURLOPT_HEADER, false and all this 
    # goes away, but for testing we log headers
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$hold = curl_exec($ch);
#strip headers
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$headers = substr($hold, 0, $header_size);
$response = substr($hold, $header_size);
$headerArray = explode(PHP_EOL, $headers);
echo $response; #echo ourselves. Yes I know curl can do this for us.
?>

Cheesy, yes. Works for GET? Also, yes. This really is Apache's job, not ours, but kudos to Tomas for this evil idea.

An ASP.NET Core app at a host that doesn't support it

Boom. How about another page at /about? Yes.

Another page with ASP.NET Core at a host that doesn't support it

Lovely. But I had to run the app myself. I have no supervisor or process manager (again this is already handled by GoDaddy for PHP but I'm in unprivileged world.) Shooshing in and running it is a bad idea and not sustainable. (Well, this whole thing is not sustainable, but still.)

We could copy "screen" over and start it up and detach like use screen ./supercheapaspnet app, but again, if it crashes, no one will start it. We do have crontab, so for now, we'll launch the app on a schedule occasionally to do a health check and if needed, keep it running. Also added a few debugging tools in ~/bin:

[email protected] [~/bin]$ ll
total 304
drwxrwxr-x  2    4096 Feb 28 20:13 ./
drwx--x--x 20    4096 Mar  1 01:32 ../
-rwxr-xr-x  1  150776 Feb 28 20:10 lsof*
-rwxr-xr-x  1   21816 Feb 28 20:13 nc*
-rwxr-xr-x  1  123360 Feb 28 20:07 netstat*

All in all, not that hard. ASP.NET Core and .NET Core underneath it can run pretty much anywhere, just like PHP, Python, whatever.

If you're a host and you want to talk to someone at Microsoft about setting up ASP.NET Core shared hosting, email [email protected] and talk to them! If you are GoDaddy, I apologize, and you should also email. ;)


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.



© 2017 Scott Hanselman. All rights reserved.
     

The Squishy Side of Open Source

A few months back my friend Keeley Hammond and I did a workshop for Women Who Code Portland called The Squishy Side of Open Source. We’d done a number of workshops before on how to use Git and the Command Line, and I’ve done a documentary film with Rob…

The Squishy Side of Open SourceA few months back my friend Keeley Hammond and I did a workshop for Women Who Code Portland called The Squishy Side of Open Source. We'd done a number of workshops before on how to use Git and the Command Line, and I've done a documentary film with Rob Conery called Get Involved In Tech: The Social Developer (watch it free!) but Keeley and I wanted to really dive into the interpersonal "soft" or squishy parts. We think that we all need to work to bring kindness back into open source.

Contributing to open source for the first time can be scary and a little overwhelming. In addition to the technical skills required, the social dynamics of contributing to a library and participating in a code review can seem strange.

That means how people talk to each other, what to do when pull requests go south, when issues heat up due to misunderstandings,

Keeley has published the deck up on SpeakerDeck. In this workshop, we talked about the work and details that go into maintaining an open source community, tell real stories from his experiences and go over what to expect when contributing to open source and how to navigate it.

Key Takeaways:

  • Understanding the work that open source maintainers do, and how to show respect for them.
  • Understanding Codes of Conduct and Style Guides for OSS repos and how to abide by them.
  • Tips for communicating clearly, and dealing with uncomfortable or hostile communication.

Good communication is a key part of contributing to open source.

  • Give context.
  • Do your homework beforehand. It’s OK not to know things, but before asking for help, check a project’s README, documentation, issues (open or closed) and search the internet for an answer.
  • Keep requests short and direct. Many projects have more incoming requests than people available to help. Be concise.
  • Keep all communication public.
  • It’s okay to ask questions (but be patient!). Show them the same patience that you’d want them to show to you.
Keep it classy. Context gets lost across languages, cultures, geographies, and time zones. Assume good intentions in these conversations.

Where to start?

What are some good resources you've found for understanding the squishy side of open source?


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.



© 2017 Scott Hanselman. All rights reserved.
     

How to set up Kubernetes on Windows 10 with Docker for Windows and run ASP.NET Core

Docker for Windows is really coming along nicely. They have both a Stable and Edge channel and the Edge (beta, experimental) one just included a lovely new feature – Kubernetes support. Per their docs, Kubernetes is only available in Docker for Windows…

Docker for Windows is really coming along nicely. They have both a Stable and Edge channel and the Edge (beta, experimental) one just included a lovely new feature - Kubernetes support. Per their docs, Kubernetes is only available in Docker for Windows 18.02 CE Edge. They set most everything up nicely and put Kubectl into your path and setup a context. If you use kubectl for other things - like your own Raspberry Pi Kubernetes Cluster, then you'll need to be aware of switching contexts. Same thing applies if you have one in the cloud, like the Kubernetes Cluster I made in Azure AKS.

Got Docker for Windows? If you have not yet installed Docker for Windows, see Install Docker for Windows for an explanation of stable and edge channels, system requirements, and download/install information.

It's easy to get started, just click "Enable Kubernetes" and Docker for Windows will download and start the images you need. I clicked "show system containers" because I like to see what's hidden from me, but you decide for yourself. Do be aware - there's a TON.

Enabling Kubernetes in Docker for Windows

By default, you won't get the Kubernetes Dashboard - of which I'm a fan - so you may want to install that. If you follow the default instructions (and you're a noob like me) then you'll likely end up with a Dashboard that is pretty locked down. It can be somewhat frustrating to get access to your own development dashboard, so I use the alternative (read: totally insecure) dashboard, like this:

C:\> kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard.yaml

I also like charts and graphs so I added these as well:

C:\> kubectl create -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/influxdb.yaml
C:\> kubectl create -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/heapster.yaml
C:\> kubectl create -f https://raw.githubusercontent.com/kubernetes/heapster/master/deploy/kube-config/influxdb/grafana.yaml

I can access the dashboard by default by running "kubectl proxy" then visiting this http://localhost:8001/ui and I'll get redirected to the dashboard:

Kuberenetes Dashboard

Now I can run through all the cool Kubernetes tutorials like the Guestbook Kubernetes Sample Application from the convenience of my Windows 10 machine. (I'm running a SurfaceBook 2 on the current non-Beta Windows 10.)

There are a lot of nice samples on running .NET Core and ASP.NET Core apps with Docker up at https://github.com/dotnet/dotnet-docker-samples/

I made a quick ASP.NET Core app called kubeaspnetapp:

C:\Users\scott\Desktop>dotnet new razor -o kubeaspnetapp
The template "ASP.NET Core Web App" was created successfully.
...snip...
Restore succeeded.

Then added a two-stage build DockerFile that looks like this:

FROM microsoft/aspnetcore-build:2.0 AS build-env
WORKDIR /app
# copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
# copy everything else and build
COPY . ./
RUN dotnet publish -c Release -o out
# build runtime image
FROM microsoft/aspnetcore:2.0
WORKDIR /app
COPY --from=build-env /app/out .
ENTRYPOINT ["dotnet", "kubeaspnetapp.dll"]

And built and tagged the image with:

C:\Users\scott\Desktop\kubeaspnetapp>docker build -t kubeaspnetapp .

Then I create a quick Deployment that manages a Pod that runs the Container:

C:\Users\scott\Desktop\kubeaspnetapp>kubectl run kubeaspnetapp --image=kubeaspnetapp:v1 --port=80
deployment "kubeaspnetapp" created

Now I'll expose it to the "outside." Again, this is usually done with .yaml files but it's a good learning exercise and it's all local.

C:\Users\scott\Desktop\kubeaspnetapp>kubectl get deployments
NAME            DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
kubeaspnetapp   1         1         1            1           1m
C:\Users\scott\Desktop\kubeaspnetapp>kubectl get pods
NAME                             READY     STATUS    RESTARTS   AGE
kubeaspnetapp-778f6d49bd-rct59   1/1       Running   0          1m
C:\Users\scott\Desktop\kubeaspnetapp>
C:\Users\scott\Desktop\kubeaspnetapp>
C:\Users\scott\Desktop\kubeaspnetapp>kubectl expose deployment kubeaspnetapp --type=NodePort
service "kubeaspnetapp" exposed
C:\Users\scott\Desktop\kubeaspnetapp>kubectl get services
NAME            TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubeaspnetapp   LoadBalancer   10.98.234.67   <pending>     80:31756/TCP     5s
kubernetes      ClusterIP      10.96.0.1      <none>        443/TCP          1d

Then I'll hit http://127.0.0.1:31756 in my browser...note how that port is brokering to the internal port 80 where the app listens...and there's my ASP.NET Core app running locally on Kubernetes, set up with Docker for Windows. Nice.

My ASP.NET Core app running in Kubernetes local on my Windows 10 machine

Here's me getting the startup logs from that pod:

C:\Users\scott\>kubectl get pods
NAME                             READY     STATUS    RESTARTS   AGE
kubeaspnetapp-7fd7f7ffb9-8gnzd   1/1       Running   0          6m
C:\Users\scott\Dropbox\k8s for pi\aspnetcoreapp>kubectl logs kubeaspnetapp-7fd7f7ffb9-8gnzd
Hosting environment: Production
Content root path: /app
Now listening on: http://[::]:80
Application started. Press Ctrl+C to shut down.

Pretty cool. As all the tooling and things across Windows, Docker, Kubernetes, Visual Studio (all flavors) continues to get better and better, I can only imagine this experience will get better and better. I look forward to a time when I can freely mix containers from different OSs and easily push them all en masse to Azure.


Sponsor: Get the latest JetBrains Rider for debugging third-party .NET code, Smart Step Into, more debugger improvements, C# Interactive, new project wizard, and formatting code in columns.



© 2017 Scott Hanselman. All rights reserved.
     

Building a Raspberry Pi Car Robot with WiFi and Video

Last year I found a company called SunFounder that makes great Raspberry Pi-related kits and stuff. I got their Raspberry Pi 10″ Touchscreen LCD and enjoyed it very much. This month I picked up the SunFounder PiCar 2.0 kit and built it with the kids. T…

The SunFounder Raspberry Pi Car kit comes wtih everything you need except the 18650 batteries. You'll need to get those elsewhere.Last year I found a company called SunFounder that makes great Raspberry Pi-related kits and stuff. I got their Raspberry Pi 10" Touchscreen LCD and enjoyed it very much. This month I picked up the SunFounder PiCar 2.0 kit and built it with the kids. The kit includes everything you need except for the Raspberry Pi itself, a mini SD Card (the Pi uses that as  hard drive), and two 18650 rechargeable lithium batteries. Those batteries are enough to power both the Pi itself (so the car isn't tethered) as well as provide enough voltage to run the 3 servos AND motors to drive and steer the car around. You can also expand the car with other attachments like light sensors, line followers, and more.

The PiCar 2.0 includes the chassis, a nice USB WiFi adapter with antenna (one less thing to think about if you're using a Raspberry Pi  like me), a USB webcam for computer vision scenarios. It includes a TB6612 Motor Driver, PCA9685 PWM (Pulse Width Modulation) Servo Driver with 16 channels for future expansion. The kit also helpfully includes all the tools, screwdriver, wrenches, and bolts.

Preparing to build the SunFounder Raspberry Pi car

All the code for the SunFounder PiCar-V is on GitHub and while there can be a few hiccups with some of the English instructions, there are a bunch of YouTube videos and folks online doing the same thing so we had no trouble making the robot in a weekend.

Building a SunFounder Raspberry Pi Car

PRO TIP - Boot your new Raspberry Pi up with ssh enabled and already joined to your wifi

You'll need to use a tool like Etcher.io to burn a copy of the Raspbian operating system on to a mini SD card. I prefer to save time and avoid having to connect a new Raspberry Pi to HDMI and a mouse and keyboard, so I get the Pi onto my wifi network and enable SSH by copying these two files to the root of the file system of the freshly burned mini SD card. This will cause the Pi to automatically join your network when it boots up for the first time. Then I used Ubuntu on Windows 10 to ssh into the Pi and follow the instructions.

  • Make a 0 byte file called "ssh" and copy it to the root of the new PI disk
  • Make a file called "wpa_supplicant.conf" with just linefeeds at the end and make it look like this. Copy it to the root of the new PI disk.
country=US
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
    ssid="YOURWIFI"
    scan_ssid=1
    psk="yourwifipassword"
    key_mgmt=WPA-PSK
}

I like to use Notepad2 or Visual Studio Code to change the line endings of a file. You can see the CRLF or the LF in the status car and click it. Unix/Raspbian/Raspberry Pi likes just an LF (line feed) for the lineending, while Windows defaults to using CRLF (Carriage Return/Line Feed, or 0x13 0x10) for text files.

Changing the line endings to Unix

The default Raspberry username is pi and the default password is raspberry. You may want to change that. SunFounder has a decent "install_dependencies" script that you'll run on the Pi:

Installing the PiCar dependencies

Once you've built the PiCar you can ssh in and run their development server that gives you a little WebAPI to control the car. The SunFounder folks are pretty good at web development (less so with mobile apps) and have a nice Django app to control the PiCar.

Here's the view from the front camera of the PiCar as viewed through local website on port 8000. It's looking at my computer looking at itself. ;)

Viewing the PiCar camera through the Django website

You're able to control the PiCar from this web interface with the keyboard. You can move the car and steer with WASD, as well as move the head/camera independently. You will need to enter the settings area (upper right corner) and calibrate the back wheels direction. By default, one wheel may go the opposite direction because they can't be sure how you mounted them, so you'll need to reverse one wheel to ensure they both go in the same direction.

They also included a client application, also written in Python. On Windows you'll need to install Python, and when you run client.py you may get an error:

ImportError: No module named requests

You'll need to run "pip3 install requests" as that module isn't installed by default.

Additionally, Python apps aren't smart about High-DPI displays, so I went to C:\Users\scott\appdata\local\Programs\Python\Python36 and right click'ed the Python.exe and set the DPI setting to "System (Enhanced)" like this.

Overridding DPI settings to System (Enhanced)

The client app is best for "Zeroing out" the camera and wheels, in case they are favoring one side or the other.

image

All in all, building the SunFounder "Raspberry Pi Video Car Kit 2.0" with the kids was a great experience. The next step is to see what else we can do with it!

  • Add a speaker so it talks?
  • Add Alexa support so you can talk to it?
  • Make the car drive around and take pictures, then use Azure cognitive services to announce what it sees?
  • Or, as my little boys say, "add weapons and make another bot for it to fight!"

What do you think?

* I use Amazon affiliate links and appreciate it when you use them! It supports this blog and sometimes gives me enough money to buy gadgets like this!


Sponsor: Unleash a faster Python! Supercharge your applications performance on future forward Intel® platforms with The Intel® Distribution for Python. Available for Windows, Linux, and macOS. Get the Intel® Distribution for Python Now!


© 2017 Scott Hanselman. All rights reserved.