Create a Windows Docker Host and connect to it without Visual Studio

I have been exploring Docker on Windows for a while because I think it is very promising that containers can run on the Windows platform as well. I’ve also written  a few posts already about Docker but this one is all about creating a Windows Docker Host on Azure and connect to it. Why does this deserve a blog post? Isn’t that next next finish and straightforward??

Unfortunately no. And that probably exactly why Microsoft created this extension for Visual Studio which does that for you.

The extension in Visual Studio allows you to publish a ASP.NET 5 application to Docker. When choosing that option you can create a new Virtual Machine on Azure. When you do that, a whole bunch of stuff is done. Magic. But in the end you can connect to your new and shiny Windows Docker Host.

I did that, and it worked fine. But then I wanted to allow a colleague to connect to my machine. And I wanted to connect to another Windows Docker Host of someone else. Or publish my ASP.NET application to a Windows Server Container Image I had already created….And then it started to hurt. The magic did more than creating a VM, it also created certificates, injected them into the Virtual Machine and reconfigured the Docker host on this machine.

Because I wanted to understand what happened I made it my goal to be able to connect to a Windows Docker host that I created without Visual Studio. At Xpirit we have an Innovation Day every 2 months. During this day you can work on whatever you like and in the end present it to your colleagues. This was my project, and here are the results.

A  bit background on certficates

The Windows Docker Host is running in the cloud. Because you want to do things on this server remotely, your workstation (with the docker tools for windows installed, which you get with the VS extension), must have a connection to this machine. Because you need to interact with a remote machine, you need certifcates on the host machine and your own machine to be able to talk to each other.

Creating a fresh Virtual Machine and open the firewall

First I created a fresh Virtual Machine based on the Windows Server 2016 Core with Container Tech Preview 4 image on Azure

• Login to the azure portal –> portal.azure.com
• Click New | Compute | Windows Server 2016 Core with Container Tech Preview 4
• Choose Resource Manager as your deployment Model.
• Fill in the details. Notice the region where you are creating this, because you need that later.
  
• When the machine is created, open the Network Security Group and add an inbound rule for Docker. Port 2376 should opened up for communication with our Docker Host
  
• Your machine is now up and running and ready to do some Docker stuff. When you connect with a RDP session, you can run the [docker] command and see that it lists the available commands

Generate certficates for the region

Now we need to generate certificates. In the Docker extension in VS, this was all done in the background, but I want to make it explicit. We need to generate certificates that we will use for communication to Azure. I have created a Powershell script that generates the certificates for the location you specify. The script can be downloaded from Github here. It also includes a directory Tools which contain the tool OpenSSL.exe and OpenSSL.cnf. This tool will handle the certificate generation

• Create a new directory on your PC. e.g. c:\temp\dockerhost
• Run GenerateCerts.ps1 with the following parameters
  • -CertificatePassword <password>
  • -OpenSSLExePath <full path> e.g. c:\temp\dockerhost\tools\openssl.exe
  • -OpenSSLConfigPath <full path> e.g. c:\temp\dockerhost\tools\openssl.cnf
  • -ResourceGroupLocation <the location you specified for your Docker Host VM> E.g. westeurope
• After you have run the script, you have a directory certs with files in it. If you want to rerun, first delete the certificates in this folder otherwise they won’t be re-generated

  .\GenerateCerts.ps1 -CertificatePassword "password" -OpenSSLExePath 
  "c:\temp\dockerhost\tools\openssl.exe" -OpenSSLConfigPath 
  "c:\temp\dockerhost\tools\openssl.cnf" -ResourceGroupLocation "westeurope" 
  

Upload certifcates to the Docker Host

Now that we have created the certificates, we want to upload them to the Docker Host. But how? Because we need certficates to communicate with the server and we first need to upload them. This part took me the longest to figure out, but I will try to explain.

When Microsoft creates the Virtual Machine with the Docker extension, it uses a ARM (Azure Resource Manager Template). This template contains a parameter for a certificate and some scripts. When creating the image, it uploads the generated the generated certificates to a temporary Azure Storage Account and then “injects” them in the image that is created, This means the certificates are already in place when he image is generated.

In our case we have a problem. But we can use Powershell Custom Script Extensions to “inject” stuff in to our Virtual Machine from a location that is trusted. So what we will do

• Create a Storage Account where the generated certificates can be stored
• Create a script that is executed on the host machine and copies the certificates to the right location
• Upload the certificates + PowerShell script to the storage account
• Use the PowerShell Custom Script Extension to inject the files from the storage account

To make it a bit convenient I created a script (UploadFiles.ps1 in the Git Repo) for this as well, that looks like this (gfragment)

Write-Host "Create Storage Account"
$storageAccount = New-AzureRmStorageAccount -ResourceGroupName 
$ResourceGroupName -Name $StorageAccountName 

-Type Standard_LRS -Location $Location 

Write-Host "Get Storage Account"
$storageAccount = Get-AzureRmStorageAccount -ResourceGroupName 
$ResourceGroupName -Name $StorageAccountName

Write-Host "Retrieve Key"
$key = Get-AzureRmStorageAccountKey -ResourceGroupName 
$ResourceGroupName -Name $StorageAccountName

Write-Host "Create Storage Container"
New-AzureStorageContainer -Name scripts -Context $StorageAccount.Context
Write-Host "Upload Certificates + Init Script"
Set-AzureStorageBlobContent -File ".\Helpers\Init.ps1" -Container 
scripts -BlobType Block -Context $storageAccount.Context

Set-AzureStorageBlobContent -File ".\certs\ca.pem" -Container 
scripts -BlobType Block -Context $storageAccount.Context

Set-AzureStorageBlobContent -File“.\certs\server-cert.pem”-Container

scripts-BlobType Block -Context $storageAccount.Context

Set-AzureStorageBlobContent -File“.\certs\server-key.pem”-Container

scripts –BlobType Block-Context$storageAccount.Context

Write-Host“Push into VM and execute Init.ps1 Script”

Set-AzureRmVMCustomScriptExtension –Name CSEDocker -Location$Location -containername

scripts -ResourceGroupName $ResourceGroupName -VMName $VMName -StorageAccountName

$StorageAccountName -FileName ‘ca.pem’,‘server-cert.pem’,‘server-key.pem’,‘Init.ps1’

-StorageAccountKey $key.Key1 -Run‘Init.ps1’
The script that will be executed on the host is in the “helpers” directory that you pulled from Github and looks like this

netsh advfirewall firewall add rule name="Open Port 80" dir=in 
action=allow protocol=TCP localport=80
netsh advfirewall firewall add rule name="Open Port 5000" dir=in 
action=allow protocol=TCP localport=5000
netsh advfirewall firewall add rule name="Open Port 6000" dir=in 
action=allow protocol=TCP localport=6000
netsh advfirewall firewall add rule name="Open Port 7000" dir=in 
action=allow protocol=TCP localport=7000
netsh advfirewall firewall add rule name="Open Port 7050" dir=in 
action=allow protocol=TCP localport=7050
netsh advfirewall firewall add rule name="Docker Secure Port" dir=in 
action=allow protocol=TCP localport=2376

Copy-Item "$PSScriptRoot\ca.pem" "C:\ProgramData\Docker\certs.d\"
Copy-Item "$PSScriptRoot\server-cert.pem" "C:\ProgramData\Docker\certs.d\"
Copy-Item "$PSScriptRoot\server-key.pem" "C:\ProgramData\Docker\certs.d\"

Restart-Service Docker 

what this script (helpers/init.ps1)  does:

• We open some ports on the Windows Docker host
• Copy the certificates to the right location for the docker daemon (c:\programdata\docker\certs.d\)
• Restart the docker service

Now you know the inner workings of the scripts. Just execute it

• Start a powershell window and navigate to your directory
• Make sure you have the powershell module AzureRM installed
  • install-module AzureRM
  • install-AzureRM
• Run the command Login-AzureRMAccount and login in to your account
• Execute the script UploadFiles.ps1 with the following parameters
  • -ResourceGroupName the name of the resource group from your Windows Docker Host machine
  • -VMName the name of your windows Docker host VM
  • -Location the location of your Windows Docker Host machine
  • -StorageAccountName the name of the storage accoount where the certficates will be uploaded to. This will be created in the same resource group (only lowercase no special characters!!)
• The certficates are now uploaded

Copy your certificates to the right folder on your local PC

Now the only thing that is left (but is crucial so do not forget), is to copy the generated certificates to the right folder on your PC. The docker certficates are located under the directory

%userprofile%\.docker or the full path c:\users\username\.docker

If there are certficates already, make sure you make a backup

Test the Docker Connection

Then the moment of truth. Test if it works.

• Open a Command prompt
• Check if the docker tools are installed by typing [docker]
• If so, type the following command

docker -–tlsverify  –H tcp://machinename.westeurope.cloudapp.azure.com:2376 images

• -H is the hostname, the public DNS name of your virtual machine and the port 2376
• –tlsverify ensures you use the certifcates
• As a shortcut you can set DOCKER_HOST to the tcp://dnsname:2376 in your environment variables. Then you can remove the –H switch

If you see the two windows server core images listed, you are good to go !

Get the sources from GitHub repo

References

• Docker Tools Extension in Visual Studio
• Docker tools documentation
• Docker Channel 9 videos
• Windows Containers
• Running Visual Studio Build Agent in a Linux Docker Container
• Get started with Docker on Azure for Microsoft developers and/or Linux noobs
• Deploy Docker on Windows
• Automating VM Customization tasks using Custom Script Extension
• Understanding Azure Custom Script Extension