Import Machines Keys – Visual Studio Team Services – Unit Tests in Build


In my previous post I talked about how to Encrypt an App.config file and export the machine keys needed to deploy the application to different machines and import them, all using our old friend aspnet_regiis.exe

This breaks my build

If you are using Visual Studio Team Services build definition package, and you run Unit Tests during the build which rely on using the encrypted credentials, they will fall over which an error similiar to this one:

System.Configuration.ConfigurationErrorsException: Failed to decrypt using provider ‘DataProtectionConfigurationProvider’. Error message from the provider: Key not valid for use in specified state.

This (as explained previously) is beause the machine keys wont be present on our Azure VM, exactly the same reason if you ran the application on a desktop that didnt have the keys imported.

The answer is……

The VSTS has a handy build step Batch Script, which allows you do run batch files as part of the build process., example here:

batchpng

What I did was create an area in the repository with a directory called encrypt, and leave my install_keys.bat file there. Then the first step I run is this script, which will then install the keys from the file (keys.xml) created previously.


GOTO EndComment
This batch file will install a pre-made RSA machine key file onto a machine.
Things to note:
MyCustomKeys is an container I have specified and you can specfify whatever you want hen you create & export the keys.
The applicaton is presuming that .NET 4 is installed. For other versions please see my post:
https://mywebanecdotes.com/2016/09/17/encrypting-credentials-in-app-config-for-multiple-machines/
:EndComment
%windir%\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis -pi MyCustomKeys keys.xml

view raw

install_keys.bat

hosted with ❤ by GitHub

My build order then in VSTS looks something like this:

buildstep

 

 

 

 

 

 

 

 

 

Yes should mean your Unit Tests can access and decrypt the sections in the app.config for the credential data.

Security Hole

The only issue with the multi-machine-to-one-RSAkey approach, is the keys.xml is left on the VSTS server. Now it is left in a private repository, but it is still somewhere. We cannot delete it, because we may need it for more machines in the future.

Apart from that, the beauty of this approach is you can deploy your application with encrypted app.config credentials to any machines, as long as the machine has had these RSA keys installed.

Encrypting Credentials in App.config for Multiple Machines


We should all care as developers about security and how we store and use sensitive data, to either connect to databases, login to domain accounts etc.

Today I’m going to talk about how to encrypt usernames and passwords that are stored and saved to via an applications app.config. This article will use a custom configuration section called EncryptUserCredentials. I wont discuss how I created that here, but here is a sample app.config showing it, please not:

  • service: key value to the record.
  • userName: username.
  • password: password.


<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="CustomConfig" type="garfbradaz.Common.CustomConfigSections.EncryptedUserCredentialsSection, garfbradaz.Common" />
</configSections>
<appSettings>
</appSettings>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0" />
</startup>
<CustomConfig>
<EncryptedUserCredentials>
<EncryptedUserCredential service="keyToCredential" userName="firstname.lastname" password="pwd12345@" />
</EncryptedUserCredentials>
</CustomConfig>
/configuration>

view raw

app.xml

hosted with ❤ by GitHub

I will not show you the implementation details and how you would access this in code, until another post. Today i will talk about how you can encrypt the EncyptedUserCredentials themselves, because at the moment they are plain text for all to see!

The way you accomplish this is using aspnet_regiis.exe, which all you ASP.NET web developers will know registers your web applications with IIS.

But wait, there are other functions this fine and dandy binary brings and that is encrypting sections in web.configs…..

But I’m using an App.Config silly.

Thats right, that doesnt matter. They are just config files to .NET, but with different names. So let me explain what you need to do, but before that, here is where aspnet_regiis is located on your Windows box:

Version of .NET Framework Location of Aspnet_regiis.exe file
.NET Framework version 1 %windir%\.NET\Framework\v1.0.3705
.NET Framework version 1.1 %windir%\Microsoft.NET\Framework\v1.1.4322
.NET Framework version 2.0, version 3.0, and version 3.5 (32-bit systems) %windir%\Microsoft.NET\Framework\v2.0.50727
.NET Framework version 2.0, version 3.0, and version 3.5 (64-bit systems) %windir%\Microsoft.NET\Framework64\v2.0.50727
.NET Framework version 4 (32-bit systems) %windir%\Microsoft.NET\Framework\v4.0.30319
.NET Framework version 4 (64-bit systems) %windir%\Microsoft.NET\Framework64\v4.0.30319

Before we move on, I must tell you we are focusing on a multi-machine configuration file encryption using RSA. If though your application is running on one machine only then you can use DPAPI and its provider DataProtectionConfigurationProvider. DPAPI is handled by Windows itself and uses specific machine keys and containers. These are not transferable to different machines. If you wanted to use the DPAPI method for a multi-machine scenario, aspnet_regiis would need to be run on a app.config on each machine it is deployed on.

Why is that a bad thing?

Simple, you would need to store a plain text app.config file as either part of the Continous Integration process or someone would need to manually keep a copy and run it on each machine or even include the plain copy in the installer if that was your method for deploying. This just adds a security weak  point. You could include scripts to delete the plain text files, if this is the route you wanted to go down. But just so you know, DPAPI exists and could be a better option for you.

RSA route

So aspnet_regiis allows you to create containers of asymmetric private/public keys and export them to other machines, allowing you one global config file to be used.

Step 0 – Preperation is (RSA) key

Yes yes, Step 0 exits because I got half way and forgot this step, thank the stars it was meant to be Step1! Add a configProtectedData section to your config with provider. Please note:

  • keyContainerName – should be the name of the RSA container you will create later.
  • name – Can be anything. Im naming mine MyEncryptionProvider.


<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="CustomConfig" type="garfbradaz.Common.CustomConfigSections.EncryptedUserCredentialsSection, garfbradaz.Common" />
</configSections>
<appSettings>
</appSettings>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0" />
</startup>
<CustomConfig>
<EncryptedUserCredentials>
<EncryptedUserCredential service="keyToCredential" userName="firstname.lastname" password="pwd12345@" />
</EncryptedUserCredentials>
</CustomConfig>
<configProtectedData>
<providers>
<add keyContainerName="MyCustomKeys"
useMachineContainer="true"
name="MyEncryptionProvider"
type="System.Configuration.RsaProtectedConfigurationProvider"/>
</providers>
</configProtectedData>
</configuration>

view raw

app.xml

hosted with ❤ by GitHub

Step 1 -Espionage….

Yes i said aspnet_regiis wont have a problem with an App.config – it wont, but first you need to rename/copy said App.config file to web.config.

copy app.config web.config

Step 2 – Rise and Serve

Create a public/private RSA key pair with a specfic container name. They should also be marked as exportable (otherwise what is the point!). MyCustomKeys can be anyname you desire.

aspnet_regiis.exe  -pc MyCustomKeys -exp

Step 3 – Let me in!

Grant permissions for accounts to access the container. Example here is the network service say IIS uses.

aspnet_regiis.exe  -pa MyCustomKeys "NT AUTHORITY\NETWORK SERVICE"

Step 4 – Encrypt and Protect

Now the magic happens. The following line will now encrypt your section (my EncryptedUserCredentials are wrapped in section CustomConfg). The -pef switch is telling the application to look for a web.config file and to use my provider I declared in Step 0 (which is using type RsaProtectedConfigurationProvider).

aspnet_regiis.exe  -pef CustomConfig . -prov MyEncryptionProvider

You web.config file should now have transformed. Gone is the CustomConfig section with plain text credentials, now there is a nice CyperValues. Please note mine below have been replaced with hard coded text, but you will see what i mean when you do yours. Also note your CustomConfig section now declares it uses a configProtectionProvider=MyEncryptionProvider.


<?xml version="1.0" encoding="utf-8"?>
<configuration>
<configSections>
<section name="CustomConfig" type="garfbradaz.Common.CustomConfigSections.EncryptedUserCredentialsSection, garfbradaz.Common" />
</configSections>
<appSettings>
</appSettings>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0" />
</startup>
<CustomConfig configProtectionProvider="MyEncryptionProvider">
<EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element&quot;
xmlns="http://www.w3.org/2001/04/xmlenc#"&gt;
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#tripledes-cbc&quot; />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#"&gt;
<EncryptedKey xmlns="http://www.w3.org/2001/04/xmlenc#"&gt;
<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5&quot; />
<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#"&gt;
<KeyName>Rsa Key</KeyName>
</KeyInfo>
<CipherData>
<CipherValue>longstringofdata</CipherValue>
</CipherData>
</EncryptedKey>
</KeyInfo>
<CipherData>
<CipherValue>anotherlongstringofdata</CipherValue>
</CipherData>
</EncryptedData>
</CustomConfig>
<configProtectedData>
<providers>
<add keyContainerName="MyCustomKeys"
useMachineContainer="true"
name="MyEncryptionProvider"
type="System.Configuration.RsaProtectedConfigurationProvider"/>
</providers>
</configProtectedData>
</configuration>

view raw

app.xml

hosted with ❤ by GitHub

Step 5 – Export those Keys

So now we have created our web.config file you can rename it to app.config and use this in your application. To use it on different machines though, you will need to export the keys from the machine that you created the encrypted web/app.config file and import them onto each machine. Firstly on your machine run the following which will create the key file for your container, including the private keys (-pri).

aspnet_regiis.exe -px MyCustomKeys keys.xml -pri

Step 5 – Import those Keys

Log into the machine(s) you wish your application to work on and run the following

aspnet_regiis -pi MyCustomKeys keys.xml

I would do this as part of your Release or Installation process making sure you delete the keys.xml file from the installed machines. The only place the keys.xml should be kept is in your code repository store but somewhere safe where it is restricted. This is the security issue for the RSA approach.

 

The full encrypt and export script can be found here. Amend it to include your custom container, section and provider names.


GOTO EndComment
This batch file will encrypt a app.config file using aspnet_regiis. You can use other windows directories to run
the application depending on which version you have installed.
The batch file assumed you have app.config file incuded in the direcorty you are running the batch file AND has been amended to
include the appropriate provided.
See this article on details on how to setup the config etc and what each command does
https://mywebanecdotes.com/2016/09/17/encrypting-credentials-in-app-config-for-multiple-machines/
The article describes on how to import the keys to different machnes as well.
:EndComment
cd %~dp0
copy app.saved.config web.config
%windir%\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe -pc MyCustomKeys -exp
%windir%\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe -pa MyCustomKeys "NT AUTHORITY\NETWORK SERVICE"
%windir%\Microsoft.NET\Framework64\v4.0.30319\aspnet_regiis.exe -pef CustomConfig . -prov MyEncryptionProvider
%windir%\Microsoft.NET\Framework64\v4.0.30319\v4.0.30319\aspnet_regiis.exe -px CustomKeys keys.xml -pri
pause

view raw

encrypt.bat

hosted with ❤ by GitHub

 

  1. https://msdn.microsoft.com/en-us/library/k6h9cz8h.aspx

How to….Item Templates in Visual Studio


For some time now I have wanted to find out if it was possible to create a C# Template with the formatting I use when writing code. For example I always add regions that I break down into:

  • Fields
  • Properties
  • Constructors
  • Methods

After posting a Stack Overflow question which gained no reply, I went on C-Sharp Group and got inspiration from a lovely chap Juri, His answer was correct but there was a better way of doing it using the Export Template function in Visual Studio I talk about in the above youtube video.

I hope this helps you.

Clarizen.net – Rethink & Refactor


This slideshow requires JavaScript.

Rest Easy.

I have reflected today on what I need to do first and I really do need to ship the Clarizen.net project out properly, so I have focused the areas that are needed to make the product whole (image below) and the main part is getting the HttpClient done properly, so I have decided to to create RESTIsEasy HTTP Client.

version 0.0.1 – The following areas are being implemented (All the following will feed in to Microsofts HttpWebRequest):

  • PUT, DELETE, GET and POSTClarizen_Net_Structure
  • HTTP Settings
  • Authentication
  • HTTP Request
  • HTTP Response (Json only at the moment)
  • HTTP Error Handling

Once this is in then moving onto the Query Engine which will pass the CZQL-JSON to RESTIsEasy HttpClient. This will mean a major refactoring of the current implementation, but it will be worth it.

Clarizen2Trello – – Beta v1.1.0.3 Released


Another Post, Another Beta…..

As detailed in another post, I submitted Clarizen2Trello as my Appathon 2015 submission. Today I have released an update, which allows for:

  1. Update to show more feed back to the user while the application is running.
  2. Input for Clarizen credentials.
  3. Input for the Trello Board ID that the app is meant to insert the Clarizen Tasks.
  4. Input for the maximum amount of Tasks to pull down from your Clarizen Implementation.

The Release can be found here:

https://github.com/garfbradaz/clarizen2trello/releases/tag/v1.1.0.3-beta

What Next?

I need now to concentrate on Clarizen.NET and getting that ready for beta for people to use. I have hit a crossroad where I cannot add more features to Clarizen2Trello (and PlainClarizen), until this API wrapper is completed. Plus I want the API ready for use for Windows 10 UWP so time is of essence! More on Clarizen.NET soon.

PlainClarizen – Beta 2.0


As per my previous post, I released PlainClarizen for Clarizen Developers. My plan is to finish all my Clarizen developments ready for Windows 10 release on the 29th July, so I can concentrate on learning Windows 10 UWP.

With that in mind, please let me introduce PlainClarizen Beta 2.0:

https://github.com/garfbradaz/PlainClarizen/releases/tag/v0.2.0-beta

This release includes:

  1. Ability to enter Clarizen Username/Password.
  2. Now can request all or a portion of Entities to save on your daily calls.
  3. Turn off creating Custom Field properties in your POCO’s so we are just creating the core system
  4. Integrated new version of Clarizen.Net.

Enjoy.

Open Source Clarizen Projects


Pull,Push,Pull,

Push,Release…..

Since starting (and finishing) my Clarizen Appathon application, I have decided to carry on and release them so anyone can use them – i.e. fix and release the brute code of the Clarizen.NET wrapper. Today I have move forward with that goal by completing the console version of my tool PlainClarizen, which will help me rapidly implement the required entities for Clarizen.Net quickly. So here are the links if anyone would like to follow the repos.

  1. Clarizen.net: Wrapper for the RESTful API – this is nowhere near finished and I wanted to just get some simple methods in place to pull down and query tasks. There is a simple fluent CZQL creator as well. The coming weeks this will gather pace, so if you are interested watch the repo.
  2. PlainClarizen: Simple application that pulls down a list of available Clarizen Entities and serializes them into POCO C# class files. I started building this so I easily create strongly typed objects for the Clarizen.NET wrapper.
  3. Clarizen2Trello: My Appathon submission. Once the Clarizen.Net is in Beta, my next task is to make this a good and robust application.

All Clarizen, and no play……

With Windows 10 looming, I’m really only going to work on this for the next month, desperately trying to ship Clarizen.Net out so other developers can start hacking and using it. Reason: Working on a Raspberry Pi 2 IoT App in August for Windows 10……

Clarizen Appathon 2015 – Clarizen.NET & Clarizen2Trello


clarizen-appathon

Finally finished it.

It has been a busy 2 months personally, with my wife’s 30th birthday and her various parties, upping running training to start tackling Marathons (and hopefully at the back end of 2016, Ultra 50ths), plus all my myriad of coding projects, including Windows 10 development and Windows 10 Internet of Things (IoT) for a Raspberry Pi App – more on that soon.

One of my coding projects is an App for the Clarizen Appathon. For anyone who does not know, Clarizen is a Project Management collaboration and tool, and honestly I’m a big fan. Clarizen comes with a a very good Custom Engine for building:

  • Custom Fields
  • Custom Work flows
  • Custom Visualisations using HTML/JS (Called Panels)
  • Custom Business Logic
  • Not really part of the Customisation Engine but a robust RESTful API

This is very good, because there are some areas which Clarizen struggles with, one being their Agile/Scrum implementation. There are various bespoke add-ons for this area but they are not very good.

Which is where I come in.

I decided to try and create an interface to my favourite (free, as Clarizen is an Enterprise implementation) tool outside of work – Trello. This project grew larger than I expected, mainly due to the fact that I also decided to write a Clarizen .NET code library to wrap around their RESTful services. So that is what I did. Because I only wanted to pull down Tasks from Clarizen, I concentrated on the necessary actions from the list to accomplish this:

* Authentication / Session ID

*  Metadata (DescribeEntities)

* Query and a simple Clarizen Query Language (CZQL) implementation.

I have of course used great frameworks like Newtstonsofts JSON.NET and json2csharp to create my POCO’s and built a brute-force alpha-stage API. I enjoyed doing this very much and plan in the following month to get these issues completed:

issues_clarizendotnet

So once I have the bare basics of this done, I moved onto the quick and easy Clarizen2Trello console application, that pulls down tasks and updates Trello (Tasks as cards within various Todo/Doing/Done lists). here is a snippet:

client.Data.Query.Select("Name,Work,CreatedOn,ActualCost,RemainingEffort,DueDate") 
                 .From("Task")
                 .Where("StartDate > 2015-06-01");
            client.ExecuteQuery();

            foreach (Bradaz.Clarizen.API.Models.Task t in client.Tasks)
            {

                if (t.RemainingEffort.Value == t.Work.Value)
                {
                    Card newCardTodo = trello.Cards.Add(new NewCard(t.Name, todo));
                    trello.Cards.ChangeDueDate(newCardTodo, t.DueDate);
                    trello.Cards.AddComment(newCardTodo, "Remaining Effort is : " + t.RemainingEffort.Value);
                    trello.Cards.ChangeDescription(newCardTodo, t.Id);
                }

                if (t.RemainingEffort.Value == 0)
                {
                    Card newCardDone = trello.Cards.Add(new NewCard(t.Name, done));
                    trello.Cards.ChangeDueDate(newCardDone, t.DueDate);
                    trello.Cards.AddComment(newCardDone, "Remaining Effort is : " + t.RemainingEffort.Value);
                    trello.Cards.ChangeDescription(newCardDone, t.Id);
                }
                else
                {
                    Card newCardDoing = trello.Cards.Add(new NewCard(t.Name, doing));
                    trello.Cards.ChangeDueDate(newCardDoing, t.DueDate);
                    trello.Cards.AddComment(newCardDoing, "Remaining Effort is : " + t.RemainingEffort.Value);
                    trello.Cards.ChangeDescription(newCardDoing, t.Id);
                }




            }

Easy! Its the Trello.Net and Clarizen.Net API’s that are doing the grunt. Here is a quick video explaining how it works:

Code can be found here:

A little bit of Clarity.

That isn’t it by a long stretch, I want to really flesh out the Clarizen.NET code so it can be used by other developers, plus I’m also working on a Chrome Extension (Clarity) that enchances the Configuration page within Clarizen itself, and I may even try and convert Clarizen.NET to Javascript AND Windows 10 UWP and create a Windows 10 Clarizen Client (for Desktop, Phone and possibily Hololens – haver some ideas on 3D data visualisation).

Watch this space.

Fresh new Theme, Fresh New Direction


making sweet code.

I have had the old Theme for this blog since its inception a long time ago, and it has not aged well. I have updated with a fresh new look which hopefully reflects my own life at the moment and the direction it is going, which is good. I’m currently extremely busy creating my own site (quick Single Page Application using Knockout.js), learning Windows 10 UWP development, especially for the Raspberry Pi and the IoT branch, plus a number of small projects (like the Clarizen Appathon), so alot going on,

If anyone is interested, my projects are hosted on my Trello Page.

Source of Cover Image.
Related articles