Setting up build automation for .Net open source projects
In the last weeks I worked on a .net client library for SDL Studio GroupShare. This new client library is open source and meant to simplify developers work in terms of consuming resources available in GroupShare, but more about this on a future article. As I was coming closer to make a new release of this library I was thinking that it would make perfect sense to automate the build. There are many reasons to do that but for me the most important one was to have a painless mechanism to verify contribution from other developers. I've setup build automation for a few projects but this was my first attempt to do it for an open source project. While the automation concepts are exactly the same as for any other project the tooling involved is a bit different, especially if you are coming from a Microsoft world.
Cloud hosted building automation for .NET
Open source means that your entire source code is publicly available to everyone interested, if it's not then what's the point of having it open source? Being public it probably means that you are hosting your code in places like Github, honestly nowadays this is probably your best choice. My choice to host the code for the GroupShare client library was Github and based on my previous experience my first idea to setup build automation was to prepare a server somewhere that would be able to grab my code and run the build script. While I could have setup a server pretty easy I immediately step backed since I was having the feeling that this approach doesn't fit the purpose, somehow this doesn't fit on the open source world. The reason I was having this feeling is because somebody (me) had to invest time, effort and some money to keep the server run. Also there was no simple option to share this with the other developers interested in using or contributing to the library. So I started doing some research and I found out there are a bunch of cloud hosted services that provide build automation and each of them was integrating nicely with Github. The list is pretty long so I'll share just a few that I found most interesting and popular among other developers: Travis CI, Codeship and Drone.io. Awesome, just that none of them had support for Windows environments and my library was .net and I was not using either Mono or .Net core. So in my mind I was thinking that I should turn my attention to what Microsoft has to offer in terms of build automation since I'm pretty sure they'll support Windows environments. I knew Microsoft had TFS Build as part of their on-premise enterprise solution and I also knew about their relatively new visual studio online offering which should include also build automation. Visual studio online does offer this capability and can also be configured to grab the code from github, more about his here. Awesome again, just that this feature is not completely free and unfortunately the pricing model is not simple and friendly to understand. So again I had to continue looking for other options and I finally found AppVeyor which is completely free for open source projects and it's supporting Windows environments, actually they market themselves as ... Continuous Integration and Deployment tools to every .NET developer .... It's pretty obviously that my choice was Visual studio online because ... errr not. I choose AppVeyor because of obvious reasons, but it's a shame that Microsoft doesn't have a simple free offering for open source projects given the recent strong movement towards open source.
Build scripting
At this point I had my source code on Github and choose AppVeyor to run my builds. Linking them together was easy since AppVeyor allows you to log in with your Github credentials and once you authorize AppVeyor it's just a matter of choosing the repository for which you want to build. At this point I highly recommend looking to the AppVeyor documentation if you are not familiar with the platform.
After the linking was done AppVeyor requires that you repository to have appveyor.yml file from which you can configure which build environment, deploy mechanism and build scripts to run. Since in my case the requirements where simple I pretty much left the default configuration. I only specified my build script:
As you can see from my above script I basically run my cmd Windows batch script specifying two different targets one for building my library and one for run a few integration tests. The batch script is a bit more complicated at the first look but is really simple actually. The first step is to use Nuget to install to additional libraries that I need to run the build. First is Fake, more about this in a bit, and the XUnit test runner console that I need to run my integration tests since I'm using XUnit for testing instead of MS Tests.
Creating build automation script is not something that the .Net developer has to do and this is because the wonderful job done for him by Visual Studio. That is slowly changing because of the CI practice. To build a .net project/solution you need run msbuild (this is what Visual Studio does behind the scenes). MSBuild is extensible and you can alter the build process based on your need but the syntax is pretty complicated and verbose (completely written in xml), personally I don't necessarily like it. Apparently I was not the only who didn't like working directly with msbuild so the community created libraries to simplify the build process. This type of libraries are using a domain specific language based on a well know programming, like rake using ruby or grunt/gulp using java-script. Any of this libraries can be used to create a .net build script since they have dedicated plugins from msbuild but you'll have to add a new dependency on your build system. For rake you will have to install ruby and for grunt/gulp you will need nodejs. Based on the same concept I found 2 other libraries based on .Net programming language which didn't require additional dependencies. One is named fake using F# and sake using C#. Initially I was tempted to use sake because I feel much comfortable writing in C# and I've seen it's used by Microsoft in their open source repositories but it's not well documented so I decided to use fake. I'm not an F# developer so the syntax seems kind of weird to me but it doesn't matter that much since you only need some basics and you can of course copy from other examples. For the GroupShare client library I ended up with this:
Looks a bit scary at first but just walk through it line by line and it will start make sense, especially if you've done some work with msbuild. First 23 lines are just setting a bunch of values and then defining some msbuild targets using fake dsl language. Name of the targets are meaningful for what they are doing.
Handling sensitive data
In an open source project all your information has to be public including your configuration settings, like api keys, user name or passwords. For example your application might use a mailing platform like mailigun for which you have your own account which you don't want to be shared with everyone. In a private project you don't need to worry to much about this because typically this type of information end up in configuration file that can be accessed only by authorized people. Also you might run some integration tests as part of your build for which you might need some api key's or credentials. The solution I found for handling sensitive data is to use environment variables. This are easy to configure on your machine, testing machine(s) and all of the build platforms allow configuring environment variables from and administrative panel.
Conclusion
I hope this will give you a much clearer picture about options you have in terms of cloud hosted build platforms you can use for your project, especially open source. One thing I have to mention that each of the build platforms have offerings for non open source projects but for those you'll have to pay.
Please let me know if you have any comments or questions.