Weather you’re using AWS Lambda, Azure Functions, or Google Cloud’s Firebase, you’ll want to re-think how you approach configuration management for serverless projects. The number of projects tend to grow fast because a microservice architecture is most commonly used. This introduces a new set of configuration management problems. Manual tasks to create and manage these projects grows exponentially. Without proper configuration management these projects can quickly spiral into the wild wild west. I’d like to review the principals of SCM and share some solutions for accomplishing these goals when working with serverless/microservice projects.
The majority of developers spend most of their time in Visual Studio and don’t have to understand what happens under the hood with MSBuild. Click the play button and magic happens! However, you may need to troubleshoot a failing build or extend the build to automate things. For those scenarios I’d like to provide a quick intro to what MSBuild is made up of and some tips on troubleshooting. Note: I’ll be using some of the code snippets from the Microsoft docs.
The trace-ability features when linking between Octopus and TFS/VSTS work pretty well out of the box, especially when using the Octopus Deploy build extension. When Releases are created by a build, there is a link in the Release description that points back to the build that created it. There is also a section in the build results that points to the Octopus release that was created. This met most of my needs, but to take it a step further and reference builds/files/artifacts during deploy time, I decided to use the Octopus Deploy API. I wanted to update Octopus variables as part of the build process before a Release is created so that deploy steps could link back to builds/files/artifacts that were versioned during build.
Earlier this year I decided to create some Visual Studio Marketplace build extensions. I was re-using the same tasks in multiple build definitions and there wasn’t anything published to the marketplace that accomplished what I needed. The first extension I created is used to upload files to AWS S3; this was the very first AWS related extension in the marketplace which was kind of cool. It took me longer than I expected because of all the ceremony in setting up the metadata and manifest json files, and packaging with tfx-cli. More details on creating a build task here: https://www.visualstudio.com/en-us/docs/integrate/extensions/develop/add-build-task Creating my second extension went a lot faster since I could re-use and modify most of the files. Now that these extensions were published, I was interested to see metrics on them.
On March 29th, 2017 the VSTS team released a new feature that added the ability to specify conditions for running a build task. This provides more control over your build tasks, for things such as a task to clean things up (even if a build fails), or send a message when something goes wrong, under conditions such as certain branches or with certain triggers. Custom Conditions appear in the new build definition editor under Control Options. Custom conditions are not currently available for Task Groups because they still use the old build definition editor (this is on the backlog).
Last month I migrated our TFS collection to VSTS using Microsoft’s Database Import Service and migration guide. To be frank, it was a long process and it took a lot of going back and forth to make sure I fully understood the guide which is a PDF that is 58 pages long. The guide comes with several checklists and things you need to check and prep before your migration.
Update: Unlimited private pipelines are now free for Build!!
Microsoft caused a lot of confusion when they announced that you will need to pay to run private build agents, since this was free in TFS. Why should we have to pay to use our own server resources to run a build agent?!? You can see the outcry in the reviews on the Marketplace: 1.7/5 stars: https://marketplace.visualstudio.com/items?itemName=ms.build-release-private-pipelines#review-details
Once I fully understood how and why the billing is setup this way, I wasn’t as frustrated.
When I first got started with Octopus Deploy I would setup projects to have one long deploy process that ran steps in serial. Each step would wait until finished before another one would start. Deploy times got longer and longer, so I set out to find a way to cut down deploy times.
This last year our team made the decision to move from Team Foundation Version Control (TFVC) to Git.
Currently the TFS/VSTS build system has a pretty big bottleneck: tasks run in serial. For build steps that run PowerShell, I’ve implemented runspaces to run processes in parallel (Invoke-Parallel is amazing). However, that only works for one build step. The rest have to wait until it’s finished.
Enter Queue Build(s) Task. This extension sounds pretty simple, right? Queue a build as part of a build…. But it has potential to do much more. It has the ability to use configurations (JSON) to pass variables from the build that’s queuing other builds, including system variables (source branch name, build id, build name, etc.).