DevOps Days Hackathon Toronto

One day DevOps Hackathon event is coming to Toronto on May 25th. It's a full day event where you will get a chance to chance for some hands-on learning experience with DevOps concepts and automation using tools from Microsoft & Chef. Here is an agenda for hackathon event:

Agenda:

9:00 AM – 9:30 AM Registration, Breakfast & Introduction

9:30 AM – 10:00 AM Session I – DevOps and the Cloud: Microsoft Azure

10:00 AM – 10:45 AM Session II – Chef + Microsoft Azure, Managing your cloud consistently, securely and transparently

11:00 AM – 12:00 PM Hackathon Begins

12:00 PM – 1:00 PM Working Lunch

1:00 PM – 3:00 PM Hackathon Continues

3:00 PM – 4:45 PM Hackathon Ends & Group Presentations (5 min/group)

4:45 PM – 5:00 PM Closing Remarks & Winners Announced

 

If you are interested in participating, fill free to register at https://www.eventbrite.com/e/devops-days-toronto-2016-tickets-17038721274?access=DEVOPSTOHACKATHON. This event will take place a day before DevOps Days Toronto kicks off. It's an excellent event, which is unfortunately already all booked, but hackathon event has only recently opened up and promises to be very informative. So, go ahead and register. It will be fun… See you there.

 

TFS chart size limits

TFS is an excellent ALM/DevOps tool. Very flexible ALM/DevOps tool. You can do a lot with it. A lot. Once in a while though, like with any other tool, you hit a hurdle that prevents you from doing what you and/or the customer is trying to do with TFS. For example, recently I encountered the following error: VS402395: The daily number of work items returned exceeds the trend chart size limit of 1000. An error was caused because the customer was trying to build a chart from a query that returns more than 1,000 work items. I questioned why would they need a chart for a query that big, and the customer seemed to have a good reason to have chart for that many items. I might not agree with that reason, but the customer thought it was important, so I've promised to look into it.

And, of course, if there is a will, there is a way. Especially, if you're dealing with a flexible and feature rich TFS. TFS has a very rich API. And, almost always, if there is something that you cannot do in UI, you can do via API. Just like in this case. Even though, there was no way to by part chart size limit, there is a very straightforward to get it done using API. OK, enough talk. Here is how you can change chart size limit using PowerShell and TFS API:

 

# Load TFS snapin

if ( (Get-PSSnapin -Name Microsoft.TeamFoundation.PowerShell -ErrorAction SilentlyContinue) -eq $null )

{

Add-PSSnapin Microsoft.TeamFoundation.PowerShell

}

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Common")

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.WorkItemTracking.Client")

 

$tfsCollection = New-Object -TypeName Microsoft.TeamFoundation.Client.TfsTeamProjectCollection TFS_COLLECTION_URL

$hiveCollection = $tfsCollection.GetService([Microsoft.TeamFoundation.Framework.Client.ITeamFoundationRegistry])

 

Write-Output "The current value of chart size limit is:"

$hiveCollection.GetValue("/Service/WorkItemTracking/Settings/MaxTrendChartTimeSliceResultSize")

Write-Output "Changing the chart size limit to 1500 (maximum)…"

$hiveCollection.SetValue("/Service/WorkItemTracking/Settings/MaxTrendChartTimeSliceResultSize", 1500)

Write-Output "The new setting for a chart size limit is:"

$hiveCollection.GetValue("/Service/WorkItemTracking/Settings/MaxTrendChartTimeSliceResultSize")

 

 

That's it. Have a good day J

Excluding weekends in Azure Automation Runbooks

Getting Azure Automation runbooks to shut down your virtual machines (or turn them on) automatically is not new. There are a lot of scripts out there that could do it for you. You can also write one yourself. It's not that complicated. I did it J Just kidding…

There are a couple of ways my PowerShell scripts are different:

  1. First, the scripts that automatically start/stop Azure virtual machines take the weekend into account. Scripts will not turn your machines on or off on the weekends. After all, you probably do not want to automatically turn on your virtual machines in Azure early in the morning on the weekend, just so that you can turn them off at the end of the day on the weekend. Seems like a waste, right? Anyways, this small change should save you a few bucks.
  2. Second, the scripts adjust the schedule from UTC to the time zone you need. It looks like when the scripts that are part of Azure Automation runbooks run, they use UTC time. So, if you're in Toronto, script will think that the weekend starts 5 hours earlier. It's not bad, I guess. But, it also means that the weekend will end 5 hours earlier, and that just not right and need to be fixed.

Below is a code snippet that makes the above mentioned happen:

$UTCTime = (Get-Date).ToUniversalTime()

$TZ = [System.TimeZoneInfo]::FindSystemTimeZoneById("Eastern Standard Time")

$LocalTime = [System.TimeZoneInfo]::ConvertTimeFromUtc($UTCTime, $TZ)

$day = $LocalTime.DayOfWeek

if ($day -eq 'Saturday' -or $day -eq 'Sunday')

{

Write-Output ("It is " + $day + ". Cannot use a runbook to start VMs on a weekend.")

Exit

}

 

The complete scripts that start or stop Azure virtual machines can be downloaded from OneDrive. Enjoy.

Accessing Git from Behind the Proxy

So, you have installed Git client and trying to connect to Git server (on Visual Studio Team Services, Github, or whatever), but you're getting "fatal: unable to connect a socket (Invalid argument)" error. One of the reasons could is that you're behind the proxy. For example, you're at work and your employer requires all internet traffic to go through the proxy. ~/.gitconfig global config file is the key here. In this case, to get Git client to work with the proxy, you need to configure http.proxy key in git config using one of the following commands:

git config --global http.proxy http://proxyuser:proxypwd@proxy.server.com:8080

or

git config --global https.proxy https://proxyuser:proxypwd@proxy.server.com:8080

  • change proxyuser to your proxy user
  • change proxypwd to your proxy password
  • change proxy.server.com to the URL of your proxy server.
  • change 8080 to the proxy port configured on your proxy server

If you do not need to authenticate to proxy, then just specify proxy server name and port number and skip proxy user and password.

 

If you decide at any time to reset this proxy and work without (no proxy), use one of the the following commands:

git config --global --unset http.proxy

or

git config --global --unset https.proxy

 

Finally, to check the currently set proxy, use one of the following commands:

git config --global --get http.proxy

or

git config --global --get https.proxy

 

By the way, to retrieve the proxy settings you're using, you can use one of the following commands:

reg query "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings" | find /i "proxyserver"

or

netsh winhttp show proxy

That's all I got to say about Git and proxy server.

tf.exe (Team Explorer) vs. tf.exe (Team Explorer Everywhere)

I recently had an interesting experiencing writing post build PowerShell script for a client. The client wanted to check in certain files into source control after the build is finished. Sounds easy, right? You can use either good old tf.exe command line utility from Visual Studio command tools. Or, you can use something more current like PowerShell to write a simple script that will check in pending changes for you. The problem is that the client also wanted to associate work items with the check in. Not a big deal, right? Well, apparently it is a big deal. You cannot associate work item with the check in using tf.exe command tool. And, what's even stranger, I could not find a way to associate work item with the check in using PowerShell. I got stuck with figuring out how to make WorkItemCheckinInfo[] parameter in Workspace.Checkin method to work properly.

This is how I learned that apparently you can associate work item with TFS check in, but you have to use tf.exe command from Team Explorer Everywhere. Apparently, even though the names are the same, those are very different command line utilities. When you use tf.exe from Team Explorer Everywhere, you can associate work item with the check in using a simple command:

tf checkin ItemSpec -associate:WorkItemIds

It's that easy. I just wish –associate option was available in common tf.exe command from Visual Studio command tools. I would also wish that those two seemingly identical tf.exe commands would actually do the same thing (the same way), or at least that those commands would have different names to avoid the confusion. By the way, there are also other differences between those two commands with the same names. You can get them form the links provided in the post. I'm too upset to list myself L

Toronto Enterprise DevOps User Group

I have started a new user group with the focus on Enterprise DevOps. DevOps is getting significant attention in the industry. Many organizations don't understand what DevOps is, how to adopt DevOps practices effectively within the organization, and are not aware of what DevOps tools to use. Toronto Enterprise DevOps User Group is focused on applying DevOps practices in the enterprise environment. This group is for people in the Greater Toronto Area who are interested in continuous deployment/integration, release management, infrastructure as code, change/configuration management, load testing & auto-scale, performance/availability monitoring, capacity management, automated testing, automated environment provisioning/de-provisioning, self service environments, automated recovery (rollback & roll-Forward), and many more of constantly evolving DevOps practices. Every level of experience is welcome, all we ask is that you come with an open mind and are excited to share your knowledge.

The first meeting is on September 10th, 2015. We'll start with a discussion of What is DevOps? DevOps is a term for a group of concepts that, while not all new, has catalyzed into a movement and is rapidly spreading throughout the technical community. Like any new and popular term, people have somewhat confused and sometimes contradictory impressions of what it is. Is it "Quality" or "Agile,"? Well, DevOps is a large enough concept that it requires some nuance to fully understand. DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through to the development process to production support. We will cover what DevOps is and is not during our first user group meeting.

Visit our website for more info. See you there.

OpsHub Visual Studio Online Migration Utility

I recently got to use OpsHub Visual Studio Online Migration Utility to help the client move from on premises TFS environment into the awesomeness of Visual Studio Online. OpsHub Visual Studio Online Migration Utility is actually pretty good and solid tool. The migration was very smooth and relatively painless. I thought I share some of the things I have come across when using VSO using OpsHub Visual Studio Online Migration Utility:

  • OpsHub Visual Studio Online Migration Utility is free and can be downloaded from Visual Studio Gallery here (http://aka.ms/OpsHubVSOMigrationUtility)
  • OpsHub Visual Studio Online Migration Utility can only be installed on 64 bit Windows machine
  • During the installation, you will be required to fill out the registration info. Please make sure that you have provided a valid email address since you will need a verification code to proceed with the installation that will be sent to the email address you provide during the install. Usually, it takes a couple of minutes for an email to come through. Check your Spam folder in case if the email was mislabeled by your spam filter
  • If you are using proxy servers to connect to the internet, you will need to do the following http://opshub.com/main/index.php/ovsomu-proxy to get the installer working. Please note that even if you follow the instructions, and even if you get passed the registration page to next page (Verify Email page), which to a normal person mean that the registration process was successful and that you should receive verification email, it does not necessarily mean that you will get an email. The reason for it could be that your proxy is blocking the installation wizard from sending the registration info to Opshub. Very frustrating. As a workaround, you can run the installation wizard on any other machine that is not going through the proxy, fill out the registration email, wait for an email, then use the verification code that you have received to install OpsHub Visual Studio Online Migration Utility on any other machine in your network. From what I can tell, verification code is not tied to machine that you're installing on in any way
  • If you have installed OpsHub Visual Studio Online Migration Utility while your machine was going through the proxy to get to the internet, and then managed to convinced your IT to allow you to bypass proxy, you will need to uninstall OpsHub Visual Studio Online Migration Utility, remove any changes you've made as per http://opshub.com/main/index.php/ovsomu-proxy and then reinstall OpsHub Visual Studio Online Migration Utility. Sounds silly, but it's true. By the way, I strongly recommend that you spent some time and convince your IT to allow the machine that is running OpsHub Visual Studio Online Migration Utility to bypass the proxy, it just makes things a lot easier. Especially when you start configuring the migrations.
  • Before you start configuring migrations, you will need to:
    • Pre-create empty team projects (with matching process templates) for the team projects that you will be migrating from on premises TFS
    • Add users to your Visual Studio Online account and grant them some permissions on the team projects. One of the steps in configuring the migrations is map local users to VSO users, so you'll need users to be in place before you start the migration
    • Add VSO account that you will be using to migrate to Project Collection Service Accounts group in VSO
  • Remember that you can migrate one project a time. You don't have to move the entire project collection from on premises to VSO
  • If you're migrating work items then you might have to deal with template discrepancies. Be patient.
  • OpsHub Visual Studio Online Migration Utility is provided by a company called OpsHub, Microsoft partner. Their support is pretty good. You can reach them via email ovsmy@opshub.com or via StackOverflow using hashtags #opshub and #visual-studio-online. Please keep in mind that the company is located in California, USA, and take into account the time difference when awaiting a response.

That's all.

Schedule TFS releases

I have been asked if TFS 2013 Release Management allows you to schedule TFS releases. Yes, you can schedule TFS release. What I mean is TFS allows you to schedule the deployment time of the release during the acceptance step of the release path. It's that easy.

But what if you want to schedule release to happen on a regular basis, for example you would like to automatically deploy/release the latest code to the development environment every Monday/Wednesday/Friday nights. TFS 2013 Release Management does not really have that feature. Luckily, we have reach TFS API and PowerShell. So, here is a PowerShell script that triggers release of the last successful build with the build quality set to "Ready for Deployment" (obviously, you can use any other filter to get the build you want):

param

(

    [string] $tfsCollectionPath = "http://SERVERNAME:8080/tfs/COLLECTIONNAME",

    [string] $tfsProjectName = "PROJECTNAME",

    [string] $buildDefinitionName = "BUILDDEFINITIONNAME",

    [string] $releaseTemplate = "RELEASETEMPLATENAME"

)

 

# Clear Output Pane

clear

 

# Enforce coding rules

Set-StrictMode -version 2.0

 

# Loads Windows PowerShell snap-in if not already loaded

if ( (Get-PSSnapin -Name Microsoft.TeamFoundation.PowerShell -ErrorAction SilentlyContinue) -eq $null )

{

    Add-PSSnapin Microsoft.TeamFoundation.PowerShell

}

 

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Common")  

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")  

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Common")  

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client")  

 

[Microsoft.TeamFoundation.Client.TfsTeamProjectCollection] $tfs = get-tfsserver $tfsCollectionPath

 

$tfsCollection = New-Object -TypeName Microsoft.TeamFoundation.Client.TfsTeamProjectCollection -ArgumentList $tfsCollectionPath

 

$server = new-object Microsoft.TeamFoundation.Client.TfsTeamProjectCollection(New-Object Uri($tfsCollectionPath))

$buildServer = $server.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])

$buildDetail = $buildServer.QueryBuilds($tfsProjectName, $buildDefinitionName)

 

$foundBuilds = $buildDetail | where {($_.Quality -eq "Ready for Deployment") -and ($_.Status -eq "Succeeded")} #| sort $_.BuildNumber

 

if ($foundBuilds -eq $null) 

{

    Write-Host "No builds found"

}

else

{

#$foundBuilds = $build | select BuildNumber, SourceGetVersion, Quality, DropLocation

 

    $LastReadyForDeploymentBuildNumber = $foundBuilds[-1].BuildNumber

    $LastReadyForDeploymentBuildDropLocation = $foundBuilds[-1].DropLocation

    Write-Host "Triggering build " $LastReadyForDeploymentBuild

 

    &"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementBuild.exe" release -rt $releaseTemplate -pl $LastReadyForDeploymentBuildDropLocation

}

 

As you can see from the script, script first connects to TFS, finds the proper build definition, then looks for last successful and "Ready for Deployment" build, then uses ReleaseManagementBuild.exe to trigger the release for that build. Obviously, the script could be improved, for example, to include the exception handling, but it should enough to get you started.

Now that you have the script, you can use Windows Task Scheduler to trigger the release outside of the Release Management client. As often as you need it, whenever you feel like it.

TFS/Visual Studio Online API

As I was looking for API reference for TFS Release Management, I came across this useful link that I wanted to pass along. Visual Studio Online REST API Reference page (https://www.visualstudio.com/en-us/integrate/api/overview) includes API reference for TFS components like build, work item tracking, version control/GIT, load test, test management, shared services, etc. Basically, it includes reference for all existing API hooks for TFS/VSO.

I have also found this blog post (http://blogs.msdn.com/b/aseemb/archive/2014/12/23/how-to-program-against-release-management.aspx), which includes the API reference for TFS release management. It includes sample code that allows you to create a user, environment, release path, release definition, etc. Please note that release management (like build) is changing quite a bit in TFS 2015, so it's safe to assume that the API for build and release management will also be changing. Visual Studio Online REST API Reference page should have the latest changes, so just use it as a reference :)

If you haven't already seen it, look at the information on getting started with these APIs.

DevOps Self Assessment

Microsoft has released a DevOps self-assessment online tool that allows you to gauge your readiness in the 7 key DevOps practice areas. Complete the form below to step through the assessment and see your results including guidance on which areas to focus on next. Total time commitment is approximately 10 minutes. DevOps self-assessment tool can be found at http://devopsassessment.azurewebsites.net/