How to run tests cross domain with TFS 2010 from the “cloud”

If you need to run load tests (or any other tests) against your web application, the most efficient way to do it is to use "the cloud": Amazon Elastic Cloud or Windows Azure servers. Why? Two simple reasons:

  1. It's easy spin up as many servers as you need, which make it easier to scale up or down your test rig environment
  2. It's external to your environment, which is necessary to make your tests closer to a real life scenario

I won't bore you with details on how to set up your TFS environment and all. We're all adults; we know how to read install guides. I will only cover the things are not in any of those guides.

Server instance setup in Amazon EC2 (applicable to Windows Azure as well):

  • Pick large instance type or better. Small instance just is not powerful enough.
  • Use Windows Server 2008 operating system as host. Amazon EC2 does not support Windows Server 2008 R2 yet, and there is no good reason to still use Windows Server 2003 operating system.
  • Use an instance with SQL Server 2008 Express pre-installed to save time, unless you think your databases will be larger than 4Gb (very unlikely scenario in this cases)
  • When going through "Launch an instance" wizard, at the Security Groups screen, pick Default group and create new security group (you don't have to configure new security group now, this can be done later). Make sure both of those groups were selected as this will save you a lot of headache later.
  • Assign public IPs to your "cloud" servers and make sure that ISP that hosts your web application does not block or quarantine those IPs.

Host OS setup:

  • Disable Simple File Sharing. I am not sure why this feature is enabled on the servers.
  • Add your service account(s). To keep these notes simple, we will stick with one service account: tfsservice. Make sure all of your server instances have the same user accounts added (with the same passwords, of course)
  • If you're feeling lazy, grant your service account a local administrator's permissions
  • Make sure all of your "cloud" servers can see one another. If not, tweak your security groups in AWS Management Console.

TFS 2010 server tweaks:

  • Make sure that the user account with the same name and password as the service account on the cloud servers exists in the environment that hosts TFS 2010. Could be a local account on the TFS server or domain account.
  • Make sure that this service account is a member of Project Collection Administrators group, Project Collection Service Accounts group, and Project Collection Test Service Accounts group. Use tfssecurity.exe to add your user to those groups, if GUI does not let you.
  • In some cases, using SSL when connect to your TFS server does not work, so configure your TFS server to accept connections on HTTP port as well (for example, port 8080)

Test Controller configuration:

  • Make sure that the account you're using to run Test Controller service is a member of TeamTestAgentService local group
  • Always use ".\" instead of a domain name or machine name when provide credentials. For example, ".\tfsservice", instead of "SERVERNAME\tfsservice"
  • Do not register your test controller with any TFS project collection just yet
  • Enter SQL Server instance that will be hosting your Load Test database
  • Add virtual user licenses

Test Agent configuration:

  • Always use ".\" instead of a domain name or machine name when provide credentials. For example, ".\tfsservice", instead of "SERVERNAME\tfsservice"
  • Enter the machine name of your test controller. Use port default port 6901, unless you have configured your test controller to listen on a different port

I hope you found this info useful. If you have any questions or comments, shoot me an email at max [at] objectsharp [dot] com.