Run Functional (Coded) UI Tests in your VSTS Release Pipeline

Today I was trying to run some CodedUI test from my VSTS Release pipeline. The process is quite straightforward but there were some tiny catches I’d like to share with you

Let me first show the pipeline I have created

First of all, WINRM. It is always a search for me to see what’s wrong. In my Deploy Test Agent, I first pointed to the IP address of my machine, but that gave me the following error message

017-02-01T09:46:54.3930910Z ##[warning]DistributedTests: Task 
'PopulatingMachinesPresentState' for machine 10.32.6.4:5985's Error : 
System.Management.Automation.Remoting.PSRemotingTransportException: Connecting to
 remote server 10.32.6.4 failed with the following error message : The WinRM client
 cannot process the request. Default authentication may be used with an IP address 
under the following conditions: the transport is HTTPS or the destination is in the 
TrustedHosts list, and explicit credentials are provided. Use winrm.cmd to configure
 TrustedHosts. Note that computers in the TrustedHosts list might not be 
authenticated. For more information on how to set TrustedHosts run the following 
command: winrm help config. For more information, see the 
about_Remote_Troubleshooting Help topic.

With telnet I checked if the right port was open

telnet ipaddress 5985

That seemed ok, but when testing the remote PowerShell session within a PowerShell command window I got the same error.

Enter-PSSession -ComputerName 10.32.6.4

Then I tried the same with computer name and the right Credential and it worked, so that problem was fixed !

http://stackoverflow.com/questions/37317468/enter-pssession-to-remote-server-fails-with-access-is-denied

After my Test Agent was deployed (don’t forget the Interactive Process checkbox), and I copied some files to the machine, it was time to run the Functional Tests. The pipeline found my tests perfectly, but somehow I got an error.

System.Exception: Some tests in the test run did not pass, failing the task.

Not a very useful message, but when looking at the Test Runs in the Test Hub, I got some more info.

Automation engine is unable to playback the test because it is not able to 
interact with the desktop. This could happen if the computer running the test is 
locked or it’s remote session window is minimized.

I tested with an open Remote Desktop session and it indeed worked, but this is not something I wanted. Then my colleague Marcel de Vries attended me on the AutoLogon tool from SysInternals.

This logs in to the server automatically and creates the interactive desktop. But you do not need a remote desktop or something.

After I installed the autologon tool, configured the tool to use the account to run the tests, and rebooted the server, everything worked fine !