Dear future Stefan,
When running IE8 UI-tests on a build server, always remember to login to the machine with the same user account that is used by your CI framework of choice. There are many different reasons why selenium can have a hard time communicating with IE such as running in protected mode and/or having other browsers installed which triggers the “use IE as your standard browser”. When the tests fail it’s not always clear as to what the exact error is since all selenium usually have to say is something along the lines of “OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server X timed out after 60 seconds”.
Connecting via RPD (using the same credentials as your CI server) and actually starting IE locally can usually clear up all these problems in a few minutes. Please remember this next time.