Stefan Forsberg

coding is my boyfriend

SpecFlow scoped bindings

Some background: on the site I’m currently working with searching is a key part and a search might exist in many different context and differ somewhat depending on the context. When writing a feature file I’d typically end up with something like this

Feature: Main search

Scenario: ...
  Given I search for "bla"
  And I select search result 1
  Then...
Feature: Search widget

Scenario: ...
  Given I search for "bla"
  And I select search result 1
  Then...

Duplicated steps

When running these scenarios SpecFlow will execute the same method for the steps defined above. This is a good thing since it allows us reuse steps defined in other features and thus avoid duplication. However, in this case it causes a bit of a pickle since while the scenarios are similar in a natural language sort of way what will be executed in the actual step differs quite heavily. For instance, the html structure might be different and the widget could have some additional ajax functionality to take into consideration. The bottom line is that the code we want executed in the Given I search… step differs between the two features.

A less than optimal solution

One way to make sure we can execute different methods depending on the context is to make sure the step name are unique.

Feature: Search widget

Scenario: ...
  Given I search for "bla" on the search widget
  And I select search result 1 on the search widget
  Then...

While this solves the problem of making the steps unique, this just adds a lot of noise to the steps. The feature is already called search widget and including this part in most of the steps feels very redundant since we already know we are in the context of describing the search widget.

Anti-patterns to the rescue!

SpecFlow comes with a concept of scoping bindings to either feature, scenario or tags. This allows us to inform SpecFlow that certain steps should only be associated with the scope in question. For the examples used above I scoped the step definitions to the feature since that seemed the most appropriate.

[Binding]
[Scope(Feature = "Search widget")] 
public class SearchWidgetSteps

Keep in mind that coupling your step definitions to features is considered a gherkin anti-pattern but in this case it felt like a good enough compromise. But perhaps there is a better solution?

EPiServer 7 patch 2, MVC and session state

When my colleague Niklas Melinder applied patch 2 for EPiServer 7 he ran into trouble when the site was deployed to our acceptance server. The site started to throw “System.InvalidOperationException: The SessionStateTempDataProvider class requires session state to be enabled.”. Since our acceptance environment runs with sessions state switched off it was pretty clear what caused the error but why it happened was not.

After some reflection the following in PropertyExtensions seemed to be the culprit

html.ViewContext.TempData["propertyName"]

Using TempData requires session state if the value is not used in the same request (see the answer to this q on SO). This code is used for the PropertyFor extensions methods so it seems webform pages are not affected by this but we haven’t really looked into that.

Waiting for knockout.js bindings to be complete before starting UI-test

As always when UI-testing javascript heavy code with ajax calls going around you might end up with some timing issues. The problem is that it’s not easy for the poor driver to know when the page is finished and the test can start. Recently I had the task of writing some UI-tests for a page that used knockout.js and this is how I solved the waiting part.

applyBindings

All the bindings are triggered through the page and it’s viewmodel with the standard applyBindings function. This will bind the viewmodel itself and all the other views the page contains.

ko.applyBindings(pageViewModel);

Since this call is blocking when it’s finished the bindings are applied. So I added a simple bool to the object that represented the page that tracks whether the page has finished it’s bindings.

SomeObject.BindingsAreApplied = false;
// Lots of setup
ko.applyBindings(pageViewModel);
SomeObject.BindingsAreApplied = true;

Selenium WebDriver side

When the driver browses to the page in question it issues a WebDriverWait with the condition that the bool we just added to the javasript is true.

new WebDriverWait(driver, TimeSpan.FromSeconds(secondsToWait))
  .Until(d =>
  {
    var result = (d as IJavaScriptExecutor).ExecuteScript("return window.SomeObject.BindingsAreApplied");

    if (result == null)
    {
      return false;
    }

    return (bool)result;
  });

API when creating page data used for tests

Some context: when using an in memory implementation of IContentRepository (most likely in a test scenario) you need to create pages that are involved in the tests. I’m not quite sure which approach I fancy so I thought I’d ask you. Given that these two snippets produces the same result, do you find one “better” (be it that it’s easier to understand, easier to write or something else) than the other?

_oldParent = ContentRepository.GetDefaultPageData<StartPage>(_startPage.PageLink);
_oldParent.PageName = "OldParent";
_oldParent.LinkURL = "oldparent";
ContentRepository.Save(_oldParent, SaveAction.Publish, AccessLevel.NoAccess);
_oldParent = ContentRepository
    .GetDefaultPageData<StartPage>(_startPage.PageLink)
    .With(p =>
        {
            p.PageName = "OldParent";
            p.LinkURL = "oldparent";
        })
    .SaveAndPublish(ContentRepository);

One reason why I don’t like service locator

Let’s say I want to use the new (brilliant) ISelectionFactory to easily create a list of something for the editor to choose. The data the editor uses is fetched from EPiServers IContentRepository.

public class SomeDataSelectionFactory : ISelectionFactory
{
    readonly IContentRepository _content; 

    public SomeDataSelectionFactory(IContentRepository content)
    {
        _content = content;
    }

    public IEnumerable<ISelectItem> GetSelections(ExtendedMetadata metadata)
    {
        return _content
            .GetChildren<PageData>(ContentReference.StartPage)
            .Select(c => new SelectItem { Value = c.ContentLink.ID, Text = c.PageName });
    }
}

Unfortunately this won’t work since the class is not created from the container. Instead we get an error in edit mode saying the page that has a property that uses the selection factory can’t be previewed and if we follow the link in the javascript console log that reports a 500 error we find the expected “No parameterless constructor defined for this object”.

Service Locator

Since constructor injection doesn’t work we can use service location, either via the ServiceLocator or Injected.

public class SomeDataSelectionFactory : ISelectionFactory
{
    public IEnumerable<ISelectItem> GetSelections(ExtendedMetadata metadata)
    {
        return ServiceLocator.Current.GetInstance<IContentRepository>()
            .GetChildren<PageData>(ContentReference.StartPage)
            .Select(c => new SelectItem { Value = c.ContentLink.ID, Text = c.PageName });
    }
}
public class SomeDataSelectionFactory : ISelectionFactory
{
    Injected<IContentRepository> _content;

    public IEnumerable<ISelectItem> GetSelections(ExtendedMetadata metadata)
    {
        return _content
            .Service
            .GetChildren<PageData>(ContentReference.StartPage)
            .Select(c => new SelectItem { Value = c.ContentLink.ID, Text = c.PageName });
    }
}

These two options are basically equivalent since Injected uses the ServiceLocator behind the scenes. My beef with any of these approaches become apparent when trying to initiate the class, for instance in a test scenario.

The test

When writing a test for the selection factory in the first example it become very clear that I need to supply a IContentRepository because of the constructor. The service locator based approaches happily create the object but will fail at runtime because it can’t resolve the dependency to IContentRepository. This means I need to make sure that before the test is run I have set a service locator and make sure that the relevant dependencies are registered.

var container = new Container();
container.Configure(x => x.For<IContentRepository>().Use(contentRepository));
ServiceLocator.SetLocator(new StructureMapServiceLocator(container));

This is not complicated to do but that’s beside the point. When a new up something I can clearly see what the object needs in the constructor. Service location hides all those details away. That’s not good infrastructure magic that just works. That’s magic where I need to know a lot about the involved rabbits and hats and that’s one reason why I don’t like the service locator.

UI-testing (EPiServer): SpecFlow

This post is not going to be about what BDD / Specification by example is and the benefits of it but more of a quick start with doing UI-testing in a SpecFlow manner.

Installing SpecFlow

Although it’s not technically necessary I’d advice you to add the SpecFlow extension to Visual Studio. This is added through TOOLS / Extensions and Updates.

sf01

Go to the online tab and search for SpecFlow and install it

sf02

The first feature

A feature is something we use to describe what we’re about to test and why. Since we’ve installed the SpecFlow extension we can simple add a new feature by adding a new item and select the SpecFlow Feature file. Since we’re going to continue with our header test let’s call it Header.feature.

sf03

Back to the feature

Our feature now contains the template feature so let’s take a look at it since it describes the anatomy of a feature quite nicely

Feature: Header
	In order to avoid silly mistakes
	As a math idiot
	I want to be told the sum of two numbers

@mytag
Scenario: Add two numbers
	Given I have entered 50 into the calculator
	And I have entered 70 into the calculator
	When I press add
	Then the result should be 120 on the screen

At the top of the feature is a description of the feature in a user story format. After the description comes one or more scenarios which is used to describe various aspects of the feature. The scenarios are what are actually executed as test(s). Each of the sentences that starts with a gherkin (that’s the name of the language) keyword (given, and, when, then) is called a step and is roughly equivalent to a method that executes that given task. (The example also has a tag, (@mytag), which we’ll talk about in another post.)

In our case of the header it’s likely that it does more than simply indicate the current active page so in real life we’d probably have more than the one scenario we’re going to focus on in this example.

Feature: Header
	In order to navigate the site
	As a visitor
	I want to be able to access the main sections of the site

Scenario: Active page
	Given I am on the page "/"
	Then the active link in the header should say "Start"
	When I am on the page "/Alloy-Plan"
	Then the active link in the header should say "Alloy Plan"

There’s much to be said about how the descriptions and scenarios should be formatted but since that’s a bit of a rabbit hole and (at least) a post in its own I’m gonna say “Look behind you, a three-headed monkey!” and get on with it.

Step definitions

With out first scenario written we need to make sure that there are steps corresponding to the ones specified in the feature file. Luckily the SpecFlow-tooling we installed earlier makes it dead simple to generate a skeleton to use. Simply right click on the scenario and choose “Generate step definitions”

sf04

In the window opening we can preview the missing steps SpecFlow found as well as some other options we can ignore for now. Click generate and save the file somewhere in your project.

sf06

The file that gets generated looks like this

[Binding]
public class HeaderSteps
{
    [Given(@"I am on the page ""(.*)""")]
    public void GivenIAmOnThePage(string p0)
    {
        ScenarioContext.Current.Pending();
    }
        
    [When(@"I am on the page ""(.*)""")]
    public void WhenIAmOnThePage(string p0)
    {
        ScenarioContext.Current.Pending();
    }
        
    [Then(@"the active link in the header should say ""(.*)""")]
    public void ThenTheActiveLinkInTheHeaderShouldSay(string p0)
    {
        ScenarioContext.Current.Pending();
    }
}

A step method

Let’s dissect the Given method to see what’s going on here.

[Given(@"I am on the page ""(.*)""")]
public void GivenIAmOnThePage(string p0)
{
    ScenarioContext.Current.Pending();
}

The method is marked with a attribute containing a keyword (Given in this case) and a regular expression. This regexp is used to match what is written in the feature file. SpecFlow was also clever enough to realize that what we typed inside the quotation marks (an url in this case) is probably going to be used as a parameter for the test. This is captured in a group in the regexp and then sent to the method as the (generated) parameter p0.

When SpecFlow executes the step ‘Given I am on the page “/”‘ it will find a method marked with an attribute that matches that regexp (GivenIAmOnThePage) and then use the group in the regexp to give the parameter p0 the value of “/”.

Implementing the steps

We’ll make use of the SeleniumServer class from the previous post and as usual we’ll need both the Nunit and the Selenium.WebDriver package. We’ll also add the NUnit for SpecFlow package so that we can use NUnit for asserts in the steps.

Install-Package Selenium.WebDriver
Install-Package NUnit
Install-Package SpecFlow.NUnit

Previously we used NUnit hooks to open and close the browser before any test is run and after all tests are run as well as clearing cookies between tests. SpecFlow also contains these type of hooks and they are implemented by adding the Binding attribute to a class and then implement methods that are decorated with one of the hooks attributes.

[Binding]
public class SpecFlowHooks
{
    [BeforeTestRun]
    public static void Start()
    {
        SeleniumServer.Start();
    }

    [AfterScenario]
    public static void AfterScenario()
    {
        SeleniumServer.ResetBrowser();
    }

    [AfterTestRun]
    public static void Shutdown()
    {
        SeleniumServer.Stop();
    }        
}

With that infrastructure in place we can finally implement the steps and also refactor the parameter names a bit for clarity.

[Given(@"I am on the page ""(.*)""")]
public void GivenIAmOnThePage(string slug)
{
    SeleniumServer.Driver.Navigate().GoToUrl("http://alloy.demo.episerver.com" + slug);
}
        
[When(@"I am on the page ""(.*)""")]
public void WhenIAmOnThePage(string slug)
{
    SeleniumServer.Driver.Navigate().GoToUrl("http://alloy.demo.episerver.com" + slug);
}
        
[Then(@"the active link in the header should say ""(.*)""")]
public void ThenTheActiveLinkInTheHeaderShouldSay(string expectedActiveLinkText)
{
    var activeLinkText = SeleniumServer.Driver.FindElement(By.CssSelector("ul.nav li.active")).Text;
    Assert.That(activeLinkText, Is.EqualTo(expectedActiveLinkText));
}

And our scenario is now able to run and pass

sf07

This concludes our first look at SpecFlow and UI-testing. While I haven’t touched upon the subject in detail in this series of post there are ways to mitigate code duplication and page layout details leaking into your test code. One such way is using the Page Object pattern. During the last years I (and the rest of the team) used Cucumber I liked how the gherkin language tied into UI-testing and at times it can be useful almost in the same manner as a page object but on a different level of abstraction. It might not be very clear in this introduction post but hopefully I can come back later and elaborate more on this subject.

UI-testing (EPiServer): the need for speed

Executing our two test takes around 20 seconds. If you’ve run the test you’ve noticed that a lot of the test time is actually spent on opening a new browser window. One way to mitigate this is to use the same browser throughout all the test which opens before the first test and closes after the last test. Note that there are downsides with this since your test may now affect other tests because of the shared browser data (cookies etc.). We can try and mitigate this by clearing the state between tests but the absolutely full proof way of ensuring a clean slate is to fire up a new browser before each tests.

Sharing the browser

We need to refactor our test code a bit to make this happen. Let’s start with adding some code that runs before any tests are run and after all the tests are completed using the SetupFixture attribute.

[SetUpFixture]
public class SeleniumServer
{
    public static IWebDriver Driver { get; private set; }

    [SetUp]
    public static void Start()
    {
        Driver = new FirefoxDriver();
        Driver.Manage().Window.Maximize();
    }

    [TearDown]
    public static void Stop()
    {
        Driver.Quit();
    }

    public static void ResetBrowser()
    {
        Driver.Manage().Cookies.DeleteAllCookies();
    }
}

The TestBase class now looks like this

public class UiTestBase
{
    protected IWebDriver Driver = SeleniumServer.Driver;
    protected const string BaseUrl = "http://alloy.demo.episerver.com";

    [TearDown]
    public void After()
    {
        SeleniumServer.ResetBrowser();
    }

    public void Visit(string relativeUrl)
    {
        Driver.Navigate().GoToUrl(BaseUrl + relativeUrl);
    }
}

Bring me the head of… oh wait

A great way to speed up your tests is to use what is called a headless browser. A headless browser is basically a browser without a GUI and since it doesn’t need to show a user anything it’s considerably faster than a conventional browser. This has been missing from the .net space but now PhantomJS is available and it’s also integrated into the Selenium code! To use it we need to download the phantom js executable either via the PhantomJS page or by fetching the NuGet package

Install-Package phantomjs.exe

To use phantom we only need to adjust our SeleniumServer class to initiate a phantomjs driver instead of a firefox one. The constructor needs to know the path the phantomjs executable.

[SetUp]
public static void Start()
{
    //Driver = new FirefoxDriver();
    Driver = new PhantomJSDriver(@"(yourPathHere)\phantomjs.exe.1.8.0");
    Driver.Manage().Window.Maximize();
}

Time spent executing tests

As usual the numbers themselves are not important but rather how they compare to each other.

2 tests

  1. New browser: 20 seconds
  2. Shared browser: 13 seconds
  3. Headless browser: 7 seconds

6 tests

  1. New browser: 44 seconds
  2. Shared browser: 20 seconds
  3. Headless browser: 8 seconds

14 tests

  1. New browser: 1 minute 48 seconds
  2. Shared browser: 29 seconds
  3. Headless browser: 9 seconds

UI-testing (EPiServer): improving the test(s)

The code we ended up with in the previous post works but it’s hardly optimal.

There are some tasks that are performed in each and every UI test. For instance, before the test start up the browser and maximize it and after the test close the browser. If we put this common tasks in a base class we end up with this

public class UiTestBase
{
    protected IWebDriver Driver;
    protected const string BaseUrl = "http://alloy.demo.episerver.com";

    [SetUp]
    public void Before()
    {
        Driver = new FirefoxDriver();
        Driver.Manage().Window.Maximize();
    }

    [TearDown]
    public void After()
    {
        Driver.Quit();
    }
}

The test itself now looks like this

[Test]
public class HeaderTest : UiTestBase
{
    public void Alloy_Meet_should_be_marked_in_header_when_browsing_Alloy_Meet_page()
    {
        Driver.Navigate().GoToUrl(BaseUrl + "/Alloy-Meet/");
        var activeLinkText = driver.FindElement(By.CssSelector("ul.nav li.active")).Text;
        Assert.That(activeLinkText, Is.EqualTo("Alloy Meet"));
    }
}

Not a whole load has changed compared to the previous version but we now have a common base that we can use for all out UI-tests. If we were to add another (very) similar test that simply checks that another page is marked in the header the test would look like this

[Test]
public void About_us_should_be_marked_in_header_when_browsing_About_us_page()
{
    Driver.Navigate().GoToUrl(BaseUrl + "/About-us/");
    var activeLinkText = driver.FindElement(By.CssSelector("ul.nav li.active")).Text;
    Assert.That(activeLinkText, Is.EqualTo("About us"));
}

As you can tell we have some duplication between the tests to let’s refactor a bit

public class HeaderTest : UiTestBase
{
    private string GetActiveLinkText()
    {
        return Driver.FindElement(By.CssSelector("ul.nav li.active")).Text;
    }

    [Test]
    public void Alloy_Meet_should_be_marked_in_header_when_browsing_Alloy_Meet_page()
    {
        Driver.Navigate().GoToUrl(BaseUrl + "/Alloy-Meet/");
        Assert.That(GetActiveLinkText(), Is.EqualTo("Alloy Meet"));
    }

    [Test]
    public void About_us_should_be_marked_in_header_when_browsing_About_us_page()
    {
        Driver.Navigate().GoToUrl(BaseUrl + "/About-us/");
        Assert.That(GetActiveLinkText(), Is.EqualTo("About us"));
    }
}

UI-testing (EPiServer): up and running

I’ve blogged previously about UI-testing. The purpose of that series of posts was to share information I’d gathered in the process of testing but I never really got around to writing about how to get started. Inspired by the Selenium testing tools cookbook I want to write some very hands-on posts so let’s get started.

Example to test

We’ll be using the episerver demo site and the test we’re going to run is that we want to make sure the header menu correctly marks the active page. By that I mean that when browsing the Alloy Meet page that corresponding menu choice is selected

Skärmavbild 2013-01-15 kl. 21.35.38

We’ll start with the simplest thing that could possibly work and then as this series goes along we’ll tweak various things and look at alternatives.

First test

Start by creating a new class library in VS (I’ll be using 2012). To drive the browser we need a driver (Selenium WebDriver) and to create the tests we need a testing framework (NUnit). Both of these are on NuGet so let’s add them

Install-Package Selenium.WebDriver
Install-Package NUnit

When everything is downloaded we add the class HeaderTest.

public class HeaderTest
{
    [Test]
    public void Alloy_Meet_should_be_marked_in_header_when_browsing_Alloy_Meet_page()
    {
        using (IWebDriver driver = new FirefoxDriver())
        {
            driver.Manage().Window.Maximize();
            driver.Navigate().GoToUrl("http://alloy.demo.episerver.com/Alloy-Meet/");
            var activeLinkText = driver.FindElement(By.CssSelector("ul.nav li.active")).Text;
            Assert.That(activeLinkText, Is.EqualTo("Alloy Meet"));
        }
    }
}

The code is pretty self explanatory (at least if you know a bit about css). There are many ways to locating elements and interacting with them, many of which I’m not going to touch in this series since it’s not about how Selenium WebDriver works. Reading Selenium testing tools cookbook, exploring the FindElement(s) method or googling will be very helpful however.

What’s in EPiServers container?

I was curious what’s registered in the StructureMap container by EpiServer themselves. StructureMap comes with the helpful method WhatDoIHave() which spits out everything registered. I formatted that list to make it slightly prettier

_container
    .WhatDoIHave()
    .Split(new[] { "--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------" }, StringSplitOptions.None)
    .Where(l => l.Contains("EPiServer"))
    .Aggregate((current, next) => 
        current + "<br /><br />" + next
            .Replace("Configured Instance of", "<br />Configured Instance of")
            .Replace("Instance is created by", "<br />Instance is created by")
        );

Here’s the list from the above code. Rumor has it that you need to have this list memorized if you are to be a certified developer for EPiServer 7.