Testing strategies for EPiServer now and in the future

This post discusses some different approaches you can take to testing your EPiServer site. Unfortunately it won’t give you answers to the eternal question “what to test?”. While it’s always boring to do the typical consultant “it depends” it’s very hard to give general guidelines what to test, when to do it and what approach to take. Things like size of team, time-to-market, how long the site should run etc all are part of the equation. Personally I like the guideline I heard from someone I can’t remember that went along the lines of “When all tests passes we’re confident enough to release a new version”.

Unit / Integration tests without DB

These types of tests are often referred to as unit tests. Depending on your definition of unit and / or integration tests they can be one or the other and quite frankly I’m bored by discussion of their difference. The main part here is that these tests don’t talk to the database but rather some faked or mocked version of it.

PageData (or derived types) are typically setup using the property array.

[Test]
public void Test()
{
    var page = new StandardPage();
    page.Property.Add("PageName", new PropertyString("Hello"));

    Assert.That(page.PageName, Is.EqualTo("Hello"));
}

Changes with Epi 7

EPiServer introduces lots of interfaces (IContentRepository and so on) making it much easier to work with these types of tests without needing to create your own abstractions. Many of the page data properties are now virtual which make it possible to work with mocks of pages instead of setting the properties as shown above.

[Test]
public void Test()
{
    var pageMock = new Mock<StandardPage>();
    pageMock.Setup(x => x.PageName).Returns("Hello");
    var page = pageMock.Object;

    Assert.That(page.PageName, Is.EqualTo("Hello"));
}

Pros

Fast.
Easy and a great way to get started with testing.

Cons

MockHell (eg 2312312 line of mocking code and one assert. Can feel more like you’re testing the mocking framework than actual functionality you’ve built.
There’s always a possibility that something happens between your abstractions and the actual EPiServer code that, for instance, saves something to the database that is not tested with this approach.

DB integration tests

These types spin up EPiServer in a none-webapp mode and all tests are run against the actual DB. Resetting test data between tests is typically done manually (that is, if a property is set to foo, than changed to bar during the test the tear down sets it to foo again) or by utilizing snapshots (see EpiTest for instance).

Changes with Epi 7

Not much. The big game changer here would be the possibility to run EPiServer DB in one of the in memory databases for speed reasons. The setup code used for EPiTest is written against CMS 6 R2 and does not work for EPiServer 7 but it shouldn’t be too much of a hassle to upgrade it.

Pros

For data-centric tasks that are not triggered by UI you control these tests can give a great end-to-end test result.

Cons

Doing the setup can be somewhat tricky.
Slow.
Resetting the database can be tedious (when done manually).

UI-tests

Using one of the drivers and perform actual actions from an actual browser. These tests are also great end-to-end tests. Let’s take an unit test against a controller (in a MVC-project) that asserts that the model property ShowNews is true for some scenario. This tests says little about what is shown to the user compared to a UI-test that actually asserts that the content controlled by ShowNews is actually there. So the unit test may well pass but the site may not function the way it should anyway.

Changes with Epi 7

Not much.

Pros

Great way to perform end-to-end tests that don’t lie (too much).

Cons

Slow.
They can make you want to pull your hair out.