Waiting for elements when UI-testing with WebDriver and EPiTest

Something that is more common than not on a webpage is that data on your page is fetched without loading the whole page but rather using something like ajax. A normal page load is quite easy for your driver of choice to handle since it’s quite clear when all data has been fetched. When performing an Ajax call it’s a whole other story. An example of this in EPiServer could be to switch which visitor group you should fake view as in edit mode on a page. On the standard Alloy Tech template package the start page has some text that varies depending on what type of visior you are.

The specification

The bases for our specification is that we want to make sure that when we switch visitor group to Job Seeker the page displays the appropriate text.

Establish context = () =>
    {
        Login("sf", "password");
        VisitPageWithIdInEditMode(3);
    };

Because of = () =>
    {
        InEditPanel();
        Within("#visitorGroupDropDown", e =>
            {
                e.Click("li a");
                e.ClickLabel("Job Seeker");
                e.Click("#applyPreview");
            });
    };

It should_print_text_tailored_to_the_choosen_visitor_group = () =>
    {
    };

This logs a user in, goes to edit mode for the start page, selects the user group and clicks apply preview button. This is a fairly EPiTest specific API but it’s still a work in progress so don’t dwell too much on it. Now, the interesting part is how we should assert that the text we want to be there actually is. The simplest approach would of course be

It should_print_text_tailored_to_the_choosen_visitor_group = () =>
    {
        InPreviewFrame();
        Driver.Content().ShouldContain("We are recruiting");
    };

If you try this there’s quite a good chance that the test will pass. But occasionally it will fail. Perhaps with saying something about how it can’t find a body tag (Content() grabs all text inside the body tag). What’s happening is that when we click that apply preview button the new page (viewed as the chosen visitor group) is loaded and WebDriver has a hard time figuring out when it’s completely loaded causing it to go right ahead and try and find that “We are recruiting” text. If the page hasn’t been loaded yet then obviously the text can’t be found and the tests fail.

This leads to one of the biggest pain points in UI-testning, the brittleness of test sometimes passing and sometimes failing because of outside affect that’s not really tied to the test.

The road to UI-testing hell is paved with Thread.Sleep

So if we have a problem with something happening too soon one approach could be to give the page a little time to load before we try to find the text.

It should_print_text_tailored_to_the_choosen_visitor_group = () =>
    {
        InPreviewFrame();
        Thread.Sleep(2000);
        Driver.Content().ShouldContain("We are recruiting");
    };

And this will take care of 95% of the “page not being loaded” type of errors. But you still have brittle test. And since this functionality is used in 83 of your test you notice that 83*2 seconds of waiting time really hurts. Especially since your UI-tests weren’t that quick to begin with. So to summarize you’ve just made your tests slower by an order of magnitude without really solving the problem.

Polling with WebDriverWait

There’s a better alternative to waiting for a set amount of time and then try to find the element. This is to still have a timeout but “poll” the page and see if the condition is satisfied and if it is then hunky dory, otherwise try again a little later until the time out occurs. Luckily this is built into WebDriver.

It should_print_text_tailored_to_the_choosen_visitor_group = () =>
    {
        InPreviewFrame();
        var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(5));
        wait.Until(d => d.Content().Contains("We are recruiting"));
    };

I’m not too crazy about the built in API surrounding WebDriverWait and will probably create something around it for EPiTEst. Perhaps take the Capybara approach where the waiting is built into every operation and you don’t really need to think about whether you’re asserting on something that’s loaded partially after the page is loaded or not.

But won’t timeout problems still exist?

As you can imagine if, for some reason, the page above hasn’t been able to load in 5 seconds we will have a timeout exception and the test will fail. This can still happen but it will be clearly logged as a timeout exception instead of some cryptic message about not finding an element that’s clearly there when you look at the page.