Archive for the ‘Agile Testing’ Category.

Moq Sequences

I have recently started to use Moq for mocking and I really like it. The fluent interface and Lambda support makes it very easy and natural to use.

However, I quickly ran into a situation where I wanted to ensure that methods on a mock object were called in a particular order. I have provided a simpler example below where I want to check that BlogPresenter.Show() shows blogs in reverse chronological order:

public class Post
{
    public DateTime DateTime { get; set; }
}

public class BlogPresenter
{
    private readonly BlogView view;

    public BlogPresenter(BlogView view)
    {
        this.view = view;
    }

    public void Show(IEnumerable posts)
    {
        foreach (var post in posts.OrderByDescending(post => post.DateTime))
            view.ShowPost(post);
    }
}

public interface BlogView
{
    void ShowPost(Post post);
}

To check this I used a callback to increment a counter like this:

[Test]
public void Should_show_each_post_once_with_most_recent_first()
{
    var olderPost = new Post { DateTime = new DateTime(2010, 1, 1) };
    var newerPost = new Post { DateTime = new DateTime(2010, 1, 2) };
    var posts = new List { newerPost, olderPost };

    var mockView = new Mock();

    var viewOrder = 0;

    mockView.Setup(v => v.ShowPost(newerPost)).Callback(() => Assert.That(viewOrder++, Is.EqualTo(0)));
    mockView.Setup(v => v.ShowPost(olderPost)).Callback(() => Assert.That(viewOrder++, Is.EqualTo(1)));

    new BlogPresenter(mockView.Object).Show(posts);

    mockView.Verify(v => v.ShowPost(newerPost), Times.Once());
    mockView.Verify(v => v.ShowPost(olderPost), Times.Once());
}

This works code but is not very intentional. I wanted to express the intent the there is a required ordering of method calls. After searching around I found a nice code snippet from Max Guernsey, III that was promising. I thought I would push a little further to see if I could get something like this:

[Test]
public void Should_show_each_post_with_most_recent_first_using_sequences()
{
    var olderPost = new Post { DateTime = new DateTime(2010, 1, 1) };
    var newerPost = new Post { DateTime = new DateTime(2010, 1, 2) };
    var posts = new List { newerPost, olderPost };

    var mockView = new Mock();

    using (Sequence.Create())
    {
        mockView.Setup(v => v.ShowPost(newerPost)).InSequence();
        mockView.Setup(v => v.ShowPost(olderPost)).InSequence();

        new BlogPresenter(mockView.Object).Show(posts);
    }
}

So, I created Moq.Sequences and you download Moq.Sequences.dll from github. Simply, add Moq.Sequences.dll as a reference in your .Net project and add a using Moq.Sequences; in your test class. Moq.Sequences supports the following:

  • checks order of method calls, property gets and property sets
  • allows you to specify the number of times a call is made before the next one is expected
  • allows intermixing of sequenced and non-sequenced expectations
  • thread safe – each thread can have its own sequence

Sequences

Sequences are added using the Sequence static class and extension methods. You create a sequence by calling:

using (Sequence.Create())
{
    ...
}

Sequences that do not fully complete are detected when the sequence is disposed. So, all the setups and mock calls should be done within the lifetime of the sequence.

Steps

Within a sequence you set the expectations for ordering via an extension method InSequence<(). I call these steps. For example,

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence();
    mock.Setup(_ => _.Method2()).InSequence();
    ...
}

This sets an expectation that Method1() will be called once followed by a single call to Method2. You can set more sophisticated expectations by using a Times parameter:

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence(Times.AtMostOnce());
    mock.Setup(_ => _.Method2()).InSequence(Times.Between(1, 10, Range.Inclusive));
    ...
}

Steps can be created for method calls, property gets and property sets:

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence(); // method call
    mock.SetupGet(_ => _.Property1).InSequence().Returns(0);  // property get
    mock.SetupSet(_ => _.Property2 = 0).InSequence(); // property set
    ...
}

Also, sequenced steps can be intermingled with non-sequenced expectations.

Loops

Loops are used when you want to check that as group of method calls are called in order several times. An example of this could be some resources which you expect to be opened, operated on and then closed, one after the other.

Loops are created via Sequence.Loop():

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence();

    using (Sequence.Loop())
    {
        mock.Setup(_ => _.Method2()).InSequence();
        mock.Setup(_ => _.Method3()).InSequence();
    }
    ...
}

The above checks that Method1 is called by any number of calls to Method2 followed immediately by a call to Method3. You can constrain the number of times the loop can be executed by adding a Times parameter such as the following example where the combination of Method2 and Method3 should be called exactly twice:

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence();

    using (Sequence.Loop(Times.Exactly(2)))
    {
        mock.Setup(_ => _.Method2()).InSequence();
        mock.Setup(_ => _.Method3()).InSequence();
    }
    ...
}

Wrap-Up

Moq.Sequences provides a simple, intentional way of checking that things are are done in a specific order. Feel free to check it out at github. I welcome any feedback!

FitNesse with Finesse

I like to make agile training as realistic as possible by having the team build working software in a realistic agile setting. So I like to use the game of TicTacToe because everyone knows how to play it and the team can quickly be productive in building and enhancing it.

It recently occurred to me that the requirements for TicTacToe could be readily explained through tables. It seemed very natural to use Fit/FitNesse as the vehicle to describe the game functionality. As a result I wrote some Fit fixtures and FitNesse pages to describe the TicTacToe game for my most recent training session.

And I think it worked out rather well …

Fit & FitNessse Background

Fit is a tool for enhancing communication and collaboration on agile teams. It does this by describing how the software should work through examples written in tables. There are many different types of tables which are backed by “fixtures” which serve as a bridge between the tables and underlying production code. Fixtures are quite easy to write.

FitNesse is a Wiki front-end for Fit providing an easy way to write, manage and execute Fit tests.

The diagram below shows a high level architecture for Fit and FitNesse:

Fit FitNesse Architecture
 

TicTacToe FitNesse

The page below describes the winning combinations for TicTacToe as described in FitNesse:

Winning Combinations

Grey cells are inputs or descriptive text and the green cells are passed assertions. Failed assertions would be shown in red and any exceptions would show up as yellow. The Fit engine uses reflection to instantiate fixtures which then act as a bridge between the Fit tables and the production code. Fit handles all the table parsing

So What?

I find Fit to be a great way to express and radiate business intent. When I look at the above page it is easy for me to understand what the winning combinations are and the green cells let me know that the software understands too. Perhaps the “Turn” row which indicates whose turn it is shouldn’t be there so I should remove it. But other than that, the above does a great job of describing the winning combinations.

Fit story tests are a primarily a great collaboration tool between your product owner and the rest of the team. Ron Jeffries talks about the the 3 C’s of a story: card, conversation and confirmation. Fit story tests are the confirmation. The product owner and the team can collaborate and clearly define what “done” means for a story. This focuses the team on getting story tests to pass and helps avoid scope creep and getting side-tracked with other distractions.

Another benefit of Fit story tests is that they help drive ubiquitous language. The terms used in the story tests should be domain terms that are used consistently through the Fit fixtures and on into the production code. The need to express working examples in the constrained language required by Fit and the underlying fixtures requires the team to think carefully and consistently about the terms they use. I would expect the TicTacToe production code to have classes named “cell”, “mark”, “player” and so on.

What makes Fit even more powerful is that story tests can be run individually or as a suite. Tests can be run from build tools like Ant and NAnt and from continuous integration tools like CruiseControl and CruiseControl.Net. In other words, you can build up a regression suite of story tests and run them many times a day.

What Next?

If you have tried Fit or FitNesse I would be interested in hearing how it worked or did not work for you.

If you haven’t either one why not take them for a test drive? No doubt about there are challenges setting it up and getting it part of the team’s culture … hmmm sounds like a good idea for another blog post ….

 

Taking Stock of Your Software Inventory

Do you have inventory piling up between your development team and QA?

This inventory of unconfirmed functionality limits your team’s agility on a few fronts:

  • your software is not always potentially shippable
  • deferred ROI
  • delayed feedback from QA

One of the main goals of your agile team is to build potentially shippable releases every iteration. When QA testing is done after development releases your team can no longer achieve this goal.

This deferred testing also creates an inventory of unconfirmed features that need to be managed. And just like any manufacturing system, inventory is wasteful and costs your company time and money. Every day that your software is not shipped decreases the ROI.

The longer your software remains untested the more costly any corrective action becomes. The cost arises from many factors including the ramp-up time for the developers, context-switching time, overhead with issue tracking and so on.

Finally, feedback from good testers is invaluable. They think differently from developers and tend to have a more outward-looking customer perspective. If you get this feedback earlier in the process you will see a reduction in defects and less rework.

If you find your team in this situation here are a few things you could might consider doing:

Track Metrics

Select metrics such as defect counts or time to close that you feel best highlights the obstacles your team faces. Chart these metrics in big visible charts

Consider translating technical metrics into bottom-line financial data. For example, you could use daily expected revenue of the product to calculate the revenue loss due to each day of delay.

Get Developers and Testers Working Together

There has been much talk about getting quality assurance infused throughout the product cycle. Agile teams are tailor-made for this transition. Get a tester on your team and allow her to contribute throughout the iterationAside from traditional testing, the tester can bullet-proof the story tests, suggest testing tasks, pair with developers on testing and so on

This inter-personal contact will do more to promote quality within your team than any formal QA process ever will.

Stagger Iteration Testing

Staggered testing

If you must maintain a separate system test phase, consider having the QA team test the previous iterations release. This way, the QA team has a stable release for testing and feedback is delayed at most one iteration

Over time, you may be able to move additional QA activities into the development iteration.

What does QA Mean on Agile Teams?

I have struggled with getting traditional QA professionals to work effectively with agile teams. Testers are frustrated because there is scant coverage of “tester” practices in much of the agile literature. For instance, the XP bill of rights covers the rights of the customers and the developers. Testers don’t even get a mention.

I think this stems from an emphasis within agile processes on test-driven design and automated acceptance tests. Done well, these practices result in much higher software quality. Agile developers then question the value of “monkey” manual tests which are often covering the same ground as the developer and customer facing tests that are already passing. Now, to be fair, there certainly is information out there on the role of agile testers such as work done by Lisa Crispin, Janet Gregory, Brian Marick, Jonathan Kohl and Michael Bolton to name a few.

And there are groups and forums out there such as the agile-testers Yahoo group. Michael Bolton recently posted that perhaps the confusion arises from what we mean by QA:

One important step is to stop thinking about testers as “QA” — unless the A stands for “assistance” or even “analysis”, but not “assurance”. We’re testers. Management (in one sense) and the whole team (in another) provide the “assurance” bit, but since we don’t have control over schedule, budget, staffing, project scope, contractual obligations, customer relations, etc., etc., we can’t assure anything. We should be here to help — not to slow the project down, but to aid in the discovery of things that will threaten value.

I think Michael has it right. In a private email he provided a great distinction between a developer’s and a tester’s perspective:

In the developer’s world, the job is to create value. In the tester’s world, the job is to defend value by looking and seeing where it might be missing. The developer’s job is to discover how it can work. The tester’s job is to discover how it might fail — and by disproving a failure hypothesis, getting one more piece of evidence but never certainty that it will work under these sets of assumptions — ‘cos the assumptions are never sure things.

I couldn’t agree more. I see the role of a tester then as supporting quality infusion into agile teams.