Archive for the ‘Agile Testing’ Category.

The Presentation Game

I have recently started paired agile coaching with Jason Cheong-Key-You. Jason suggested this ‘game’ for an Agile Acceptance Testing Workshop I was leading. He first experienced as an introduction to agile session with James Shore and Diana Larsen.

I have dubbed it ‘The Presentation Game’. It takes a minimum of 90 minutes – here’s how it works:

The presentation game

Set-Up
I did this myself and it would have worked better with two facilitators. If you have two, I suggest that you each support one team during estimating and planning and so on.

[10 minutes] Introduction
I suggest a quick ice-breaker followed by a description of the how the workshop will proceed. You will need to establish two teams. With a group that already knows each other you could skip the ice-breaker and use additional time for debrief. Have each team select someone to ‘pitch’ their session. Identifying this person early will allow them to construct a mental model of their pitch in advance.

[8] Create
Each team will create stories where each story is a topic to be covered. Instruct people to write one topic per index card. if you are doing this by yourself rotate around and answer any questions they may have about the session. If the opportunity arises, try to get participants to break-down big presentation topics into smaller ones.

I was doing this session focusing on acceptance testing so I asked the group to define their acceptance criteria on the back of the cards. This proved to
be a little problematic as I could not easily see them. If you decide to include acceptance criteria I suggest you get participants to write them on separate cards.

[8] Estimate
Before starting, I suggest handing out three index cards with large H, M, L on them and instruct attendees to do T-shirt sizing based on the *value* of the topic. This helps because we can be sure to estimate the higher priority topics first – in case time gets tight. Go to each group for four minutes and estimates their stories via points. Write the points on the card – I used a red marker so the estimate stood out. I also used a scale of 1 point per minute of presentation time. Be sure to act like a development team estimating: ask questions, clarify, negotiate, challenge, try to split the story etc.

[8] Select
Based on the estimates and your target velocity each team should select the stories that they want you to present.

[5] Pitch and Vote
Each team should then pitch their presentation to the entire group – timebox this to 2 minutes each. Then take a vote from all attendees as to which presentation they would prefer to hear. Be ready for a tie-breaker if you have an even number of attendees – e.g. flip a coin.

[5] Preparation
Review the cards and arrange them in a logical presentation order. If you have a co-facilitator I suggest creating a quick task board with “not started”, “in progress” and “done” columns. Tape each topic card and its acceptance criteria to the task board.

[30] Deliver
Move each topic card as you go. For people new to agile this will help reinforce the use of a story, acceptance criteria, “done” and so on. Remember to ask after each topic is presented if the acceptance criteria has been met. Have someone, either a co-facilitator, or a volunteer run a timer for you and let you know at regular intervals how much time is left.

[15] Debrief
I debriefed using a simple fishbowl and this has mixed results. This format works great if you have people that are comfortable enough to share openly.

My Take-Aways from This Format

  • arrange for two facilitators – this was hard to facilitate alone
  • excellent demo of courage – look to underscore this in debrief it it comes up
  • excellent focus on delivering value – I kept stressing that I would would estimate any topic they came up but theior presentation would be more likely to succeed if it was aligned with what was most valuable to *everyone* there.
  • I found it hard to present concisely on some topics. For instance, one topic I spoke on was “What is Agile?”. It made me realize that we should have short mini-presentations available to answer such questions. Perhaps lightning talks would be a good way to sharpen the saw on this!

Moq Sequences

I have recently started to use Moq for mocking and I really like it. The fluent interface and Lambda support makes it very easy and natural to use.

However, I quickly ran into a situation where I wanted to ensure that methods on a mock object were called in a particular order. I have provided a simpler example below where I want to check that BlogPresenter.Show() shows blogs in reverse chronological order:

public class Post
{
    public DateTime DateTime { get; set; }
}

public class BlogPresenter
{
    private readonly BlogView view;

    public BlogPresenter(BlogView view)
    {
        this.view = view;
    }

    public void Show(IEnumerable posts)
    {
        foreach (var post in posts.OrderByDescending(post => post.DateTime))
            view.ShowPost(post);
    }
}

public interface BlogView
{
    void ShowPost(Post post);
}

To check this I used a callback to increment a counter like this:

[Test]
public void Should_show_each_post_once_with_most_recent_first()
{
    var olderPost = new Post { DateTime = new DateTime(2010, 1, 1) };
    var newerPost = new Post { DateTime = new DateTime(2010, 1, 2) };
    var posts = new List { newerPost, olderPost };

    var mockView = new Mock();

    var viewOrder = 0;

    mockView.Setup(v => v.ShowPost(newerPost)).Callback(() => Assert.That(viewOrder++, Is.EqualTo(0)));
    mockView.Setup(v => v.ShowPost(olderPost)).Callback(() => Assert.That(viewOrder++, Is.EqualTo(1)));

    new BlogPresenter(mockView.Object).Show(posts);

    mockView.Verify(v => v.ShowPost(newerPost), Times.Once());
    mockView.Verify(v => v.ShowPost(olderPost), Times.Once());
}

This works code but is not very intentional. I wanted to express the intent the there is a required ordering of method calls. After searching around I found a nice code snippet from Max Guernsey, III that was promising. I thought I would push a little further to see if I could get something like this:

[Test]
public void Should_show_each_post_with_most_recent_first_using_sequences()
{
    var olderPost = new Post { DateTime = new DateTime(2010, 1, 1) };
    var newerPost = new Post { DateTime = new DateTime(2010, 1, 2) };
    var posts = new List { newerPost, olderPost };

    var mockView = new Mock();

    using (Sequence.Create())
    {
        mockView.Setup(v => v.ShowPost(newerPost)).InSequence();
        mockView.Setup(v => v.ShowPost(olderPost)).InSequence();

        new BlogPresenter(mockView.Object).Show(posts);
    }
}

So, I created Moq.Sequences and you download Moq.Sequences.dll from github. Simply, add Moq.Sequences.dll as a reference in your .Net project and add a using Moq.Sequences; in your test class. Moq.Sequences supports the following:

  • checks order of method calls, property gets and property sets
  • allows you to specify the number of times a call is made before the next one is expected
  • allows intermixing of sequenced and non-sequenced expectations
  • thread safe – each thread can have its own sequence

Sequences

Sequences are added using the Sequence static class and extension methods. You create a sequence by calling:

using (Sequence.Create())
{
    ...
}

Sequences that do not fully complete are detected when the sequence is disposed. So, all the setups and mock calls should be done within the lifetime of the sequence.

Steps

Within a sequence you set the expectations for ordering via an extension method InSequence<(). I call these steps. For example,

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence();
    mock.Setup(_ => _.Method2()).InSequence();
    ...
}

This sets an expectation that Method1() will be called once followed by a single call to Method2. You can set more sophisticated expectations by using a Times parameter:

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence(Times.AtMostOnce());
    mock.Setup(_ => _.Method2()).InSequence(Times.Between(1, 10, Range.Inclusive));
    ...
}

Steps can be created for method calls, property gets and property sets:

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence(); // method call
    mock.SetupGet(_ => _.Property1).InSequence().Returns(0);  // property get
    mock.SetupSet(_ => _.Property2 = 0).InSequence(); // property set
    ...
}

Also, sequenced steps can be intermingled with non-sequenced expectations.

Loops

Loops are used when you want to check that as group of method calls are called in order several times. An example of this could be some resources which you expect to be opened, operated on and then closed, one after the other.

Loops are created via Sequence.Loop():

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence();

    using (Sequence.Loop())
    {
        mock.Setup(_ => _.Method2()).InSequence();
        mock.Setup(_ => _.Method3()).InSequence();
    }
    ...
}

The above checks that Method1 is called by any number of calls to Method2 followed immediately by a call to Method3. You can constrain the number of times the loop can be executed by adding a Times parameter such as the following example where the combination of Method2 and Method3 should be called exactly twice:

using (Sequence.Create())
{
    mock.Setup(_ => _.Method1()).InSequence();

    using (Sequence.Loop(Times.Exactly(2)))
    {
        mock.Setup(_ => _.Method2()).InSequence();
        mock.Setup(_ => _.Method3()).InSequence();
    }
    ...
}

Wrap-Up

Moq.Sequences provides a simple, intentional way of checking that things are are done in a specific order. Feel free to check it out at github. I welcome any feedback!

FitNesse with Finesse

I like to make agile training as realistic as possible by having the team build working software in a realistic agile setting. So I like to use the game of TicTacToe because everyone knows how to play it and the team can quickly be productive in building and enhancing it.

It recently occurred to me that the requirements for TicTacToe could be readily explained through tables. It seemed very natural to use Fit/FitNesse as the vehicle to describe the game functionality. As a result I wrote some Fit fixtures and FitNesse pages to describe the TicTacToe game for my most recent training session.

And I think it worked out rather well …

Fit & FitNessse Background

Fit is a tool for enhancing communication and collaboration on agile teams. It does this by describing how the software should work through examples written in tables. There are many different types of tables which are backed by “fixtures” which serve as a bridge between the tables and underlying production code. Fixtures are quite easy to write.

FitNesse is a Wiki front-end for Fit providing an easy way to write, manage and execute Fit tests.

The diagram below shows a high level architecture for Fit and FitNesse:

Fit FitNesse Architecture
 

TicTacToe FitNesse

The page below describes the winning combinations for TicTacToe as described in FitNesse:

Winning Combinations

Grey cells are inputs or descriptive text and the green cells are passed assertions. Failed assertions would be shown in red and any exceptions would show up as yellow. The Fit engine uses reflection to instantiate fixtures which then act as a bridge between the Fit tables and the production code. Fit handles all the table parsing

So What?

I find Fit to be a great way to express and radiate business intent. When I look at the above page it is easy for me to understand what the winning combinations are and the green cells let me know that the software understands too. Perhaps the “Turn” row which indicates whose turn it is shouldn’t be there so I should remove it. But other than that, the above does a great job of describing the winning combinations.

Fit story tests are a primarily a great collaboration tool between your product owner and the rest of the team. Ron Jeffries talks about the the 3 C’s of a story: card, conversation and confirmation. Fit story tests are the confirmation. The product owner and the team can collaborate and clearly define what “done” means for a story. This focuses the team on getting story tests to pass and helps avoid scope creep and getting side-tracked with other distractions.

Another benefit of Fit story tests is that they help drive ubiquitous language. The terms used in the story tests should be domain terms that are used consistently through the Fit fixtures and on into the production code. The need to express working examples in the constrained language required by Fit and the underlying fixtures requires the team to think carefully and consistently about the terms they use. I would expect the TicTacToe production code to have classes named “cell”, “mark”, “player” and so on.

What makes Fit even more powerful is that story tests can be run individually or as a suite. Tests can be run from build tools like Ant and NAnt and from continuous integration tools like CruiseControl and CruiseControl.Net. In other words, you can build up a regression suite of story tests and run them many times a day.

What Next?

If you have tried Fit or FitNesse I would be interested in hearing how it worked or did not work for you.

If you haven’t either one why not take them for a test drive? No doubt about there are challenges setting it up and getting it part of the team’s culture … hmmm sounds like a good idea for another blog post ….

 

Taking Stock of Your Software Inventory

Do you have inventory piling up between your development team and QA?

This inventory of unconfirmed functionality limits your team’s agility on a few fronts:

  • your software is not always potentially shippable
  • deferred ROI
  • delayed feedback from QA

One of the main goals of your agile team is to build potentially shippable releases every iteration. When QA testing is done after development releases your team can no longer achieve this goal.

This deferred testing also creates an inventory of unconfirmed features that need to be managed. And just like any manufacturing system, inventory is wasteful and costs your company time and money. Every day that your software is not shipped decreases the ROI.

The longer your software remains untested the more costly any corrective action becomes. The cost arises from many factors including the ramp-up time for the developers, context-switching time, overhead with issue tracking and so on.

Finally, feedback from good testers is invaluable. They think differently from developers and tend to have a more outward-looking customer perspective. If you get this feedback earlier in the process you will see a reduction in defects and less rework.

If you find your team in this situation here are a few things you could might consider doing:

Track Metrics

Select metrics such as defect counts or time to close that you feel best highlights the obstacles your team faces. Chart these metrics in big visible charts

Consider translating technical metrics into bottom-line financial data. For example, you could use daily expected revenue of the product to calculate the revenue loss due to each day of delay.

Get Developers and Testers Working Together

There has been much talk about getting quality assurance infused throughout the product cycle. Agile teams are tailor-made for this transition. Get a tester on your team and allow her to contribute throughout the iterationAside from traditional testing, the tester can bullet-proof the story tests, suggest testing tasks, pair with developers on testing and so on

This inter-personal contact will do more to promote quality within your team than any formal QA process ever will.

Stagger Iteration Testing

Staggered testing

If you must maintain a separate system test phase, consider having the QA team test the previous iterations release. This way, the QA team has a stable release for testing and feedback is delayed at most one iteration

Over time, you may be able to move additional QA activities into the development iteration.

What does QA Mean on Agile Teams?

I have struggled with getting traditional QA professionals to work effectively with agile teams. Testers are frustrated because there is scant coverage of “tester” practices in much of the agile literature. For instance, the XP bill of rights covers the rights of the customers and the developers. Testers don’t even get a mention.

I think this stems from an emphasis within agile processes on test-driven design and automated acceptance tests. Done well, these practices result in much higher software quality. Agile developers then question the value of “monkey” manual tests which are often covering the same ground as the developer and customer facing tests that are already passing. Now, to be fair, there certainly is information out there on the role of agile testers such as work done by Lisa Crispin, Janet Gregory, Brian Marick, Jonathan Kohl and Michael Bolton to name a few.

And there are groups and forums out there such as the agile-testers Yahoo group. Michael Bolton recently posted that perhaps the confusion arises from what we mean by QA:

One important step is to stop thinking about testers as “QA” — unless the A stands for “assistance” or even “analysis”, but not “assurance”. We’re testers. Management (in one sense) and the whole team (in another) provide the “assurance” bit, but since we don’t have control over schedule, budget, staffing, project scope, contractual obligations, customer relations, etc., etc., we can’t assure anything. We should be here to help — not to slow the project down, but to aid in the discovery of things that will threaten value.

I think Michael has it right. In a private email he provided a great distinction between a developer’s and a tester’s perspective:

In the developer’s world, the job is to create value. In the tester’s world, the job is to defend value by looking and seeing where it might be missing. The developer’s job is to discover how it can work. The tester’s job is to discover how it might fail — and by disproving a failure hypothesis, getting one more piece of evidence but never certainty that it will work under these sets of assumptions — ‘cos the assumptions are never sure things.

I couldn’t agree more. I see the role of a tester then as supporting quality infusion into agile teams.