Finding important, messy, and hard-to-nail-down bugs is easier than by using the old blindly-run-test-cases-anyone-at-all-could-do-because-little-thinking-goes-on variety.  I personally thrive in being able to use my experiences, curiousity, a variety of test techniques, and brain power to test software (see more in my recent post Be An Explorer!).

Moving into using session-based exploratory testing to help manage testing across a team and provide metrics to management was easy for me.  It was straightforward to write charters that were not so specific as to be test cases, but also not so generic that they were not helpful.  I was able to quickly write a charter that identified the mission and goal of a test session with appropriate depth and breadth.

In working with many dozens of testers over the years, I discovered that creating appropriate charters was not easy for many of them.

Many ‘charters’ I saw were a test-case in disguise, and in some cases, a really bad one that you could see right through to the meat-and-bones of it.  I wondered if these testers did not have the right mindset to be able to understand what made a charter different from a test case.  Or maybe they weren’t given the coaching they needed to be able to write a good charter.  Or maybe something else entirely.

Other ‘charters’ I came across were akin to a movie title or headline, like ‘User Phone Home’, ‘Save The File’, or ‘When Files Go Bad’.  Sure, these ‘charters’ could be appropriate in some situation, but they leave something to be desired in communicating the mission of a test session.

What should a Charter look like?

James Bach says that Charters:

Are intended to communicate the mission of a test session clearly and succinctly to testers who have already been trained in the expectations, vocabulary, techniques and tools used by the organization. Remember, in ET we make maximum use of skill, rather than attempting to represent every action in written form.

Elaborating and combining with my own experiences, the basic questions a charter is likely to address are:

• Where do you want to go?
• What do you want to learn about?
• What should you use to get there?

I recently worked with a group of testers in a workshop to practice different testing skills, paired testing, and in writing test charters and session notes.  Day-to-day, many of the testers were writing test cases instead of charters, which was in opposition to their work environment.  So one key objective for the workshop was to help them learn how to write appropriate charters that clearly communicated the mission of a test session.

Participants finished with a brainstorming session to identify what constitutes a good charter – results are included below.  The applicability of some attributes will depend on the skill and experience level of the tester, so consider your context and what fits for you.

Attributes of a Great Test Session Charter:

• Defines objective, a clear purpose
• Identifies the area of focus
• Identifies general scope of the test coverage intended (features, test approaches, depth of testing)
• May provide guidance on what to look for (if someone else will test the session, and is inexperienced)
• Uses language that encourages exploring!
• It is clear, concise, accurate, and to the point
• It is generic and non-precise in the domain of the focus area (NOT a specific test case – it wouldn’t be exploratory testing then!)
• May associate application under test with another similar application (as an oracle – this is helpful if application very new, and there is little information available about how it should work).
• Contains NO mention of expected results
• Does NOT direct the testing that should occur, or identify what the tester should or should not find
• Generalized, leave open to interpretation to who is running it (don’t assume who will test it)
• Provide guidance for things to look out for
• Bullets not numbered lists (if you are going to use them)
• Consider though that lists may be limiting for people (not mind expanding)
• Examples can be helpful if generic enough, helps guide and think of other ideas
• Be careful of being too generic!  Don’t want to necessarily create a charter that can be applied to anything
• All depends on context – whether people are experienced or not, whether you have information or not, etc..

What do you think?

Is it helpful to apply these types of attributes to help people create better test charters?  Are there any that you would add, or some you would take off the list?  Or, do you think that the writing of charters themselves is limiting and unhelpful?

I would love to hear from you about this!

For some it is a passion, and the only way to do testing critically, skillfully, and effectively.  It is an opportunity to use your brain, not only in day-to-day life, but also at work (imagine!).  …To be paid for questioning and thinking, not to blindly do what someone else said should be done.  …To feel alive!

After being introduced to exploratory testing by Michael Bolton several years ago, I never looked back.  It supported all the feelings I had about the dire write-tests-run-tests-ho-hum-drudgery I found myself in early in my career in testing software.

In those days I believed there was a better way to do testing, and so I did it amongst the management-mandated ‘test plans’ (word documents with lists of test cases to be executed).  I explored the software, aiming to find the next bug all on my own, using my wits and smarts.

I never knew that others were doing this too.  I never knew that other people were alive on the high of being software detectives just as I was; that they were hunting down the elusive bugs that everyone knew existed, but only a small few could find.

I especially did not know that people like James Bach and Michael Bolton had put a name to the testing I enjoyed – Exploratory Testing.  How wonderful it was to discover that a revolution was afoot!

I was even more excited when I discovered that James had created something called Session-Based Test Management (SBTM) to help manage the exploratory testing that people did for a project. Wow!  It was exactly what I needed at the time to take exploratory testing into the main stream in my department.  It created an opportunity to work with team members to more formally and effectively coach on and track the work we were doing and the coverage we were obtaining.  It was just enough data collection to satisfy management, yet not so much that it was seen as unruly overhead for the testers.

In the years since, I have moved away from doing a lot of hands-on testing myself.

Instead I have enabled dozens of people to become skilled testers, and transitioned several departments over to using exploratory testing (for small projects through to large programs).

I have found ways to meld the management of exploratory testing into uber-SBTM spreadsheets, home-grown applications created by co-op students, small test management applications, and even popular large scale test management programs.

I have continued to coach and teach testers to think, to be critical, to question, to become skilled at a variety of test techniques, and to find passion in their work (or find it elsewhere!).  And I have learned so much from each of these people along the way.

I did all this by working and collaborating extensively with a number of equally passionate, skilled, questioning, and thinking testers.  I feel privileged to have worked with so many talented people.

While I love the work that I have done, and am doing now, I admit that I sometimes miss the thrill of the hunt.

• I miss tracking the scent of a bug, and the thrill of a capture.
• I miss solving the riddles of elusive bugs that are so challenging to locate that it takes a collaboration of several people to figure it out.
• I miss the quiet satisfaction in unearthing a problem that is both disturbing and unexpected, where team members stand in awe of your accomplishment.
• Most of all, I miss the day-to-day collaboration and learning from other testers while working together on the latest and greatest software.

But I love what I do, and I am passionate about my work.  I am an explorer by nature, yet there are more technically capable testers out there than I.  So instead, I am fortunate in being able to enable their abilities as skilled, thinking, exploring, questioning, passionate testers.

So now I ask you….  Are you a software explorer yet?    If not, what are you waiting for?!