The Software Test Professionals Conference is quickly coming up Oct 19-21.

Have you registered yet?

If so, I look forward to seeing you there!

If not, what are you waiting for?!

Read on for 5 reasons to attend:

1) You are a Software Test Professional – if so, you need to be there! This conference is all about people like you who want to learn, improve themselves, connect with like-minded people, and grow themselves professionally. If any of these apply to you, sign up now!

• You are a tester, a manager of test teams, a project test lead, a programmer who tests, someone who manages beta testers or usability testers, or someone just interested in increasing the value of testing.
• You spend time improving your skill sets (or want to), and believe that continuous learning and challenging yourself is a necessary part of your job.
• You want to increase the value of testing in your organization.
• You are not interested in being a robot who blindly does what you are told, but finds ways to improve the way you do your own work so you can do your best work possible.  You also encourage this in others, by leading by example.

2) Learn by Doing, Not Listening in the Hands-On Testing Techniques Lab. The best way to learn new skills is to practice them hands-on while you are learning them. Come and challenge yourself with new methods in a safe environment with instructors and other students. This is a rare opportunity at any conference, so be sure to take advantage of the opportunity (and bring your laptop!).

3) Attend My Sessions & Talk With Me! There are many other great presenters to check out as well – but I know you’ll want to attend mine. ?

My pre-conference workshop, Test Management Clinic, will help you explore and learn techniques for solving real-life test management challenges. You may even leave with appropriate solutions to challenges you are currently experiencing. Register ASAP to reserve your seat now!

My session Explore Beyond the Obvious in the Hands-On Testing Techniques Lab will allow you to try different testing approaches to dig deeper into seemingly simple applications, and learn new information about products you test.

The panel I am speaking on about Reducing the Cost of Testing will help you gain ideas in reducing test costs, understand tradeoffs in doing so, and learn how to talk about testing costs with management in a reasoned, articulate manner.

4) Improve Your Career via the multitude of opportunities awaiting you. Network with people who are driving changes in the industry and understand their strategies so you can too. Connect with people around the globe who can help you with some of your challenges, and vice-versa. Learn leadership strategies to champion initiatives and changes in your workplace. Improve your skills in working with management, and making testing a valuable part of the organization.

5) STP Conference is in Las Vegas! Seriously, how much fun will that be? If you ever needed an excuse to get yourself to Vegas, this is it. Imagine – spending days learning to your hearts content and talking with fellow test professionals, then enjoying evenings in the fun of Las Vegas while connecting with fabulous testers from around the globe.

Register Now – The Early Bird Deadline ends this Friday August 20, which provides a $500 savings off the conference package.

See you in Vegas!

Yesterday my husband, son, and I took a road trip to seek out interesting places in our area. We have found it is easy to take local attractions for granted when they are so close, and often think “we’ll get there one day”, but that day rarely comes.  It was time to correct that for some select places of interest.

Castle Kilbride

The first stop was Castle Kilbride, a heritage site located in the small town of Baden, Ontario. We had driven past signs for it countless times over the last 16 years but had never been there. It proved to be more of a large opulent house than a castle, but it was interesting to see.

The Puzzling Tombstone

Our next stop was a small cemetery north of Wellesley, Ontario, which was purported to contain a tombstone with an inscription that was in itself a puzzle. Readers had to decipher the puzzle in order to understand what was written upon the tombstone. Being a lover of puzzles, we spent time sitting in the cemetery in front of the infamous gravestone, and proceeded to decipher the message. It took some time, but we did figure most of it out. What fun!

Take a look at the photo below, and see if you can decipher it’s message. No cheating!

So… Did you solve it?

What techniques did you use to get started?

What methods did you apply as you deciphered the message?

Did you use any heuristics, oracles, design, or testing techniques to help you solve this puzzle?

Consider how this exercise can help you in your day job, whether you are a tester, a programmer, or general agilist.

Please share your insights and thoughts!

To finish my story…

The last stops on our road trip were searching for Pioneer Tower in south Kitchener, Ontario, and Sheave Tower in Blair, Ontario. Unfortunately we did not have exact locations, and were unable to locate them.

After returning home, we did some searches online and discovered we drove right past the access points for them. They turned into an adventure for today, instead.

PS. You can see the solution to the tombstone puzzle, and the history of the origins of the tombstone at the link provided here. Please do not post this information on this blog. If you are trying to solve the puzzle, do not look at this page until you have finished. Tombstone Puzzle, Rushes Cemetery, Woolwich Township

Having been a black box software tester for a few years already, I still knew there was a lot I could learn from this course specifically. Hands-on courses with exercises requiring active observation, critical thinking, and collaboration with peers were in short supply back then. I relished the opportunity to challenge myself and to learn, so I was hooked.

Oh, and it was free. That made it easy to get approval from my manager to take the course (ha! Free = no approval necessary!).

Unfortunately, I consistently had obligations that kept me away from the course. Excuses? No. That’s just how it was.  Starting and raising a family while moving up the ranks in responsibility at work took priority.

Fast-forward five years.

The Black-Box Software Testing Foundations course was moved to be offered and taught through the Association for Software Testing. Cem is actively involved with the AST, as are many other bright, active, and thinking testers who do a lot to contribute to, and further, the craft of software testing.The BBST Foundations course is offered for free to all AST members. There are subsequent courses offered as well, with a nominal fee for both members and non-members.

All course material is still available on the Center for Software Testing Education & Research website, so you can take it on your own time, though you won’t benefit from the peer and instructor interactions (which are a BIG piece of the learning throughout the course).

In March I finally found the time to register for the May 2010 offering of the BBST Foundations course, as I had a lighter workload than usual in May. I admit that I was somewhat nervous as I had not actively applied my skills at a “doer” level in quite some time, due to numerous roles at management, project/program management, and other similar, levels.

Without giving away pertinent course details that will require course instructors/designers to have to devise new materials to evaluate student comprehension, I can say that I enjoyed the BBST Foundations course immensely.

Reflections on BBST Foundations

The course was advertised to be about 8 hours of work, per week. While that seemed like a fair amount, I was quite surprised to find myself spending 20-25 hours on course work per week. It was well worth it though.

Highlights of my experiences during the BBST Foundations course include:

• Online course experience. I found this difficult at first, as I prefer to work with people in person to observe body language, and to form bonds. It is more challenging to do via email and forum posts, but possible with a concerted effort. This experience gave me hope for future online learning experiences. The course designers encouraged a lot of peer interaction to promote and facilitate learning, which did wonders for everyone’s learning.

• I met phenomenal testers, both new to the field, and many years experienced; both instructors, and students; located here in Canada, North America, Europe, and Asia. I appreciate the learning I obtained and experienced with all of them.

• I learned, a lot! Having been in the software field for 10 years, I wondered how much I would learn from an introductory black box software testing course. I discovered that there was a lot to learn still. Without providing specific details, I can tell you that whether you are new to testing, a small number of years in, or a decade or more, this course will get you thinking critically, and from different perspectives.

• I still don’t like multiple choice tests. Yes, there are multiple choice quizzes in the course. These are intended to help you think critically while reading text to correctly understand intent and respond with the appropriate answer. Sometimes only small nuances make the difference between a correct and incorrect answer. I have never been very good at multiple choice, and discovered I still am not. That said, the exercises certainly helped me slow down and try to think the through questions using different approaches.

• The assignments were incredibly valuable to explore the concepts of oracles and heuristics. For whatever reason, I have often struggled with the terms ‘oracle’ and ‘heuristic’, despite that I can use them appropriately in context. Regardless, the exercises really nailed home these terms and improved my application of them in practice.

• The exam has a study guide, which is a series of questions that you are able to attempt and discuss with fellow students. What an idea! Imagine: Time to think through questions and improve upon answers based on peer comments before the actual exam. This really helped me better understand the questions and provide more context-appropriate answers.

So, a lot of work?

Absolutely.

Well worth it?

Yes.

I love to learn, and apply learned skills, to add more value to a business (and subsequently), more value to my immediate organization. I found this BBST Foundations course to be challenging, educational, and appropriately balanced in introductory concepts and applications.

I highly recommend the BBST Foundations course to all levels and experiences of software tests (heck, even non-software testers).

Be sure to sign up for upcoming BBST course offerings on the AST website.

.. and hey, I passed the BBST Foundations course! Yay!

Finding important, messy, and hard-to-nail-down bugs is easier than by using the old blindly-run-test-cases-anyone-at-all-could-do-because-little-thinking-goes-on variety.  I personally thrive in being able to use my experiences, curiousity, a variety of test techniques, and brain power to test software (see more in my recent post Be An Explorer!).

Moving into using session-based exploratory testing to help manage testing across a team and provide metrics to management was easy for me.  It was straightforward to write charters that were not so specific as to be test cases, but also not so generic that they were not helpful.  I was able to quickly write a charter that identified the mission and goal of a test session with appropriate depth and breadth.

In working with many dozens of testers over the years, I discovered that creating appropriate charters was not easy for many of them.

Many ‘charters’ I saw were a test-case in disguise, and in some cases, a really bad one that you could see right through to the meat-and-bones of it.  I wondered if these testers did not have the right mindset to be able to understand what made a charter different from a test case.  Or maybe they weren’t given the coaching they needed to be able to write a good charter.  Or maybe something else entirely.

Other ‘charters’ I came across were akin to a movie title or headline, like ‘User Phone Home’, ‘Save The File’, or ‘When Files Go Bad’.  Sure, these ‘charters’ could be appropriate in some situation, but they leave something to be desired in communicating the mission of a test session.

What should a Charter look like?

James Bach says that Charters:

Are intended to communicate the mission of a test session clearly and succinctly to testers who have already been trained in the expectations, vocabulary, techniques and tools used by the organization. Remember, in ET we make maximum use of skill, rather than attempting to represent every action in written form.

Elaborating and combining with my own experiences, the basic questions a charter is likely to address are:

• Where do you want to go?
• What do you want to learn about?
• What should you use to get there?

I recently worked with a group of testers in a workshop to practice different testing skills, paired testing, and in writing test charters and session notes.  Day-to-day, many of the testers were writing test cases instead of charters, which was in opposition to their work environment.  So one key objective for the workshop was to help them learn how to write appropriate charters that clearly communicated the mission of a test session.

Participants finished with a brainstorming session to identify what constitutes a good charter – results are included below.  The applicability of some attributes will depend on the skill and experience level of the tester, so consider your context and what fits for you.

Attributes of a Great Test Session Charter:

• Defines objective, a clear purpose
• Identifies the area of focus
• Identifies general scope of the test coverage intended (features, test approaches, depth of testing)
• May provide guidance on what to look for (if someone else will test the session, and is inexperienced)
• Uses language that encourages exploring!
• It is clear, concise, accurate, and to the point
• It is generic and non-precise in the domain of the focus area (NOT a specific test case – it wouldn’t be exploratory testing then!)
• May associate application under test with another similar application (as an oracle – this is helpful if application very new, and there is little information available about how it should work).
• Contains NO mention of expected results
• Does NOT direct the testing that should occur, or identify what the tester should or should not find
• Generalized, leave open to interpretation to who is running it (don’t assume who will test it)
• Provide guidance for things to look out for
• Bullets not numbered lists (if you are going to use them)
• Consider though that lists may be limiting for people (not mind expanding)
• Examples can be helpful if generic enough, helps guide and think of other ideas
• Be careful of being too generic!  Don’t want to necessarily create a charter that can be applied to anything
• All depends on context – whether people are experienced or not, whether you have information or not, etc..

What do you think?

Is it helpful to apply these types of attributes to help people create better test charters?  Are there any that you would add, or some you would take off the list?  Or, do you think that the writing of charters themselves is limiting and unhelpful?

I would love to hear from you about this!

It was beneficial as I was able to help employees learn new concepts and improve skills,  in a manner that fit for them.  The added bonus for me was that I learned something new from them as well.  What powerful experiences those were!

Now, as a manager for 18 testers who are spread across several different agile teams, I find it difficult, if not impossible, to coach testing skills one-on-one.  On a weekly basis I also spend time coaching the agile teams, management-level initiatives to improve company-wide practices, and the usual management administration.  Such is life in a ‘start-up’ type of company.  As a result, my one-on-one time with each person is limited to 30-60 minutes over two weeks.  This seems a miniscule amount given they are working a minimum of 80 hours in that same time period!

I am someone who is motivated and energized in working with others to learn new skills, to understand and be able to apply new skills appropriately, and to have those ‘light bulb’ moments where things just click.  Given my work situation, I have been frustrated with how little I have been able to do this.  This left me with a problem!

How can I increase the amount of coaching and teaching time each individual is receiving?

Solutions:

• Forget about it: Nope! I’m too invested in helping others learn and grow.
• Clone myself: Not feasible, as in I don’t know how, and it is kind of icky to think about.
• Shift responsibilities to other people: The possible people to offload to are just as overloaded as me.
• Find additional ways to teach and coach skills so each individual gets more opportunities to learn and improve: There are lots of ways to do this!  I just need to invest some more time to create these opportunities.  Some possibilities:
  • Coach a smaller group, and have them coach small subsets of the larger group.
  • Direct them to other people who I trust will coach them appropriately.
  • Move our 1:1′s to their work space & turn it into a paired testing session.
  • Create workshops where they can learn and practice skills.
  • Coach other people to create workshops where they can learn and practice skills.

What I’m Doing:

Ongoing learning and growth are important to me.  Happily, I discovered that they are important to my department as well.  They requested regularly scheduled workshops/training sessions to be reinstated after being cancelled during the agile training the entire engineering organization went through over the summer.  They missed them!  They missed the learning, new perspectives, and much needed ‘down time’ at work.

So, regular department workshops / training sessions were reinstated on a biweekly basis.  We are currently shifting to make them weekly.

Our first workshop back was a brainstorming session to come up with a list of loosely prioritized workshops the team members would like to have provided.   They came up with a lot of great topics, which we are adding to on an ongoing basis, and are adapting priorities as needed.

The workshops we are doing are typically 2 hours long, and are created by myself or a team member, and contain both teaching of concepts, and time to explore, practice and apply concepts.  We customize workshop focus and content based on the people involved, what they want to learn about, and their current knowledge-base and skill sets.  A blend of ‘teaching’ time and ‘experience’ time are proving vital to help people go back to their day-to-day work and integrate new learnings – which when applied to hands-on work become new-found skills when they invest time in practicing them.

The workshops also allow me to engage with everyone both as a group and individually, and determine how I can add more value for them when we have our regular one-on-one conversations.

Final Thoughts:

I am not yet satisfied with the amount of coaching and teaching I am able to provide to each individual. However, I am happy with the progress I am making in increasing the amount of learning opportunities they are getting from me.  These workshops have allowed me to stretch the limited time I have to teach, coach, and add more value for all of my department members.   This is an improvement, and a step forward to incorporating other learning opportunities for my department.

I plan to share some of the workshop experiences in future blog entries, so keep an eye out for those!

On Teaching Software Testing:

My thoughts on this have been bouncing around in my head for a while – how can we improve on how testing skills are taught?  What is the best way to teach and coach them?

For myself, there are things I am doing now which are helpful, but there is so much more I would like to do, and I’m sure could be doing that I haven’t considered yet.  I know I am not the only person in that position.  So where do we begin?

Matthew Heusser recently blogged on this very topic.  He invited people to comment on what types of teaching and training are wanted by organizations for their employees, and what would be most effective for the employees.   I added my comments to that entry, alongside some brilliant thinkers in this field.  Thank you to Matt for starting a great conversation thread!  You can read it at http://blogs.stpcollaborative.com/matt/2009/11/18/you-say-you-want-a-revolution/.

Your turn!

What are your experiences and thoughts on how to coach and teach software testing skills?  As a coach/teacher/trainer, what have you done to do this when you have had little time to do it well?  As a tester, how do you want to be coached and taught – what would be most effective for you?

I welcome your comments, and encourage you to share them both here and on Matthew’s blog.

When projects get chaotic it is easy for people to get caught up in the little details, and forget the big picture.   Our projects have been quite chaotic lately, so I have been seeing this happen regularly with everyone.

One major thing that happens as a result is that people lose sight of what their primary goals are.  When this happens in my department, they lose sight of their primary goals as testers – one of which is to find important defects fast!

So a couple of weeks ago I revisited activities that emphasized observational skills.  I find that activities like these really help rebalance and refocus people, and that their test efforts improve thereafter.

Thanks to Adam White’s blog, I was able to use some exercises I haven’t used before.  These are the ones we looked at:

1. Which way is she spinning?
2. Are the two blocks different colors?
3. Spot the Differences

For each exercise, we used the following process:

• Load the exercise
• One minute to observe and make any notes of observations
• No discussion allowed
• When minute passed, everyone stopped making notes.
• One-by-one, each person read the notes that they made.
• No additional comments could be made until after everyone had read their notes.
• Group debrief and discussion followed.

We used this process for several reasons:

• To allow each person to have their own unique experience, without influence from other people’s experiences.
• To hear what other people observed afterwards, so you can determine how they view things differently than you.
• To begin to understand our own observational bias’
• To collaborate afterwards in techniques we can each apply to increase our abilities to observe more things!

Note:  Try the exercises now before you read any further!

All three exercises lead to sharing some interesting observations, and subsequent discussions.    Below, I have shared some of the observations people noted during each exercise, and any subsequent learnings we took away.  You may notice that the second exercise (blocks) resulted in the most active discussion, with valuable insights that we could relate back to our daily work activities.  Is this some sort of observational bias in itself? 

Spinning Silhouette Illusion

Observations

• The girl is spinning clockwise.  She is using her arm and leaning slightly to help keep herself balanced.  She appears to be bouncing.  Her leg seems to come down on an angle.
• The girl is spinning on her heels and her toes never touch the ground.  Her body is bent backward to help stay balanced, and her right arm is always up.  She is jumping in the air while spinning.  The shadow for her right foot only shows when she is facing forwards.
• The girl is spinning clockwise, then counterclockwise.  After I noticed the title of the exercise, I watched her foot disappear in shadow.  When I looked back up and she was spinning the other way!
• The girl is spinning clockwise with her hands open, and leaning slightly.  She is possibly nude and has a ponytail.  She is landing on her left foot after jumping.
• The girl is spinning to her right, and never spins to her left arm.  Her body does not move relative to her other body parts (as if she’s frozen), even though she is moving up and down on her heel.  Her head is tilted to the right and her knees are locked.
• There are lines in background behind the girl!

Debrief: How does this exercise relate to the everyday?

• We all notice different things in the same situation.
• Some people were not able to see the girl spin in both directions.  This reflects limits that we each have in day-to-day life.  We may never be able to notice particular things that other people are able to notice.
• Sometimes the things we notice are not important.  How do we learn to notice the important things before the trivial ones?

Block Colours

Observations

• They are the same colour, if you mean the letters (A & B).
• No they are not.   B is not as dark as A, based on the cast of the shadow.
• We know they are supposed to be the same colour, but we can’t see that they are!

Debrief: How does this exercise relate to testing?

• Testing components in isolation without interacting with surroundings isn’t necessarily valid!  In this exercise, looking at just the block colours, without taking into account the shadow casts gives a different perspective.
• In testing you may not be able to deduce things correctly on your own, but may need to harness other tools to help you see what is really going on.
• In testing, I would file a bug against the image because it doesn’t meet the spec (they’re the same colour).
• Sometimes when you think something is happening, it isn’t actually happening at all.
• Without the letters, you wouldn’t even stop and think that the blocks should be the same colour.
• In reality, the blocks aren’t the same colour, the shadow is causing them to be the same.
• What is a shadow?  Which concept is correct?
• Is this a 3D image, or a flat image?
• Hey.. what is reality here?  Which reality is correct?  Whichever one makes the business the most money!
• The spec should have been more clear as to what the expectation is.
• The environment plays an important role – what we are testing behaves differently under different circumstances.
• Your experience is all relative, and is based on your frame of reference.

Spot the Differences – Castle Picture

Observations

• All of the identified 13 differences were noted, plus one extra one!

Debrief

• No one was able to spot all the differences.
• Most people saw different combinations of the differences (no one noted the exact same differences).
• Very interesting that someone was able to find a difference that was not noted on the exercise webpage.
• Seeing things at eye level is different than seeing things from a different perspective (from below, or up higher).   This is called Parallax.

Wrap Up

All said and done, everyone thought this was a very insightful session.  They left feeling more open-minded and positive about their work, and how to approach their work differently. It was also nice to get out of the day-to-day grind, and have an experience that could be more generally applied.  Final remarks and reflections that were made as the session wrapped up were:

• Observation is crucial.
• Finding ways to be more self-aware to turn off our blinders can help us be more observant.
• Looking at things as a team, or a group, and comparing results helps get more coverage and notice more things.
• Everyone applies a different method when experiencing something, and sees different things.

I hope you found value in these activities as well.  I would love to hear about other exercises you have done too!