Chartering Your Way From Developer Mindset To Tester Mindset

Chartering your way from developer brain to tester brain

The illustration above was generated by Bing Image Creator, and was so strange and bizarre I decided to use it. I have determined post-hoc that the bad spelling on the two boats is an indication of the in-between state of chartering between two brains. Sure, why not.

Switching that brain out of developer mode

You can find much talk about there being a difference between a developer mindset and a tester mindset. A long time ago I used to describe them as “developer brain” and “tester brain,” mostly for the humor of it. The discussion seems to flow along the following tracks:

• do either even exist
• the difference between the two
• can you switch between them

Select any one of the above, and you are going to find yourself picking sides of a heated egg toss. Mostly rotten eggs. You could try to straddle, but that just maximizes the number of eggs hitting you.

My side chosen, in short:

• I believe the mindsets exist
• I believe the different mindsets are better at finding different kinds of bugs
• I believe you can switch, but it isn’t easy

I am going to share something I experienced this week which seemed to make switching easier for me. I used a test charter.

What is a test charter?

I am going to use a simple definition that conflates test mission and test charter. I don’t believe the difference matters for this discussion, as I believe the development of both helps in the mindset shift. A test charter is what you intend to test. I prefer a somewhat open-ended, broad charter rather than detailed specifics. I prefer larger test problems, feature areas that might cover dozens of test ideas, test cases, questions, notions, investigations.

The key point is that the test charter takes you away from the “how do I make the code do what we need it to?” and toward “what do we need to learn about the code behavior?” It is a step away from implementation and a step closer toward things that might expose product risk.

Some charter examples

Consider the following charters:

• Check how the web UI performs across different browsers and OS platforms
• Investigate load characteristics of the stored procedures
• Perform penetration testing on the new product APIs
• Construct new data sets to test the import and migration features

Notice how the scope of the charters are a little bit open-ended. They imply planning and analysis that precedes the activity. The suggest what might possibly be multiple sessions of work and testing, which might lead to some charter splitting. It is common you realize a single charter is better covered with three or four.

My test charters

I have an open source project called DupIQ. This prior week, I implemented authentication and authorization in the web layer, along with varying levels of role enforcement. For this, I created a couple of test charters:

• Update automation suite to check basic authorization claims token requirement against existing functionality
• Update automation suite to check tiered authorization enforcement against existing functionality

I spent probably four days covering each charter. I found myself splitting the charters across sub features of the app. Specifically, I was taking different controllers in the REST API tier one at a time and adding the authorization support for each one. There are four controllers, so all told I wound up with eight testing charters that followed eight coding sessions, one after the other.

I found that while I committed to completing the work as described, I was unable to cleanly separate my testing and bug fixing. I hit too many blocking bugs mid-session that rendered continued testing useless.

My example described, let’s talk about how the mental switch played out. Everything I describe below is subjective. I talk about it as if I am referring to the human mind in general, but it is probably better to acknowledge that all of this is how it felt to my mind specifically, so your mileage may vary.

What does the charter do to your brain?

Contemplating the charter, creating it, implementing it, executing on it, forces you to switch away from developer mindset and into tester mindset.

Choices, choices, and more choices

The first step is coming up with testing charters. There will be many. Testing is about compromises against infinity and the number of charters to choose from is infinite.

If picking a charter is difficult, go for low hanging fruit. Testers jeer each other over simple confirmation, but it has to happen at some point, so let’s put this down first:

• check the basic functionality
• check the user stories

If you have been doing TDD and BDD, you might already covered these. Don’t punish yourself for duplicating your prior thinking, or for not having done this analysis yet. Pat yourself on the back either way.

From here, up the ante with some simple generic test investigations. This is like enumerating testing buzzwords. Never mind what they mean, you can look them up later when you flesh out your charters:

• Valid and invalid data
• Security testing
• Error states
• Performance testing
• Compatibility testing
• etc…

All the testers are probably eye-rolling at me right now. They are doing that because they know every one of those is more like a full on courseware of testing rather than something one can squeeze into a charter. It feels like rote category matching because it is rote category matching. Never mind that, because it is doing something that might not have happened otherwise.

Your brain is starting to shift modes. Instead of “what do I have to build into the product?” you are forcing it to contemplate “what kinds of things can I do with this product?” This may or may not fill you with a sense of dread. Some of these word-association handwaves are much bigger than they sound. Do you know all the various flavors of valid and invalid data for your product? Where would you put it? How would you submit it? How do you check the responses? How about the security testing? Is this a simple investigation, or does it demand a larger infrastructure and testing environment?

Let’s think about that feeling of dread. Do you know what that is? That is the developer part of your brain trying to avoid switching to the tester part of the brain. All that work, all that analysis, all that preparation building data sets and creating test environments, standing up fixtures and harnesses, all of those keep you from making changes to the product code.

That sense of dread is your first evidence of switching to a tester mindset. If you want to test your own code, you need to push through it.

At this point, hopefully, your imagination has picked up enough momentum from these easy pickings starter methods for coming up with charters and you start crafting ideas that are more relevant, more interesting, more creative. Perhaps your distributed caching system hasn’t been evaluated for how well it handles live schema migrations. Does the multi-player online game graphics engine handle updates coming from players with different laggy network latencies? Maybe nobody has scanned the middle-tier to data tier access permission handling for security risk. The ideas will start getting better the more you think about it.

Lean on your developer sensibilities for a little bit

Here is a trick that I find works splendidly.

Stop thinking about the testing as a thing which keeps you from working on the product code. It does keep your from working on the product code, but you need to shift your head.

Think about the testing as a problem to solve. You are building a testing solution. The testing is the product.

I find that when developers think about their tests the same way they think about building a financial workflow, or a distributed database load balancer, or a streaming video compressor, or a mobile card trading game, that they apply a lot of brilliant design and creativity.

Define the charter - getting further distance from the code

Once a charter is chosen, you need to flesh it out. You probably did some of that already narrowing down the choices. Going from “Security testing the cache” is ambiguous, but “Checking front-tier to caching-tier to storage tier access permission settings on deployment and account roll” gets the brain going and inspires something far more meaningful.

Charters will vary a lot based on product, problem, situation, but in general you should consider:

• what in detail does the charter cover
• what preparation needs to happen to be ready to act on the charter
• what kind of outputs do we expect from the charter

Some charters describe easy and simple activity that might require very little preparation (e.g. “Do quick behavioral survey of product UI to understand feature behaviors”) and others might demand a lot of pre-work (e.g. “Data import and update testing of directory to user profile store” - a real problem I worked on with SharePoint).

It is almost impossible to do this much work preparing a test charter for whatever testing activity follows without gaining some degree of critical distance.

Execute the charter - test sessions anyone?

A testing idea I learned about far too late in my career is Session Based Testing. It was developed by James Bach and Jon Bach, and I find it to be an elegant way to talk about managing testing time.

There are a lot of things to say about testing sessions and their advantages, but I want to single out one, very important aspect. Test sessions are about dedicating a focused block of time toward a given testing charter. James Bach recommends one to two hour blocks. I find that a little bit small for some activities (I have had some gnarly testing problems that took me more than a day to pound through), but it works well for probably most work one would ever do. It works well physically for managing energy. It works well mentally on the brain. It fits easily in the middle of a work day. All that is good, but the most important part to me is that you commit fully to testing for that session.

This is important to the testing mindset. Your test charter defines a set of objectives, goals for the testing. Your time is meant to satisfy those objectives. It is very difficult to do that if you stop the testing to fix a bug or make any other product change.

Remember the sense of dread we talked about earlier? That was your developer brain realizing how much work it was going to take to do all that testing. If you got this far, you managed to put that part of your brain to bed, but like a child who gets bored, that brain wants to wake up and do something. Just like parents in another room watching a TV show wakes the kid up, so too do bugs found during testing wake up the developer brain. “I get to go back to product code now? REALLY!!?”

This part is really difficult, because if you let the developer brain go back to the product code, you have put whatever remains of the testing charter back on the shelf. Will you get back to it?

Maybe that is fine. Maybe what is left of the charter is lower interest, lower priority. Maybe picking it up in fifteen minutes, an hour later, tomorrow is easily done. Or - and this is a big one that comes up a lot - maybe the bug you found testing so impedes the charter that continuing is pointless without a fix.

Or maybe it is not fine. Maybe finishing the charter is difficult. Maybe the fix is easily handled later. Maybe what remains of the charter is critically important, more so than whatever bug you have found so far.

At this point, lean on that charter. Look at it. Look at what you completed so far. Do you need to get back into developer mode, or would that distract you from important testing work? Ask yourself how you felt about the charter before starting. Let it be your guide.

Should a developer test their own code? What does the charter suggest?

Of course they should. Nobody serious about product development or testing in the last 30 years has suggested otherwise.

The real question is whether or not it is sufficient for the developer to do all the testing. To this, I offer the following considerations:

• is the developer time better used for what is described in the testing charter, or for fixing whatever is discovered from acting on the testing charter?
• is the developer the best talent to apply toward the given test charter, or would someone else with a different set of skills, resources, or perspectives be the better person?

Rather than answer this question for all problems and all situations, let the charter guide the answer. Many testing charters put the developer squarely centered as the best possible person to act on the testing needs. Some testing charters describe a problem where schedule, budget, skills, resources or expertise suggest someone else ought to do the testing work. Drop ideologies and dogma and consider the problem in context.

Need more ideas to get in the testing mindset? I have a book…

The book is really about authoring test plans, but part of that covers applying generic, well known topics to the test problem. It also describes different ways of describing a test problem, which I find helps a lot to organize the mind and shift the brain out of one mode or another. It is also a short, inexpensive read.