Thursday, January 19, 2017

Re-testing without the tester

Some days we find these gems we, the testers, like to call bugs. Amongst all kinds of information, bugs are often things we treasure in particular. And we treasure them by making sure they get properly communicated, their priority understood and when they're particularly valuable, reacted on with a fix.

We're often taught how the bug reports we write are our fingerprints and how they set our reputation. And that when something relevant was found and fixed, the relevant thing is worth testing again when a fix is available to see that the problem actually has gone away.

We call this testing again, the precisely same thing as we reported the bug on, re-testing. And it's one of the first things we usually teach to new people that there is a difference in re-testing (precise steps) and regression testing (risk around the change made).

Today I got to hear something I recognize having said or felt many times. A mention of frustration: "they marked the bug closed as tested, and it turns out they only checked that a big visible error message had vanished, but not the actual functionality".

This was of course made even more relevant with external stakeholders coming back with the feedback that something had indeed been missed.

What surprised me though was the quickness of my reaction to mention that it was not *just the tester* who had failed to retest the fix. It was also the programmer who did the fix, who had completely closed their eyes on the actual success of whatever change they did. And to me, that is just something I want to see different.

It reminded me on how much effort I've put on teaching my close developers that I will let fixes pass into production without testing them - unless they specifically ask me to help because they are concerned of the side effects or don't have access to the right configuration that I would have.

Re-testing needs to happen, but re-testing by a tester is one of these relics I'd rather see more careful consideration when it is done. 

Two people who get me to do stuff

Sometimes, I feel like I'm a master of procrastination. For some types of tasks (usually ones requiring me to do something small by myself that has little dependency or relevance to other people) just seem much bigger than they realistically should. I wanted to make note of two people I respect and look up to for making me do things I don't seem to get done.

'I'll just sit here until it's done'

There's a team next door here, that works on the same system but we could easily organize our work so that we don't share that much. I had decided however I wanted to try out running their test automation, maybe even extending that when things I want to test benefit from what they've built. And I got the usual mention: there's instructions, just three steps. So I went and followed the instructions, only to be there in (typically) unlucky day when they had changed everything except their instructions while upgrading their test runner.

So a day later, I hear they've improved the instructions and we're back to just three steps. As I work on something else, I don't really find the energy to go back and see how things are. I gave it change, it did not work out, not like I really need it anyway. So my favorite colleague from that team comes into my team room, with his laptop and sits on the corner of my desk saying: 'Try it. I'll just sit here until it's done'. And I try it, and five minutes later we have delightful discussions on my feedback on making sense of this as someone new.

Thinking back to this, I realize this is a tool he uses all the time. Actively deciding something needs to be done and committing his time to insert positive pressure by just being there. Setting an expectation, and making himself available.

'Let's pair'

Another person I admire takes it further. He volunteers to pair and actively schedules his time to get more out of the shared work. Sometimes his 'Let's pair' attitude feels like pushy, but the results tend to be great. It takes time to get used to the idea that someone is there with you on you doing something you know you sort of could do by yourself.

As one of the organizers for European Testing Conference, he has paired with every one of us. The pairing has both supported timely doing of things, but also created contents we wouldn't create without pairing.

There was a task that I needed to do, and I was trying to find time in my busy schedules to do it. With him proclaiming 'Let's pair on it', it got done. And while I was sure I had the best skills for the task, I was again reminded on the power of another person on identifying things I could be missing.

From Envy to Stretching

I find it extremely hard and energy consuming to force myself on people who are not actively and clearly inviting my participation. So I envy people who, with a positive attitude just go and do it, like these two people. Recognizing the envy gives me a personal stretch goal. Try it, do more of it, find your own style.

It's not about doing what they do, but knowing if doing what they do would help you in situations you experience. 

Wednesday, January 18, 2017

A day in the life of bug report avoider

I'm doing really well on my "No Jira" project at work. Four months in, and I've so far been forced to write one bug report in Jira, even if there's been many many things I've helped identify, address and fix.

Today is one of my particularly proud days. I was testing this new thing that the developer just introduced as soon as it was declared available. And as I was puzzled and did not seem to quite understand enough to get the response out of the software that I wanted, I immediately asked him. He walked up to my desk from the other corner of the room, was just as puzzled as me until he told me to check the version number. The number does not (yet) tell much to me but made him realize that I was on the wrong version. And a minute later, he realized his build job had failed without letting him know about it.

So I get the working version, and my main concern is these five similar switches that are not really similar. They're on/off, and one of the five is a master switch to three and the combinations are pretty hard to grasp. So I turn them all off, and then just try turning the master switch on. And I end up with confusion, again calling the dev to show me the basics. This time he is puzzled a little longer. It worked for him. It does not seem to work for me. It takes a while before he realizes that of course I have got the combination in that causes the most confusion. I thank him for help and watch him say five minutes later: "I will change that master switch. It can't work that way."

I put a lot of personal energy on trying to figure out not just how to find the problems, but how to create experiences around those problems that make people want to fix them. Jira (or similar bug tracking tools) were long in my way for this.

But I realize I can do this because I have no fear where I work, whether the fear is real or perceived. I trust that I can drive things in ways that I believe make things better. I don't feel the need of leaving a track of bug reports to show my work. And I'm grateful to be in this position as it sets us up, together as a team, better for success. 

Tuesday, January 17, 2017

Step Up to the Whiteboard

You hear an idea somewhere and realize it's something you could do but aren't doing. You decide to try it out, but feel awkward at first on changing the dynamic of how you work - after all people are used to you filling your role in a particular way. But over time, you get the courage. And when you get started, you don't want to stop because it gives you more than you could have anticipated.

I've had several of these experiences. One of the very valuable ones coming from a session in some agile conference I can't even pinpoint to correctly attribute is the use of whiteboard. I believe it is one of those things that I kept seeing over and over again without anyone particularly emphasizing, and at some point it became the thing I needed to try.

The transformation of how I am in meetings has been quite significant. From the person on the computer making detailed notes (and often ending up as the meeting secretary), I've moved my role to be the person on the whiteboard, making conceptual notes. And where in my written detailed notes I never got feedback if we understood things the same way, my notes on the whiteboard invite corrections, additions and depth. The visual nature of the boxes and arrows helps us discuss things, remember things and go deeper into the things.

My phone is full of pictures that remind me of discussions in the past, and they work better than any of the detailed notes I used to have.

The big part for me was to dare to change myself and try out something I wasn't sure if I knew how to do. In doing, I discovered that the best way to learn to do it is doing it.

And every meeting, I step up to the whiteboard. It helps me keep track where we are in discussions. It makes the people changing topics in middle less confusing, as the image on the board must change. And it makes sure I'm not alone with my confusion.

Sunday, January 15, 2017

The Overwhelmingly Helpful Comments

I'm going through a bit of an emotional flashback for a discussion I saw on twitter and tried to dismiss. Did not succeed to well with the dismissing, so I'm blogging to offload.

One of the 49 % of the 454 people (woman) I follow on twitter posted her Selenium code sample from a short training she had just delivered. And two of the 51 % of the 454 people (man) I follow on twitter posted comments criticizing her code.

I truly believe these comments of critique were made in good faith, and with the intention to help improve. Both comments introduced concepts that were missing from code ('page object factory', 'java 8 features like lambda') and you could even assume the authors knew there could be a reason things are excluded even if they could be there too.

What brought me to blogging is that it took me back to the time when I started coding decades ago, and when I stopped coding for precisely these kinds of lovely, helpful people who were suffocating me.

Talking in metaphors

I love dancing but I'm not a dancer. When I go to dancing lessons, on the basic level my teachers usually correct only the most relevant things, and a lot of times, they don't correct anything. They let me be surrounded with the joy of dancing, encourage continued practice without critique.

They could also do things differently. They could start right away telling me to pay attention to my dancing position. But they could also point out continuously everything that I could do differently and better. I could hear of my facial expressions (smile!), the positioning of every part of my body and the fact that you know, there's all these subtle differences to rhythms I'm not yet ready to pay attention to.

The joy comes first. And the other stuff comes layered. And sometimes, the feedback just sucks the joy out of dancing because that's all I want to do.

Being a Woman who Codes

When you are minority, the positive and helpful people around you tend to all want to pitch in to the feedback. The style of comments may be very constructive, but the amount of it can be overwhelming. You see that the amounts are overwhelming just for you because they just don't care as much for the others. The others are lucky if they get helped, but you get helped by everyone.

Everyone looks at what you do in a little more detail. Everyone wants to help you succeed with their feedback. And then there's someone, usually a very small minority, who uses all the feedback you're getting that others don't as evidence that 'women are not meant for coding'.

In projects with crappy code from everyone else, I always felt the feedback was asking me to be more perfect. Good intentions turn into sucking the joy out of the whole thing. I dropped coding for 20 years. And even as I've come back, I'm still overly sensitive to being helped in overwhelming amounts.

The environment matters

Recently, experiencing projects where pull requests get, regardless of gender, criticized and improved in detail are places I find safe again. It's not special treatment, it's feedback for everyone. And it comes from a place of putting every line to production pretty much as soon as it gets committed to the main branch.

Back to the twitter incident

So with the twitter incident of commenting in particular for this piece of code, I would ask:

  • Are these same people giving same attention to every other public speaker's code?
Selective helping is one of the things I've experienced that drove me away from coding. I can't speak for anyone else, but I surely know that at a younger age, it made a difference to me. I would not be back without (strong-style) pairing and mobbing. 

Saturday, January 14, 2017

Thinking in Scopes

The system I'm testing these days is very much a multi-team effort and as an exploratory tester looking particularly into how well our Windows Client works, I find myself often in between all of these teams. I don't really care if works as designed on my component, if the other components are out of synch failing to provide end users the value that was expected. 

Working in this, I've started to experience that my stance is more of a rare one. It would appear that most people look very much at the components they are creating, the features they assign to those components and the dependencies upstream or downstream that they recognize. But exploring is all about discovering things I don't necessary recognize, so confirming and feature focus won't really work for me. 

To cope with a big multi-team system, I place my main focus on the two end points that users see. There is a web GUI for management purposes, and there's a local windows client. And a lot of things in between, depending on what functionality I have in mind. As an exploratory tester, while I care most for the end-to-end experience, I also care in ways I can make things fail faster with all the components on the way and I have control over all the pieces in between. 

I find that the decomposition of things into pieces while caring for the whole chain may not be as common as I'd like it to be amongst my peers. And in particular, amongst my peers who have chosen to pay attention to test automation, from a manual system tester background.

Like me, they care for end to end, but whatever they do, they want to do in means of automation. They build hugely complicated scripts to do very basic things on the client, and are inclined to build hugely complicated scripts to do very basic things on the web ui - a true end-to-end, automated. 

There's this almost funny thing for automation that while I'm happy to find problems exploring and then pinpoint them into the right piece, I feel the automation fails if it can't do a better job at pinpointing where the problem is in the first place. It's not just a replacement of what could be done manually while testing, it's also a replacement for the work to do after it fails. Granularity matters. 

For automation purposes, decomposing the system into smaller chains responsible for particular functionality gets more important. 

I drew a picture of my puzzle.


Number 6 is true end-to-end: doing something on a windows client 'like a user', and verifying things on the the web guide 'like a user'. Right now I'm thinking we should have no automated tests in this scope.

Number 1 is almost end to end, because the Web GUI is very thin. Doing something on the windows client and verifying on the same rest services that serve the GUI. This is my team's system automation favored perspective, to an extent that I'm still struggling to introduce any other scopes. When these fails (and that is often), we talk about figuring out in the scope of about 10 teams. 

Number 2 is the backend system ownership team's favored testing scope. Simulating the windows client by pushing in the simulated messages in from one REST API and seeing them come out transformed from another REST API. It gives a wide variety of control through simulating all the weird things the client might be sending. 

Number 5 is something the backend system ownership team has had in the past. It takes REST API as a point of entry simulating the windows client, but verifying end user perspective with the Web GUI. We're actively lowering the number of these tests, as experimenting with them shows they tend to find same problems as REST to REST but be significantly slower and more brittle. 

I'm trying hard right now to introduce scopes 3 and 4. Scope 3 would include tests that verify what ever the windows client is generating against what ever the backend system ownership team is expecting as per their simulated data. Scope 4 would be system testing just on the windows system. 

The scopes were always there. They are relevant when exploring. They are just as relevant (if not more relevant) when automating. 

The preference to the whole system scope is puzzling me. I think it is learned in the years as "manual system tester" later turned into "system test automation specialist". Decomposing requires deeper understanding of what and how gets built. But it creates a lot better automation. 

Telling me there are unit tests, integration tests and system tests just isn't helpful. We need the scopes. Thinking in scopes is important. 




Friday, January 13, 2017

Overnight Changes

There is a discussion that I keep going back to, begging to be unloaded off my mind. This morning I said the words: "It's like I joined a different project this week". There's been a sudden change in the atmosphere and in the things we do, and thinking back the last week makes me realize some changes can be really fast.

In a week, my team transformed from a team working on its own components to a team that works with other teams on shared goals. We transformed from a team that seeks product owner acceptance and prioritization into a team that checks priorities but works actively to identify the next steps with one another. And we changed from a team that was quiet and not sharing, into a team that talks and plans together.

I can see three changes in the short timeframe:

  1. We did our first end-to-end demo across two teams and it resulted in a lot of positive reinforcement of customer value over team output. 
  2. Our product owner moved out of the team room and took a step back leaving more room for the team to decide on things. 
  3. A new-old developer joined the team. 
We weren't bad before, but this week has been amazing. We've accomplished a lot. We've learned a lot. 

Experiences this week remind me again on how changes in the environment change the system. And I'm delighted to be in a place that is willing to play with the environment to find even better ways to work together.