Is everyone a tester?
This is one of the greatest debates about Agile testing: who does it? One camp claims that everyone on an Agile team is a tester: there should be no “tester” role. The Scrum camp is perhaps most adamant about this. Another camp claims that there are testers, and that the separate role is very important.Again, the right answer depends. Even Jeff Sutherland – the inventor of Scrum – has complimented the performance of projects that had separate test teams. This one stands out. So if it is ok with Dr. Sutherland, it should be ok for Scrum adherents. The question is, when does it make sense, and when does it not make sense?
For Morticia’s website (see Part 1), Thing did most of the testing but we all chipped in, and that was fine. But for the EFT Management Portal, the testing is so complex that it would sure make sense to have a test lead, and several people to focus on pulling together the performance testing, the security testing strategy, the testing of the multiple back end partner interfaces, the testing of the legal compliance rules, and so on. These things are like projects in their own right and so they need leads. But don’t forget about learning: some people might want to learn about new types of testing and test automation, even if they have not done it before, so allow team members to change roles with appropriate supervision (e.g., through pairing).
Saying “You should never have a test lead” is not very Agile,
and saying
“You should always have a test lead”
is not very Agile either.
If unsure, use common sense: don’t use doctrine. Agile is first and foremost about applying judgment: that is why the Agile Manifesto is written the way it is, with phrases like “While there is value in the items on the right, we value the items on the left more.” In other words, it is not prescriptive. They wanted us to keep our thinking flexible: e.g., saying “You should never have a test lead” is not very Agile, and saying “You should always have a test lead” is not very Agile either.
Who needs to understand these things
One of the most important aspects of Agile in general is the conversations among the team – the exchange of ideas, the helping each other, and the talking through of issues. This is critically important as it relates to testing and understanding when we have met the needs of the end users. Ensuring that the team understands – not just reads – the testing strategy, is critical, so that the testing strategies are ever present in the minds of the developers. Developers need to think about testability as they design and code, and they need to be thinking about failure modes and how those are going to be tested, because developers often identify situations that that testers have overlooked. Communicating testing concerns across team roles is extremely important.In Part 1 of this article we pointed out that an Agile test strategy is developed and maintained by the team, in collaboration with external stakeholders. The team leader(s) (e.g., Scrum Master, coach, project manager, tech lead, etc. – however the team(s) is/are constituted) need(s) to understand what an Agile test strategy is for, so that it is accounted for during iteration planning. If the organization has support functions such as Security, Testing, Architecture, etc., those support functions need to understand how an Agile test strategy is different from a traditional test plan: the collaborative nature of Agile testing, the need to test continually, the need to automate as much as possible, and the need to evolve the testing strategies as the team learns more about the application and its requirements. The support function managers need to know all this so that they can make sure that they provide staff to collaborate with the team in the initial development of the testing strategies and throughout software development.
The support functions will need to come to terms with the fact that their role changes significantly with respect to waterfall development: waterfall teams obtain services from support functions, services such as Testing, Architecture, etc., and those services operate largely independently. In an Agile setting, the support functions need to operate in a collaborative way, working side by side with the team. In fact, much of their work should shift from “doing” to “teaching” – i.e., the support functions need to coach the team in how to perform the things that the support function used to do – to the extent that that is practical. Thus, support functions become coaching centers and resource centers. (In the article How To Rapidly Infuse Technical Practices Into Your Agile Teams we talk about how to transition waterfall oriented support functions to Agile support functions.)
Agile coaches need to work with the various support functions to help them to think through these changes. Agile will impact the types of people who work in the various support functions – they need to be more people-oriented, with an interest in helping others instead of doing the work themselves. The suppot function staff will also have to learn about Agile practices and automation tools. Agile will impact how the support functions are measured by senior management: they will need to be measured on how effectively they help teams to become self sufficient in technical practices, and the support functions also need to be measured in terms of whether they stay current in the rapidly evolving landscape of Agile tools. Given these changes, Agile will therefore impact funding for these functions. It will shift the balance of power in the organization, and that is why the CIO needs to be the driver for these discussions. In a successful Agile transformation, the support functions are not eliminated: they are transformed and reorganized. Knowledge must increase – not decrease – and to make continual learning a sustainable practice, it really helps to have organizational functions that focus on helping practitioners – the teams – to continue to learn new things in an endless cycle of learning, doing, and improving.
Creating a learning organization
In a discussion thread in the LinkedIn group Agile and Lean Software Development, Claes Jonsson, a Continuous Deployment architect at TPG Objektfabriken in Sweden, asked this pertinent question:How is [assurance achieved] in an organization that is committed to delivering the right thing, with extremely high quality, minimal waste and with the shortest possible time to market using Lean Startup principles and Continuous Release practices? And do note that this does NOT mean unstructured, or disorganized, it instead relies on high organizational alignment, and extreme discipline.
The only way for an organization to preserve assurance – that is, manage risk – while becoming more Agile is to elevate people's knowledge. E.g., consider security: the process of having "gates" for security review is antithetical to Agile because gates impose risk management at discrete points instead of integrating it into the development process itself. But if you teach developers how to write secure code, then you don't need the gates anymore! The same thing applies to other areas of assurance.
But wait: I am using a little bit of hyperbole here: gates are a form of oversight, and it is not really that you don’t need any kind of oversight – you do – but it takes a different form. In Part 2 of this article we talked about how the concept of test coverage, and the role that a quality assurance (QA) function might play in an Agile setting. We explained that an Agile form of QA is still independent, but that it works alongside a development team – not as a gated phase. Again, Morticia’s website probably doesn’t need such a setup, but the EFT management portal probably does: you need to make a judgment about how much independent quality oversight is necessary to properly manage all of the risks in your project or program. Agile is about transparently and collaboratively making those kinds of judgments – not blindly following a plan or procedure.
Many large organizations utilize gated software development processes due to historical reasons. These “gates” typically include a review phase for things such as security and regulatory compliance adherence. What we find when working with these organizations – and this is different from small companies and startups – is that the gates are used in lieu of conversations about how the software will meet compliance requirements: i.e., the transparent and collaborative discussions about what process to use for the project do not take place. By shifting to a culture of learning through conversations, gated processes can eventually be reduced to a few minimal stages or eliminated entirely.
There is an artificial comfort in gated processes.
There is an artificial comfort in gated processes: one feels secure because the gates are in place, but the comfort is naïve because the gates do not address the underlying reason the gates were created in the first place: that those who are building systems either do not know how to implement the compliance and risk management requirements, or they are not testing sufficiently for these things. Learning organizations move past this dilemma by ensuring that there is a much broader understanding of the requirements and how to test for them.
Agile transformation is really about systematically creating a learning organization. You have to identify the things that people need to know, and make sure that there are people who know those things embedded in the development process, by creating a system for people to learn and share that knowledge (here is one approach). Ideally, everyone knows everything, but that is not practical, so there is a balance that needs to be achieved between specialists and generalists. But all need to be involved in real time or near real time.
The chart at the end of this article lists some of the things that each part of the IT organization will need to learn. As you can see, it is a-lot, and that is why learning – not process re-engineering – is the “long pole in the tent” for Agile transformation. Learning is one of the first steps on the long road of changing to an Agile culture.
Conclusions
Morticia’s website and the EFT Management Portal are two extremes – as business systems go. Most business applications are somewhere inbetween. That is why there is no single answer to how one should plan and execute testing under Agile.Trust the team – that works fine for Morticia’s website. Top down planning – quite a bit of that is needed for the EFT Management Portal, although we try to approach it in an Agile way by putting the team in the driver’s seat, by keeping the documentation light, by allowing things to evolve, to a degree, by doing testing repeatedly and with as much automation as possible, and by implementing a learning strategy built around coaching to help teams to learn what they need to know to address all of the things that need to be tested.
In the case of the EFT Management Portal we also found that external parties demand – have a right to – oversight, and their risk management team will want to talk to our development leaders – hence we need to have development team leaders if only for the purpose of interfacing to these risk management folks – and the risk management folks will want to review our testing strategies and the ways that we measure test coverage. They will also be watching closely when our first release is deployed for demonstration: by that time, we should have already tested the application at scale in our cloud based test environment and so we should know what the outcome will be – there should be no surprises – but the first release is still a major visible milestone, even if it is not a production release, and people’s credibility rides on it to a large degree.
The fact that credibility rides on a first release is cultural – but it is also human, and to a large extent independent of culture – and so even though Agile encourages tolerance for failure, that has its limits: the idea is to fail early to learn so that you do not fail when it counts and everyone is watching – including the risk management people who are tasked with finding out if you know what you are doing. Testing is Agile’s primary tool for ensuring early failure (feedback), so it is crucial to do the early planning needed to make sure that testing is thorough.
One sign that your organization is embracing early contained failure as a strategy for ensuring long term success is when teams start using demonstrations as a way of “proving” that compliance requirements are met. Teams often look forward to being able to do this. This then continues to strengthen trust within the organization. The more an organization implicitly trusts its teams, the less process rigor is needed – but this will only occur if continual learning is implemented as a strategic way of ensuring that the teams have and continue to have the knowledge that they need.
Authors (alphabetically):
Scott Barnes
Cliff Berg
No comments:
Post a Comment