How Does Agile Deal With Requirements Traceability?

Feature traceability is a hold over practice from waterfall, where requirements are checked against tests. If one is using an Agile testing practice such as, say, Behavior Driven Development (BDD), then feature traceability is redundant and a great time waster. This is because in an Agile process, each Agile story has a set of acceptance criteria, and in the BDD process, each acceptance criteria has one or more associated test scenarios. Thus, there is a clear one-to-many relationship between acceptance criteria—which you could think of as requirements—and tests.

On the other hand, there is no table of this mapping, and so organizations that use traditional (waterfall) controls will often insist on a requirements traceability matrix (RTM). This is a real nuisance, because it is make-work for an Agile team and adds zero value; however, if you are forced to create an RTM, and you use BDD, there are techniques that you can use to lessen the burden. I’ll explain some below.

First of all, create a unique ID for each story (stories usually have IDs anyway, so this is easy), and also for each acceptance criteria. Then, given that you are using BDD, each story has a Gherkin feature file (that is not actually true, but assume it is for the moment): tag the feature with the story’s ID, and tag each scenario with the ID of the acceptance criteria that it pertains to.

You now have an explicit linkage between an acceptance criteria and one or more test scenarios. The challenge is to provide that linkage as a deliverable. If the linkage must be provided as a table, then you will need to either write a script to parse the feature files and assemble the table, with each table column an acceptance criteria ID and each row a Feature/Scenario ID combination. A much better situation, however, is if you can get the organization to accept a different process, whereby a test lead or business analyst reviews each story’s feature file and attests, via a comment in the Agile story tool, that the acceptance criteria are all covered by the feature file. That approach does not require creating a separate RTM.

Of course, any such process needs to accommodate a situation in which a story’s acceptance criteria or feature file change after the feature file has been attested to. Unfortunately, the tool for creating a feature file is usually a text editor (some BDD tools allow you to use a Word document), and so there is probably no workflow built into the tool. The feature file should be in the project’s source code repository, so you can create a trigger using a build tool to perform some action whenever the feature file is changed, such as posting an email to the team lead or the test lead.

Adding features over time

Another consideration is what happens over time, as features accumulate and tests accumulate. Eventually you get to a point at which a new story impacts an existing feature. The question then is, Do you modify the current feature file, or do you add a new one? When your project started, you created a feature file for each story, but now some stories are changing the features that were implemented by prior stories.

If you continue to create a new feature file for each story, you will have good traceability between a requirement and a feature file. However, it will become difficult to maintain because the programmers will have to find out which feature files are impacted by a code change. That is generally impractical, and so teams prefer to update existing feature files, and this breaks the correspondence between story and feature file. For this reason, contrary to what I said earlier, do not create a feature file for each story: instead, for each story, identify and name a set of one or more features that the story implements or changes; then, make sure there is one and only one feature file for each such feature. In each feature file, tag the feature with the IDs of the stories that affected that feature. This makes it possible to search and find features pertaining to stories and stories pertaining to features - without needing a table.

A ramification of this is that the analyst who reviews feature files will now have to sometimes review multiple feature files for a story, and so each story should state which features are impacted by the story.

Assessing coverage

One thing that is very important to acknowledge is that programmers write terrible behavioral tests. I am not talking about unit tests - I am talking about functional integration tests, which are what behavioral tests are. It is therefore essential to have a testing expert or QA person examine the feature files and assess the coverage to determine if there are enough test scenarios, covering edge conditions, error paths, etc. One test scenario for each acceptance criteria is not enough!!! Having an independent person who has a testing mindset assess the test scenarios will increase your quality enormously.