Agile validation in the context of an existing V-model CSV framework Part II

Fitting an agile development project into a waterfall-driven CSV framework and quality system can be a challenge. The first part of this series describes thorough preparation of some key topics before the start of the project to set a good foundation for this endeavor. Like the agile ceremonies and artifacts, the CSV tasks should be an integral part of the overall agile project planning. Likewise, the agile development approach is recommended to be described in the Validation Plan. This article shares best practices and observations and gives insights to some concrete activities that should be considered during the course of such a project.

1. Agile validation ≠ Less validation

I have observed a general misconception that with agile validation less effort or less documentation is expected (compared to a waterfall CSV approach). But this is not true, agile validation cannot compensate inefficient CSV processes per se, which are defined in the own Quality System. In fact, as it is the agile nature that a product is evolving over time, an increased focus and effort on coordination, alignment, regular reviews, and impact analysis is needed from validation perspective. The validation manager must work even closer together with the entire project team, e.g., by active participation in the scrum ceremonies or by parallelizing different CSV deliverables activities.

2. Use DoR and DoD as CSV Quality Gates

Quality Gates are important for ensuring that a project is on track with the planned deliverables. In this context, the agile concepts of ‘Definition of Ready’ (DoR) and ‘Definition of Done’ (DoD) are also quality gates, i.e., a defined checklist for a common alignment on the status of product backlog items at a certain stage. For a closer integration of CSV with the agile development and to ensure that the validation activities are on track with the sprint-wise development, DoR and DoD should be extended with some CSV relevant criteria. DoR is elaborated for a product backlog item as part of the backlog refinement, prioritization, and sprint planning processes. In the Sprint Planning Meeting, which kicks-off the next sprint, the DoR is officially confirmed by the team and product owner, then considered as sprint backlog item (SPI). Tasks to achieve DoD on sprint level are completed within the sprint. This includes CSV related activities next to development work. DoD is confirmed and approved in the Sprint Review meeting.

On release level, the DoD criteria should be defined as well. I recommend documentation of all DoR/DoD criteria with different levels needed in the Validation Plan. The Validation Report then verifies and confirms whether the defined DoD criteria on release level were met.

3. Fully exploit the risk-based approach

For a system’s validation in an agile setting, the main CSV deliverables will mostly be the same compared to waterfall model. The Validation Manager must know existing guardrails and constraints of the current quality system. At the same time, those parts of the quality system should be identified and adapted on a risk basis, which are flexible to be trimmed to agile project needs. A risk-based approach is not new and is applied in “waterfall world” as well. However, for agile endeavors this practice is even more important to handle changes that will occur over the time efficiently.

For applying risk-based approach, a system’s risk evaluation is recommended at project initiation, i.e. with an assessment of the until then known high-level scope and critical key areas and capabilities. During the project this evaluation should be revisited on a more detailed level as recurring activity, e.g., as integral part of the backlog refinement or sprint planning process. Depending on the outcome, the validation planning is regularly reviewed and CSV deliverables are added or changed as needed with justification. The functional risk assessment is one other form of applying risk-based validation, by tailoring the validation test scope and test extent, where reasonable, to the risk-level of a function or feature. In an agile project it is recommended to conduct this assessment at the very start of each sprint for the defined SPIs in scope. With that, the test strategy is at a very early stage defined for each sprint, so that the preparation of test scripts can start immediately and in parallel to development.

The “adjusting screws” that the quality system offers for risk-based approach should fully be exploited to gain efficiencies. A more critical thinking needs to take place when assessing risks and evaluating hazards behind a product, process, or system features. A resolute approach should be taken if features are considered with a very low risk and therefore defined to be tested without a formal documentation. This would imply having a closer look at the ‘informal’ testing scope and procedures. All mitigations activities derived from the risk assessment must seriously be actioned so that this is not ending as a pointless pro forma exercise only.

4. Welcoming any changes

When it comes to changes in an agile development project, the Agile Manifesto Principle #2 is key: “Welcome changing requirements even late in development. Agile processes harness change for the customer’s competitive advantage”. In this context, Validation Manager should keep in mind that this is not contradictory to their work or to any regulations. Of course, changes must be controlled and reproducible, but regulations do now outlaw changes. Business Owners are changing their minds from time to time due to various reasons, processes are changing over time as well as technologies. Obviously, a change always implies an impact which must be known and considered, but a change is not necessarily negative. It is a mindset thinking in some people’s heads that changes only bring inefficiency and additional efforts and it is not recognized that it also brings a chance for improvement. If welcoming a change is seen as ‘complicated’ in a validation project, I would rather recommend reviewing the companies Change Management procedures, instead of declining a change which might bring opportunities for a product.

5. The right timing for creation, approval and sequencing of CSV deliverables

The right timing for drafting and approval of CSV deliverables is very important, also with regards to managing changes within an agile development project. In this context, I recommend a simple approach: Draft planning or specification deliverables as early as possible, i.e. sprint by sprint, and do early reviews in small packages, especially from Subject Matter Experts, Quality Assurance and Product Owners. Good and stepwise preparation is half the battle. I recommend keeping the documents in draft mode as long as possible, so that the work can be resumed if needed to reflect any changes when the product evolves over time. Approval of the specification documents should take place as late as possible at the end of development sprints. This approach brings especially benefit for bigger projects with multiple sprints defined and sprint durations of 3-4 weeks. A clear structuring of the in-sprint CSV activities should be defined with clear roles, responsibilities and dependencies is recommended as with this approach multiple CSV deliverables are created in parallel by various people to have them in a final draft state in time.

With regards to sequencing, it is important to keep in mind that regulations mostly do not require a strict sequence for CSV deliverables, which is beneficial for applying an agile way of working. Example: User Requirements do not necessarily need to be approved before Test Cases are approved. The apparent requirement for a specific sequencing of CSV deliverable is often artificial and “homemade” and defined in the own quality system, but not preceded from the regulations. This mindset is again historically grown and originate from the classical waterfall approach. However, if the own quality system requires a certain sequence of CSV deliverables this can be adhered to when the mentioned approach above for drafting is considered. Then the approval could take place at development end in the required sequence.

6. Testing Strategy

For hybrid agile validation approaches it is proved to be reasonable to execute an entire formal validation testing in one block only after completion of the last development sprint and before the release (“Validation Sprint”). This is especially reasonable for major releases. As described in the previous section all specifications and test cases should have already been final drafted by then once the development is completed.

Execution and documentation of the formal validation tests after the last development sprint does not mean that there is no testing during the development at all. To the contrary, informal, scripted testing within each sprint is essential. Additionally, exploratory and unscripted testing are accompanying measures that should be performed. Lastly, regular demo sessions during the sprint development process increase the product quality, as user feedback is directly given back to the development team. With that a lot of deviations and defects in formal testing phase can be avoided upfront. The involvement of key users in exploratory testing or demo sessions will also enhance the user knowledge, experience and acceptance of the product. It is also beneficial as preparation for the formal user acceptance testing to already have knowledgeable key users, who know how to use the system.

Sometimes the concept of a hardening sprint and end-to-end testing at the very end of the development cycles and before start of the validation sprint can bring advantages, especially for complex products. It will make the product more robust for release and will minimize even more the risk of encountering any defects in formal validation testing. Finally, I also recommend to describe on high-level all informal testing activities and the concept of product demo sessions in the validation plan, as they contribute to the overall product quality as well and provide, along with formal testing activities, a holistic picture of the whole testing strategy.

7. Align the toolchain(s) for the system and agile development repositories

There are often multiple IT repositories for managing the agile development activities, the system’s lifecycle documentation, a system’s change management or service management processes or documentation archival. All these activities and information assets must be considered from validation point of view as they are relevant for the system and its validated state. And sometimes the information that is managed within these systems overlap and can be found in multiple repositories. Complex toolchains should be avoided in general and should be kept to a meaningful minimum. It might appear obvious to set up and use platforms like JIRA, which are used for agile development also for maintaining information, like validation testing or requirements documentation. One should have in mind, that companies which have not yet implemented a fully agile validation and development approach, mostly have not validated their agile development platforms itself for that use, so this is not a valid option. Having more than one tool for managing this kind of information is mostly inevitable. Moreover, the processes for managing the information and the overlaps must be well defined. The project team should be trained on the toolchain setup and usage.

My Conclusion: Be open, creative and willing to make compromises

Insisting on an ideal image of agile practices and methodologies is counterproductive for an agile development project which needs to be validated with an existing V-Model CSV framework. Likewise Quality Managers who are not open for finding creative ways to stretch the current framework in a compliant way and bringing it in this way a bit closer to the agile world is not promising as well. I would recommend starting your agile validation journey with a GxP pilot project first. This could be an agile software development project which is not too complex, where business processes are quite known, and the risk of scope creep is reduced. The experiences made and learning collected in this project can later be used as valuable input for an extension of the own CSV framework to fully define and support agile development projects.