fasttrait: Quality Assurance in Agile Scrum Organisation using CMMI

Agile Scrum

Relating to or denoting a method of project management, used especially for software development, that is characterized by the division of tasks into short phases of work and frequent reassessment and adaptation of plans.

Key Benefits of Agile Methodology

1. Smaller Shipments à huge cost of product is broken down into smaller shipments so it is easy to manage the risks related to cost/budget overrun.

2. Frequent Releases à frequent releases reduce risk of high schedule slippages or duration overrun.

3. Faster Delivery Cycles à faster delivery cycles help in getting quick customer feedback so cost of poor quality is reduced to a great extent

4. Competence Building à teams to focus on improvements faster and on continuous basis which results in building higher technical competence.

5. Customer Involvement à Customer involvement helps in better addressing risks related to scope creep or changing requirements.

Limitations of Agile:

Agile Methodology has certain limitations in working with following scenarios

Highly complex projects
Big-size life-critical projects
Fixed price contracts
Lesser customer involvement
Lesser Technical Competence
New Technology

Agile Process Consultant Role:

Assist Business / Delivery Head in process model selection for the projects
Provides feedback to the management that agile approach is working in given projects scenario.
Acts as the process coach to the teams and guides in implementing agile principle and practices.
Brainstorms with project teams to identify ways to improve agility to achieve desired quality goals.
Helps management to scale up agile practices at an organization level

Quality Assurance plan in Agile Projects:

This is a Program or Project QA Plan which contains:

Quality Goal : Based on Baselines Velocity, Code Quality, Test Coverage, Defect Density, And Defect Removal Efficiency
Process Model : Development Life Cycle Used in the project mentioning processes in the form of Flow Charts in ETVX (Entry-Task-Verification-Exit)format
Configuration Management : When done at program level, this may require detailing at project level
Test Plan : When done at program level, this may require detailing at project level
Resource Management
Training
Internal Audit – Frequency
Metrics Analysis Plan : Frequency of data collection and analysis for the chosen quality goals

This may be done at the broad level such as the program level that is why the name Program QA Plan

CMMI to achieve sustainable agility

If Agile gives the momentum, Agile CMMI gives sustainable momentum.

CMMI can be used in such way that eventually process improvement journey will lead you towards a highly capable and mature agile process.

A highly capable and mature agile process means how best you are in adopting the agile framework in the organization and to what extent you are able to confidently say that you are going to fulfill the customer needs in the given sprint/iteration.

Level 2 - Managed

Practices are defined and implemented at project level.

1. Project Planning:

Ø Sprint / Iteration Planning: Done by Customer (Product Owner and Scrum Team )
Program / Project QA Plan (Refer to Quality Assurance Plan in Agile Projects)

2. Project Monitoring:

Ø Monitoring as per the program QA Plan

Ø Daily Scrum Meetings (Done by Scrum Team),

Ø Sprint Reviews with Customer (Product Owner)

3. Requirements Management:

Ø Product and Sprint Backlog Management,

Ø Requirements Traceability Matrix Management

4. Subcontract Management:

Ø If outsourcing is done, vendor quality analysis and process evaluation as per all applicable CMMI practices

5. Product and Product Quality Assurance:
Ø Product Quality Assurance

o Done by Customer and Scrum Team during Sprint Review Meeting

o Internal quality audits to ensure this is being done

Ø Process Quality Assurance:

o Internal Quality Audits to provide objective feedbacks about all processes

6. Configuration Management:

Ø No specific change for the agile scenario

Ø May be done at a program level

7. Measurement Analysis:

Ø No specific change for the agile scenario

Ø May be done at a program level

Ø Need to identify data collection mechanism, and what type of measures and metrics will be captured for agile ( e.g. Velocity, Code Quality, Test Coverage, Defect Density, Defect Removal Efficiency etc)

Level 3 - Defined

At level 3, there are Standardization for Level 2 Practices performed in projects (as already discussed above)
In addition, there are engineering practices, organisation practices, support areas and project management areas. The practices for these areas are also defined (Standardized) at Organisation level

8. Organisational Process Definition

Ø Defining all standard practices at organisation level

Ø Organisation develops OPAL (Organisation Process Asset Library) and OSSP ( Organisation Standard Set of Processes)

Ø In Agile scenario, Organisation will need to include Agile Specific practices, life cycle models, metrics templates as well as templates for Agile Scrum Practices such as Sprint Backlogs, Burnt down charts, User Stories, and Design.

9. Organisational Process Focus

Ø Process Improvement Suggestions are generated

Ø Process Improvement Opportunities are identified based on SWOT and Cost Benefit Analysis and conducted on process improvement suggestions.

Ø Process Improvements are implemented

Ø No specific change here for agile projects but when there is a transition to agile model the organisation need to conduct these practices.

10. Organisational Training

Ø Organisation Training practices remain the same

Ø No specific change here for agile projects but when there is a transition to agile model the organisation need to conduct trainings related to Agile Scrum

11. Integrated Project Management

Ø Key expectation here is that organisation the project needs to tailor the practices as per OSSP.

Ø If the project is doing something different from OSSP then it will be required to define it in the project specific process model. That could be an additional section in the Program/Project QA Plan.

12. Risk Management:

Ø This necessitates identification, mitigation and resolution of risks as per Risk Management Plan

Ø This may be performed at broad level like Program level for agile projects

Ø For Agile Scenario, project planning, execution and quality related risks are addressed in agile scrum practices to a great extent.

13. Requirements Development:

Ø Requirements development practices are addressed in agile scrum practices.

o Behaviour driven development (BDD)

o Product and Sprint Backlog

o User Stories

Ø If not addressed, they need to be defined specifically in Process / Life Cycle Model

14. Technical Solution

Ø Technical Solution practices are addressed in agile scrum practices

o Behaviour driven development (BDD)

o Evolutionary Design

o Test driven development (TDD)

o Refactoring --> Technical Solution

Ø If not addressed, they need to be defined specifically in Process / Life Cycle Model

15. Product Integration

Ø Product Integration practices are addressed in agile scrum practices

o Continuous Integration,

o Continuous Deployment and

o Continuous Delivery and Release

16. Verification

Ø Engineering Verification practices are addressed in agile scrum practices

o Sprint Planning – Requirements Review

o Sprint Review

o Test Driven Development (Covers unit tests)

Ø If not addressed, they need to be defined specifically in Process / Life Cycle Model

Ø Other Verification practices are required like Review of Program/Project QA Plan , Review of Functional and Non Functional Test Cases, Release Plans, Release Notes and User Documentation

17. Validation

Ø Engineering Validation practices are addressed in agile scrum practices

o Sprint Review – Requirements Review and User Testing

o Sprint Retrospection

o Behaviour Driven Development

Ø If not addressed, they need to be defined specifically in Process / Life Cycle Model
Ø Other Validation practices are required to be performed like Testing of Disaster Recovery Plan, Vulnerability Assessment Test, and Backup Restoration Validation etc.

Ø Validation activities related to prototypes, tools, performance models and environment testing are also need to be performed exceptionally.

18. Decision Analysis and Resolution:

Ø This necessitates structured decision making.

Ø This may be performed at broad level like Program level for various decisions

Ø For Agile Scenario, it can have relevance as Selection of Agile Methodology can lead to such a structured decision making process. Organisation may need to include decision making criteria for agile process selection.

Level 4 – Quantitatively Managed

At level 4, there are Defined practices of Level 2 and 3 are performed in projects as well as organisation level (as already discussed above)
In addition, there are organisation area, and project management areas. The practices for these areas are also defined (Standardised) at Organisation level
Special causes of problems are addressed and performance is quantitatively managed for all defined practices.

19. Quantitative Project Management

Ø The Sprint/Iteration goal should be the quality goal which needs to be monitored when you are trying to improve an agile-scrum process.

Ø Examples of metrics

§ Velocity: The number of story points delivered in each sprint.

§ Defect Density = Number of defects / Story Points

§ Test Coverage % = (Number of test cases / Number of User stories)*100

§ Defect Removal Efficiency % = (Internal Number of Defects / Total Number of Defects)*100

Ø Quantitative analysis of velocity and other metrics to identify process problems which has special causes associated with specific sprints/iteration.

Ø Targeted sub processes to be given special attention for attaining the Sprint Goals.

20. Organisation Process Performance

At Organisation Level:

Ø Baselines for metrics (e.g. Velocity, Test Coverage, Defect Density, And Defect Removal Efficiency)

Ø Performance Models for metrics (e.g. Velocity, Test Coverage, Defect Density, And Defect Removal Efficiency)

Ø Apart from agile specific practices there will be other practices which are common or shared among the projects:

§ Configuration Management

§ Risk Management

§ Training

§ Decision Analysis and Resolution

§ Process and Product Quality Assurance

§ Organisation Process Definition

§ Organisation Process Focus

§ Integrated Project Management

For these areas, there may be quantitative analysis and process performance activities need to be performed.

Level 5 - Optimizing

Defined practices of Level 2 , 3 and 4 are performed in projects as well as organisation level
In addition, there are organisation area, and support areas. The practices for these areas are also defined (Standardised) , quantitatively managed and optimised at Organisation leve
Common causes of problems are addressed and process performance is optimised to meet the goals.

21. Organisation Performance Management:

Ø Organisation Level analysis for performing process and technology innovation towards achieving or exceeding goals for

o Velocity

o Customer Satisfaction

o Product Quality Metrics such as code quality, defect density, defect removal efficiency, test coverage

o Employee Satisfaction

o Goals for Risk Management and other processes or practices at project or organisation level for e.g. training, reviews etc.

22. Cause Analysis and Resolution:

Ø To reduce common causes of variation in the process and thereby improving the capability of the process

o Project Level

· Sprint Retrospectives is one of those effective methods towards early Causal Analysis & Resolution.

· Defects root cause analysis

· Root Causes Analysis of process problems, risk failures and metrics data such as Velocity, Code Quality, Test Coverage, Defect Density, And Defect Removal Efficiency

o Organisation Level

· Analysis of retrospectives to capture learnings across projects

· Defects analysis to capture learnings across projects

· Root Causes Analysis of process problems, risk failures and metrics data such as Velocity, Code Quality, Test Coverage, Defect Density, And Defect Removal Efficiency