Software Quality Assurance (SQA) should be considered an integral part of any software project. Adopting SQA efforts early in product development will involve meeting specified requirements, fixing bugs early, and preventing bugs. SQA should not be an afterthought or project phase that the product must pass through in order to ship. Nor should SQA be viewed solely as a testing effort, though testing is a big piece of it. When SQA is properly implemented throughout the Software Development Life Cycle (SDLC), the results are better products at a lower cost due to well defined requirements, improved software quality, and bug prevention.
There are differences between Software Quality Assurance and Software Quality.
Software Quality Assurance can be defined in a couple of ways:
- “A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements.” (IEEE, 1991).
- “A systematic, planned set of actions necessary to provide adequate confidence that the software development process or the maintenance process of a software system product conforms to established functional technical requirements as well as with the managerial requirements of keeping the schedule and operating within the budgetary confines” (Galin, 2004).
Software Quality is defined as:
- “The degree to which a system, component, or process meets specified requirements. The degree to which a system, component, or process meets customer or user needs or expectations” (IEEE, 1991).
- “Quality consists of those product features which meet the needs of customers and thereby provide product satisfaction. Quality consists of freedom from deficiencies” (Juran, 1988).
The SDLC is comprised of many different phases that historically follow the order of: Requirements, Design, Development, Testing, Deployment, and Maintenance. Notice, testing falls in the middle of these phases, which is an important element of SQA. The benefits to initiating SQA efforts further upstream in the SDLC include meeting specified requirements, fixing bugs early, and preventing bugs.
Clearly not all software projects and teams are the same. Software projects come in a variety of complexities and so do teams. However, some form of SQA can be adopted to any project which will address the specific needs of the project and the unique capabilities of the team. SIGMADESIGN strives to provide SQA to all software projects in order to meet the client’s specified requirements. Here’s how we do it:
BETTER REQUIREMENTS
SQA can be especially helpful during the early stages of the SDLC, specifically the Plan/Design phases as it focuses the design around the client’s requirements. Also, it can ensure the requirements provide clear product testability measures. A good development team will assess if the product requirements contain evaluation criteria, if the requirements are consistent, and if the requirements are measurable. During the Design phase, the SQA will also help identify any missed requirements or lost product functionality.
SQA brings a unique perspective to requirement reviews as developers check evaluation criteria, measurability, and consistency involved in meeting those requirements. During the requirement phase, SQA addresses any concerns regarding incomplete or missing requirements that will be needed to both plan and execute testing.
SQA focus areas during the Requirement/Design phases include:
- Requirement reviews for completeness/consistency
- Specification reviews for completeness/consistency
- Design reviews for testability
- Risk study/mitigations
- Bug prevention measures
- Test Plan (Master Test Plan)
- Establishing bug tracking process/procedures
BUG FIXES
Identifying and fixing bugs early in the SDLC will, in most cases, will be less expensive. According to data gathered by the System Sciences Institute (SSI) at IBM, “the cost to fix a bug found during implementation was around 6 times costlier than one identified during design, bugs found during the testing phase could be 15 times more costly than during design.” Using these estimates, a bug found and fixed during the Plan/Design phases would cost $100 to fix. In these early phases, the bug would be considered a requirement/specification bug, not a coding bug. If, however, that same bug escaped detection in an earlier phase and was not identified until the Testing phase began, that bug would now cost $1,500 to fix.
Additionally, other costs attributed to bugs can be found after a product has shipped. These costs can be direct or indirect. Direct costs may include lost revenue for the current product or future products. Indirect costs may include reduced company or brand reputation and customer product loyalty.
Interestingly, there are a few examples of real-world cases where companies learned the hard way that identifying and fixing bugs early in the SDLC will cost less. Here are a couple of examples documented by SmartBear:
NASA – The Mars Climate Orbiter was a robotic space probe that NASA Launched in 1998 to study the climate and atmosphere of Mars. All was well until communication got cut-off a year into the mission when the Orbiter got lost in space. When dealing with spacecraft engineering, there’s a lot that could go wrong — it’s literally rocket science. However, the error could have been an easy fix if caught on time. The $125 million spacecraft presumably orbited too close to Mars’ surface and disintegrated because the engineering team failed to convert their measurements from U.S. units to metric.
AT&T – In January of 1990, AT&T failed to meet the basic needs of their customers as thousands attempted to call their friends, family, and airports. Long distance calling was down for nine hours, and customers sparked outrage after experiencing a total of 75 million missed phone calls and 200,000 lost airline reservations. Initially, AT&T thought they were being hacked, but the issue was due to a software bug. AT&T had just recently “updated” code, but the change of speed ended up being detrimental to a system that couldn’t keep up. AT&T lost an estimated $60 million from long distances charges that didn’t go through after being down less than a full day in an error that was the result of one line of buggy code.
BUG PREVENTION
Bug prevention does not mean that every bug is prevented from occurring. Complete bug prevention is not feasible. The goal in bug prevention is to minimize the number of bugs introduced during the life of a project, starting at an early phase. Finding bugs in the later phases of the project, such as the testing phase, or worst case after shipping, will be far more detrimental to the project budget and potentially that company’s reputation than if found earlier in the SDLC or in the best scenario prevented in the first place.
There are multiple areas where SQA plays an active role in bug prevention early on in a project, including the following:
- Architecture risk analysis
- Setting quality goals
- Setting quality measurements
- Education/training
- Requirement standards
- Requirement reviews
- Design reviews
- Coding standards
- Code reviews
- Unit tests
- Continuous building and integration
- Test automation
Moving SQA upstream on your next software project might seem like an insurmountable undertaking. However, it may be easier than you think. The goal is to identify and fix bugs earlier by including SQA in the planning/design reviews and minimizing the introduction of bugs during earlier phases of the project. Ensuring SQA is an early foundational cornerstone on your next software project will help to lower costs in both the short and long term.
SQA should not be considered a single person or a constrained ideology. SQA should be considered a quality mindset that each project team member can and should embrace for the overall success of any software project. When stakeholders see and appreciate the value and benefits SQA brings to a project when it is adopted earlier in SDLC, they will more likely become advocates for early adoption on future projects.
References:
https://www.techopedia.com/definition/4363/software-quality-assurance-sqa
