Until a few years ago, quality assurance was generally considered low-level R&D. As a result, quality assurance (QA) teams were characterized by a low budget, lack of attention and, more often than not, poor attitudes. Moreover, QA engineers were considered the enemies of developers and the product.
However, today, this is no longer the case, and QA is part of the entire software life cycle. Today, organizations understand how a product’s quality affects the brand’s reputation and customer retention. In the Agile ecosystem, QA engineers are no longer solely focused on testing; they also fill an important role in the planning and delivery domains. The shift left approach—where testing is performed earlier in the life cycle—requires QA engineers to have higher technical abilities and involvement in the whole CI/CD process. Nevertheless, QA managers, who usually encourage assimilation in the scrum team, as well as rapid feedback for user stories and automation, must not forget the main objective of quality engineers: to ensure a high-quality product and a seamless user experience for customers.
High quality demands testing and verifications. There are different types of testing that can be conducted to meet quality goals. In one of its blog posts, Testing Excellence describes almost 60 types of testing that can be performed on a product. We’re going to look at one that is quite clearly one of the most important types of testing: functional testing.
Different than other forms of testing, functional testing is considered one of the most sophisticated test type, and the most comprehensive one.
One important fact every QA engineer must remember is that developers develop and QA engineers test. Even though QA and developers are part of the same scrum team, their work, methodologies and state of mind are completely different. In cases in which the significant contribution of QA isn’t visible to the whole team, testing might be considered an easy and clear task. Quite the contrary; testing an application is much more complicated and entangled than developing it. When planning tests and executing them, QA engineers must take into account several aspects developers likely never consider.
The primary distinction between dev and QA is state of mind. While developers write pieces of code that later become features in the application, QA is expected to understand how real users think, engage customers to learn business scenarios, and confirm that the application satisfies customer needs.
Let’s look at eBay, for example. Every veteran R&D member will agree that eBay isn’t just a simple shopping application, but a comprehensive system that includes a whole set of capabilities on both web and mobile that requires attention and deeper understanding to determine whether they really function correctly. In such a rich and mature online platform, we can identify several types of users and many cases in which the application may be utilized. A seller performs different steps than a buyer and requires different functionality, while a support agent needs an entirely different version of the application. When planning tests, all these personas must be taken into consideration with a particular test plan for each. In other words, testing functionality is not just simulating user operations.
In addition, eBay is not a standalone application—its web application lives inside the browser and subsequently depends on that browser’s rules and behaviors. This is true for eBay’s mobile apps, as well, which must meet the OS constraints to function correctly, whether it’s navigating to the seller’s store or making a purchase. This makes compatibility testing an integral part of functional testing. Most developers don’t think about the project availability matrix when they code, so QA is required to contemplate all the ways the customer will engage with the application and verify functionality for each.
Dev and QA differ in their point of view—an obvious but important distinction. While developers focus on developing user stories, QA engineers test the user story, the feature it is part of and the application behavior with the new code. Integration testing is a crucial part of the application verification effort and requires a broader view of the application and how the new functionality is part of it. End-to-end testing is the fulfillment of the above two aspects, validating that real user scenarios in the context of multiple regions in the application behave as expected. End-to-end tests increase test coverage and reduce the risk of making the new code part of the application.
As noted, testing the application functionality is merely one step in functional testing. Testing the operations available in the application encompasses another type of testing—the user experience. If a new feature is developed, but users do not have easy access to it or cannot understand how to use it, it is as if the feature doesn’t exist. QA engineers must incorporate user experience as a main metric of features so users will not have room for error, which can lead to misuse and frustration.
For instance, eBay’s “all categories” page previously displayed all the categories by default, even though some of the categories had no items. While this was the correct behavior from a functionality standpoint, it caused user confusion and mistrust. Consequently, the “all categories” view was changed to show only categories with items.
Looking at the narrow current task, developers might implement features as they are told without questioning or contemplating the specifications. In the same way, testers who test features without realizing how customers perceive them are missing an important benefit of the test.
The above three aspects—thinking like customers, testing the application as a whole and paying attention to user experience—revolve around the way developers and QA engineers relate to new features. While developers tend to follow the specification guidelines, QA engineers must “think outside the box” when it comes to user consumption of the feature.
There are two other ways in which QA work is more complicated and yet invaluable—test maintenance and test automation.
In general, developers split their time between developing new features and fixing defects. QA work is much more diverse. A substantial portion of the release is spent on testing functionality that already has been released and used. While the main effort of designing and writing test plans for this functionality was invested in past releases, every new feature developed requires a test plan and a maintenance effort on existing tests. One of the main goals for QA managers is reducing this effort to a minimum. Teams that succeed in creating clear and reusable tests simplify the maintenance struggle and optimize the test registration time.
However, for QA, the most cost-effective solution is test automation. At release end—the time when QA is at its most stressed and every commit might introduce a new risk—automation testing can eliminate risk and supplies early feedback on every change. Agile teams strive for the highest automation coverage percentage so the development team can have confidence in committing changes late in the release. That said, developers often aren’t interested in investing their time in automation. They support the QA automation effort, but they don’t want to “dirty their hands” by doing testing. Whether it’s UI automation—during which achieving element uniqueness is essential yet tricky and requires developers’ intervention—or API testing for APIs without proper documentation, QA faces intricate challenges that require sophisticated solutions, which developers are, in most teams, not part of. While the dev team focuses on the status of the new feature, QA attends to the application as a whole with extra attention to identify regressions created by a change.
Testing an application requires a completely different mindset than developing it. One of the main challenges in the Agile methodology is to create developer awareness of quality, involving other stakeholders in the quality assurance activities and making the QA process visible to everyone. QA managers must understand how day-to-day QA work is different than the work of other team members so they may truly represent the complexity, efforts and eventual success. R&D groups that join forces and understand that QA extends far beyond the actual tasks of the QA team have a much higher chance of improvement and set high standards for their product’s quality. Implementing the appropriate methodologies and utilizing the right tools lead to an increase in customer satisfaction and retention rate.
GitLab Duo Chat is a natural language interface which helps generate code, create tests and access code summarizations.
Expect attacks on the open source software supply chain to accelerate, with attackers automating attacks in common open source software…
The emergence of low/no-code platforms is challenging traditional notions of coding expertise. Gone are the days when coding was an…
Datadog today published a State of DevSecOps report that finds 90% of Java services running in a production environment are…
Linux dodged a bullet. If the XZ exploit had gone undiscovered for only a few more weeks, millions of Linux…
We're going to send email messages that say, "Hope this finds you in a well" and see if anybody notices.