As the Delivery Manager of Arcadier, Gligor Milosevic is the man in charge of ensuring that new features are built on time and existing features are optimised. Together with his quality assurance (QA) team, Gligor goes through the QA process and tests the system before pre-release to ensure that it runs smoothly. In his Arcadier Inspire summit talk, Gligor brings us behind the scenes to take a closer look at Arcadier’s testing process prior to each feature release.
What is Quality Assurance?
QA is essentially the process of maintaining a minimum level of quality in a service or product, hence the name quality assurance. When Gligor mentions QA, he refers to Arcadier’s team of automation testers.
The QA process is usually kickstarted by a request from the Product Manager or the Business Development (BD) team. These requests may be due to increased demand for new features or different feature behaviour from customers. In response, the QA and Developer teams will build a new feature or alter an existing one. Gligor’s role in this process is to liaise with the Product Manager, QA team and Developer team to discuss how the feature should be built. This is known as the solution design phase. The solution design phase usually takes around a day to be completed, where the teams will define the minimum viable product (MVP) version of the features. After the skeleton has been drafted, enhancements are then gradually added on until the feature is ready to be built.
Afterwards, the Developer team starts building the feature based on the specifications given to them by the Product Manager. The QA team then picks up the stories that are written and prepare test cases and automation to make sure that they have sufficient time to complete regression tests before the feature release.
Testing Process
Gligor goes on to elaborate on the testing process that each new feature undergoes because it is pushed live. The process starts in the testing environment, which is where code is pushed once the Developer team deems it ready to be tested. The team uses JIRA, an issue and project tracking software to create cards for different features. After the Developer team has built a particular feature or has enhanced an existing one, a test case and card is created for it. Once the card is marked as ready for QA, the QA team will start running the test cases and check if they are complete.
A task is defined as complete when it no longer yields any bugs during the testing process. If the task is complete, the QA team changes its status to ready for regression. Otherwise, if bugs are found, they create a bug card in JIRA. The Developer team will then look through and rank all the bug cards from highest to lowest priority. Given time constraints, it is difficult to fix all bugs. Nonetheless, the team makes sure that severe bugs are fixed before the feature goes live.
After the Developer team has fixed the bug, they pass it back to the QA team for checking where the team runs test cases again. If they find any defects, the card will be passed back to the developers and this loop will continue until the bug has been completely fixed.
Once all the bugs have been fixed and tested, the feature will be ready for regression and will be pushed into the staging environment. One final round of regression is then performed. If any bugs are detected, they will be assessed in terms of priority. Depending on how they will affect customers, the bug will need to be fixed. The next step is to determine if the bug can be fixed in staging. If it is too severe to be fixed directly, the QA team will have to cancel regression and push the feature back into the testing environment. As regression takes up to three days to be completed and time is usually quite tight, the team does its best to avoid such situations.
How Arcadier Prioritises Bug Fixes
Arcadier usually releases one new feature per month, though two features may be released simultaneously if they are complementary. With the speed at which Arcadier launches new features, Gligor says that it’s inevitable for bugs to appear. Hence, it is important to manage and prioritise which bugs to fix.
Bugs are divided into three levels of priority, priority one (P1), priority two (P2) and priority three (P3); P1 being the highest priority and P3 being the lowest. Onboarding-related bugs are classified as P1 along with bugs related to checkout, item display, adding items to the cart and payment issues. Bugs which have lower priority can be upgraded to P1 as well if they are recurring or customers have given feedback about them.
Striking a Balance between the Quality Assurance and Developer Teams
Gligor revealed that Arcadier’s QA team is actually smaller than the Developer team. One of the biggest challenges of this situation is managing and estimating the number of features that Arcadier is planning to build such that they are within the QA team’s testing capacity. As the Developer team is larger, they have greater manpower to build more features. To prevent the QA team from being overwhelmed with testing new features, instead of having the Developer team send new features to QA for testing, the QA team defines the limits of what they can test and the Developer team works within those boundaries.
Nevertheless, having a larger Developer team does pose several advantages. Instead of being fully preoccupied with building new features, the team can work on fixing or optimising existing features as well. In this way, they can help to improve the quality of the product. Furthermore, extra developers can also handle sudden feature requests without affecting the main roadmap.
Interested in finding out more about QA? Watch the entire Arcadier Inspire Summit talk here:
