In the spirit of Halloween, this past month testing experts from around the globe answered the call to join in a “crowd-sourced” article highlighting their most frightening testing QA experiences. These are real-life examples of testing gone wrong—leading to all sorts of mayhem in some cases. We allowed these testers to hide their identity, and to keep their organizations anonymous to solicit the most behind-the-scenes responses.
In no particular order here are the top five Testing Nightmares, as told by the testers themselves:
The Importance Of Making Sure Your Hosting Company Gets Paid
By Ivar (from a medium sized company/ now acquired)
The company I was working for at the time had bet its existence on signing an enterprise contract that it had already been in negotiations for more than year with. We were building a digital asset management system that controlled the licensing and delivery of assets to corporate devices. (Think ‘private app store’.) I had written backup tooling that wasn’t ready for use but was included, unused, in the branch that was being deployed. Ten minutes after the final deployment (in preparation for client review!) all of our assets disappeared. Nothing new in the deployment was related to asset storage except my code. There was an eight-figure contract on the line and we had no backups. Fueled by the mother of adrenaline rushes, I reviewed my work and remained convinced that my code was not the cause of the loss. I was already into investigating other possible causes when another developer informed our team that after calling our asset host, he discovered that we had neglected to update our billing credit card. The host had turned off access to all of our assets ‘to get our attention’. They succeeded, but at oh what a price…
Not The Best Way To Start As The New Test Automation Engineer
By James Eisenhauer
This happened my first week as the new and only Quality Engineer at a health and wellness startup. I was on a mission to get test automation created and show them what I could do for them! My first test was to invite all the members of a test company to a new company wellness challenge. It just so happened the company President’s test account was a member of that test company. I scheduled my tests to run at 5 a.m. At 5:38 a.m. we all received an email from the President that read, “Emails every few minutes telling me that QA wants me to walk by the damn river. Make it stop!” “Walk by the River” was a copy of a real wellness challenge we had in our test system. Something in my auto-retry logic had gone horribly wrong. First impressions are a bitch!
Gaming Fail
By Abe Chiu
While working for a gaming app company, we pushed out a build with an engine update that reverted all of our users back to levels that were saved on their device. This as opposed to the one that was saved on our server side. (Example: If logged in to create an ID and was at level 50 on your device, that was the save that popped up even though on our server you had previously reached level 125). All of our users were affected. That’s not the worst part. Some of our users reverted back to level 1 when they were above level 100 and up. This all happened on a Friday and the build was pushed on Thursday night. On Friday we were only able to fix some issues with a small percentage of our user base but the ones that were reverted back to level 1, those were not able to be helped until the following Monday.
The White Screen of Death
By Anonymous
I work as a Software Engineer in test at a media company. This company has its own video player that is embedded either as a single standalone video or a playlist. The player was re-done completely, and I was responsible for testing it before it went out to production. I tested the player as a single video and found a number of issues so the release to production was pushed out. After the issues had been fixed, I tested it again as a video and then gave my “go-ahead” to release it to production. The problem was I had COMPLETELY FORGOTTEN to test the playlist! We had figured that everything looked good and since the release had already been pushed off once, we made a decision to release it to production at the end of the day, around 5:30 p.m. I double-checked it on production as a single video and it looked fine. An hour later, when I got home, we started receiving urgent messages and emails that the playlist wasn’t working!!! You could only watch the first video and then it became a blank white screen! Since the playlists are embedded so deeply, this was a major problem. At 6:30 pm that night only one engineer was around and online but he wasn’t sure how to revert it. Finally after a very stressful hour and a half, he figured out how to do it and I verified that everything was back and working again with the old player. I still can’t believe that I had forgotten to test the playlists, which is such an important part of the feature.
Global Co
mmunication Breakdown
By Joe Nolan
We were trying to create a mobile on-line conferencing feature to include in our education app, much like Google Hangouts. Testing had gone well in controlled environments. Then we pulled all of the teams together to have a large group call using and testing the feature. We immediately had reports of crashes and all sorts of issues. It was impossible to talk over the people trying to diagnose the problems, while at the same time others were randomly pushing buttons to try the feature. Even worse, participants were located around the world with different networks and connectivity issues. It was worse than herding cats!
About the Author/Sauce Labs
Sauce Labs is the leading cloud-based web and mobile application automated testing platform. Its secure and reliable testing infrastructure enables users to run JavaScript unit and functional tests written with Selenium and Appium, eliminating the time and expense of maintaining a test grid. With Sauce Labs, organizations can achieve success with continuous integration and delivery, increase developer productivity, and reduce infrastructure costs for software teams of all sizes.