DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • Azure Migration Strategy: Tools, Costs and Best Practices
  • Blameless Integrates Incident Management Platform With Opsgenie
  • OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
  • Red Hat Brings Ansible Automation to Google Cloud
  • Three Trends That Will Transform DevOps in 2023

Home » Blogs » DevOps Toolbox » Crowdsourced Software Testing

Crowdsourced Software Testing

Avatar photoBy: Chris Riley on January 27, 2015 2 Comments

As a developer or technical architect, there is no better feeling than taking pride in giving a quality product or service result to your customers.  Now, with computer trends going rapidly towards cloud services and mobile devices, regular company IT development teams are finding their testing capabilities more and more limiting. They are faced with faster releases with more complex products. How do they keep up?

Recent Posts By Chris Riley
  • Using Incident Response for Continuous Testing
  • What Is Resilience Engineering?
  • Moving from NOC to the SRE Model
Avatar photo More from Chris Riley
Related Posts
  • Crowdsourced Software Testing
  • Testing Mobile Apps? Consider a Mobile Device Cloud
  • You Asked for Quality, You Got Alienated QA
    Related Categories
  • Blogs
  • DevOps Toolbox
    Related Topics
  • automation
  • crowdsource
  • QA
  • testing
Show more
Show less

TechStrong Con 2023Sponsorships Available

To bring this kind of quality assurance requires strong testing throughout the entire development process from end-to-end delivery. With mobile devices beginning to outnumber desktop and laptop systems combined, and software application variation types now averaging in the millions. Testing quality with best practices and a local QA toolset and resources is getting harder and harder to work with for the average development department.

There is a growing trend to leverage crowdsourcing to augment aspects of your QA processes. Crowdsourced software testing toolsets and services is becoming a popular alternative with current Quality Assurance teams. Transforming internal teams into strategy units that generate test plans, approach, automation, and manage the crowd source tool.

Instead of hiring 5 or 10 outsourced testers, in-house, you can leverage 50, 100 or even thousands more crowd sourced testers online without the typical budgetary overhead such as extended compensation packages, or for onsite temporary testers the overhead of providing them facility office services, software, PCs, etc.

Crowdsource testing toolsets helps provide testing-as-a-service functionality for many types of software applications using web-based, mobile, or front-end applications.  It also can provide test-team results simulating the capacity of a dozen QA development test teams for the fraction of the price it would cost years ago to manage and support that kind of Quality Assurance infrastructure.  Thanks to automated script-testing, synthetic user transaction functionality, mass crowd testing participation and a wider user and system base of compatibility testing, Crowdsource Testing Toolset services can help provide easy IT budgetary needs for justifying Return-on-Investment or Total-Cost-of-Ownership business support.

With growing audit control regulations and shrinking IT budgets, Crowdsourced Testing solutions can be the best alternative when working with team resource challenges.  Focusing and targeting the problem with Crowdsource testing helps streamline the code review and mitigation process, considerable.   Many of these services are also providing streamline paying systems using Bitcoin, Credit Card, Kudos and other popular online payment systems which automates or sometimes eliminates the need for invoice management and other extensive overhead procedures in the payment process.  Here are some popular solutions that help provide strong value in the industry, today:

** Note from the author I have no affiliation or connection to any of the products below.

Rainforest QA

The Rainforest QA group provides “On-Demand” QA team functionality by using their comprehensive workflow setup tools to simulate individual or hundreds of synthetic user transaction flows which report back performance and response results in detailed screen-captured dashboards or in multiple result producing reporting formats.  Rainforest interfaces with GitHub Repository services for collaboration access.  They offer a 14-day trial period with Startup, Business and Enterprise pricing based on the number of “steps” you want to purchase and increased testing functionality depth you want in your workflow.

Toolsets include:

  •   Unique online dashboard web-based applications to configure test setups
  •   Identify breakage before and after deployment through online interface reporting
  •   Browser-type online problem reporting
  •   Extensive Filtering options to help focus on important testing areas
  •   Automated Screenshot reporting

Crowdcurity

Crowdcurity uses a “Bug Bounty Program” approach which is both innovative and effective for current security quality testing validation.  They provide the choices of either using mass Crowd-approach testing or specific dedicated researchers, as an option.  Crowdcurity will provide their systems and technical staff to stage and setup penetration testing contests to analyze websites and their services for security holes.  Customers sets up the rules and scope of the contest while payment scale is based on the severity or exploit risk level of the bug.  Vulnerability Reports are generated after each activity which developers can leverage to implement security bug fixes as needed.

Toolsets include:

  •   Crowdcurity Central ® provides a central GUI interface for browsers and iDevices
  •   Reporting Management Tools for Performance and Vulnerability reporting
  •   Setup options for both Bug Bounty and Audit-On-Demand testing types where you can choose by paying per bug, not per hour, and the size of your testing and/or research crowd for the test activity

TestBirds

Gartner classified TestBirds under “Cool Vendors in Enterprise Mobility, for 2014”!  TestBirds is a Munich, Germany based company providing global services which specializes in mobile device Crowdsourced testing. Their focus is on application testing for iOS, Windows, Blackberry, and Android operating systems.  Both Screencasts and Screenshot services is also provided from tester devices.  “BYOC”, (Bring-Your-Own-Crowd of testers such as employees or company developers), is another option they help provide in their template services. TestBirds also provides stronger Project Management tools such as survey questionnaire setups, demographic framework configuring, and approval workflow reporting which can help streamline any enterprise-level initiative needing quantitative and qualitative measuring of reporting metrics.

Toolsets include:

  •   Birdflight ® : Used for distributing iOS and Android apps to the test teams
  •   SDK: Crash reporting for Android devices, (iOS solution under development)
  •   Self-Service: Web-based online tools for accessing crowd setups and configuring test parameters, and define reporting types for the initiative

passbrains

Passbrains uses the approach of online request proposals.  They offer much more options in Desktop, mobile and online web service system and application crowdsource testing.  Functional Testing, Usability Testing, Load Performance Testing, Localization Testing, and Security Testing on all these platforms which provides extensive versatility to quality assurance teams, needing this.

Toolsets include:

  •   Web-based online control dashboard used for:
    • Ticket Request and Acceptance Management
    • Project Participation – Status Reporting
    • Feature Management Request

Crowdsourcing is not for everyone, and it is not an all or none proposition. But it is a good way to build in pseudo automation into your processes, take on the increasing workload. And most importantly transform your QA/QE team a first rate citizen able to focus on broader aspects of QA, and become part of DevOps strategy.

Filed Under: Blogs, DevOps Toolbox Tagged With: automation, crowdsource, QA, testing

« CFP for DevOps Connect: DevOpsSec @RSA Conference is now open
Docker tinkers with Docker Hub for performance improvements »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Automating Day 2 Operations: Best Practices and Outcomes
Tuesday, February 7, 2023 - 3:00 pm EST
Shipping Applications Faster With Kubernetes: Myth or Reality?
Wednesday, February 8, 2023 - 1:00 pm EST
Why Current Approaches To "Shift-Left" Are A DevOps Antipattern
Thursday, February 9, 2023 - 1:00 pm EST

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

Azure Migration Strategy: Tools, Costs and Best Practices
February 3, 2023 | Gilad David Maayan
Blameless Integrates Incident Management Platform With Opsgenie
February 3, 2023 | Mike Vizard
OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings
Red Hat Brings Ansible Automation to Google Cloud
February 2, 2023 | Mike Vizard
Three Trends That Will Transform DevOps in 2023
February 2, 2023 | Dan Belcher

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

New Relic Bolsters Observability Platform
January 30, 2023 | Mike Vizard
OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings
Jellyfish Adds Tool to Visualize Software Development Workflows
January 31, 2023 | Mike Vizard
Cisco AppDynamics Survey Surfaces DevSecOps Challenges
January 31, 2023 | Mike Vizard
Let the Machines Do It: AI-Directed Mobile App Testing
January 30, 2023 | Syed Hamid
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.