Crowd Testing News and Trends

The perfect recipe of testing methods for your software application

Posted by Thomas Grüderich

Aug 29, 2014 3:52:00 PM

The perfect recipe of testing methods for your software application

Every comprehensive software testing strategy uses a combination of different testing methods. Which one is going to be used, strongly depends on the kind of software that needs to be tested. From completely automated testing methods and scripted test cases to explorative tests and private or public beta tests: the right application of quality and succession will insure the best possible examination of your software before its official launch.

An overview of the three most important stages on the path to a software that is free of bugs and explanation of each testing method’s value, their strengths and weaknesses, as well as the right time to use them:

Test Cases

A technological product (Software) is inspected according to exact parameters. Test cases do primarily guarantee that a software can be correctly used, just as the developers been imagining it. The main advantage of this kind of testing is a quick dulcification and reproducibility – that way they can be repeated identically in every phase of the development process.

A simple example could be e.g testing a shopping cart of an online shop. Questions that can be answered using test cases are among the following: Can every product of the shop be added to the shopping cart? Appear all prices of the products correctly?

This kind of testing can verify crucial insights but can also be fairly limited. What if the shopping cart would show wrong pictures or wrong product descriptions? If those parameters are not considered while setting up the test, they won’t have an effect on this test run.

Exploratory Testing

Every kind of software and every single product profits from the dialogue with stakeholders and target group to gain important feedback before and during the development process. Exploratory tests checks the system in depth by not requiring the tester to adhere to pre-defined parameters. Each tester investigates a software individually to discover aspects that the developer might have overlooked.

In our shopping cart example they would also test the possibility to remove things or change the number of items.

Usability Testing

Software testing can be done regularly before a release – but as soon as it hits the market you will be confronted with a variety of unexpected things. It doesn’t matter if you decide involving the target group as potential testers on a live software or provide an public access for the software’s beta version: Important note – collecting feedback before the official product release is essential.

There is no appropriate substitute for real world conditions – but at least it’s worth a try. Usability tests can help to reveal shortcomings of a system and teach us a lot about the expectations of your potential users at the same time - how your product will likely be perceived in the real world.

Regarding the shopping cart, users might fill it up but return one week later. In this case he would expect that the shopping items are still stored in the shopping cart ready for checkout. If not he would decide against continue shopping in the worst case scenario. Different users will do things that seem intuitive to them, while it seems completely implausible to you. Users will always produce scenarios that developer and QA personnel did not consider. 

The mix matters!

The choice for the combination of the different test methods depends on the type of system as well as the future users. Generally, business softwares are mostly tested automatically, e.g. verification of processes, whereas usability tests are more recommended for consumer-oriented systems like online shops or social games.


A good approach is to focus on test cases, until the core functions of your software are working properly. Shift to exploratory testing thereafter to get relevant feedback to functions you might have missed in your development.

Round out your testing process with usability tests to find out your user´s acceptance and get your product into an optimal form for your target group before the official release.


Read More

Topics: Software Testing, Crowd Testing, Releases, Beta Testing, testing methods

Black-Box Testing vs. White-Box Testing

Posted by John Faucett

Aug 14, 2014 9:49:00 AM

Black-Box Testing vs. White Box Testing - Possibilities and Limitations of both Testing Methods

If you've ever been involved with Software Testing in any way, you've probably heard the Terms being thrown around and perhaps on first thought they were next generation game consoles from Microsoft.

But what are they and how can these two testing methods ensure that our Software does what's its suppose to do?

Let's take White-Box testing first, since historically it`s the most wide-spread method in use. If you're a programmer and write tests for your own code, you're white-box testing. White-Box testing means the tester has full access to the source code and intimately understands the software. 
This is a key point, since as a White-Box Tester you know the inner workings, you're going to test the software based on that knowledge, and the assumptions made by the code itself. Unit testing, Integration testing, regression testing, and coverage tests fall into this category. It can be used to assert that a part of the system does exactly what the programmer/tester expects it to do. Without this, you have no way of knowing your upload form even works - except by trying it yourself of course. But change one line months later and the whole system might break without you knowing it - unless of course you've done your white-box unit tests. Which is why this method is so beloved and so widely adopted.

At the opposite end of the spectrum, there's Black-Box testing. This is where the software testers have no idea about the internal workings of the system, though they usually have some document telling them what the Software is supposed to do.Functional Testing, and Test Cases fall into this category.It`s much more open-ended, since testers don't know the intimate details of the system or exactly what the programmer expected as inputs. 
Because of this, an entire new set of errors and mistakes can be discovered. This is the area where Crowd-Testing thrives. Even if you have 100% Code-Coverage and a massive barrage of Unit Tests, all this still won't let you know that no end-user can find your 'Buy Now' button, or that your css layout explodes in a rotated iPad2, or that users expect the download link to download an image and not a word document.

And unfortunately Black-Box testing is the most oft neglected, Project Managers are thinking ship, ship, ship, and programmers think, all my unit tests are green so this software must do what its intended to do. But in the end, only your users will tell you if your software is good or not, so if you want to save time, wasted effort, and money - let them in from the beginning - do Black-Box testing.


Read More

Topics: Crowd Testing, quality assurance, testing methods

Why is Crowd Testing of higher value than simple testing techniques?

Posted by Thomas Grüderich

May 21, 2014 12:30:00 PM

Why is crowd testing of higher value than simple inhouse testing techniques?

Almost every company sooner or later faces the question of whether to test inhouse or outsourcing the process to preferably a testing service expert. Software testing can get very intense and time consuming. That’s exactly the reason why developers often get distracted from their core activity: developing and bug fixing!

No matter what kind of game/software/website you develope, it won’t reach its financial potential, if it isn’t user friendly. Since you and your team test the staging or live software many times over and over again, you’ll already be familiar with it's unique usability and features – but an external user wouldn’t. A new user may have a hard time finding buttons or instinctively using the software solution. This is exactly the reason for many companies to outsource or rather 'crowdsource' their e.g. mobile app testing to external software testers.

Crowdsourcing enables direct contact with a high number of high profile testers who are familiar with testing complex software solutions, using multiple devices. Instead of postponing your launch for an undisclosed amount of time to test it yourself, crowdsourcing companies will deploy multiple testers to check funktionality and usability from several devices at the same time. Important bugs will be filtered, the quality level adapts to a higher level and missing UI elements as well as additional features/spelling mistakes will be found. With an elaborated time management, the project will still reach the set launch date. Help getting your Quality Assurance at it's finest level. If you want a successful technological product, crowd testing is a must

Read More

Topics: New Features, Crowd Testing, Releases, quality assurance

Why is daily software testing of importance?

Posted by Thomas Grüderich

Apr 24, 2014 12:23:41 PM

The Importance of Software Testing

Most likely, everyone already made the experience of buying a product online, but unfortunately the purchasing process did not work. This is not only extremely frustrating and annoying for the customer, but also very costly and reputation damaging to the company. A recent, prominent example of software failures would be the Alitalia ticket promotion on Facebook, which enabled thousands of users eventually getting their tickets for free. The Italian airline just wanted to increase their social media presence and promoted a 25% discount on all international flights. However, soon it got to attention that it was now possible to book the flights for free, due to a software mistake. 

This of course, is a pleasant situation to customers, but an absolute catastrophe to corporations since they face extremely high losses due to software malfunction. Software testers can spare such negative experiences!  

Especially when the functionality of all applications of the Website/App has to be constantly ensured, daily testing is beneficial. Why? Because you want to guarantee that customers can purchase all your offered products, all the time, from everywhere and you don’t want to lose them just because the payment processes didn't work, for instance. For more information, please view or recent blog post 'Todays Software Testing Dilemma'

Daily testing is a software testing service, which is usually handled by a small group of testers, who identify potential bugs that could be harmful for the business. Fast reporting via the unique platform Cirro allows you and your developer team, to access and view identified bugs immediately or as soon as you enter the office in the morning. 

bild_ohne_hgCorporate bug fixing must be undertaken before any harm is caused. With daily testing it is possible to rotate between different browsers, operating systems (e.g. life or staging) and even countries. Especially when different product websites are offered and several developers are involved, it gets very complex to overview all software applications and bugs are difficult to detect. With daily testing, you can monitore all basic functions of your website/app and check if they operate correctly and create corresponding reports. testCloud arranges the crowd testing entirely to your requirements – testing within different German federal states would be a common example. Check our seven tips for explorative crowd testing.

If you want to know more about our services, watch our video, check the website or contact our team:

+49 30 6098 499 23

We are happy to answer any further questions.

Your testCloud Team


Read More

Topics: New Features, Crowd Testing, Events, Tester Community, Releases, Beta Testing

Software Testing Event Agenda 2014

Posted by Thomas Grüderich

Apr 16, 2014 6:34:00 AM

Event Agenda 2014

Topic: Software Testing 

Due to the increasing relevance software testing is gaining, testCloud most recently attended the Ministry of Testing Conference on March 28th 2014 in Brighton as micro sponsor. This event enabled meeting highly skilled people within the crowd testing industry, as well as getting to know new testing methods and technologies. It is overwhelming seeing tester communities constantly growing and developing to an industry section, which emphasizes the associated relevance of quality assurance to business applications. Strong team spirit and a mutual knowledge sharing sense constitute the heart of the tester community. 

Future European testing events, which seem being a promising opportunity, would be:

• The National Software Testing Conference 2014 ( 20-21. May, London, The British Museum ). This conference is a vehicle for any professional or business aligned with software testing to network, learn, swap and share advice and keep up-to-date with the latest industry trends.

• iqnite ( 20.-22. May, Düsseldorf, Congress Center Düsseldorf South ). The iqnite serves as a knowledge and experience sharing conference, which gives you insight andsupport into QA topics.

• Expo: QA Conference ( 26.-29. May, Madrid ). This conference offers interesting speeches, tutorials, exhibitors and workshops. It's a great opportunity to meet speakers and attendees in a warm and relaxed atmosphere.

• QA & Test 2014 ( 22.-24. October, Bilbao ). QA & Test is a unique conference that brings together professionals and experts from different sectors and intends to diffuse the latest technological developments.

• Euro Star - Software Testing Analysis and Review ( 24.-27. November in Dublin ). The test lab gives attendees the opportunity to test real systems with various kits and tools in a live, practical environment.


With the help of a tester crowd, corporations can significantly benefit through the view of 'the many'. Each tester usually has a different approach to your product and recognizes only distinctive features, so that this enables a broader spectrum for identifying bugs. Because testers have different professions, ranging from housewife to technician for instance, a deeper insight can be obtained, as well as various points of view because a housewife will most likely look at other aspects of the website than a technician. /seven-tips-for-explorative-crowd-testing According to studies, the wisdom of a crowd always beats the intelligence of a single software tester, even if it is competing with the smartest one.

Flexibility - testers being ready 24/7 to test any kind of software, because they are passionate about it, testing active within their leisure time, ensures both - qualified and prompt documentations. Because of long-term work experiences with us, testers are confident with typical testing procedures and can totally focus on identifying bugs. 

If you're keen on testing with our crowd, please don't hesitate to contact us and safe your 5 free-bug-test already today, as a special condition!

For more information, watch our video in english or german & visit our website

Read More

Topics: Crowd Testing, Events, Tester Community

Mobile App Testing Via Crowd Testing by Real Tester - A Necessity

Posted by Thomas Grüderich

Apr 11, 2014 12:18:00 AM

The Importance of Having Your Mobile App Crowd Tested

Software testing in general increasingly raised importance over the past few years. For to many people it is still regarded as cost-intensive and useless, but with advanced technology, this holds absolutely no true. Aside from the obvious benefit of having your app tested (discovering technical glitches), there is a much bigger benefit in having your software e.g. game tested – user-friendliness.

No matter what kind of game you create or how much fun it is to you and your developers, if it isn’t user friendly, it won’t reach its financial potential. Since you and your team test the game many times over and over again, you’ll already be familiar with how the game works and where everything is situated on the app – but a new user wouldn’t. A new user may have a hard time finding buttons or instinctively click on a common area that doesn’t work on your game (such as clicking the back arrow to return to the home screen). This is why so many large and successful gaming manufacturers outsource or rather 'crowdsource' their mobile app testing to external software testers.

Crowdsourcing puts you in direct contact with multiple high profile game testers who are familiar with games in your category, and with playing them on multiple devices. Instead of halting your launch for an undisclosed amount of time to test it yourself, crowdsourcing companies will deploy multiple testers to check your game from several devices at the same time. This helps you to discover bugs and quality assurance issues as well as poor UI elements and missing elements in time enough, to still hit your projected launch date. If you want to have a successful, technological product, crowd testing is a must.

Game Testing

There’s nothing quite like playing a fun and challenging game to pass the time. In the past, one could only enjoy this sort of activity in front of a television, but now gaming is virtually cord-free. As long as you have access to the internet, you can download games directly from the manufacturer to your mobile handset or tablet.

Both Android and iOS devices allow users to download free and paid mobile app games from their stores. This has become such an insanely popular way to play and access games, that the gaming company King has already made over $500 million in sales since its release of “Candy Crush” back in 2012.

Judging by King and other mobile app gaming companies, the profit margin made from creating these apps is quite high – if you get it right. This means the game has to have appeal, some paid features and be glitch free. While the former is easy to nail with a little market research, the latter isn’t so easy – this is where crowd testing comes in. Thereby testers diligently investigate into your app and report everything in the minutest detail, so that you can fix all bugs in time before the app's release. Need an idea how to set up an test? Seven tips for explorative crowd testing

Read More

Topics: Game Testing

Did you ever think about beta testing prior to launch?

Posted by Thomas Grüderich

Apr 3, 2014 7:04:00 PM

If you’re looking for a way to advertise your products and services while also building your brand and expanding your audience then the internet is the way to go. By creating a website and optimizing it with quality content, you won’t even have to find the audience, they will find you. These days, business owners are using apps alongside their website and blogs to maximize their visibility on the web and to promote easier engagement with their audience. Since the majority of mobile phone users download apps and use the mobile web, this seems to be the best way forward for businesses that wants long-term growth. However, one small glitch could cause an otherwise successful plan to crumble. This is why it’s so important to have your website, apps or created software tested before advertising it.

Why Beta Testing is Important

Have you ever gone on a website and got the dreaded 404 message? If you did and you were trying to make a purchase, chances are you went to a competing company and made your purchase without looking back. Right? Well, this is the same action that another consumer would take if they navigated to your website or app and it was down; this is why you need to have your product tested before launch.

Of course most software and web developers test the product themselves before releasing it to go public (alpha testing), but this has its cost as we indicated last week. In order to identify the real glitches people from outside of the organization need to provide feedback. Instead of using your audience as software guinea pigs, use beta testers to try out every aspect of your software and website. Beta testers will use the site in the same manner as regular web surfers/software users and note any bug found. Although this sounds easy, it requires an eye for detail – and of course, time. But, by running your software/website through a round of beta testing, you’ll be able to fix the bugs and launch your product successfully.

Benefits of Crowdsourcing Your Beta Testing

Crowdsourced software testing comes with a host of benefits. If you’re planning to reach the maximum number of testers possible, you’ll need people to test your product in multiple situations and under multiple conditions. Since you won’t have this number of testers at the ready, you can crowdsource your software testing.

Since usability and functionality are the two main elements that keep users coming back to your website or app, it’s vital that you have it tested before launch. It is true that bugs can occur at any stage during the launch, but these bugs are relatively easy to pinpoint and fix. On the other hand, early stage bugs may require reprogramming or a complete redevelopment plan. To be on the safe side, crowdsource your beta testing to save time, money and your business reputation.

Read More

Topics: Beta Testing

Four Tips for explorative Software Testing

Posted by Thomas Grüderich

Mar 25, 2014 11:55:00 PM

expl_testingIn order to create a software testing environment that covers a wide range of platforms, locations and languages, companies can rely on crowdsourcing.  It can be used to create scenarios as realistic as possible. Here are four handy tips:

1. Utilize the Wisdom of the Crowd

Explorative testing unleashes the full potential of the crowd as the name already implies.  Crowd testers thoroughly examine the entire web or app property without any specific guidelines. This unscripted and open process generally covers a wide range of the software and leads to the discovery of almost every imaginable bug. Ranging from simple spelling mistakes to critical bugs, which can have a severe bias on the software's performance and might affect the user experience in a negative way. 

2. Represent the Properties Target Audience Through Handpicked Testers

Depending on the exact case, testers should be carefully recruited, mainly based on their language skills, profession or testing experience. This selection process ensures a close match with the properties target audience and consequently also with submitted bug quality. 

3. Cover a Wide Range of Platforms

Covering as many platforms as possible is a critical aspect. People should especially focus on those platforms that are mostly utilized by the target audience. The crowd enables testing with a close sample perfectly representing it. No other technology offers a wider range of digital device and software application combinations as crowdsourced software testing.

4. Test "in the wild"

While your app or software runs smoothly inside your testing lab, this doesn't mean that it works everytime for everyone outside of your organization. Testing with a crowd also means, testing it where users are most likely to use the software. At home, at work, in rural or urban areas, users expect the software to function as intended. Like test driving a car on a real road instead of a simulator,  crowd testing is like putting the software in the hands of the target audience, stress testing it and receiving qualified feedback.

Read More

Topics: Crowd Testing

Best Practice Crowd Management in Software Testing

Posted by Thomas Grüderich

Mar 18, 2014 10:11:00 PM


After a while in the crowd testing space, we've had the chance to learn from numerous different scenarios and customers. Over time we identified several triggers that worked particularly well for managing an international crowd of software testers.



Enable the most Natural User Behavior

As mentioned frequently, the idea of crowd testing rests in the fact that all testing takes place under real conditions and not in the lab. But this particular behavior needs to be enabled, for example through an explorative testing approach. An explorative approach encourages the most natural user behavior under real conditions as the tester's mission is to explore the entire property without any predefined guidelines. No other commonly deployed software testing method offers such an unbiased point of view on products. Testers are usually guided in order to achieve certain results, which on the one hand generates more predictable results, but on the other hand also in less insights. But why are testers encouraged to search and document bugs? 

Ensure Superior Quality Through Pure Performance-based Compensation

While recruiting a representative tester panel is crucial, adequate motivation of each tester usually does not arise on its own. Mainly, because you need to provide an attractive incentive, in our case this would be a certain bonus depending on the type and severity of the found bug. This implies that crowdsourced software testers should be solely compensated based on their performance. In doing so, only relevant bugs comprising a thorough documentation will be submitted, hence only reproducible bugs are sent to developers. This ensures a high level of motivation paired with superior quality, as bug bonuses are only paid after the customers final approval.  

Challenge the Crowd Community

As a continuously high motivational level of every individual tester is a critical success factor, attractive projects and also incentives should be offered. Besides monetary incentives intangible factors such as the tester's social standing in the crowd can also be leveraged to generate superior results. Consequently, a well-managed community among others comprising a leaderboard – ranking the most successful bug hunters – or a point-based system – issuing points for superior performance – will generate extraordinary outcomes. This way you will always ensure the best possible experience for all parties, whether this is the tester, customer or your team.  

Read More

Topics: Crowd Testing, Tester Community

The Ideal Software Testing Environment

Posted by Thomas Grüderich

Mar 12, 2014 8:22:00 PM

What does the ideal environment look like?

This certainly depends on each individual case as well as the circumstances software will be used in the field. Overall, we can identify similarities that help in creating an ideal environment for software testing.


First of all, the test environment should consist of multiple unbigoted, highly motivated and heterogeneous testers, not just a hand full of in-house testers. This will ensure, that the properties target audience is mapped accordingly, meaning that certain locations, languages, device/OS configurations, carriers and so on… are covered in a test. This setup could be described as 'real conditions' and clearly contradicts commonly simulated 'in the lab' test environments.

Second, making sure that testers are dedicated to their tasks and handle all testing efforts strictly confidential is a further factor that should not be underestimated. Especially, when testing unreleased beta versions or security relevant product features. Consequently, software testers must be properly incentivized to deliver high quality bug reports and at the same time keep all information undisclosed. 

At this point, it is pretty obvious that these conditions are usually uncommon when testing software and can only be provided by a fairly large group of people. This diversified and fairly large group of people may be called a crowd. Hence, the correspondent testing type is called 'crowdsourced software testing' or simply 'crowd testing'. It enables companies to create exactly the test environment they need at a fraction of the cost that would be accumulated by an in-house solution. 

Feel free to contact us - if your're interested in learning more about crowd testing.

Read More

Topics: Software Testing, Crowd Testing

Subscribe to our blog