Blog

clouds-5bf297e1

Crowd Testing News and Trends

How do I choose the best Bug Tracker for my needs?

Posted by Jan Schwenzien

Sep 11, 2014 9:35:00 AM

How do I choose the best Bug Tracker for my needs?

Anyone who was ever developed a software product knows of the advantages of bug trackers. Bug trackers conveniently support developers and their teams wanting to fix bugs by saving them valuable time. The availability of various bug trackers makes it hard for software developers to choose the right one since there is no universally perfect bug tracker. Which one is perfect for you and your development team depends on a set of criteria we have laid out for you below.

While unnecessarily high complexity can slow down the productivity of your development process, a bug tracker that is too simplistic can delay and possibly harm the development of your software. This is why testCloud has shifted the focus of this away from the sheer functionality of the tools to the characteristics and technological possibilities of your team.

The technical capabilities of your users

In every team there are always technical and non-technical members. The non-technical ones may involve financial analysts, customers and also end-users. In our experience, non-technical users may not be so happy about using a complicated bug tracker. Even the most advanced features are worthless if they aren’t used right and a tool that is more easy to use would be more applicable in the case of non-technical team members.

The complexity of your workflow 

In large, highly-specialized teams bug trackers with a simple workflow will not suffice as there may be many layers of bureaucracy involved in making a final decision about a product before its release or launch. On the other hand in small teams the same workflow can lead to redundancy and ambiguity. That is the reason why for smaller teams bug trackers with a simple workflow might be more applicable.

The size of your team

It is not always clear at the start of a project that will be a part of the team. Since many bug trackers charge per user, the uncertainty of the team size can further complicate the choice of the right bug tracker as unnecessary costs can arise for infrequently used accounts. On the other hand if there are not enough accounts available for the project when they are needed, delays in the development process might be a result. That is why you should precisely plan your project and involve all of the team members, even if their accounts will be used more infrequently than those of the development team.

The structure of your team

How are all the participants connected and what is their role in the project? Do you have an internal Quality Assurance team or do you work together with external stakeholders that need access to the bug tracker? These questions among others also influence your bug tracker choice.

For your convenience we have ranked the different bug trackers according to their complexity:

Simple

These tools have limited options for configuration which will make them easily usable, however at the cost of flexibility:

- BugHerd

- Redmine

- Mantis

Advanced

These tools allow a more sophisticated configuration and have a steep learning curve:

 - JIRA

- Bugzilla

- YouTrack

Professional/ Project Management

These tools incorporate bug trackers but can also be used for the whole project management:

- Trello

- Pivotal Tracker

-  Assembla

“Ultimately, personal preferences and previous experiences with such tools always play a big role when deciding on bug trackers. Knowing this we offer all established bug trackers with our crowd software testing service,” says Jan Schwenzien, CTO at testCloud.

For those looking for expert testing help to free up their time, SaaS crowd software testing providers like testCloud offer the right solution. By having permanent access to a network of thousands of software testers, your application can be screened for bugs quickly and highly efficiently. You will be provided with a detailed bug report you can easily import into your bug tracking system.

Interested? Just sign up here to explore our SaaS platform.

 

Read More

Topics: Software Testing, Crowd Testing, Beta Testing, Bug Tracker

The perfect recipe of testing methods for your software application

Posted by Thomas Grüderich

Aug 29, 2014 3:52:00 PM

The perfect recipe of testing methods for your software application

Every comprehensive software testing strategy uses a combination of different testing methods. Which one is going to be used, strongly depends on the kind of software that needs to be tested. From completely automated testing methods and scripted test cases to explorative tests and private or public beta tests: the right application of quality and succession will insure the best possible examination of your software before its official launch.

An overview of the three most important stages on the path to a software that is free of bugs and explanation of each testing method’s value, their strengths and weaknesses, as well as the right time to use them:

Test Cases

A technological product (Software) is inspected according to exact parameters. Test cases do primarily guarantee that a software can be correctly used, just as the developers been imagining it. The main advantage of this kind of testing is a quick dulcification and reproducibility – that way they can be repeated identically in every phase of the development process.

A simple example could be e.g testing a shopping cart of an online shop. Questions that can be answered using test cases are among the following: Can every product of the shop be added to the shopping cart? Appear all prices of the products correctly?

This kind of testing can verify crucial insights but can also be fairly limited. What if the shopping cart would show wrong pictures or wrong product descriptions? If those parameters are not considered while setting up the test, they won’t have an effect on this test run.

Exploratory Testing

Every kind of software and every single product profits from the dialogue with stakeholders and target group to gain important feedback before and during the development process. Exploratory tests checks the system in depth by not requiring the tester to adhere to pre-defined parameters. Each tester investigates a software individually to discover aspects that the developer might have overlooked.

In our shopping cart example they would also test the possibility to remove things or change the number of items.

Usability Testing

Software testing can be done regularly before a release – but as soon as it hits the market you will be confronted with a variety of unexpected things. It doesn’t matter if you decide involving the target group as potential testers on a live software or provide an public access for the software’s beta version: Important note – collecting feedback before the official product release is essential.

There is no appropriate substitute for real world conditions – but at least it’s worth a try. Usability tests can help to reveal shortcomings of a system and teach us a lot about the expectations of your potential users at the same time - how your product will likely be perceived in the real world.

Regarding the shopping cart, users might fill it up but return one week later. In this case he would expect that the shopping items are still stored in the shopping cart ready for checkout. If not he would decide against continue shopping in the worst case scenario. Different users will do things that seem intuitive to them, while it seems completely implausible to you. Users will always produce scenarios that developer and QA personnel did not consider. 

The mix matters!

The choice for the combination of the different test methods depends on the type of system as well as the future users. Generally, business softwares are mostly tested automatically, e.g. verification of processes, whereas usability tests are more recommended for consumer-oriented systems like online shops or social games.

Conclusion

A good approach is to focus on test cases, until the core functions of your software are working properly. Shift to exploratory testing thereafter to get relevant feedback to functions you might have missed in your development.

Round out your testing process with usability tests to find out your user´s acceptance and get your product into an optimal form for your target group before the official release.

 

Read More

Topics: Software Testing, Crowd Testing, Releases, Beta Testing, testing methods

Black-Box Testing vs. White-Box Testing

Posted by John Faucett

Aug 14, 2014 9:49:00 AM

Black-Box Testing vs. White Box Testing - Possibilities and Limitations of both Testing Methods

If you've ever been involved with Software Testing in any way, you've probably heard the Terms being thrown around and perhaps on first thought they were next generation game consoles from Microsoft.

But what are they and how can these two testing methods ensure that our Software does what's its suppose to do?

Let's take White-Box testing first, since historically it`s the most wide-spread method in use. If you're a programmer and write tests for your own code, you're white-box testing. White-Box testing means the tester has full access to the source code and intimately understands the software. 
This is a key point, since as a White-Box Tester you know the inner workings, you're going to test the software based on that knowledge, and the assumptions made by the code itself. Unit testing, Integration testing, regression testing, and coverage tests fall into this category. It can be used to assert that a part of the system does exactly what the programmer/tester expects it to do. Without this, you have no way of knowing your upload form even works - except by trying it yourself of course. But change one line months later and the whole system might break without you knowing it - unless of course you've done your white-box unit tests. Which is why this method is so beloved and so widely adopted.


At the opposite end of the spectrum, there's Black-Box testing. This is where the software testers have no idea about the internal workings of the system, though they usually have some document telling them what the Software is supposed to do.Functional Testing, and Test Cases fall into this category.It`s much more open-ended, since testers don't know the intimate details of the system or exactly what the programmer expected as inputs. 
Because of this, an entire new set of errors and mistakes can be discovered. This is the area where Crowd-Testing thrives. Even if you have 100% Code-Coverage and a massive barrage of Unit Tests, all this still won't let you know that no end-user can find your 'Buy Now' button, or that your css layout explodes in a rotated iPad2, or that users expect the download link to download an image and not a word document.


And unfortunately Black-Box testing is the most oft neglected, Project Managers are thinking ship, ship, ship, and programmers think, all my unit tests are green so this software must do what its intended to do. But in the end, only your users will tell you if your software is good or not, so if you want to save time, wasted effort, and money - let them in from the beginning - do Black-Box testing.

 

Read More

Topics: Crowd Testing, quality assurance, testing methods

Why is Crowd Testing of higher value than simple testing techniques?

Posted by Thomas Grüderich

May 21, 2014 12:30:00 PM

Why is crowd testing of higher value than simple inhouse testing techniques?

Almost every company sooner or later faces the question of whether to test inhouse or outsourcing the process to preferably a testing service expert. Software testing can get very intense and time consuming. That’s exactly the reason why developers often get distracted from their core activity: developing and bug fixing!

No matter what kind of game/software/website you develope, it won’t reach its financial potential, if it isn’t user friendly. Since you and your team test the staging or live software many times over and over again, you’ll already be familiar with it's unique usability and features – but an external user wouldn’t. A new user may have a hard time finding buttons or instinctively using the software solution. This is exactly the reason for many companies to outsource or rather 'crowdsource' their e.g. mobile app testing to external software testers.

Crowdsourcing enables direct contact with a high number of high profile testers who are familiar with testing complex software solutions, using multiple devices. Instead of postponing your launch for an undisclosed amount of time to test it yourself, crowdsourcing companies will deploy multiple testers to check funktionality and usability from several devices at the same time. Important bugs will be filtered, the quality level adapts to a higher level and missing UI elements as well as additional features/spelling mistakes will be found. With an elaborated time management, the project will still reach the set launch date. Help getting your Quality Assurance at it's finest level. If you want a successful technological product, crowd testing is a must

Read More

Topics: New Features, Crowd Testing, Releases, quality assurance

Why is daily software testing of importance?

Posted by Thomas Grüderich

Apr 24, 2014 12:23:41 PM

The Importance of Software Testing

Most likely, everyone already made the experience of buying a product online, but unfortunately the purchasing process did not work. This is not only extremely frustrating and annoying for the customer, but also very costly and reputation damaging to the company. A recent, prominent example of software failures would be the Alitalia ticket promotion on Facebook, which enabled thousands of users eventually getting their tickets for free. The Italian airline just wanted to increase their social media presence and promoted a 25% discount on all international flights. However, soon it got to attention that it was now possible to book the flights for free, due to a software mistake. 

This of course, is a pleasant situation to customers, but an absolute catastrophe to corporations since they face extremely high losses due to software malfunction. Software testers can spare such negative experiences!  

Especially when the functionality of all applications of the Website/App has to be constantly ensured, daily testing is beneficial. Why? Because you want to guarantee that customers can purchase all your offered products, all the time, from everywhere and you don’t want to lose them just because the payment processes didn't work, for instance. For more information, please view or recent blog post 'Todays Software Testing Dilemma'

Daily testing is a software testing service, which is usually handled by a small group of testers, who identify potential bugs that could be harmful for the business. Fast reporting via the unique platform Cirro allows you and your developer team, to access and view identified bugs immediately or as soon as you enter the office in the morning. 

bild_ohne_hgCorporate bug fixing must be undertaken before any harm is caused. With daily testing it is possible to rotate between different browsers, operating systems (e.g. life or staging) and even countries. Especially when different product websites are offered and several developers are involved, it gets very complex to overview all software applications and bugs are difficult to detect. With daily testing, you can monitore all basic functions of your website/app and check if they operate correctly and create corresponding reports. testCloud arranges the crowd testing entirely to your requirements – testing within different German federal states would be a common example. Check our seven tips for explorative crowd testing.

If you want to know more about our services, watch our video, check the website or contact our team:

+49 30 6098 499 23

We are happy to answer any further questions.

Your testCloud Team

 

Read More

Topics: New Features, Crowd Testing, Events, Tester Community, Releases, Beta Testing

Software Testing Event Agenda 2014

Posted by Thomas Grüderich

Apr 16, 2014 6:34:00 AM

Event Agenda 2014

Topic: Software Testing 

Due to the increasing relevance software testing is gaining, testCloud most recently attended the Ministry of Testing Conference on March 28th 2014 in Brighton as micro sponsor. This event enabled meeting highly skilled people within the crowd testing industry, as well as getting to know new testing methods and technologies. It is overwhelming seeing tester communities constantly growing and developing to an industry section, which emphasizes the associated relevance of quality assurance to business applications. Strong team spirit and a mutual knowledge sharing sense constitute the heart of the tester community. 

Future European testing events, which seem being a promising opportunity, would be:

• The National Software Testing Conference 2014 ( 20-21. May, London, The British Museum ). This conference is a vehicle for any professional or business aligned with software testing to network, learn, swap and share advice and keep up-to-date with the latest industry trends.

• iqnite ( 20.-22. May, Düsseldorf, Congress Center Düsseldorf South ). The iqnite serves as a knowledge and experience sharing conference, which gives you insight andsupport into QA topics.

• Expo: QA Conference ( 26.-29. May, Madrid ). This conference offers interesting speeches, tutorials, exhibitors and workshops. It's a great opportunity to meet speakers and attendees in a warm and relaxed atmosphere.

• QA & Test 2014 ( 22.-24. October, Bilbao ). QA & Test is a unique conference that brings together professionals and experts from different sectors and intends to diffuse the latest technological developments.

• Euro Star - Software Testing Analysis and Review ( 24.-27. November in Dublin ). The test lab gives attendees the opportunity to test real systems with various kits and tools in a live, practical environment.

Bildschirmfoto_2014-04-15_um_16.40.08

With the help of a tester crowd, corporations can significantly benefit through the view of 'the many'. Each tester usually has a different approach to your product and recognizes only distinctive features, so that this enables a broader spectrum for identifying bugs. Because testers have different professions, ranging from housewife to technician for instance, a deeper insight can be obtained, as well as various points of view because a housewife will most likely look at other aspects of the website than a technician. /seven-tips-for-explorative-crowd-testing According to studies, the wisdom of a crowd always beats the intelligence of a single software tester, even if it is competing with the smartest one.

Flexibility - testers being ready 24/7 to test any kind of software, because they are passionate about it, testing active within their leisure time, ensures both - qualified and prompt documentations. Because of long-term work experiences with us, testers are confident with typical testing procedures and can totally focus on identifying bugs. 

If you're keen on testing with our crowd, please don't hesitate to contact us and safe your 5 free-bug-test already today, as a special condition!

For more information, watch our video in english or german & visit our website

Read More

Topics: Crowd Testing, Events, Tester Community

Mobile App Testing Via Crowd Testing by Real Tester - A Necessity

Posted by Thomas Grüderich

Apr 11, 2014 12:18:00 AM

The Importance of Having Your Mobile App Crowd Tested

Software testing in general increasingly raised importance over the past few years. For to many people it is still regarded as cost-intensive and useless, but with advanced technology, this holds absolutely no true. Aside from the obvious benefit of having your app tested (discovering technical glitches), there is a much bigger benefit in having your software e.g. game tested – user-friendliness.

newsletter_with_testcloud
No matter what kind of game you create or how much fun it is to you and your developers, if it isn’t user friendly, it won’t reach its financial potential. Since you and your team test the game many times over and over again, you’ll already be familiar with how the game works and where everything is situated on the app – but a new user wouldn’t. A new user may have a hard time finding buttons or instinctively click on a common area that doesn’t work on your game (such as clicking the back arrow to return to the home screen). This is why so many large and successful gaming manufacturers outsource or rather 'crowdsource' their mobile app testing to external software testers.

Crowdsourcing puts you in direct contact with multiple high profile game testers who are familiar with games in your category, and with playing them on multiple devices. Instead of halting your launch for an undisclosed amount of time to test it yourself, crowdsourcing companies will deploy multiple testers to check your game from several devices at the same time. This helps you to discover bugs and quality assurance issues as well as poor UI elements and missing elements in time enough, to still hit your projected launch date. If you want to have a successful, technological product, crowd testing is a must.

Game Testing

There’s nothing quite like playing a fun and challenging game to pass the time. In the past, one could only enjoy this sort of activity in front of a television, but now gaming is virtually cord-free. As long as you have access to the internet, you can download games directly from the manufacturer to your mobile handset or tablet.

Both Android and iOS devices allow users to download free and paid mobile app games from their stores. This has become such an insanely popular way to play and access games, that the gaming company King has already made over $500 million in sales since its release of “Candy Crush” back in 2012.

Judging by King and other mobile app gaming companies, the profit margin made from creating these apps is quite high – if you get it right. This means the game has to have appeal, some paid features and be glitch free. While the former is easy to nail with a little market research, the latter isn’t so easy – this is where crowd testing comes in. Thereby testers diligently investigate into your app and report everything in the minutest detail, so that you can fix all bugs in time before the app's release. Need an idea how to set up an test? Seven tips for explorative crowd testing

Read More

Topics: Game Testing

Did you ever think about beta testing prior to launch?

Posted by Thomas Grüderich

Apr 3, 2014 7:04:00 PM

If you’re looking for a way to advertise your products and services while also building your brand and expanding your audience then the internet is the way to go. By creating a website and optimizing it with quality content, you won’t even have to find the audience, they will find you. These days, business owners are using apps alongside their website and blogs to maximize their visibility on the web and to promote easier engagement with their audience. Since the majority of mobile phone users download apps and use the mobile web, this seems to be the best way forward for businesses that wants long-term growth. However, one small glitch could cause an otherwise successful plan to crumble. This is why it’s so important to have your website, apps or created software tested before advertising it.

Why Beta Testing is Important

Have you ever gone on a website and got the dreaded 404 message? If you did and you were trying to make a purchase, chances are you went to a competing company and made your purchase without looking back. Right? Well, this is the same action that another consumer would take if they navigated to your website or app and it was down; this is why you need to have your product tested before launch.

Of course most software and web developers test the product themselves before releasing it to go public (alpha testing), but this has its cost as we indicated last week. In order to identify the real glitches people from outside of the organization need to provide feedback. Instead of using your audience as software guinea pigs, use beta testers to try out every aspect of your software and website. Beta testers will use the site in the same manner as regular web surfers/software users and note any bug found. Although this sounds easy, it requires an eye for detail – and of course, time. But, by running your software/website through a round of beta testing, you’ll be able to fix the bugs and launch your product successfully.

Benefits of Crowdsourcing Your Beta Testing

Crowdsourced software testing comes with a host of benefits. If you’re planning to reach the maximum number of testers possible, you’ll need people to test your product in multiple situations and under multiple conditions. Since you won’t have this number of testers at the ready, you can crowdsource your software testing.

Since usability and functionality are the two main elements that keep users coming back to your website or app, it’s vital that you have it tested before launch. It is true that bugs can occur at any stage during the launch, but these bugs are relatively easy to pinpoint and fix. On the other hand, early stage bugs may require reprogramming or a complete redevelopment plan. To be on the safe side, crowdsource your beta testing to save time, money and your business reputation.

Read More

Topics: Beta Testing

Four Tips for explorative Software Testing

Posted by Thomas Grüderich

Mar 25, 2014 11:55:00 PM

expl_testingIn order to create a software testing environment that covers a wide range of platforms, locations and languages, companies can rely on crowdsourcing.  It can be used to create scenarios as realistic as possible. Here are four handy tips:

1. Utilize the Wisdom of the Crowd

Explorative testing unleashes the full potential of the crowd as the name already implies.  Crowd testers thoroughly examine the entire web or app property without any specific guidelines. This unscripted and open process generally covers a wide range of the software and leads to the discovery of almost every imaginable bug. Ranging from simple spelling mistakes to critical bugs, which can have a severe bias on the software's performance and might affect the user experience in a negative way. 

2. Represent the Properties Target Audience Through Handpicked Testers

Depending on the exact case, testers should be carefully recruited, mainly based on their language skills, profession or testing experience. This selection process ensures a close match with the properties target audience and consequently also with submitted bug quality. 

3. Cover a Wide Range of Platforms

Covering as many platforms as possible is a critical aspect. People should especially focus on those platforms that are mostly utilized by the target audience. The crowd enables testing with a close sample perfectly representing it. No other technology offers a wider range of digital device and software application combinations as crowdsourced software testing.

4. Test "in the wild"

While your app or software runs smoothly inside your testing lab, this doesn't mean that it works everytime for everyone outside of your organization. Testing with a crowd also means, testing it where users are most likely to use the software. At home, at work, in rural or urban areas, users expect the software to function as intended. Like test driving a car on a real road instead of a simulator,  crowd testing is like putting the software in the hands of the target audience, stress testing it and receiving qualified feedback.

Read More

Topics: Crowd Testing

Best Practice Crowd Management in Software Testing

Posted by Thomas Grüderich

Mar 18, 2014 10:11:00 PM

crowd

After a while in the crowd testing space, we've had the chance to learn from numerous different scenarios and customers. Over time we identified several triggers that worked particularly well for managing an international crowd of software testers.

 

 

Enable the most Natural User Behavior

As mentioned frequently, the idea of crowd testing rests in the fact that all testing takes place under real conditions and not in the lab. But this particular behavior needs to be enabled, for example through an explorative testing approach. An explorative approach encourages the most natural user behavior under real conditions as the tester's mission is to explore the entire property without any predefined guidelines. No other commonly deployed software testing method offers such an unbiased point of view on products. Testers are usually guided in order to achieve certain results, which on the one hand generates more predictable results, but on the other hand also in less insights. But why are testers encouraged to search and document bugs? 

Ensure Superior Quality Through Pure Performance-based Compensation

While recruiting a representative tester panel is crucial, adequate motivation of each tester usually does not arise on its own. Mainly, because you need to provide an attractive incentive, in our case this would be a certain bonus depending on the type and severity of the found bug. This implies that crowdsourced software testers should be solely compensated based on their performance. In doing so, only relevant bugs comprising a thorough documentation will be submitted, hence only reproducible bugs are sent to developers. This ensures a high level of motivation paired with superior quality, as bug bonuses are only paid after the customers final approval.  

Challenge the Crowd Community

As a continuously high motivational level of every individual tester is a critical success factor, attractive projects and also incentives should be offered. Besides monetary incentives intangible factors such as the tester's social standing in the crowd can also be leveraged to generate superior results. Consequently, a well-managed community among others comprising a leaderboard – ranking the most successful bug hunters – or a point-based system – issuing points for superior performance – will generate extraordinary outcomes. This way you will always ensure the best possible experience for all parties, whether this is the tester, customer or your team.  

Read More

Topics: Crowd Testing, Tester Community