Killer Bugs And Their Effect On Your Conversion Rates

0aaaPhil Soffer TestIo

One of the biggest issues in e-Commerce today (and every day) is conversion. Data scientists and marketers actively influence and track every stage of the sales process, down to the movement of items successfully into shopping carts. Then something goes awry and the sale isn’t completed, leaving teams to question where their carefully planned experience went wrong. Often it’s a pesky little bug in the software that went unnoticed in initial testing or subsequent updates. Left unchecked, that tiny bug wreaks havoc.

E-Commerce managers call these bugs “conversion killers,” because these issues can block potential customers from making a purchase when they want to or frustrate them enough that they stop attempting to buy it. Sale killed — and opportunity lost.

To hone in on where most conversion killers occur in the checkout process, my company recently investigated. We concentrated our efforts on critical-level issues that prevented a main function of the app or web site, or that caused a potential loss of income for the company at the checkout level. Here’s what we found:


Payments Are An Achilles Heel

More than one-third (36.3%) of critical bugs in our sample were payment related. Nearly half (49.5%) of problems were due to credit cards because of outside dependencies and integration issues, including the ability to securely verify card numbers, dates and security codes as well as the ability to connect with credit card networks like Visa and Mastercard. There were also web site issues on the seller’s credit card payment page. PayPal accounted for another 29.0% of issues during checkout, with bugs ranging from authentication to integration of various PayPal services into the e-Commerce platform to problems with the data transfer in between. The remaining 21.5% of payment errors had to do with issues like accepting and applying vouchers and coupon codes as well as direct debits.

What is clear is that accepting online payments requires a large number of fields and seamless, secure integration with a large number of outside organizations. Creating and testing what happens with each interaction in each field across multiple payment systems can be difficult and unwieldy. It calls for familiarity with the implementation details of the respective payment systems, particularly for software testers.

For example, one recurring bug documented in multiple checkout sections prevented testers from using Visa cards with 19 digits. While credit cards with 16-digit payment card numbers are most common, the lengths of card numbers range from 12 digits up to 19 digits. Through an expansion on Visa’s VPAY sub-brand, 19-digit cards are becoming more common, so imagine how many sales could be lost if the bug were not quickly detected and addressed.

Is That Right?

Consumers are known for changing their minds. Items go into carts and right back out, opening software up to numerous possibilities for a glitch. This is why pricing and shopping cart issues ranked behind payments as a notable source of critical errors, at just over 10%. Between discrepancies in item quantities, missing items and faulty transitions between the cart and checkout and calculation errors, incorrect tax rates, delivery fee misapplication, prices not updating when the quantity of items changes and incorrect application of promotions, there is enormous margin for error.

Robust and thorough software testing of all possible combinations is essential, and the complexity involved should not be underestimated. For example, if a potential customer changes the item quantity in the shopping cart, but the tax rate and shipping costs don’t update correctly to reflect the new total price, that loss of confidence about what the final cost will be leads directly to a lost sale or distrust in the vendor. Companies simply cannot afford to get slack in making sure that shoppers pay the correct amount at checkout.

I Live In Athens…Greece, Not Georgia

Errors around shipping and addresses also caused problems. If there are difficulties entering a shipping address or calculating what the shipping cost is, the potential customer won’t feel assured that their purchase will be sent to the correct address — if they’re able to enter it at all. Some of the shipping-related errors uncovered in our investigation included not being able to select a state, failure to check out with an international address, inability to add or save billing or shipping addresses, and/or shipping to multiple addresses causing phantom items. Validating customer entered data, such as emails and phone numbers, also proved to be critical to conversion rates.

These are areas where the average programmer rarely can plan for all edge cases and how they interact with each other. Choosing the right validation methods (libraries), modeling complicated interactions like shipping and checkout with care, and working with a broad range of testers who know where real-world problems could pop up are a few strategies to avoid these conversion-killing software bugs that can seriously impact revenue.

The (Back) Button

Button-related issues account for a very small percentage (4.6%) of revenue-related critical errors (even though they are prone to making mobile apps crash). Nonetheless, these software issues can be hard for automated tests to detect, since many of them are caused by user interface display problems. These are the kinds of problems that human testers uncover exceedingly well, whereas automated testing can’t tell that the button is out of sight of the screen, invisible, or covered by another UI element. For example, one bug report submitted by a human tester documented how all products he/she added to the shopping cart were removed when browser back button was used. This is not the intended behavior, nor is it conducive to helping potential customers complete transactions successfully.

To avoid some of these issues, developers can store the state of the user shopping cart. By doing so, if the user clicks the browser back button, the contents of their cart doesn’t get lost. This has the added benefit of allowing users to visit other sites and return to finish their purchase. It’s also recommended not to break the back button functionality with automatic redirects. These redirects prevent users from navigating through a site using the normal browser buttons and history.


It can seem like the checkout process is fraught with pitfalls, and it can be. But quality software testing, combining automated and human testing, makes the difference. Automated tests are great for finding bugs on the happy path. Human testers can then check any complicated interactions and graphical user interface issues to ensure your checkout process runs smoothly.

Your team may not be able to fix all the bugs that are uncovered, but through comprehensive testing, you’ll be working from a position of knowledge: you can prioritize the issues that have the biggest potential impact on the bottom line and your business. It’s possible to prevent significant revenue loss, offer a seamless experience to customers and avoid losing customer trust.

Philip Soffer is the CEO of test IO, which helps software teams ship high quality software faster. As a global leader in software crowdtesting, test IO enables fast-moving software development teams with a platform for on-demand QA testing throughout the entire development cycle. Soffer began his career as an early employee at Plumtree Software, which founded and led the market for enterprise portals. Working with Plumtree founder Glenn Kelman, Soffer built out the marketing and product management functions, eventually growing the product line to include collaboration, content management, search, and developer tools. After Plumtree, Soffer joined Lithium Technologies, where he led engineering, product management, and occasionally marketing. 

Feature Your Byline

Submit an Executive ViewPoints.

Featured Event

Join the retail community as we come together for three days of strategic sessions, meaningful off-site networking events and interactive learning experiences.


Access The Media Kit


Access Our Editorial Calendar

If you are downloading this on behalf of a client, please provide the company name and website information below: