Whether you’re just starting out or you have numerous apps in the market, it’s important to be aware of how much quality can differentiate your product in the marketplace. In the increasingly competitive app ecosystem, quality really is key, as only 16% of people say they will try a failing app more than twice. Not to mention the fact there are over a million other apps in the various markets out there for your customers to choose from.
It’s a competitive space, but it’s certainly possible to give yourself an edge. There are three key areas where quality control can be engineered into your app: testing, crash reporting, and feedback management. In this article you’ll get a high-level overview of each category, some examples of tools and services available, and why that specific area is important in building a quality experience for your customers.
Testing and Test Automation
If you’re using Test Driven Development, Behavior Driven Development, or any number of other strategies, you’re likely familiar with unit testing. This is a way of engineering code with testing in mind. For Android, JUnit is the weapon of choice, which goes very well with the free tool, Robolectric, an open-source project for running Android unit tests in a headless environment. This is great because it’s very fast (way faster than the notoriously slow Android emulator). For iOS, OCUnit is a popular choice, and, because the simulator uses hardware acceleration, it’s fast enough out of the box to run without the aid of a Robolectric-like harness.
Functional tests, tests that exercise basic functionality and flows within an app, are prime candidates for automation. Given the thousands of device models on the market, manually running such tests is a daunting task, especially because these types of tests are most valuable when run on real, physical hardware. Tools like Robotium, a library that integrates with JUnit, and Calabash, a cross-platform framework, help make writing automated, scalable test scripts easy that can be run on emulators, simulators, and real hardware.
A popular strategy is to run tests locally on emulators and simulators, then run them on local devices with tools like Spoon, and then run those same tests on hundreds of devices hosted by services like AppThwack. Rather than spend time building automation harnesses, device labs, and other necessary components of an end-to-end test environment, services like AppThwack remove the burden by providing cloud-based device labs for rapid, parallel test automation execution.
Exploratory and UX testing is very important as well, and this is where humans excel. A good strategy is to automate your mundane tests so your time can be spent exploring new functionality, conducting labs where you can observe real people as they interact with your app, and so on. Having a human touch is paramount in measuring how an app actually feels from a customer perspective.
Keeping Track of Errors
Let’s face it. No matter how well you code and how much you test, we all write bad code, meaning some errors are bound to make it through. In fact, with the speed of today’s development only getting faster and the increasing adoption of agile development practices, it’s necessary now more than ever to engineer in anticipation of unexpected errors.
There are numerous tools for tracking crash reports in the wild for both Android and iOS. For Android, ACRA is a popular free option. There are commercial services available that support multiple platforms and add bells and whistles to crash reporting, such as pattern identification, tracking issues over time, and analyzing performance metrics, but at their core the biggest value of any crash reporting library is to provide insight into how your app is behaving once it’s in the market.
Once you know how and where your app is crashing, you can more quickly fix the reported issues, avoiding uninstalls and bad reviews. While it’s best to capture bugs before customers encounter them, at least you won’t be in the dark until someone decides to tell you about a problem. For every one complaint you receive there are 26 others you don’t, which should emphasize just how important this step is.
What’s more accurate, your own impression of how your app performs or your customers’? The answer is obvious, but time and time again we, as developers, assume we know best. Your customers’ feedback should guide your decisions, whether that means acknowledging their comments and consciously dismissing them, or restructuring your app so it better fits their needs.
The obvious question, then, is how do you interact with your customers so you get valuable information while not seeming needy, overbearing, or paranoid? It’s important to form dialogs with your customers and encourage them to communicate with you, rather than create a confrontational relationship that leaves you both unhappy.
Services such as Apptentive do the work for you, inserting feedback prompts into your app in a way that gathers pertinent information and, ideally, avoids the most damaging form of feedback: poor reviews on the market.
By keeping these three categories in mind, building app quality into the app lifecycle should be a clear strategy going forward. Coding with testing in mind (unit tests) and automating tests that make sense (functional and performance tests), tracking app performance (crash reporting), and managing feedback (customer dialogs) will all help differentiate your app in the market. With more than a 1.5 million apps on the various platforms, that kind of positive differentiation can’t hurt.
Trent Peterson is a founder at AppThwack, a fully automated service that helps developers and QA teams test their Android, web, and iOS apps on 100s of real devices in minutes, gathering high-level results, low-level logs, pixel-perfect screenshots, and performance trends along the way. You can keep up with Trent and AppThwack on Google+ and Twitter.
If there is one thing that all app developers should know now, it is that obtaining user feedback from your current customers is much more helpful than reading their negative comments in the app store reviews.
No one enjoys seeing his or her hard work be lambasted by people who cannot grasp the scope of its existence. Although it is disheartening, and ego deflating, it also turns the app into something of little service to others. Before someone downloads an app, there is a good chance that they are going to read the reviews first. If they download the app without bothering with another’s opinion and find it hard to use, they may even leave their own negative feedback. Countering this negativity is possible with the next version or damage control marketing measures, but ideally, the creator should want to avoid them to begin with. After all, this ominous circle of negativity can send an app into the black hole of App Stores everywhere, never to be seen again.
Get better ratings by avoiding negative reviews
Instead of envisioning an app world of mayhem, developers everywhere are realizing that with a simple testing period of a minimum viable product, consumer test groups can provide feedback before the app launches. But why stop there? You should keep gathering feedback even while your app is live.
What is a Minimum Viable Product?
A minimum viable product does not mean that it is less of a product than it will be in its finalized form. In the app world, it simply means that it has enough functionality to deploy its capabilities, but is not coded to complete operation.
Keep your v1.0 simple and iterate with feedback from the people using your app
This means you can open it, review its contents and show it off to others as a prototype, without spending an exhausting amount of time or money insuring its overall functionality through coding and programming. It allows you to gauge interest, excitement, feedback, and the overall need for your app, before you take it to the completion stage.
A minimum viable product is tested on a controlled group of people, whose comments and conclusions on its testing can be accounted for. This means you can let people know they are responsible for helping you create a better product together. That is the important word “together.” People love to be included in the creation of things. The idea is to find a forgiving audience, one who knows that this is only a beginning but could benefit from the final product’s existence.
Get Your Users To Help You Create The Best Product Possible
Think of it this way: If you give someone an app and say, “This app is going to do this in v1.0. Here it is in its most minimal functionality. Play with it.” When that introduction and testing period is followed up with surveys for the consumer to address, they can provide honest and accurate feedback.
Iterate with your app customers to build a better product
This will deliver information regarding what they were expecting, and how the app lived up to those expectations. It can also tell the developer how great or how poor its performance ranked overall, which means they can take that information back to the drawing board and fine tune the application for v2.0.
How Surveys Insure a Better Minimum Viable Product
It is practically impossible to fund every idea that your genius mind creates, which means when it comes to getting app creation right the first time, surveys can be a key financial component to its success. Creating a minimum viable product allows you to display the app’s genius, without developing it completely. This means no more ill-advised turns in the wrong direction, which have to be fixed with a costly 2.0 programming option that may or not be entertained by the masses.
Better communication helps to create a better product
Surveys will provide you with all of the feedback you could possibly need to hone the app’s capabilities and smarten its aptitude. Certainly there will be answers like, “I wish it were blue” that have no bearing on its functionality, but you have to take the good information with the unrelated comments. It is all part of the process. Once the survey’s feedback has been adapted to the app, it becomes a better version of its previous existence. You are still at minimal financial output and can test it again on another audience, gathering their feedback as well.
What Intelligence can you get from Surveys and Feedback Forms?
There are a number of things you can determine with surveys and feedback forms. At the core of the forms’ content should be finding out exactly what features the people are interacting with during the test. What do they like about the app? What do they love about it? All of these things should definitely end up in the final version. What do they dislike or find confusing? Remove it, or fine tune it.
Knowing what the customers enjoy, and what they do not, can help you reduce negative feedback when the app goes to market. Once customers air their grievances, others will read it, and some of it is not transferable back to the developer, so all hope is lost for a solution.
Beat reviewers to the punch by providing them with what they are asking for through the use of surveys and feedback forms. If they happen to list items that simply are not available at the time, make notes from their requests, and adopt them to version 2.0 of the app. It is the least you can do to appease your fans.
There are many reasons an app developer would want to use metrics. But at their core, metrics are about answering simple questions:
Are we doing a good job?
Do people love our app?
How do we get better?
One of the premiere metrics of customer satisfaction is the Net Promoter® Score, that is, How likely is it that you would recommend X to a friend or colleague? on an scale from zero to ten. By subtracting the “detractors” who answer zero to six, from the promoters who answer nine or ten (the “positive passives” answering seven or eight are discarded), you get an effective measure of customer delight. It seems like a simple question, but there are reams of research to support the idea that this simple question is goldmine.
If you are able to implement such a metric, fantastic. Even if your business is selling and supporting mobile apps — where you don’t have a steady stream of walk-in or call-in customers of whom you can ask how likely they are to recommend your app — we can gather this information with in-app surveys, or by sending out an e-mail blast to paid subscribers. But in the app marketplace, we have moved beyond passive likelihood to recommend: people are actively recommending (or criticizing) apps in a public forum. What metrics, then, can you use to make day-to-day decisions about your app and your customers? For the mobile app business, I would suggest an adjunct to Net Promoter® Score. A question that can be crucial to an app’s long term success, and a metric you might be able to measure with data you already have: do people love our app?
In our work delivering customer insights and targeted messaging on mobile apps, Apptentive has found customer love to be a key indicator of a few things:
Customers who love your app are more likely to rate or review it in the app store.
Customers who love your app tend to give it higher ratings in the app store.
Customers who love your app are more likely to use the word “love” when they write a review in the app store.
One of the ways we measure customer love is with in-app feedback. In the app itself, ask the user, “Do you love [insert app name here]?” and measure the yes and no responses over time. And there is another way, using data you already have available: download the reviews from your iTunes Connect, Google Play, or other app store account, and count the percentage of reviews that use the word “love” in them. You might be surprised how many people are willing to say, I love this app, and what a powerful message that sends to other potential customers.
At Apptentive we are strong believers in empowering developers to listen and respond to users. If you are using an in-app feedback model, customer love is a great metric to see how well you are doing over time. And if you have not yet started to engage your customers directly, it is still a great time to start measuring customer love. It is one way to answer that question, “Are we doing a good job?” And once you start to listen to people and respond, you’ll also be able to answer the other question, “How do we get better.”
We recently recorded a two part conversation with Ryan Morel of PlacePlay about how app developers can increase their app revenue through customer feedback and communications.
We covered a whole host of issues and tactics, here’s the first part of the interview, talking about the basics about what we’ve built and the benefits that developers are seeing:
In the second part of the interview we dug into more of the ways in which customers who you connect with become evangelists, how we eliminate shipping app updates to fiddle with settings and survey changes and why customer lifetime values climb when you have real relationships.
Here’s a full transcript of the conversation:
Ryan: Hi. Today, we’re here with Robi Ganguly from Apptentive. So thanks for joining us, Robi.
Robi: Happy to be here. Good to see you again, Ryan.
Ryan: Yep. So before we get started, why don’t you tell us a little bit about yourself and what Apptentive does.
Robi: Sure. So, a little bit about myself and my background. I spent a bunch of time in Silicon Valley working for some of the big tech companies, most recently, Yahoo, where I was responsible for display advertising. And I had a bunch of experience with marketers and advertisers, as they were thinking about how to use the Internet and digital channels to talk to their customers. And then, after that, I actually did some work for Del Monte and Nike, with similar strategies around talking with their customers. Which leads us to what we’re building with Apptentive, which is really a tool for anybody developing apps on iOS, Android, and Mac OS10, to talk to their customers in their apps. You know, set STKs for them to do that. We think it’s just really important to have a relationship with your customers in the digital age.
Ryan: So, if I understand this correctly, you guys do two relatively specific things. So, one, you help developers get better reviews, and then the second thing is to communicate directly with consumers for feedback, relationship, et cetera. Is that fair to say?
Robi: Yes, absolutely. The ratings and reviews part is relatively straightforward, in terms of we give them tools that make it easy to prompt in their app to identify if customers are happy and they’re enjoying it, and then to prompt those people who are happy and enjoying the app to actually go to the App Store and say something wonderful about them. The flip side of that is making sure that the customers are having a great experience. So if they’re not really 100% in love with your app yet, that they get a chance to talk to that developer, and so the net result of that is your ratings and reviews are wonderful and you have a channeled action talk with people who, instead, would have gone to the App Store and ranted.
And then that second piece of really getting feedback is… A lot of apps have a “contact us” or support button, we power that and our SDKs present a native ballot box where you can solicit that feedback from an end customer and then you can also, when you get feedback from them, see the context of the device that they’re on–the carrier, OS, Virgin–and any other data that you choose to attach. So as a result, when you get feedback from a customer, it’s really rich because you understand their situation and the environment.
And finally, we also allow you to really structure some of that feedback through the use of surveys in your app, so that makes it very easy for the end customer to give feedback. It’s basically tapable responses in a survey ballot box in the app, so that they’re invested in the end customer’s investment and telling you more about what’s going on in their experience is 10-15 seconds, which then leads to a higher participation rate.
Ryan: App Store review rants have long been a problem for app developers, and specifically their inability to go back to that customer and help them solve that need. So it sounds like your solution helps prevent those App Store review rants and provides consumers and developers a channel of communication that they didn’t otherwise have before.
Robi: Yeah. That’s right. I think one of the things that we heard early on, when we started building Apptentive, was that it was super frustrating to go through the App Store and see somebody complain and then see an anonymous username. We actually heard from many developers that they’d Google those usernames and see if they could figure out a Twitter account or Facebook account in order to get in contact with them, which just really tells you how frustrated they are about not being able to go address those concerns.
I think we know, pretty obviously, at this point that on the Internet, anonymity leads to a lot more extremism in terms of comments–a lot more yelling, ranting and raving. And so, what we’ve done, by creating this rating and review process that helps qualify how somebody feels, is that we end up intercepting the rants that people who were going to go to the App Store and really shout, instead come directly to the developer. And as a result of being able to come to the developer and the developer being able to respond, the dialogue goes from this extremist yelling and screaming to an actual constructive dialogue. And we hear a lot of developers telling us about situations where the initial contact from a customer was really vitriolic–somebody was really angry–and then as soon as the developer responded back and said, “I’m sorry, are you having trouble?” The person toned it down and they got to a place where they were able to actually discuss something cordially and worked constructively to make the app better, together.
Ryan: Obviously, one of the big benefits you guys are providing is preventing developers from stalking consumers, so that’s a good thing. And so, the other thing I heard you say was that you provide a way for consumers to feel engaged in the process of the app development. So the app developer becomes kind of like a partner instead of a supplier. So maybe, you could talk a little bit about how you’ve seen those close relationships affect consumers’ engagement within an app and with the company?
Robi: Yeah. I think that it’s a really simple thing. It’s very, very easy to understand if you think about yourself as a consumer in daily life. When you come across a situation that doesn’t make sense to you or you’re frustrated, if you have the ability to raise your hand and talk about what’s going on and you feel hurt. Just by feeling hurt, I think, as a consumer, it changes the tenor in the relationship with whoever you’re dealing with. Whether that’s me going to Starbucks and getting the wrong drink and then telling them that I got the wrong thing and that he fixes it for me without a hassle, or it’s in the app, knowing that my opinion matters, right?
And then I’m actually giving you insight into what it’s like to use my app because it’s very hard, as an app developer, as anybody creating a piece of software, truly stepping into the shoes of the end consumer. And so this partnership is great for the developer as well, because they can really get advice and an understanding about what they thought it was going to do and then how it’s actually being used. So, just like you said, there’s this partnership in terms of making the app better, together.
Ryan: And I would assume, as a result of that, that some of those detractors end up becoming promoters?
Robi: That’s actually 100% correct. That’s the coolest thing. Developers, anecdotally, will come to us and say, “So and so came in. There were very upset. We went back and forth. I pushed out an update to the app. I told that person explicitly, that had complained, I pushed out an update and I listened to you.” And then shortly thereafter, that consumer becomes a person who is an evangelist–someone who’s on Twitter and Facebook sharing it with their friends. But one of the coolest things about this is actually hearing the stories where an end customer, now, because they have this relationship with the developer, will start using the developer’s name.
Like, “I was talking with Robi about their app. They responded. You should use this app, it’s amazing.” And so, it’s a way to really cut through that wall of the piece of software between you, to develop that relationship that then extends into the real world.
Ryan: Can you summarize what the really big benefits, for app developers are, of using Apptentive?
Robi: Sure. So there are a few things that people really are enjoying today, as a result. So the first thing is, the big problem with App Store is that Microsoft, Google, Apple–they own the customer relationship—and so as a result, by default, you as a developer don’t know who those customers are. Now, using us soliciting feedback, you develop relationships. You understand the email address. You understand the usage patterns. You have a way to dialogue with these people on a regular basis.
So that’s number one–direct customer relationships. Number two is better ratings and reviews. So, by and large, when people use our rating systems, they see increases in volumes and more ratings and reviews–50 to 100% more. And then they see increases in those actual ratings–another star or more. In fact, we see a lot of app developers, once they use us, get their ratings pegged pretty close to five stars, and the reviews take on a totally different tone. So people use words like “love” when they’re describing your app and that’s amazing to have in the App Store. And then the final thing is, because you have this dialogue and you’re engaging with your customers and you’re learning and you’re making the app better, your retention rate goes up.
Ryan: Do you guys do any analytics to identify what the best time is to promote, like a review or a rating feedback? Are you guys doing that right now?
Robi: Yes. So, what we have is, in our ratings prompt, everything’s instrumented so we can understand when it’s shown and then what the result is. So, we asked this question, “Do you love this app” whatever the app name is and we have all the data around when it’s shown, who clicks “yes” and who clicks “no” at each point in time.
We deliver that in our dashboard to you, so that you as an app developer can understand what’s happening with your existing settings but, importantly, the settings are all server-driven so, without having to ship an update to your app, you can act on that insight into what’s happening and change the settings of when you’re prompting and the end customer can answer that question. And, with our surveys, we also have it instrumented, and with our feedback it’s also instrumented. So, our goal is to make you, as an app developer, much smarter about interactions and the results of when you’re trying to interact with somebody because the worst thing you can do is be clumsy about it and end up hurting your app. So, that’s why we track all that data and present it to you.
Ryan: We all know that any app developer can hard code a question to ask their user, to review their app. That doesn’t cost very much. So, why would an app developer, essentially, pay you guys to handle that for them?
Robi: What we see when developers hard code that stuff in is that they don’t understand what’s actually happening. They haven’t instrumented it, so they don’t know, for example, how many times a customer has actually seen that prompt. You don’t know how many people are then clicking and going to that app store and reading it. The lack of data around that, plus the fact that you have to ship an update to your app, results in a really long cycle time around trying to figure out the appropriate time to ask somebody this question and making sure that only people who actually love your app are going to the app store and giving you a good review, as opposed to coming to you.
What we do, is we instrument that process. We ask the qualifying question, ”Do you love this app” which ends up resulting in most people who are unhappy with your app coming directly to you. That’s this notion of intercepting negative feedback that really is occurring across all of our developers. Also, importantly, because we’ve instrumented this and we’re presenting it in this dashboard and we tell you and people are saying, “Yes. I love this app versus anything else,” and because it’s connected to the server for settings, that developer can, today, launch an app with settings around what they’re prompting and then, tomorrow, change those settings without having to ship an update to the app.
So, that cycle of really getting very specific to your customer base about when you’re prompting them and trying to get as many happy customers as possible, goes from taking 6 to 12 months with not very much data to three to four weeks with tons of accurate data about all the activity around this. So, that benefit is a real reason why people are using this pegged because they have an insight into every step of the process and they don’t have to ship an update into the app store.
Ryan: Outside of the fact that they’re getting all this data and reporting, which is fantastic, it sounds like it’s like they’re paying you just because they don’t have to do updates.
Robi: Yes. I think one of the biggest smiles I get when I talk to people about our service is that we’re pretty focused on this idea that any time we can remove shipping updates to your app, in the process, we will, because people, in particular in the Apple ecosystem, are terrified of having to wait two weeks for the review process and that’s slowing down the development. So, we move that ahead.
Ryan: Can you provide any specific examples about developers who are using you who have made improvements to their app, based on the feedback they’ve gotten?
Robi: Absolutely. Here in town, there’s a great team of developers working on an iOS app called Chewsy, and Chewsy really helps you understand dishes at specific restaurants. Using their app, you can rate and review specific dishes and take pictures. So, it’s a pretty wonderful way to explore a menu through other people’s eyes and get really good data about what you should be ordering. One of the problems they run into is that, in markets where they don’t have a ton of coverage, so, for example, let’s say, Kansas City. If they don’t have a lot of users, they say that’s a great experience with any customer isn’t as good as it is here in Seattle. They’ve been using us to collect feedback and get insight from their end customers about what that experience is like and then how they would expect that experience to look, when there is volume and that’s allowed them to iterate on their product and really deliver something that is making more of their customers around the world happy. And it’s something they couldn’t have done just through metrics and analysis. They have to actually go talk to the customers in these different markets.
Ryan: And we talked about communicating with consumers is a really powerful tool for app developers. Do you have any examples of developers using you who have seen their revenue, ultimately, increase after integrating your SDK?
Robi: Absolutely. The easiest way to think about us is the idea that your app store page is really your storefront. As a result, consumers, when they’re shopping, think about downloading an app or buying an app. They’re looking at your storefront before they make that decision. The thing that they look at, most often, are the ratings and reviews. That’s what’s dynamic. Your description is important and screenshots are important. They certainly help, but thanks to Amazon and many others over the past decade, we’ve really been trained to look at what other consumers are saying about something. It helps us sort through all the noise, and as a result of having much better ratings in terms of higher volume ratings, as well as higher star ratings, people are more likely to download your app.
So, we hear from developers, of both free and paid apps, that they’re seeing more downloads, that the conversion rate of people who are seeing their apps page is just going up, that more people who see that app store page are downloading. Then, as a result, there’s a boost in revenue because you have more people using your app and they’re happier, as well, because you’re making a better app.
I think, over the long run, what we’re also seeing is that when you have relationships with customers, particularly if you’re in a free-to-play game, or something like that, you have that relationship and people trust you and they develop a relationship with you, they’re more likely to stay in your app. So, retention is really big. That difference between having a customer for a month versus 6 or 12 months, is massive from a revenue perspective, and we’re hearing from companies on a regular basis that’s true.
Ryan: So, it’s not only increasing initial downloads, but it’s increasing the retention levels and, ultimately, the lifetime value of the consumer.
Robi: Yes. Absolutely. And I think a really big point here, that a lot of folks don’t necessarily understand right off the bat is that trust is key to having lots of commercial transactions, whether or not you’re buying a car from somebody or you’re buying a latte. If you trust the business you’re dealing with, you’re more comfortable spending money with them and that’s happening in the app space incredibly fast.
Ryan: OK. Is there anything else you want to talk about before we go?
Robi: Just, the main thing is it’s really an exciting time to be an app developer. We have more customers signing on every day, and we get to talk to them about their businesses and are pretty fortunate to see across the landscape and businesses that never existed before are now springing to life because of this personal device that everybody has in their pocket. It’s pretty wonderful.
Ryan: Great. So, before we sign off, tell everybody how they can find you and learn more about Apptentive.
On a regular basis, we try to share some of the best materials shared about building your app business and communicating with your customers. Here are some of the best we’ve come across recently.
How to build your app business
Everyme’s feedback button
Capturing Feedback in your iPhone App – an awesome story from the makers of Everyme about how incorporating feedback into their app has made them more successful. From better ratings and reviews to more engagement around how to use their app, Everyme’s story really underscores the benefits to communicating with your app’s customers.
How to build an App Empire – The story of Chad Mureta’s recovery from a horrible car accident to become an “appreneur” is inspiring on its own. His tips and tricks for creating a successful niche app business is well worth the read. Be warned, however, this is definitely more self-promotional than most of the things we link to and we don’t agree with all of his tactics.
This podcast, Where has the sense of mission gone with customer service?, with Frank Eliason of Comcast is a valuable discussion about customer service in the new consumer age. In particular, they ask a really interesting question: “What would happen if you built scalable intimacy with your customers and your team?” The conversation with Frank starts about 23 minutes in and you can find the iTunes file here.