Before you set out to work on creating your own mobile research instruments, we wanted to provide a few best practices that we’ve discovered with the help of our customers. These are proven tips for creating effective mobile surveys optimized for response rates, customer experience, and the collection of actionable insights.
Best Practices: Survey Design
When designing your mobile research instrument, do:
Design with mobile in mind
Keep questions brief and concise
Allow customers to opt out at any time
Aim to address your research objective with as few questions as possible
Limit the number of options for multiple choices
Break the questions up so that only one or two appear at a time
Provide an ‘Other’ field with a textbox for fill-in answers to your multiple choice questions if you suspect that some respondents may have answers you had not previously considered
Add an option for ‘Don’t Know’ or ‘Not Applicable’ for questions that some respondents may not be able to answer
Pre-test your survey internally to identify any weaknesses and ambiguity
Create overlap in multiple choice responses. All responses should be mutually exclusive
Present rating scales with large matrices of options or questions ones that require scrolling on a mobile screen
Create vague responses that are open to the interpretation of the respondent (i.e.: If asking about use frequency, give tangible options like ‘twice a week’ and ‘once a month’ rather than ‘often’ or ‘rarely’
Frame questions in a way that leads the respondent or creates bias (i.e.: “Why do you like this app?”)
Request personal information at the start of the survey as this may lead to lower response rates. If you need this information, make the questions optional and move them to the end of the survey
Best Practices: Survey Integration
For best results integrating your research instrument within your app, we recommend you:
Use an in-app survey rather than directing mobile customers to a web survey so as to not detract from the customer experience. If you are using a web survey, be upfront about asking customers to leave the app for an external link.
Use event-based targeting that isn’t intrusive. Don’t immediately ask new users to take a survey, and only ask customers to take your survey once rather than asking each time they load an event.
Integrate the survey with your existing customer analytics to allow you to target the responses against your audience segmentation to uncover trends based on loyalty, device, etc., without having to ask customers to fill out additional questions.
We hope this guide helped, and we wish you all the best in your mobile research endeavors! Of course, this list contains just a few of our favorite tips. Please help us grow this list by letting us know in the comments below what tips worked best for you and what you’d add to the list.
Want more best practices and help thinking about your in-app surveys? Sign up for an enterprise plan for unlimited access to our dedicated customer success and support teams. We’ll work with you to create a plan specifically for your mobile app and research needs.
Revisit the highlights of this three-part guide on Slideshare:
I recently had the opportunity to represent Apptentive at the International Consumer Electronics Show (CES) in Las Vegas and wanted to share a few observations from the event that left a lasting impression and an indication of the expanding role of customer communication in an increasingly connected world.
First impressions? CES this year was HUGE.
CES 2015 hosted over 170,000 industry professionals from all reaches of the globe and celebrated innovations in every area of consumer electronics. The four-day conference took up over 2.2 million square feet of exhibit space at the beautiful Cosmopolitan Hotel on the Vegas Strip and had over 3,600 exhibitors – including a record number of startups.
Apptentive’s Red Russak and Robi Ganguly presenting on customer engagement at APPNATION, this year recognized as an official partner event of CES
The theme of this year’s conference was “The Internet of Things,” (IoT), and I’d say that’s a pretty good way to categorize the eclectic collection of gadgets we saw – from smart pacifiers, to high-tech roller skates, to Edwin the Duck.
CES boasted a “who’s who” of tech innovators. From big brands like Tesla showing off an entire selection of self-driving cars to overseas startups selling designer selfie sticks, every minute was full of humbling sights and memorable experiences.
Apptentive’s Red Russak sharing ideas (and food!) at APPNATION’s Startup Garage
On Customer Communication, Our Own Awesome Customers, and the Best Party on the Strip
Amid all the chaos of dodging connected cars and drones galore, there were three prevailing observations that made a noteworthy impression.
First, an overarching appreciation for the customer experience is here to stay. Brands didn’t come to showcase the latest and greatest innovations. They came to deliver a spectacle, an experience.
Throughout our many conversations, one theme reigned supreme: Companies are looking for customers to love the brand, just as much as they love the product. They understand that it’s more important than ever to talk to their customers and understand their needs and pain points rather than innovating for innovation’s sake.
CES proved that anything can be a connected device, from toothbrushes to rubber ducks. The big question, now, is how brands can break through the noise to effectively communicate with their customers in any sort of consistent manner – let alone provide an omni-channel experience. Customer relationships are getting harder to maintain as the proliferation of new technology continues to clutter the customer experience landscape. It’s paramount that brands think about how customer insights can be collected and shared across devices in order to build a product that customers not only want, but love.
The 3,600 exhibitors in attendance proved that product is no longer a sufficient differentiator. In order to really stand out, you need to build a strong customer brand perception into the core of your development, messaging, marketing, and ultimately, engagement. In 2015, the importance of maintaining a focus on the customer cannot be underestimated.
Ravi Ramkumar (left) with friends of Apptentive at our Customer Appreciation Dinner
Second, CES reminded me of why I’m in the business of customer communications. Talk about customer love and reciprocal loyalty – our incredible customers made a point to carve out time from their hectic conference schedules to visit our suite, share their feedback, rep our product, and proudly don our t-shirts. Our customers and key partners alike gave us an amazing reception at our customer appreciation dinner at Jaleo Las Vegas – over the best tapas in Vegas, no less!
So to all of our customers I had the privilege of meeting at CES and APPNATION, and to all of our customers I hope to meet at future events and Appy Hours, thank you.
Thanks for a great party Microsoft – Can’t wait to see how you guys top it next year!
And third, I want to give a shoutout to our friends at Microsoft for hosting the best party in Vegas, hosted by Tiesto himself! Thank you for topping off the CES experience with c-suite executives raging in backpacks and startup tees.
Overall, I had an incredible experience at CES and APPNATION. My only regret is not having enough time to play with the gadgets and meet the incredible entrepreneurs behind them.
In working with thousands of mobile developers on collecting mobile customer insights, we’ve learned a thing or two about what works – and what doesn’t work – when it comes to designing an effective in-app survey. We believe mobile research is an incredibly powerful tool for improving communication with your customers, and have made it our mission since day one to help brands create Customer Love.
We’ve therefore highlighted the steps of one of our favorite survey design roadmaps to help you think about what’s involved in creating surveys that will both deliver the data you need and enhance the overall customer experience.
1. Pick Your Survey Tool
We have an obvious bias here, but the very first thing you need to consider is your choice of survey tool. This includes both the decision of whether to use an in-app survey (like Apptentive) or a web survey (like SurveyMonkey) and the decision of whether to make your survey platform in-house or seek out a third-party solution.
We’ve done our best to highlight the pros and cons of each of these options below:
Using an In-App Survey Tool
Allows for a frictionless customer experience as customers never have to leave the app
Dramatically higher response rates when compared to web surveys
Easy to integrate into points of engagement within the app (‘mobile moments’)
May require additional engineering time to integrate into your app
Using a Web Survey Tool
Several very established survey and analytic tools available to choose from
Lower response rates
Requires your customers to leave the app to take the survey
Branding and the customer experience are often less cohesive between the app and the externally hosted survey
Using an In-House Solution
Very resource-intensive in terms of developer time and money
Necessary testing of, and updates to, the survey tool may take time away from your product roadmap
Using a Third-Party Solution
Turnkey solutions available with reporting tools and dashboards
Access to existing documentation on best practices for customer success
Requires some developer time, depending on ease of SDK integration
2. Determine Your Research Question
Before jumping into designing your survey on your chosen tool, take some time to identify a single concrete research question that you hope to answer with the results of your survey. This will provide a benchmark for your data analysis and can help to keep the survey short and concise if you constrain yourself to asking only those questions necessary for addressing your research question. (We’ll provide more on why shorter is better when it comes to in-app surveys in Part 3 of this guide.)
Research questions are generally classified as either attitudinal, behavioral, demographic, or technical. An example of a research question for each of these categories can be found below:
Attitudinal: How do new users like your app?
Behavioral: How do users interact with your app? What are the most common use cases?
Demographic: Which age bracket is your app most popular with?
Technical: How can this app be improved?
Once you have carefully selected your research question, several of the remaining steps will come naturally – including your target audience and the type of data you need to collect.
3. Identify Your Survey Audience
It’s now time to determine which customers are best suited to honestly and accurately address your research question. Once you’ve identified the customer segment, you can begin to seek out pain points within the app where the survey will provide the most value to both the customer and your team.
Common mobile research samples include:
A Sample Generated With Event-based Targeting
Delivering your in-app survey to customers who engage in pre-specified ‘mobile moments,’ i.e.:
The third time a customer uses the Search function
The first time a customer shares content via the app
After a customer updates to a new version
After a customer uses a new/beta feature
Can be used to address narrow, feature-specific research questions
Can be used to refine and beta test new versions and rollouts
Respondents inherently have familiarity with the app event they’re providing feedback on
Resource-intensive, unless using third-party mobile engagement solutions since activity is a necessary condition of the sampling selection
A Random Sample Randomly choose a percentage of your mobile customers to survey.
Diversity of responses makes the data highly representative of your overall audience
Surveys can be prompted even if mobile analytic capabilities do not collect information on customers and customer activity
Sample may be too broad to address narrow research questions that require a high level of familiarity with the app
Responses may be too generalized to uncover trends based on customers’ familiarity with the app, device used to access the app, etc.
A Sample of New Users Customers prompted during their first time using the app.
Allows you to collect information, unbiased by existing loyalty, that can be used to improve the customer experience and retention
Requesting customer information from first-time users may create a negative initial experience with the app, and has been shown to be a leading factor in app exits
A Sample of Loyal Users Prompted the n-th time a customer opens the app or a few months after installing and regularly using the app.
Can be used to uncover which features loyal users find the most valuable / what is bringing them back to your app
Can be used to gather suggestions from those already familiar with those apps – and customers can be further delighted if those suggestions are acted on
Responses may have an upward bias and have difficulty capturing equally important negative experiences with the app
An Opt-In Sample Instead of identifying sampling criteria, let your customers proactively navigate your menu to find ways to voice their feedback.
Survey can be easily integrated as a link in your app’s navigation with no third
party tools or prompts
Can lead to selection bias as those who opt-in may not share fundamentally different views/attitudes than those who do not opt-in
Opt-ins have the lowest response rate and the survey link can be difficult for potential respondents to find, making it hard to create open dialog with customers
4. Design Around the Research Question
When it comes to writing the actual questions for your in-app survey, start by asking yourself what sort of data best addresses your research question. Some research questions (particularly attitudinal) lend themselves to quantitative questions and rating scales. Others (particularly technical) are best addressed with qualitative, open-ended questions.
Survey questions can be classified as open-ended, closed-ended, or mixed. We’ve done our best below to address the use cases of each classification and their ability to address your anticipated needs:
Open-ended (i.e.: Providing a textbox for the respondent to type his or her answer)
Exploratory in nature
Less likely to result in bias from leading questions/response choices
Provides qualitative responses similar to a focus group
Can be time-consuming to answer, particularly when using a mobile device, leading to lower response rates
Closed-ended (i.e.: Multiple choice questions and rating scales with pre-defined response choices)
Can provide both qualitative and quantitative responses
Questions typically take less time to answer and experience higher response rates
Questions and response construction requires more care remove bias
Mixed (i.e.: A multiple choice question with a fill-in ‘Other’ option)
Allows room to write in answers that were not considered when the survey was designed – may uncover new customer needs/sentiments
For more on quantitative vs. qualitative questions, and their use cases, check out this post by our friends at SurveyMonkey.
5. Integrate Into the Mobile Experience
Now that you’ve determined your audience, the next step is to identify where and how to reach them within your app. If you’re using event-based targeting, seek out mobile moments in the customer journey – points of engagement where customers may feel happy, frustrated, or lost. Concentrating on these moments allows you to hone into a single part of the customer experience and can lead to more actionable results if the feedback directly relates to the point of engagement.
When prompting your in-app survey, be careful to not interrupt or annoy customers in the middle of a task. For example, if you want to ask about the in-app shopping experience, prompt your survey after checkout instead of after a customer adds an item to his or her cart. Remember, your survey should be designed to add value to both you and your customer.
If you’re targeting new customers, give them time to try out the app before prompting your survey. A ratings or feedback request upon first log-in is a sure way to create a negative first impression.
And finally, if you’re using an opt-in sample without any sort of prompt, insert a link to your survey into a logical place in your app’s navigation menu – such as the Help or Contact section.
6. Pre-Test Your In-App Survey
Before making your survey live, it’s important to pilot the survey internally or with a small sample of customers. This allows you to ensure that your survey is working as intended and identify any weaknesses or ambiguity.
While testing your survey, check that the following items are in order:
Questions proceed in the intended order, and responses are mapped to the appropriate next question or prompt if conditional branching and skip logic are in place
Questions are easily understood by those unfamiliar with the survey
Response options are clear and do not contain unintentional overlap
Optional and required fields are appropriately coded
Responses are accurately collected upon completion
After you’ve tested your survey and are satisfied with the results, we’d recommend resetting the response count so that the test data does not interfere with your actual results. You’re ready to push the survey out to your intended audience.
7. Analyze the Results
By this point, you’ve launched your survey and have collected a statistically significant number of responses. Now it’s time to dig into the results of your labor.
Data analysis should be a straightforward process for those quantitative questions directly mapped to your research question. We recommend going through each question individually and asking yourself whether each result confirms or rejects your hypothesized answer to your own research question: Are the result of customer rating scales consistent with what you’re seeing in app store ratings? Has your latest version update improved customer sentiment? Do customers feel comfortable navigating your interface and new features?
In the case of qualitative questions, you may need to do a little more work to get meaningful and generalizable data. For open response questions, we recommend categorizing the responses into themes – such as feature requests, usage or interface questions, customer testimonials, and customer complaints. You can then search through your results for the most common trends and begin to incorporate these insights into your product roadmap and QA efforts.
As a last step, decide whether you want to keep the survey running or close it. This is often a matter of saturation (if you already have the sample size you need and additional responses provide little value) and relevancy (is your research question still a priority, or is there a new question worth asking in a separate survey?).
We hope these seven steps will help kickstart your in-app survey design. Let us know if there’s anything we can do to help you create your next survey and keep us updated on how customer insights shape your app.
Continue on to Part 3 of our guide on in-app surveys and mobile research, or preview our section on best practices on Slideshare:
Through conversations with thousands of mobile developers, we’ve found that evaluation of the customer experience publishers hope to build into their apps often comes down to simple guesswork without any tested and true ways to gauge satisfaction. You have a steady stream of new installs and fantastic ratings in the app stores. And yet, your mobile customers come and go, with only a small portion continuing to use your app after the first month or even past those crucial first 30 seconds.
Fortunately, it no longer has to be guesswork. Many publishers are taking the first steps when it comes to better evaluating their in-app customer experience. Every day, we work with new CX-centric developers to integrate in-app surveys and customer communication tools into their apps. These tools, seen commonly all over the internet but rarely inside mobile apps, empower businesses to survey their customers. They unlock a wealth of insights in a much greater volume, and of a far more representative nature, than what the limited information in app store ratings seems to convey.
One such customer – Urbanspoon – used the mobile feedback prompt seen to the right to proactively ask customers for feedback with an in-app prompt. After implementing this feedback forum, Urbanspoon received over 7,000 pieces of feedback. These insights uncovered critical components that helped Urbanspoon in its strive to constantly create a better app for its customers with each new update.
Using in-app surveys to open up a channel for customer communication and gather mobile feedback has allowed Urbanspoon and countless others to save money, prioritize features, learn more about their customer base, receive structured feedback, and engage customers.
Mobile apps aren’t cheap. Every new update requires time and money, and lots of it. Mobile app developers have an inherent vision of what they want their app to be, but don’t always consider how the needs of their customers may differ from this vision. As a result, publishers are wasting valuable resources on new features and rollouts that don’t necessarily provide customer value.
However, if you can directly ask your customers what they’d like in an app using in-app surveys, you can be sure that your development time and money is being put to good use and that each new feature will enhance the customer experience. Or better yet, create a minimal viable product and test it in a mobile focus group environment to see if you’re on the right path.
As a developer, you have dozens of features on your roadmap that you’re excited to one day get to. But don’t fall into the trap of trying to do too much, too quickly. Collecting mobile feedback and giving your customers with a voice allows you to determine which of your proposed features are most important to them in the context of your app.
Once you’re armed with this knowledge, you can begin to prioritize your roadmap based on what will add the most customer delight. You’ll also be able to determine which of your current features needs a little touch-up to meet or exceed customer expectations.
Better Understand Your Customers
If we’re adamant about one thing, it’s our philosophy of Customer Love — seeing mobile customers as people, not users. Each of your customers has their own likes and dislikes, their own use cases for the app, and that one thing that will really make them love your app.
Using in-app surveys, you can begin to collect valuable demographic and psychographic information to segment your customer base. This research allows your in-app communications to speak directly to a segment’s unique interests. You can also use it segmentation to target your out-of-app marketing to potential customer groups that match one of your current customer personas and likely have common needs and interests that will draw them to your app.
Make Product-Specific Inquiries
You’ve rolled out a new update, only to find that retention is falling. Or you see customers abandoning their purchases half-way through checkout. Do you know what’s causing this behavior?
Once again, collecting mobile feedback can provide you an answer. By proactively surveying customers at key moments throughout their experience with your app (e.g., after they’ve used a new feature for the first time, or after they close out of the shopping cart platform), you can evaluate their experience with, and attitude toward, your product on-the-spot. This allows you to identify and fix any customer pain points early on and intercept negative app store reviews.
Engage Your Audience
The reviews you see on the app store ratings page represent only a small minority of your customer base. This minority tends to be skewed toward either extreme when it comes to their evaluation of your app. These are people who either love or hate your app, with a lesser proportion of those in-between making the leap to the ratings page. You’re getting only the feedback of the vocal few and not feedback particularly representative of your greater audience.
Providing your customers with the tools needed to share their thoughts – without requiring them to leave the app to take a web survey or visit the app store – allows you to unlock a much more precise perspective of customer opinion. Mobile surveys (particularly when done right) let you to quickly poll a much larger portion of your customers and have response rates dramatically higher than their web counterparts and competing market research instruments.
For more on opening up customer communication, check out MobileDevHQ’s Complete Guide to App Marketing, in collaboration with Apptentive’s Ezra Siegel and a panel of industry experts.
Not anymore. An increased number of smartphone holders worldwide and an increased demand for apps have created an extremely competitive marketplace where customers almost always have several strong options when it comes to finding that perfect app.
As customers increasingly shop around in the app stores and evaluate their options, mobile publishers are hit with dwindling customer retention rates. According to our own research, only 40 percent of customers continue to use an app they downloaded a month ago – and this number plummets to 4 percent over the course of the first year in the customer journey.
The cost of acquiring a loyal mobile customer (defined here as someone who opens your mobile app at least 3 times) is therefore at an all-time high. According to Fiksu’s Cost Per Loyal User (CPLU) Index, the average CPLU increased 34 percent from September 2013 to September 2014.
Between August 2014 and September 2014 alone, the base CPLU rose from $1.86 to $2.25. This 21 percent spike corresponds directly with the unveiling of the Apple iPhone 6 and iOS8. Both of these events led to an increase in demand for mobile devices and apps and a subsequent increase in competition among mobile app publishers vying for attention in an increasingly crowded market place.
As acquisition costs continue to rise, mobile customer retention becomes more important than ever. If done right, mobile retention campaigns can lessen the burden of the much-costlier new customer acquisitions while still allowing you to hit those monthly revenue and active customer goals.
We’ve put together a few, low-cost strategies to boost customer retention and loyalty – in turn, fueling sales, higher ratings, and organic growth through word-of-mouth and customer delight.
Give Your Customers a Voice
Before you can hope to delight, or even retain, your mobile customers, you must open up a two-way conversation where their voices can be heard. App developers constantly strive to create that perfect customer experience, but many lack the tools to evaluate whether or not their assumptions of customer opinion are really on mark.
Sure, there are the ratings on the app stores that can be used to guide future updates, but these are indicative of only a small fraction of your mobile customers – namely, the vocal minorities who either love or hate your app.
But what about the less vocal majority, those with opinions somewhere between those two extremes? This is a group that likely won’t make the journey to the app store ratings page unless they’re truly delighted or peeved. They constitute the bulk of your audience, yet their voices remain largely unheard across the much of today’s app market place.
To fill this void, developers are bringing customer service to the mobile channel – incorporating two-way chat capabilities between customers and publishers and prompting in-app surveys to gauge opinion throughout every step of the customer journey. In addition to allowing them to preemptively address customer issues before they surface publicly as a negative rating, these interactions provide developers with a wealth of data that they can use to shape future updates and delight their audiences.
Use smart, well-timed push messaging and in-app surveys
Opening up customer communication is just the first step in improving retention. Next, it’s time to actively ask for and respond to that communication.
We recommend our customers look for ways to reach out and solicit feedback during key events throughout the customer journey – such as the third time a customer logs in, or the first time someone shares content via the app. Once these events have been triggered, don’t wait for the customer to search through your menu navigation to tell you what they think. Instead, prompt a brief, non-disruptive conversation to gauge customer opinion at the broadest sense before diving into more specific sentiments in the event that the customer wishes to continue the conversation. (For more tips on asking for customer feedback, check out our 5 Tips for Writing Effective Mobile Surveys.)
Let your customers know they’re valued
The surest way to delight customers is to let them know that they are valued – and that means giving thanks and showing appreciation. If customers commonly make a similar request, consider shifting your development schedule to prioritize that need. Likewise, on releasing your next update, give a shoutout to your customers for contributing to a new feature release and remind them that you’re building a customer-first experience where their voices are not only listened to, but highly valued.
Regularly refresh your app with rich content updates
It’s no mystery that creating valuable is key to delighting mobile customers. If customers see new and interesting content each time they open the app, they are likelier to spend more time in your app consuming that content and return to your app frequently to see what’s new.
To really wow your mobile customers, align your content strategy with your customer data provides additional opportunities. For example, use your in-app prompts to suggest relevant content based on your customer communications or reward your returning customers by offering them exclusive discounts or deals out of appreciation for their loyalty.
Understand your app’s customer retention rates
And finally, it’s important to know your numbers – how to calculate retention rates and the revenue generated by each loyal customer. Of course, each app will have different retention and average revenue per user goals and benchmarks, but being able to calculate these metrics will allow you to internally compare both your marketing effectiveness over time and the ROIs of your retention and acquisition strategies to determine what makes the most sense for your business.
We’ve released a few guides to dive further into this concept and help demonstrate the importance of mobile customer retention:
Whether you’re just starting out or you have numerous apps in the market, it’s important to be aware of how much quality can differentiate your product in the marketplace. In the increasingly competitive app ecosystem, quality really is key, as only 16% of people say they will try a failing app more than twice. Not to mention the fact there are over a million other apps in the various markets out there for your customers to choose from.
It’s a competitive space, but it’s certainly possible to give yourself an edge. There are three key areas where quality control can be engineered into your app: testing, crash reporting, and feedback management. In this article you’ll get a high-level overview of each category, some examples of tools and services available, and why that specific area is important in building a quality experience for your customers.
Testing and Test Automation
If you’re using Test Driven Development, Behavior Driven Development, or any number of other strategies, you’re likely familiar with unit testing. This is a way of engineering code with testing in mind. For Android, JUnit is the weapon of choice, which goes very well with the free tool, Robolectric, an open-source project for running Android unit tests in a headless environment. This is great because it’s very fast (way faster than the notoriously slow Android emulator). For iOS, OCUnit is a popular choice, and, because the simulator uses hardware acceleration, it’s fast enough out of the box to run without the aid of a Robolectric-like harness.
Functional tests, tests that exercise basic functionality and flows within an app, are prime candidates for automation. Given the thousands of device models on the market, manually running such tests is a daunting task, especially because these types of tests are most valuable when run on real, physical hardware. Tools like Robotium, a library that integrates with JUnit, and Calabash, a cross-platform framework, help make writing automated, scalable test scripts easy that can be run on emulators, simulators, and real hardware.
A popular strategy is to run tests locally on emulators and simulators, then run them on local devices with tools like Spoon, and then run those same tests on hundreds of devices hosted by services like AppThwack. Rather than spend time building automation harnesses, device labs, and other necessary components of an end-to-end test environment, services like AppThwack remove the burden by providing cloud-based device labs for rapid, parallel test automation execution.
Exploratory and UX testing is very important as well, and this is where humans excel. A good strategy is to automate your mundane tests so your time can be spent exploring new functionality, conducting labs where you can observe real people as they interact with your app, and so on. Having a human touch is paramount in measuring how an app actually feels from a customer perspective.
Keeping Track of Errors
Let’s face it. No matter how well you code and how much you test, we all write bad code, meaning some errors are bound to make it through. In fact, with the speed of today’s development only getting faster and the increasing adoption of agile development practices, it’s necessary now more than ever to engineer in anticipation of unexpected errors.
There are numerous tools for tracking crash reports in the wild for both Android and iOS. For Android, ACRA is a popular free option. There are commercial services available that support multiple platforms and add bells and whistles to crash reporting, such as pattern identification, tracking issues over time, and analyzing performance metrics, but at their core the biggest value of any crash reporting library is to provide insight into how your app is behaving once it’s in the market.
Once you know how and where your app is crashing, you can more quickly fix the reported issues, avoiding uninstalls and bad reviews. While it’s best to capture bugs before customers encounter them, at least you won’t be in the dark until someone decides to tell you about a problem. For every one complaint you receive there are 26 others you don’t, which should emphasize just how important this step is.
What’s more accurate, your own impression of how your app performs or your customers’? The answer is obvious, but time and time again we, as developers, assume we know best. Your customers’ feedback should guide your decisions, whether that means acknowledging their comments and consciously dismissing them, or restructuring your app so it better fits their needs.
The obvious question, then, is how do you interact with your customers so you get valuable information while not seeming needy, overbearing, or paranoid? It’s important to form dialogs with your customers and encourage them to communicate with you, rather than create a confrontational relationship that leaves you both unhappy.
Services such as Apptentive do the work for you, inserting feedback prompts into your app in a way that gathers pertinent information and, ideally, avoids the most damaging form of feedback: poor reviews on the market.
By keeping these three categories in mind, building app quality into the app lifecycle should be a clear strategy going forward. Coding with testing in mind (unit tests) and automating tests that make sense (functional and performance tests), tracking app performance (crash reporting), and managing feedback (customer dialogs) will all help differentiate your app in the market. With more than a 1.5 million apps on the various platforms, that kind of positive differentiation can’t hurt.
Trent Peterson is a founder at AppThwack, a fully automated service that helps developers and QA teams test their Android, web, and iOS apps on 100s of real devices in minutes, gathering high-level results, low-level logs, pixel-perfect screenshots, and performance trends along the way. You can keep up with Trent and AppThwack on Google+ and Twitter.
If there is one thing that all app developers should know now, it is that obtaining user feedback from your current customers is much more helpful than reading their negative comments in the app store reviews.
No one enjoys seeing his or her hard work be lambasted by people who cannot grasp the scope of its existence. Although it is disheartening, and ego deflating, it also turns the app into something of little service to others. Before someone downloads an app, there is a good chance that they are going to read the reviews first. If they download the app without bothering with another’s opinion and find it hard to use, they may even leave their own negative feedback. Countering this negativity is possible with the next version or damage control marketing measures, but ideally, the creator should want to avoid them to begin with. After all, this ominous circle of negativity can send an app into the black hole of App Stores everywhere, never to be seen again.
Get better ratings by avoiding negative reviews
Instead of envisioning an app world of mayhem, developers everywhere are realizing that with a simple testing period of a minimum viable product, consumer test groups can provide feedback before the app launches. But why stop there? You should keep gathering feedback even while your app is live.
What is a Minimum Viable Product?
A minimum viable product does not mean that it is less of a product than it will be in its finalized form. In the app world, it simply means that it has enough functionality to deploy its capabilities, but is not coded to complete operation.
Keep your v1.0 simple and iterate with feedback from the people using your app
This means you can open it, review its contents and show it off to others as a prototype, without spending an exhausting amount of time or money insuring its overall functionality through coding and programming. It allows you to gauge interest, excitement, feedback, and the overall need for your app, before you take it to the completion stage.
A minimum viable product is tested on a controlled group of people, whose comments and conclusions on its testing can be accounted for. This means you can let people know they are responsible for helping you create a better product together. That is the important word “together.” People love to be included in the creation of things. The idea is to find a forgiving audience, one who knows that this is only a beginning but could benefit from the final product’s existence.
Get Your Users To Help You Create The Best Product Possible
Think of it this way: If you give someone an app and say, “This app is going to do this in v1.0. Here it is in its most minimal functionality. Play with it.” When that introduction and testing period is followed up with surveys for the consumer to address, they can provide honest and accurate feedback.
Iterate with your app customers to build a better product
This will deliver information regarding what they were expecting, and how the app lived up to those expectations. It can also tell the developer how great or how poor its performance ranked overall, which means they can take that information back to the drawing board and fine tune the application for v2.0.
How Surveys Insure a Better Minimum Viable Product
It is practically impossible to fund every idea that your genius mind creates, which means when it comes to getting app creation right the first time, surveys can be a key financial component to its success. Creating a minimum viable product allows you to display the app’s genius, without developing it completely. This means no more ill-advised turns in the wrong direction, which have to be fixed with a costly 2.0 programming option that may or not be entertained by the masses.
Better communication helps to create a better product
Surveys will provide you with all of the feedback you could possibly need to hone the app’s capabilities and smarten its aptitude. Certainly there will be answers like, “I wish it were blue” that have no bearing on its functionality, but you have to take the good information with the unrelated comments. It is all part of the process. Once the survey’s feedback has been adapted to the app, it becomes a better version of its previous existence. You are still at minimal financial output and can test it again on another audience, gathering their feedback as well.
What Intelligence can you get from Surveys and Feedback Forms?
There are a number of things you can determine with surveys and feedback forms. At the core of the forms’ content should be finding out exactly what features the people are interacting with during the test. What do they like about the app? What do they love about it? All of these things should definitely end up in the final version. What do they dislike or find confusing? Remove it, or fine tune it.
Knowing what the customers enjoy, and what they do not, can help you reduce negative feedback when the app goes to market. Once customers air their grievances, others will read it, and some of it is not transferable back to the developer, so all hope is lost for a solution.
Beat reviewers to the punch by providing them with what they are asking for through the use of surveys and feedback forms. If they happen to list items that simply are not available at the time, make notes from their requests, and adopt them to version 2.0 of the app. It is the least you can do to appease your fans.
There are many reasons an app developer would want to use metrics. But at their core, metrics are about answering simple questions:
Are we doing a good job?
Do people love our app?
How do we get better?
One of the premiere metrics of customer satisfaction is the Net Promoter® Score, that is, How likely is it that you would recommend X to a friend or colleague? on an scale from zero to ten. By subtracting the “detractors” who answer zero to six, from the promoters who answer nine or ten (the “positive passives” answering seven or eight are discarded), you get an effective measure of customer delight. It seems like a simple question, but there are reams of research to support the idea that this simple question is goldmine.
If you are able to implement such a metric, fantastic. Even if your business is selling and supporting mobile apps — where you don’t have a steady stream of walk-in or call-in customers of whom you can ask how likely they are to recommend your app — we can gather this information with in-app surveys, or by sending out an e-mail blast to paid subscribers. But in the app marketplace, we have moved beyond passive likelihood to recommend: people are actively recommending (or criticizing) apps in a public forum. What metrics, then, can you use to make day-to-day decisions about your app and your customers? For the mobile app business, I would suggest an adjunct to Net Promoter® Score. A question that can be crucial to an app’s long term success, and a metric you might be able to measure with data you already have: do people love our app?
In our work delivering customer insights and targeted messaging on mobile apps, Apptentive has found customer love to be a key indicator of a few things:
Customers who love your app are more likely to rate or review it in the app store.
Customers who love your app tend to give it higher ratings in the app store.
Customers who love your app are more likely to use the word “love” when they write a review in the app store.
One of the ways we measure customer love is with in-app feedback. In the app itself, ask the user, “Do you love [insert app name here]?” and measure the yes and no responses over time. And there is another way, using data you already have available: download the reviews from your iTunes Connect, Google Play, or other app store account, and count the percentage of reviews that use the word “love” in them. You might be surprised how many people are willing to say, I love this app, and what a powerful message that sends to other potential customers.
At Apptentive we are strong believers in empowering developers to listen and respond to users. If you are using an in-app feedback model, customer love is a great metric to see how well you are doing over time. And if you have not yet started to engage your customers directly, it is still a great time to start measuring customer love. It is one way to answer that question, “Are we doing a good job?” And once you start to listen to people and respond, you’ll also be able to answer the other question, “How do we get better.”
We recently recorded a two part conversation with Ryan Morel of PlacePlay about how app developers can increase their app revenue through customer feedback and communications.
We covered a whole host of issues and tactics, here’s the first part of the interview, talking about the basics about what we’ve built and the benefits that developers are seeing:
In the second part of the interview we dug into more of the ways in which customers who you connect with become evangelists, how we eliminate shipping app updates to fiddle with settings and survey changes and why customer lifetime values climb when you have real relationships.
Here’s a full transcript of the conversation:
Ryan: Hi. Today, we’re here with Robi Ganguly from Apptentive. So thanks for joining us, Robi.
Robi: Happy to be here. Good to see you again, Ryan.
Ryan: Yep. So before we get started, why don’t you tell us a little bit about yourself and what Apptentive does.
Robi: Sure. So, a little bit about myself and my background. I spent a bunch of time in Silicon Valley working for some of the big tech companies, most recently, Yahoo, where I was responsible for display advertising. And I had a bunch of experience with marketers and advertisers, as they were thinking about how to use the Internet and digital channels to talk to their customers. And then, after that, I actually did some work for Del Monte and Nike, with similar strategies around talking with their customers. Which leads us to what we’re building with Apptentive, which is really a tool for anybody developing apps on iOS, Android, and Mac OS10, to talk to their customers in their apps. You know, set STKs for them to do that. We think it’s just really important to have a relationship with your customers in the digital age.
Ryan: So, if I understand this correctly, you guys do two relatively specific things. So, one, you help developers get better reviews, and then the second thing is to communicate directly with consumers for feedback, relationship, et cetera. Is that fair to say?
Robi: Yes, absolutely. The ratings and reviews part is relatively straightforward, in terms of we give them tools that make it easy to prompt in their app to identify if customers are happy and they’re enjoying it, and then to prompt those people who are happy and enjoying the app to actually go to the App Store and say something wonderful about them. The flip side of that is making sure that the customers are having a great experience. So if they’re not really 100% in love with your app yet, that they get a chance to talk to that developer, and so the net result of that is your ratings and reviews are wonderful and you have a channeled action talk with people who, instead, would have gone to the App Store and ranted.
And then that second piece of really getting feedback is… A lot of apps have a “contact us” or support button, we power that and our SDKs present a native ballot box where you can solicit that feedback from an end customer and then you can also, when you get feedback from them, see the context of the device that they’re on–the carrier, OS, Virgin–and any other data that you choose to attach. So as a result, when you get feedback from a customer, it’s really rich because you understand their situation and the environment.
And finally, we also allow you to really structure some of that feedback through the use of surveys in your app, so that makes it very easy for the end customer to give feedback. It’s basically tapable responses in a survey ballot box in the app, so that they’re invested in the end customer’s investment and telling you more about what’s going on in their experience is 10-15 seconds, which then leads to a higher participation rate.
Ryan: App Store review rants have long been a problem for app developers, and specifically their inability to go back to that customer and help them solve that need. So it sounds like your solution helps prevent those App Store review rants and provides consumers and developers a channel of communication that they didn’t otherwise have before.
Robi: Yeah. That’s right. I think one of the things that we heard early on, when we started building Apptentive, was that it was super frustrating to go through the App Store and see somebody complain and then see an anonymous username. We actually heard from many developers that they’d Google those usernames and see if they could figure out a Twitter account or Facebook account in order to get in contact with them, which just really tells you how frustrated they are about not being able to go address those concerns.
I think we know, pretty obviously, at this point that on the Internet, anonymity leads to a lot more extremism in terms of comments–a lot more yelling, ranting and raving. And so, what we’ve done, by creating this rating and review process that helps qualify how somebody feels, is that we end up intercepting the rants that people who were going to go to the App Store and really shout, instead come directly to the developer. And as a result of being able to come to the developer and the developer being able to respond, the dialogue goes from this extremist yelling and screaming to an actual constructive dialogue. And we hear a lot of developers telling us about situations where the initial contact from a customer was really vitriolic–somebody was really angry–and then as soon as the developer responded back and said, “I’m sorry, are you having trouble?” The person toned it down and they got to a place where they were able to actually discuss something cordially and worked constructively to make the app better, together.
Ryan: Obviously, one of the big benefits you guys are providing is preventing developers from stalking consumers, so that’s a good thing. And so, the other thing I heard you say was that you provide a way for consumers to feel engaged in the process of the app development. So the app developer becomes kind of like a partner instead of a supplier. So maybe, you could talk a little bit about how you’ve seen those close relationships affect consumers’ engagement within an app and with the company?
Robi: Yeah. I think that it’s a really simple thing. It’s very, very easy to understand if you think about yourself as a consumer in daily life. When you come across a situation that doesn’t make sense to you or you’re frustrated, if you have the ability to raise your hand and talk about what’s going on and you feel hurt. Just by feeling hurt, I think, as a consumer, it changes the tenor in the relationship with whoever you’re dealing with. Whether that’s me going to Starbucks and getting the wrong drink and then telling them that I got the wrong thing and that he fixes it for me without a hassle, or it’s in the app, knowing that my opinion matters, right?
And then I’m actually giving you insight into what it’s like to use my app because it’s very hard, as an app developer, as anybody creating a piece of software, truly stepping into the shoes of the end consumer. And so this partnership is great for the developer as well, because they can really get advice and an understanding about what they thought it was going to do and then how it’s actually being used. So, just like you said, there’s this partnership in terms of making the app better, together.
Ryan: And I would assume, as a result of that, that some of those detractors end up becoming promoters?
Robi: That’s actually 100% correct. That’s the coolest thing. Developers, anecdotally, will come to us and say, “So and so came in. There were very upset. We went back and forth. I pushed out an update to the app. I told that person explicitly, that had complained, I pushed out an update and I listened to you.” And then shortly thereafter, that consumer becomes a person who is an evangelist–someone who’s on Twitter and Facebook sharing it with their friends. But one of the coolest things about this is actually hearing the stories where an end customer, now, because they have this relationship with the developer, will start using the developer’s name.
Like, “I was talking with Robi about their app. They responded. You should use this app, it’s amazing.” And so, it’s a way to really cut through that wall of the piece of software between you, to develop that relationship that then extends into the real world.
Ryan: Can you summarize what the really big benefits, for app developers are, of using Apptentive?
Robi: Sure. So there are a few things that people really are enjoying today, as a result. So the first thing is, the big problem with App Store is that Microsoft, Google, Apple–they own the customer relationship—and so as a result, by default, you as a developer don’t know who those customers are. Now, using us soliciting feedback, you develop relationships. You understand the email address. You understand the usage patterns. You have a way to dialogue with these people on a regular basis.
So that’s number one–direct customer relationships. Number two is better ratings and reviews. So, by and large, when people use our rating systems, they see increases in volumes and more ratings and reviews–50 to 100% more. And then they see increases in those actual ratings–another star or more. In fact, we see a lot of app developers, once they use us, get their ratings pegged pretty close to five stars, and the reviews take on a totally different tone. So people use words like “love” when they’re describing your app and that’s amazing to have in the App Store. And then the final thing is, because you have this dialogue and you’re engaging with your customers and you’re learning and you’re making the app better, your retention rate goes up.
Ryan: Do you guys do any analytics to identify what the best time is to promote, like a review or a rating feedback? Are you guys doing that right now?
Robi: Yes. So, what we have is, in our ratings prompt, everything’s instrumented so we can understand when it’s shown and then what the result is. So, we asked this question, “Do you love this app” whatever the app name is and we have all the data around when it’s shown, who clicks “yes” and who clicks “no” at each point in time.
We deliver that in our dashboard to you, so that you as an app developer can understand what’s happening with your existing settings but, importantly, the settings are all server-driven so, without having to ship an update to your app, you can act on that insight into what’s happening and change the settings of when you’re prompting and the end customer can answer that question. And, with our surveys, we also have it instrumented, and with our feedback it’s also instrumented. So, our goal is to make you, as an app developer, much smarter about interactions and the results of when you’re trying to interact with somebody because the worst thing you can do is be clumsy about it and end up hurting your app. So, that’s why we track all that data and present it to you.
Ryan: We all know that any app developer can hard code a question to ask their user, to review their app. That doesn’t cost very much. So, why would an app developer, essentially, pay you guys to handle that for them?
Robi: What we see when developers hard code that stuff in is that they don’t understand what’s actually happening. They haven’t instrumented it, so they don’t know, for example, how many times a customer has actually seen that prompt. You don’t know how many people are then clicking and going to that app store and reading it. The lack of data around that, plus the fact that you have to ship an update to your app, results in a really long cycle time around trying to figure out the appropriate time to ask somebody this question and making sure that only people who actually love your app are going to the app store and giving you a good review, as opposed to coming to you.
What we do, is we instrument that process. We ask the qualifying question, ”Do you love this app” which ends up resulting in most people who are unhappy with your app coming directly to you. That’s this notion of intercepting negative feedback that really is occurring across all of our developers. Also, importantly, because we’ve instrumented this and we’re presenting it in this dashboard and we tell you and people are saying, “Yes. I love this app versus anything else,” and because it’s connected to the server for settings, that developer can, today, launch an app with settings around what they’re prompting and then, tomorrow, change those settings without having to ship an update to the app.
So, that cycle of really getting very specific to your customer base about when you’re prompting them and trying to get as many happy customers as possible, goes from taking 6 to 12 months with not very much data to three to four weeks with tons of accurate data about all the activity around this. So, that benefit is a real reason why people are using this pegged because they have an insight into every step of the process and they don’t have to ship an update into the app store.
Ryan: Outside of the fact that they’re getting all this data and reporting, which is fantastic, it sounds like it’s like they’re paying you just because they don’t have to do updates.
Robi: Yes. I think one of the biggest smiles I get when I talk to people about our service is that we’re pretty focused on this idea that any time we can remove shipping updates to your app, in the process, we will, because people, in particular in the Apple ecosystem, are terrified of having to wait two weeks for the review process and that’s slowing down the development. So, we move that ahead.
Ryan: Can you provide any specific examples about developers who are using you who have made improvements to their app, based on the feedback they’ve gotten?
Robi: Absolutely. Here in town, there’s a great team of developers working on an iOS app called Chewsy, and Chewsy really helps you understand dishes at specific restaurants. Using their app, you can rate and review specific dishes and take pictures. So, it’s a pretty wonderful way to explore a menu through other people’s eyes and get really good data about what you should be ordering. One of the problems they run into is that, in markets where they don’t have a ton of coverage, so, for example, let’s say, Kansas City. If they don’t have a lot of users, they say that’s a great experience with any customer isn’t as good as it is here in Seattle. They’ve been using us to collect feedback and get insight from their end customers about what that experience is like and then how they would expect that experience to look, when there is volume and that’s allowed them to iterate on their product and really deliver something that is making more of their customers around the world happy. And it’s something they couldn’t have done just through metrics and analysis. They have to actually go talk to the customers in these different markets.
Ryan: And we talked about communicating with consumers is a really powerful tool for app developers. Do you have any examples of developers using you who have seen their revenue, ultimately, increase after integrating your SDK?
Robi: Absolutely. The easiest way to think about us is the idea that your app store page is really your storefront. As a result, consumers, when they’re shopping, think about downloading an app or buying an app. They’re looking at your storefront before they make that decision. The thing that they look at, most often, are the ratings and reviews. That’s what’s dynamic. Your description is important and screenshots are important. They certainly help, but thanks to Amazon and many others over the past decade, we’ve really been trained to look at what other consumers are saying about something. It helps us sort through all the noise, and as a result of having much better ratings in terms of higher volume ratings, as well as higher star ratings, people are more likely to download your app.
So, we hear from developers, of both free and paid apps, that they’re seeing more downloads, that the conversion rate of people who are seeing their apps page is just going up, that more people who see that app store page are downloading. Then, as a result, there’s a boost in revenue because you have more people using your app and they’re happier, as well, because you’re making a better app.
I think, over the long run, what we’re also seeing is that when you have relationships with customers, particularly if you’re in a free-to-play game, or something like that, you have that relationship and people trust you and they develop a relationship with you, they’re more likely to stay in your app. So, retention is really big. That difference between having a customer for a month versus 6 or 12 months, is massive from a revenue perspective, and we’re hearing from companies on a regular basis that’s true.
Ryan: So, it’s not only increasing initial downloads, but it’s increasing the retention levels and, ultimately, the lifetime value of the consumer.
Robi: Yes. Absolutely. And I think a really big point here, that a lot of folks don’t necessarily understand right off the bat is that trust is key to having lots of commercial transactions, whether or not you’re buying a car from somebody or you’re buying a latte. If you trust the business you’re dealing with, you’re more comfortable spending money with them and that’s happening in the app space incredibly fast.
Ryan: OK. Is there anything else you want to talk about before we go?
Robi: Just, the main thing is it’s really an exciting time to be an app developer. We have more customers signing on every day, and we get to talk to them about their businesses and are pretty fortunate to see across the landscape and businesses that never existed before are now springing to life because of this personal device that everybody has in their pocket. It’s pretty wonderful.
Ryan: Great. So, before we sign off, tell everybody how they can find you and learn more about Apptentive.
On a regular basis, we try to share some of the best materials shared about building your app business and communicating with your customers. Here are some of the best we’ve come across recently.
How to build your app business
Everyme’s feedback button
Capturing Feedback in your iPhone App – an awesome story from the makers of Everyme about how incorporating feedback into their app has made them more successful. From better ratings and reviews to more engagement around how to use their app, Everyme’s story really underscores the benefits to communicating with your app’s customers.
How to build an App Empire – The story of Chad Mureta’s recovery from a horrible car accident to become an “appreneur” is inspiring on its own. His tips and tricks for creating a successful niche app business is well worth the read. Be warned, however, this is definitely more self-promotional than most of the things we link to and we don’t agree with all of his tactics.
This podcast, Where has the sense of mission gone with customer service?, with Frank Eliason of Comcast is a valuable discussion about customer service in the new consumer age. In particular, they ask a really interesting question: “What would happen if you built scalable intimacy with your customers and your team?” The conversation with Frank starts about 23 minutes in and you can find the iTunes file here.