How Starbucks and Apptentive Connect Digital and Physical Customer Experiences

Keerthi: So as the introduction said in this, Veronica called out, it looks like my dress has coffee beans in it, unintentional. But I was like, “I am Starbucks apparently. That’s just how I roll.” So I’ve been with Starbucks for about two and a half years as Veronica was sharing. I lead the product management team that’s focused on the Starbucks app on both iOS and Android, as well as our web experiences. Prior to Starbucks, my entire career was in technology companies exclusively, worked for Zynga, doing product management there. And prior to that, I used to be an engineer. And like my engineers like to say, I crossed over to the dark side when I came to product management. They still like me.

So that’s been a bit of my background. And this topic is especially interesting because when I really enjoyed my time at Zynga, launched multiple games across web and mobile. And when I was looking for my next gig, I kind of felt like, you know, product management, super passionate about it, but what is like the next frontier for me personally in my career? And it was sort of taking that digital product management discipline, which technology companies do really well, and sort of trying to see how it could be applied.

And that was around the time when actually I was starting to read a lot about Starbucks and the order ahead feature, we call it mobile order and pay internally. And I was like…that was the point where I was like, this is not just a company that has a digital property, but it is actually a company that’s behaving like a digital company in terms of the sort of features it’s putting out. And it started coming into my consideration set as I’ve always liked the brand, but just didn’t think they were a digital company up until that point. So that’s a long introduction. But that’s sort of what got me into this gig.

Veronica: Great. So what do you think one of the most challenging aspects of your job is?

Keerthi: So, one of the interesting nuances of working on a product, where the digital experience is very important, but it is in service of the physical product, right? Because at its core, it’s the coffee and the tea…I’m a big tea fun, and the tea and the food and everything for which people come to Starbucks. The digital experience needs to enable that. The digital experience is not the product, right? And so one of the biggest challenges though, to do digital right, the speakers before me have talked about the need for agility, the need for iteration, the need for test and learn, and you take that in the digital world.

And then you try and think about those of you…and a lot of you come from other companies that where the digital is not the core product, right? So then what you see is, they have some very real logistic issues you need to work through, which sound very waterfally to those of us who come from the digital world, right? We are like, “I’m going to launch this thing.” “When will you have it done?” “I don’t know, two months, that’s my T-shirt size.” “No, but give me a date.” And I’m like, “I don’t know.” “Like, give me the scope.” I’m like, “Maybe this.” “Can you do it?” “I’m not sure. Let me talk to my engineer.” Like this conversation just doesn’t resonate as easily.

So the biggest challenge has often been how to bridge that gap but also how to have a lot of empathy for our store partners and our marketing folks and our folks that are getting the billboards that talk about the reserve store on Sodo 8 that any of you guys have checked out. Like, there’s a lot of physical assets that need to be coordinated. So I cannot work or think about the digital experience in isolation. That has been the biggest challenge but also the most interesting aspect of the job as you try and see, how do you bridge that gap where you try to protect the agility of a digital product innovation cycle, while also making sure you can partner well with your other stakeholders who have very real challenges that they need to work through, that need more lead time. So how do you bridge that gap? So that would be one of the big pieces.

Veronica: Yeah. So you alluded to it already, right? Starbucks has one of the most loved and talked about mobile apps. I think everybody has that app on their phone. You’ve probably all used the feature. Mobile order and pay has just blown up and you actually pioneered the way in connecting those digital and physical experiences. And I think it was in 20…

Keerthi: 2014 December, was when we ran the pilot.

Veronica: Yeah, 2014. Last year, this drove 11% of all U.S. transactions just through that feature. So what do you think…I know you weren’t there at the time, but what was the hardest part that the team said about launching a feature like this that has become just so global and watched?

Keerthi: Yeah, like you alluded to, I wasn’t in Starbucks during the launch, but I’ve talked and talked to a lot of people who were there and heard the stories. So I think the biggest challenge as I understand at the time was…so for all of you who have ever been to a Starbucks, and I’m assuming if anyone lives here you have. One of the biggest things that’s the core proposition and something we do very well is that Barista Customer Connection, and just how the baristas just do right. Like, doesn’t matter if you get your drink wrong, the Baristas will correct it, right? And they remember your drink, you walk in, they start making your drink. So it is such a core part of the value proposition and the USP and the core of who we are, that connection, that Cuban connection. So when we started having the conversations about an order ahead capability, we knew there was a latent customer need for convenience.

But you realize that it is going to…the hypothesis if it pans out, is you’re eliminating the need for that Barista connection when you walk up to the POS and place your order. So just that conversation of just talking through, us trying to test something that was going to directly compete with what was very sacred and USP and core to our success as a company, I understand took a lot of navigation. So apparently this idea came up multiple times before we finally agreed to go ahead and test it out because there was just a lot of angst about, “You’re just eliminating the Barista here.” And what is interesting is, you aren’t, it’s just in that one use case and there are customers who have different need states. In the morning, you may want convenience but in the afternoon, you wanna sit down and talk to your barista and have the conversation. So the need state changes, but anyway, I understand that was one of the hardest pieces of even trying to kick off that feature.

Veronica: I go almost every day. So in the morning, I order it on the bus…

Keerthi: Yeah, you’re…

Veronica: …so it’s waiting for me. And then in the afternoon, we actually have a group, we have a Slack channel that goes like every day at 2:00. And we just walk to the Starbucks at the next block and, like, that’s when they know all of our names, they know who we are…

Keerthi: Yeah, that’s exactly it. And even you still have the connection when you pick up your drink, right, where they kind of know.

Veronica: Yeah, they’re like, “Oh, we’ll see you later.” And they’re like, “Oh, I don’t know what to get today.” And then they make recommendations like there is that connection. Also the points make me really wanna come back for more. So it’s like, you know, you’re hooking me. But that said, obviously, you know, this is a feature that launched a few years ago, there’s still challenges popping up like can you talk a little bit about those?

Keerthi: Yeah, sure. Absolutely. So all of us product managers in the room, when you launch a feature, you have your KPIs, you have your success metric, you forecast out how this is going to move the results. And Order Ahead had its own results and sometimes features become extremely successful, sometimes much higher than you expect, and it’s a good problem to have. But one of the interesting things that it did introduce for us was, sort of trying to optimize the queue. So not all Starbucks are like this, but a lot of Starbucks essentially have three queues in which you place your order. So one is the mobile order and pay queue where you order ahead.

The second is the POS queue where you walk up interact with the barista and place your order. The third queue is essentially the drive-thru for stores that have a drive-thru, right? So what ends up happening is, in our stores, we have the concept of a playbook where you try and load balance between these different queues. So you’re trying to make sure the orders are getting made at the right time, you don’t want the order which is a hot beverage…if it’s a hot beverage to sit out and get cold, so it needs to be made again. Or if it’s a cold beverage, you wanna make sure the ice isn’t melted completely, right? So your mobile order you wanna make sure is being also made at the right time.

One of the challenges we run into is, it took us a little bit of time to figure out what are going to be the mobile order patterns we need to expect. Not all stores get equal traffic. There are some stores like, I saw a Starbucks when I was coming up the elevator here, Columbia store. Starbucks is a great example of a very high mobile order store. Like offices here, everybody places their order, not as many people walk up. So you kind of need to understand and do a lot of analysis around what is the patterns you need to expect and how do you train your baristas to have a very flexible playbook where they can flex, where sometimes there’s options, right? Do you have dedicated queues for these three things or do you put them into a First-In, First-Out kind of a queue?

But if it’s First-In, First-Out, then a mobile order person who expected that it would take 10 minutes and placed their order first, will essentially get their order made first and the person who just walked in and is going to the barista, is going to have their order made later, which is not what you want, right? Because the mobile order persons, beverages getting cold or not at the right state, in which case you’re making the order again. So there are a lot of optimizations and there’s been great partnerships again in the theme of our conversation, the digital, physical touch points.

So we’re working with the store ops teams and we’re working with other groups to try and figure out, how do we optimize these algorithms? Do we have good understanding of how we categorize stores? Do some stores have exclusive and just random pieces? The stores inside hospitals end up again getting very high mobile orders. Like, Cleveland Clinic keeps coming up because they apparently are one of the highest mobile order stores. So just some interesting nuggets there.

Veronica: Yeah. You know, I hear a lot of product managers talking about, like, the flow of employees when they’re like placing and taking orders. But that’s a part of your daily life. [crosstalk 00:11:11].

Keerthi: Yeah. No, and it has been one for everybody in my team, including me. Like, when I came in, I knew there was going to be this nuance of tweaking or like understanding the touch points with physical, but you don’t think that right off the bat, right? Like you’re sitting in your grooming sessions, writing your user stories, talking about when to get it out, and you just think about the digital. And then we are like, “Oh, did we talk to store ops? Have the partners been trained? Like, do the baristas know what happens when we roll out this test?” And a customer is going to walk in and say, “What happened here? Why is my UX different?” And if the barista doesn’t know a test is running, they don’t know how to walk the customer through it, right? So there’re a lot of things we need to think through, we constantly forget. And we’re all learning, my team is learning, and we’re all trying to get better, but that’s the interesting part.

Veronica: Do any of you ever actually go and spend time in the floors?

Keerthi: Oh, yeah. Absolutely.

Veronica: Yeah, I know you have one in the building, but…

Keerthi: One of the things that is part of the Starbucks orientation culture is, we do what we call a store immersion. And essentially, they encourage store part…Starbucks calls its employees, partners. A lot of you may know that. So we essentially encourage all our partners to, as part of their orientation, go sign up for a store immersion, where essentially you…and you’ve got to be sensitive about finding the right time so you’re not adding work to the barista, as you go there and you’re clueless about what to do. But essentially we understand what it takes. So when we come up with interesting nuances we wanna test in digital, we play out what it means to our baristas and try and make sure it’s a seamless end-to-end experience for the customer because of the barista. It trips up the barista. It has cascading implications that we need to be aware of.

Veronica: So let’s switch topics a little bit and talk about feedback funnels. We’ve talked about this with your team a lot, obviously, like a little insight into our conversations. But why is expanding that so important to both you and to Starbucks?

Keerthi: Yeah. So what Veronica was referring to when we talk about the feedback funnel…and I know a couple of speakers earlier in the morning had touched about this. As product managers, we constantly look at both qualitative and quantitative data, right? And you’re going to use qualitative and quantitative differently. And oftentimes, qualitative is at the beginning to get you directional feedback. And then you make sure you cull down your hypothesis if needed with this qualitative data. And then you design experiments, use quantitative data to then validate exactly what you saw.

One of the things when we were chatting with Apptentive, and Robi, and Mike and everybody had come in, is one of the challenges we have is, qualitative data is extremely important, but for a company of Starbucks’s scale, while a 20% customer focused panel is great to give you some insights to probe on certain topics, I can say that if I just spoke to 20 people, I understand the why. Experiments and quantitative data helped me understand the what, what is the customer doing? What are the drop offs in the funnel? Like, what am I seeing? When I put this feature out, how many people interacted with it? I get the what, the why was a bit of an area of gap for us. And, you know, we were using your survey capability that we had integrated with. But what we felt was in general, we have amazing customer insights team. We have a UX research team. I have one of my UX researchers here. We have really good qualitative work we do, but the volume of the qualitative data that we’re getting, is still pretty low.

So one of the things we brainstormed when Apptentive had come in for a discussion was, how do I make that qualitative feedback funnel big too? Like, how do I get good volume? So if we are running a test and getting quantitative data, can I also get qualitative data in the same test, where I can also understand both the what, and the why at the same time? So when we talk about the feedback funnel, I wanted to approach even the qualitative data with a funnel approach, instead of just saying it’s 20 people or 25 people, which is technically statistically significant, but I’m like, “I can’t go too much with it. I have to cut and slice the data to match with it.” So that was essentially the feedback funnel conversation and… Go ahead.

Veronica: No, I was gonna say, so we actually did an integration with Optimize to help with that, right? So we talked a lot about how we helped you inform some of these experiments. Talk to me a little bit about one of the experiments that you’re running or planning to you use Optimize for?

Keerthi: Yeah, absolutely. I kind of really…I love working with Apptentive. And they did not ask me to say that. They didn’t ask me to say that. But essentially, right? Like, I was just talking to these guys and saying, “We had really good dialogues when we talked about how we use qualitative data,” right? We were brainstorming a lot, right? And I said, “Listen, here’s the problem I have, I’m constantly running experiments and the fact that your survey capability only doesn’t let me cut data in different way… So if I have an experiment with three variants, I have a control and two variants I’m testing, and I can only serve up the same survey to everybody, and I can’t ask them different questions because the flow is different. Or, even if I want to ask them the same questions, but I wanna then cut the data differently based on the experiment variant, I don’t have a way to do that today. So even though you give me a survey capability, I’m not really getting the benefit of the qualitative data that I want back from it.”

And these guys were amazing. They were like, “Fine, we’ll integrate with your experiment tool.” So one of the things we are trying and planning out now is essentially doing that integration. I unfortunately don’t yet have that experiment in flight yet, it’s on our roadmap, it’s coming soon. But it’s just been incredible to have that tool because the one thing about qualitative data…and there is a bit of a passion, so bear with me, is, I find that while the intent of qualitative data is very valuable, people also just use it randomly to justify the things they want to do, right? They will say, “I spoke to three customers, they want this feature.” I’m like, “I don’t know how you picked those three customers.” Like, “No, sorry,” right?

So that’s one of the pieces where we want to bring a lot of discipline into also how we use the qualitative data. It’s one thing to use it directionally, but if you wanna go deeper on it, we just felt this integration was pretty useful. So what we plan to do is…we have a bit of an experiment pipeline that we have as part of our roadmap. The intent is to start having, for critical experiments where we believe, it’s really important to understand the why because we’re really testing out new features. We will be planning to use this capability and just doing more microservice, where we can also get not just the what that they do, but the why. So that’s kind of the intent.

Veronica: Yeah, I’m super excited by the way.

Keerthi: Likewise. Likewise, we are.

Veronica: We only have time for one more question. And it’s my favorite one to ask, what does customer love mean to you?

Keerthi: Other than the logo of your wonderful company.

Veronica: We’re a little corny.

Keerthi: I do like the Valentine boxes with hearts and heart shaped goodies that comes from you guys. I have to say. I think one of the things is your…I think all of us really like…just forget product managers, right? All of us as customers, word of mouth referrals is one of the most important pieces that guide our everyday action. A friend says, “Go watch that movie.” A friend says, “Try out that restaurant.” Yelp says, “Go, don’t eat there,” right? Like, Yelp is still very trustworthy. And so for me, customer love is this ability to, you have your brand evangelists. There are people that love your brand.

Customer love is really listening to them, giving an opportunity to engage with them, and finding ways in which you can bring them in early. Like, Lisa, in her talk had this example of how they had used the customer love dialogue to make exclusive access and beta access to features. That’s an incredible way to engage with your most loyal customers. They will also be honest with you. It makes them feel special, but they are a great source of information for you. So for me, customer love is just being able to tap and listen to your big evangelists and your…