Silly Things Smart Product Managers Do

Mobile product managers have some of the toughest jobs in an organization. Markets change so fast and there is often no rule book to guide us for the kinds of situations we encounter, resulting in mistakes made and valuable lessons learned along the way.

We were thrilled to host a panel of expert PMs at our Customer Love Summit, including Erika Englesby, Sr. Product Manager at Providence Health & Services; Vasantha Kostojohn, Sr. Director of Product Management at Allrecipes; and Darren Austin, Partner Director of Product Management at Microsoft; to share silly things that smart product managers do.

In the video below, our panelists share their experiences as veteran product managers. As they reflect on some of the less-flattering moments in their career and what lessons they took away from the experience, our hope is that you can learn from their mistakes.

WATCH: Silly Things Smart PMs Do: Laughing At (and Learning From) Mistakes We’ve Made

If you’d rather read than listen, you can check out the full transcript below. Enjoy!

Transcription

Darren: Thanks everybody. All right thanks everybody it’s great to be here. We’re gonna have a little fun with the panel today. I thought I’d start this out with a little bit of a quote. There’s a famous basketball coach from UCLA whose name’s John Wooden and he’s quoted as saying, “If you’re not making mistakes then you probably aren’t really doing anything.” And I kind of thought that was relevant for our world as mobile product managers because there’s no roadmap to suggest what we’re supposed to do next, right? There’s no examples for all of the various scenarios we come into, right? I mean Apple updates iOS and then something that we have breaks, Android probably happens more frequently, we have to deal with that situation. So the best we can do is to do and we learn by doing and learn by making mistakes, we take them in stride and we laugh at them in hindsight and hopefully we don’t repeat the same things again.

But I think we’re at our best when we’re sharing experiences with one another and we’re learning from one another and that’s what we’re gonna do today. So this group of folks up here is gonna share with you some of the more or rather less flattering moments that we’ve had in our career. And we’re gonna kind of laugh out of it little bit, talk about them and we’re gonna engage the audience at the end for a question and answer so that we can actually hear from you and ideas that you have about things that you’ve experienced and insights that you might wanna hear from us.

So with, that I told Vee that I was gonna put her on the spot, she was gonna be the one that was gonna hit up first. And so I thought I’d started out by asking Vee, like tell us about a situation, a lesson you learned in a difficult way, that you took away an insight, that you could share with the team.

Vasantha: Okay, you all hear me? Not used to a mic. Okay. So I think one of the big lessons is there’s really no substitute for customer feedback. A few years ago we added into our app the ability to scan grocery items in the scenarios that had we imagined were as a user throw something away they could just sort of scan it and add it to their shopping list or they could search for things just by scanning. And what we found in real life is, well the things that people actually search for generally aren’t scannable, you know, chicken, fruit and the scenario of scanning right before you throw something away sounds really cool. But, you know, thumbing it out with your, you know, finger is actually a lot faster. And so it didn’t actually ever, you know, live up to the promise that we were hoping for. So a while later we ended up cutting it and not one user complained. Yeah.

Darren: Yeah that’s cool. Yeah, I think I’ve experienced that as well where we’ve launched product features and we thought, “Oh, this is exactly what people are gonna want.” Right? “Oh, of course, this is what they want.” And then you get it out there and it just sort of, you know, like nothing. You don’t hear anything positive, negative about it. And it shows up in the user data. I had an interesting experience where earlier in my career I was working for AOL Instant Messenger, right? AIM if anybody remembers the running man? Rest in peace, rest in peace.

We had this interesting phenomenon going on in AIM where we had a status message, right? Status message was, you know, I’m out of the office, I’m on vacation whatever. We started seeing people update these things like 20 times a day, right? And we’re thinking, “What in the hell are people updating status messages 20 times a day?” And we didn’t read what they were updating because that’s way too obvious. So but we were seeing this in the data and I kinda called BS on the data. I was like, “No way, like this is ridiculous, right? I don’t understand why people do this, I would never do this.” At the same time, there was this other company coming up called Twitter.

And Twitter was…we were talking to Twitter and Twitter was like, “This is the new thing, we’re gonna do this thing and people are gonna love it and it everybody’s gonna get a text message anytime somebody, you know, makes a trip to the restroom.” And I’m thinking, “This is insane, there’s no way.” And then Twitter started growing and our team got back and we’re looking at the data and we’re saying, “This makes no sense.” We actually didn’t believe the data and so I called Twitter a fluke. We passed on a partnership opportunity with Twitter to host their back-end and if anybody remembers the whale-fail, we could have solved that but we ignored Twitter. And so that was my embarrassing moment in hindsight as I looked and I look back and I say, “Oh my God I had no idea the data is staring me in the face.” And I completely called Twitter a fluke and now we’ve got, you know, the president tweeting all the time.

Erika: So the biggest question is, do you use Twitter now?

Darren: Kind of. I kind of do I. Yeah, I do tend to. Okay, so true confessions, right? I have it set up to where it’ll post simultaneously to Twitter and LinkedIn and I don’t do Facebook for some reason, I think maybe that’s more personal. But yeah I’m using it a little bit.

Erika: It doesn’t make you like cringe every time?

Darren: Yeah. I kinda feel like I really screwed that one up because I was like, “Oh, man the data was right there.” I can’t believe it. Yeah. So lesson learned was, you know, trust the data. I don’t know if you guys ever run into a situation where you’ve had data staring you in the face and you just say, “I can’t imagine that anybody would wanna do this. It must be a fluke.”

Erika: I mean I think we’re…At Providence, we’re currently going through the opposite where we don’t have any data. So we’re part of an innovation team and we have built our product from a beta that sort of launched as a real full-fledged product but without any of the analytics that you truly want to understand what your users are doing. So we’re currently going through a period of understanding what do we wanna learn from our users and then going back in and actually adding that all those key pieces back in. And so right now we’re kind of, we have account managers that are selling to other hospital systems signer apps or white labeling to other hospital systems and they want all this data and they wanna understand it and so they’ve been going into Google Analytics and pulling out the pieces they think they want and we’re like, “No, do not look at it.” We don’t know if it’s right. So we’re actually releasing it on Friday. A new release hopefully that includes a lot more of the analytical data that we need to understand.

Darren: Nice.

Erika: So hopefully we’ll be able to make some better decisions because we’ve been kind of running blind for a while and it feels a little scary. So…

Darren: We did that too, I mean, when I first joined Microsoft to on the team a couple of years ago. When I first joined we didn’t have any analytics, I mean, we knew…I think we knew monthly active users but we had really low visibility into our retention and our churn information. And it was seat of the pants kind of decisions like, “Okay the best we can do is talk to customers and get a sense for what they’re telling us.” And even though we don’t have, you know, volumes of data that was sort of the best we can do and most of the time we guess right, I think there were a few times we guessed wrong. I think there was one time we run an alert, we wanted to promote collaboration. So we found out, “Oh yeah if people are collaborating with their notes, you know, retention will be better.” So he said, “Oh well, what if we gave an alert every time somebody updated a shared notebook?” Yeah, that’s what happened.

Erika: That sounds noisy.

Darren: Yeah it was real noisy. We ended up sort of… We tested it which sort of makes the end result even more embarrassing but we end up spamming our users for a period of time and then we had to roll that back and we’re like, “Oh yes, that’s actually gonna be kind of caddy.” We don’t want that. But we heard about it in loud voices. So, you know.

Josh: One of the screw ups embarrassed is, there’s too many that I can name here in the time we have, but it worked for a local carrier Magenta and was launching a new, I won’t name names. But it was launching a new device.

Audience: T-Mobile.

Josh: Fair. It was T-Mobile. Was launching a new device and there was a lot of new and a lot of old in this device. What I mean by that is the operating system was kind of a dinosaur at the time, I think Windows 6.5, and it did a lot but it didn’t do a lot really well. And we had a partner HTC that had this really awesome four-inch diagonal screen with, this is way back when [inaudible 00:08:39] dating myself, but Qualcomm Snapdragon processor. And so my colleagues and I looked at this device and we looked at what we had and we thought, “Well we could launch this device pretty plain and boring.” But what we wanted to do is try to turn that thing into an entertainment portal for our users and it hadn’t really been done before that way at T-Mobile.

So we partnered with a number of media app companies so Blockbuster, Netflix was doing something else with Apple at the time they said, “Thanks, but no thanks.” We ultimately got them to be on the next device preloaded. But anyway so we had Windows 6.5, coupled with this brand new device which was really powerful and we did a lot of deals with different media companies, MobiTV and Blockbuster and we rolled our own Amazon mp3 app, the T-Mobile team did and we partner with Amazon to do that. So all those apps on this device with an older operating system that wasn’t getting any love from Redmond, kind of really just came together and we thought, “Okay we have this tight timeline, are gonna make it or not?” So we ended up, we launched the device. But it had one of the higher return rates in T-Mobile’s history so we under market pressure we delayed a few times and ultimately said we just had to go. That was a big failure I think in the many millions of dollars. But more importantly the customer trust in the sales teams and the retail and then, of course, our partnership deals.

Darren: I think that’s a common thing though, right? Like there’s always pressure to ship. You’re working on something for such a long time. I feel like that’s a universal thing where you’ve got so much invested in this. And yes, we know that there are problems, it’s not perfect but if we wait on perfection we’re never gonna get it right. That seems like a common thread. Is that something that…I feel like I’ve certainly experienced that multiple times.

Josh: Yeah. Market pressure, you know, there’s plans to go and you have to make hard calls near the end and you sometimes it’s a leap of faith, other times you try to be like, “All right, can we stub in the app?” This is before the [inaudible 00:10:41], we stub it in to where they fire the app up and then it downloads so you kind of safeguard that process a little bit and update the app as it’s launching and so that the user gets the new one, you know, but now that’s all moot. But yeah you have to make the calls to either pull stuff off or say we’re gonna launch and there’s gonna be some impact here and really let the users know.

Darren: Yeah I’d be curious to hear from you guys about how you balance that pressure too. You know, because there is a…it’s a fine line and there’s no cut and dry answer like, “Do we ship or do we delay and get it a little bit better?” How do you guys handle that type of thing and what’s your decision tree look like and what are some examples of when you had to make a tough call one way or the other?

Erika: So I think from my perspective what I tend to do is I lean on my team a lot. I think that they are the ones that are really in the code, in the QA process day-to-day and they really understand all the pitfalls that are hiding behind the scenes. I do remember a time, Darren knows the story, so I’d been working at Starbucks we had re-launched the iOS and Android apps with the new design and totally rewrote them from scratch with an in-house team and the launch went off pretty great. It was delayed a few times but ultimately we got there and it was a situation where leadership kept saying, “Oh but we’d love to have this in.” I said, “Great you can have it, it’s gonna be another month.”

And so we finally got there, got over the finish line and then a couple months later we were just doing like a routine bug-fix release. But it was one of those times where we’re like, “Okay we got to get it out.” because I can’t remember what it was for but we had to get this one fix in and so we kind of rushed it. And this was when the apps were very high profile at the time because they had just released, everything was going great. So I get a call very early in the morning and it’s Howard Schultz on the other end and he was at…

Darren: Not a good thing. Usually not a good thing.

Erika: No. The update had come out the night before. He was trying to use the pay functionality at his local [inaudible 00:12:50] Park store and scan his bar code and the app kept crashing. So what we had found out is that we had an upgrade crasher that hadn’t been tested, maybe had been tested but not fully.

Darren: Not on Howard’s phone.

Erika: Not on Howard’s phone. He was the best keyway. He really was, he always had that one phone with that one problem. That’s crazy. But anyway so I get the call and it’s literally like the store had just opened like the man never sleeps. And so I was like, “How did you even get my phone number?” Number one. I found out later it was another person on the senior leadership team, who will remain nameless. But we got the call that it was crashing on the pay screen. Can you imagine? Like this is before we had launched mobile pay.

So the biggest benefit of the app was the pay screen like scanning a bar-code and so quickly I like, rally the troops. We get into work, we figure out what the issue is, we worked like day and night for like 24 hours straight trying to get the release out and really make sure it’s tested this time which was key. And then I actually had to go to Howard’s office and prove to him that the upgrade bug was fixed. Like imagine a tail between your legs like, “Oh yeah, we fixed it, I promise this time.” Anyway, we got it fixed but it was crazy because on…Like talk about customer feedback, this was like before we had implemented this was before we had done a lot of like any customer feedback. Twitter was on fire. People were pissed. It was like just grab your credit card.

Darren: Twitter ended up being a thing, didn’t it?

Erika: It did. It did yeah. Big surprise.

Darren: Big surprise.

Erika: So if we had actually looked at Twitter the night before we’ve probably would have realized then that the crashing bug was in there. But anyway long story short we got it fixed but it was definitely one of those moments where it was like, “Can I just hide under a rock right now?” Because even though like I’m not the one queuing it, I’m not the one developing it, like you are the face of the product. And so it’s really important to like understand all those pieces and really get your hands dirty too. Because I guarantee you I am such a proponent of like forced-upgrade now. Like, put that in your code because then you can flip a switch and be like guess what users you have to go on this new version.

Darren: I like that insight, the forced-upgrade is what you took away from it, while I was sitting here thinking the lesson learned for me was like, “don’t give your number to Howard Schultz.” Right? Whatever you do. But I like that better, that’s actually useful.

Erika: So now like every time I come to a new company I’m like, “Do you have forced-upgrade?” If not we’re adding that first.

Darren: Nice.

Erika: So we’ve added it at Providence. It has saved us a couple of times where we’ve been like, “Oh shoot, we really have like a bug or a version we wanted to switch over our whole back-end and so everyone really did need to update.”

Darren: Is the forced-upgrade happening every single upgrade or you just flip the switch and say, “This is the forced-upgrade?”

Erika: No, there’s a server side you can…Yeah. Switch and then we have language in there that can turn on. It’s really, really useful and we’ve actually use the upgrade tools as well to sort of softly do that. And then we usually do a forced-upgrade after that.

Darren: Yeah, I think that’s good advice. We’ve had to do that before at a previous role in it helped. Yeah. For sure. One of the things that I think is interesting too is we’ve sort of all been around before mobile, B.M., bad acronym. So before mobile but now we’ve been living in this sort of mobile-centric world and it’s different, right? I mean mobile is fundamentally different than when we were building for web and desktop apps and everything. I’m curious to get perspective from you guys on what you learned from mobile, like what are the things now in a mobile-centric mobile first world that are different, what’s more important about our job as product managers, product designers in a mobile first world?

Josh: Move very very fast.

Darren: Yeah

Josh: Because that’s what customers are expecting. And so when I go into any sort of new role I assess, you know, how often are we pushing a new app out to market. And if that cadence is, you know, two months or three months that’s just way too long in my view.

Vasantha: Okay, I would add to that. With less screen realisty the sort of harder, you have to look at your priorities and get really crisp around that. There’s a quote that I’ve heard that I love. It goes, “Your strategies fall on the battlefield of you x.” And I think that’s especially true as it relates to mobile. Were you really kind of have to get really crisp about what’s important, what’s not important. You know, and how much of this tiny space is it worth, you know, for every given thing you’re trying to do?

Darren: I think we’ve probably all tried to fit the desktop experience into mobile screen at some point. Show of hands anybody do that and made that mistake? Right? Like, don’t do that if you’ve not done that. Don’t do that. That’s yes. Definitely, a new UI is required and more focus, right? Yeah.

Erika: I think for us it’s really important like when we first re-launch the Starbucks apps we had really done an iOS-centric design. And so we basically, and I see this happen everywhere is iOS gets slammed into Android, the user experience is very different, users use their phones very differently on the two platforms. And we heard from customers immediately like, “What are you doing?” And people started just dropping the app, to be honest, our numbers on Android kinda tanked. And so we went back and had to redesign the whole app with material design in mind and it totally changed. I mean it really did change and customers were like, “Wow, this is great.” But it’s gone through a lot of evolution to try and get back to that point. Still, at Providence, you know, we are today I, you know, went through a designer review and all the other comps were for iOS and I was like, “Where’s the Android comps?” You know, like the interaction is fundamentally different. And I think we’ve all kind of made that mistake of assuming that they’re gonna be the same and users are gonna use their phones the same. But understanding the different platforms is really key. And then I think on top of that understanding really digging into usability understanding how users use those platforms differently.

Vasantha: And the demographics.

Erika: Yeah exactly.

Vasantha: No, no.

Darren: No, I think it’s a really good point though we did a… When I worked at Glympse, it was a company I was at just prior to Microsoft. We did the same thing. We spent a lot of time on a redesign and we felt it was really good and we felt like it was kinda universal. We were sort of splitting the difference between iPhone and Android a little bit. Wow what a bad mistake, right? Like the iPhone users got mad, not as mad as the Android users, because we were skewing in that direction like what you said. And we felt like, “Okay this metaphor, this navigation metaphor will carry over.” And it didn’t. It just they fundamentally use it differently. And then you’ve got the fragmentation problem with Android, so you’ve got some people on a Samsung and some people on an LG device. So you really do have to think about it differently. That’s a lesson I’ve learned the hard way as well. Yeah, design matters, design matters.

Erika: Yeah and I think also navigation really matters. You know, like you’re trying to slam a hamburger basement into a design that doesn’t work, doesn’t know what a hamburger basement is. The three little lines.

Darren: You’ve ever been to a Hamburger menu, Right?

Erika: Yeah. So trying to like slam that into a design that really doesn’t work in. My kinda key is like exposed those things that a user, like look at your analytics, see what users are using, do your usability and understand, bring to the front those key pieces that the user wants the most. Because the other stuff can be hidden and people will discover it like its mobile and people are willing to play around. I mean Snapchat give me a break like, you have to have a manual.

Darren: I also didn’t think Snapchat was gonna be a thing for what is worth. I didn’t get that.

Erika: I did give it to my 5-year-old like I handed her my app and she figured out very quickly. I can still not figure it out. But, you know, I think that’s really key to us understanding the navigation model is really different on mobile as well.

Darren: Yeah, yeah. For sure. I think you touched on something there a minute ago about, you know, looking at your analytics and see what your users are doing. That was something else I sort of learned the hard way is people will use your product in ways you don’t think. This was a very strange lesson I had to learn when I was at Glympse. So for those that don’t know Glympse is a temporary location sharing application, allows you to send a link to somebody and they can see you on a map in real time and the link has a time to live. So, you know, I’ll send any one of you guys a Glympse or maybe on Twitter and people can see me where I am. And then for 20 minutes or whatever, I send and then it blows up. So what was interesting was we were going along and our analytics didn’t look right because it looked like, “Wow, people are sending these glimpses out and some of them are getting viewed and some of them are not.” And then we started seeing that people would view the Glympse. Let’s say I send a Glympse to my wife every day when I go home from work right? She knows I’m in traffic, she knows if I’m stuck, she knows what time I’m coming home.

But after a while, we were seeing patterns where people that would receive a Glympse would look at it for a couple like the first couple of times and then stop looking at it. And we thought something was broken and we kept trying to figure it out and we were, you know, wringing our hands over the fact that nobody likes our product and it sucks and all this stuff and then when we started to dig in a little bit when we talked to users and sort of dug into exactly what they were doing we found out it was like just getting the link was almost to security enough to know something was happening for example, like with my wife, she was actually one of the people who stop looking at my Glympse when I said it. But, you know, and the reason why she was like, “Hey, all I need to know is you’re on your way.” Like if you’re not here after a certain time I’m gonna click that link. But you’re almost always here when I expect.

Josh: You just tweet that now.

Darren: I just tweet it. I get it out. I go broad. Yeah definitely, definitely.

Josh: For me the analytics and how customers use the app, I’ve run into that a number of times one was with a voice assistant called Dragon mobile assistant that was one our products, I worked on at Nuance Communications. So the data was showing us that the number of speech transactions per customer was actually lower than another product we were managing, my team was managing S voice and that’s Samsung’s speech voice assistant, and we were scratching our heads as to why and the Dragon mobile assistant interface had kind of a persona floating at the top and the mic at the bottom, you might know where this is going. And so we wired up Localytics to have a new event I said, “Are people pressing that top or so, why is this? Are they interacting with it in a way that we’re expecting?” And sure enough, we found about 45% of new users were pressing on the top persona floaty character because it was drawn their eye to it to like, you know, interact in speech and engage. So we ultimately made the call to simplify that design entirely, took the persona and kinda rolled it into the mic or behind it and the speech transactions went up drastically once we did that.

Darren: So do you find, you know, there’s always this sort of interesting debate around, allow users to discover whatever feature it is in whatever way they will, meaning it could be multiple ways to get to let’s say the microphone that you’re talking about. Do you guys find that? Where do you fall on that argument? Like is it okay to let people discover, like to have multiple ways to accomplish the same thing in an app or do you try to keep it on rails? Where do you fall on that? Just opinion wise.

Josh: Dealing with this right now with a different way to input information into the smart sheet app and the mobile app right now. And so we found that about half of the users prefer to edit it right in the grid as we call it. And about half of the users prefer to open up kind of a form with the voice.

Darren: Yes so what do you do with that? 50/50, that’s like a coin flip.

Josh: It about is. But I think we need to solve for both in a very efficient way, an intuitive way for the users and allow them to choose and empower them. So we’re working on that right now to have an answer but yeah.

Vasantha: What I would say to that I think this is where AB testing comes into play.

Darren: Oh yeah good point.

Vasantha: So you can really start understanding what is most interesting for people and when and what’s the right sort of weight of it you will. And so I think getting crisp about, you know, what are your hypotheses are and putting forward variants that you can learn think is really important and I think sort of pivoting so that you’re really leaning forward on learning and optimizing for learning. And, you know, you asked a question earlier too about, you know, “How do you know when to release versus not should delay, should we make it better and then release.” I think I would certainly say the same thing, like I think it’s all about figuring out what you can learn. So if you can get something maybe that’s not magical but is still valid from the perspective of learning something new.

Darren: I like that yeah.

Vasantha: Then go for it and learn what you can learn and then iterate.

Darren: I think you nailed it on that. That’s actually one lesson I’ve learned too. I can’t think of how I learned it the hard way but I’ve learned it over time which is, you know, we should as a team and as product managers, my metric is optimized for speed of learning. Hopefully, minimize for embarrassment like that’s good too but optimize for speed of learning like how fast can I learn something in and really forcing yourself to be hypothesis driven you use that term being hypothesis driven. I love that. And that’s one of the guiding principles I’ve taken away from my experience.

Erika: I think for me the biggest problem that I’ve had with that is getting leadership on board. Getting them convinced that you can really do a pure MVP first and then iterate on that MVP seems to be the hardest thing to kind of get through to people and until you get to that point where you are moving quickly. So when you can like release really quickly and really break your features up into something that smaller and more and that’s great. But to get leadership kind of over that hump, “But we can’t release without X, Y and Z.” It’s like, “Well, actually you can.” And that’s kind of has been the biggest lesson as a product manager, just helping kind of teach and inform how that can happen and really working with your team to understand how you can break those pieces up.

Darren: I think that you touch on something that’s really good which I think is also a universal challenge. I’ve certainly experienced it. What do you do when you have a CEO or a member of leadership team that is really driving for something that the customers aren’t asking for? Or that there’s just no data to suggest that it’s meaningful for customers like how do you deal with that?

Josh: I just I try to tease out of them and understand the context. Is it a pet project? Is it something they’re heard in the hallway? Is it a, you know, something that their daughter said to them in the car earlier? You know, feedback.

Darren: That’s amazing how often that’s the case.

Josh: It does. And so you have to parse through where they’re coming from, you know, how they got that idea, the seed of the idea in their head and to also what I do is remind them of the massive backlog of priorities zero things we’ve got to do. So where does this fall into the like if doing a, “Oh yeah, well it was just a suggestion.” When it’s like, you know, “This is just an idea.” When they say that, it takes up a lot of time to answer those sort.

Darren: So could I interpret that to say that you’re calling their bluff? Like in a way like, “Hey would you rather fix this crash issue?”

Josh: I try not to… It’s really about sorting out in their mind where it fits for them and let them empower them to make that decision relative to the other stuff we got going on.

Erika: Go ahead.

Vasantha: Okay I would just add in terms of understanding the request, figure out what’s driving the request and I think most often, you know, maybe the specific request isn’t something that is in line with kind of the road map already but if you kind of work backwards to the root of kind of the motivation behind it you’ll probably find something that is. And then sort of work to either say, “Hey, you know what, that’s actually the same idea that we’ve got coming in a month.” Or maybe adjust your plans to sort of incorporate this idea and kind of bring it. I generally don’t think there’s any new ideas like if they’re all there in your backlog somewhere.

Josh: Orthogonal sometimes but yeah. Part of it or more than you’re doing or whatever.

Vasantha: Yeah. And so there’s, you know, there’s probably a way to massage it into something that’s kind of already on plan or, you know, that needs a change of plan.

Darren: I think that’s a good point too. One of the things that saved my life a number of times is adopting Agile approach to development where you can actually say, “Look this isn’t me making the decision of whether to do this or not. Now we have a plan of record that we adhere to and the plan of record is the authority. So great idea, yes we’re gonna do this. We’re just not gonna do it right now. We’re gonna put it in the queue. We’re gonna get to it maybe in the next brand.” The good news is if you’re a sprinter coming up in one or two weeks or even three-week cycles it’s not that far away to give somebody a date.

Josh: Or do, you know, what we do as if it really comes up and it’s something we need to vet out and prioritize we validate that with user research. We run some surveys. And before, you know, as a concept and if it comes back as a high runner we’ll pay more attention to it. So…

Darren: Yeah, I like that.

Erika: We actually, so we used at Providence. We implemented a series of surveys. Right when I kinda came on board and to try to understand what our users actually wanted and it’s, from the back is laughing, she’s on my team. It’s drastically changed our roadmap because the one thing. So our app is a pregnancy and postnatal app for mom so it’s got all the baby trackers, all the pregnancy information you want, a ton of content about pediatrics and your kids and it connects to your medical record. So the biggest thing users want is to be able to message their doctor and have them message them back in the app and that can happen via My Chart pretty easily. And that was not on our roadmap. I mean it was but like, we were talking like end of next year, not even end of this year. So our roadmap’s changed now and we will be implementing that in the second half of this year. So we basically said, “Well this is what our customers,” I mean resoundingly it was like 80% of survey respondents said this is what we want.

And so it was really cool to see how it influenced it. I was gonna say something really quick about the leadership thing though.

Darren: Oh yeah, sure.

Erika: So, my favorite, first of all I love hack days like I think they’re great because I love to see what the teams come up with. But my favorite thing is like when leadership after a hack day they see all the ideas and they’re like, “Well, so that code works, right? Like we could roll that out right now.”

Darren: Ship it baby. Ship it.

Erika: We’re not shipping that. We’ve had that before. I have had twice more rogue developers like literally snuck something into the app where I was like, “Well, you know.”

Josh: It’ll be a good movie

Erika: It’s good. Right?

Daren: Yeah, it’d be a good movie. We got to talk before this, about Easter eggs. I freaking love Easter eggs.

Erika: Oh me too.

Darren: Yeah. Like I think that they should be encouraged.

Josh: If other people are aware they’re going out.

Darien: Oh I see.

Josh: I was a developer for a while, way back in the day for cheaptickets.com and I had a situation where I was testing new real time credit card processing that was out. That’s how far away that was and fraud detection. So I was rolling out the code and I pushed out the code with emoticons at that point, smiley faces.

Darren: Oh I love this story. This is great.

Josh: So if it passed the credit card authorization it had a smiley face for the call center reps so they could see it as they’re working with the customer or a frowny face or red face if it was fraud detection. But I forgot to include as part of my test harness on any text to go along with that. So 800 call center reps over five call centers as soon as I push out the code we’re seeing just a smiley face or frowny face or red face and didn’t know what to do with it.

Darren: I loved that.

Josh: That was a whoops.

Erika: How quickly did that get back up?

Josh: It was nice… Yeah, pretty quickly.

Darren: Yeah definitely, definitely. I’m doing a quick time check. I said I would open it up for questions. We have a little bit of time left. I’ll open it up to you guys, any questions from the group any interesting challenges you want insight on?

Audience: Two questions, two questions.

Darren: All right. Yeah.

Audience: Has there ever been a point where you are going throughout this whole process and you like, royally messed up and there’s like a point of no return?

Darren: Okay so the question was, has there ever been a time where you’ve gone through the process of development you figure out you royally screwed up and there’s sort of past the point of no return and then, probably yes I think, but then the question is then what do you do about it right? I’m panting on that one.

Josh: Roll forward.

Erika: Yeah roll forward. I think that’s the biggest thing I’ve had to sort of teach leadership as well is like with mobile apps there’s not a real rollback. Right? Like you’re back-end can kind of pick up some slack depending on how you’ve developed it. But it’s a roll forward plan. That’s why my forced-upgrade is key guys.

Darren: Yes. That’s a good point. It ties right into that.

Erika: But, you know…

Vasantha: Depends on if you’re in production mode.

Erika: That’s true.

Vasantha: If it never goes out, then…

Erika: Then you’re fine. I’ve been in a situation now where I have… It’s been the point of no return. And it might not be because of functionally it’s not working it might be, “Oh crap, this is really like users are gonna hate this.” And it’s been to the point you’re so far in that development cycle that we have finished it out. I don’t think I’ll do that now. If I was in the same situation I think I would maybe make a different decision because I would want that time to actually focus on something that was valuable.

Darren: I think that takes…you have to experience that to learn that lesson, don’t you?

Erika: Yeah.

Darren: Yeah I was thinking kind of along those same lines. Roll forward but then iterate fast like, “what can we do quickly to get beyond this because it’s gonna suck for about a week or two.” right? So what do we do to minimize that pain point that pain period? Anything else.

Audience: Thank you.

Darren: All right.