The Cost of Bad Mobile Data
In the film Momento, Leonard (played by Guy Pierce) is on a mission to avenge the violent murder of his wife. The only trouble is that Leonard suffers from short-term memory loss. He can’t remember what happened to him more than 15 minutes prior. Instead, he relies on informational tattoos and his habitual note scribbling. His inability to retain information severely hampers his efforts, leads to him being manipulated by acquaintances, and (spoiler alert) eventually results in him killing several men who are in no way connected to his wife’s death.
Put concisely, Leonard had bad data.
In the same regard, mobile data and data retention is vital for decision making. Without data-driven analysis, mobile app owners can end up like Leonard; angry, frustrated, and stampeding around, for the sole sake of action, with little hope of any progress. Bad data and poorly design data storage is also expensive to manage and even more expensive to fix.
In the most basic of analogies, data is like a house’s foundation. Problems there, lead to problems throughout the structure and the most severe problems can be impossible to resolve without redoing the foundation itself. This article explores the cost of having bad data and the ways it can affect your mobile app solutions.
Bad data leads to bad code
One of the forms “bad data” can take is poorly structured databases. Most modern databases should follow a very well-defined set of rules which, when followed, result in data that is organized, easy to navigate, and properly relates data items to each other within a larger data set. When data is laid out in this manner, it is more likely to result in clean and manageable code. This is because so much of a project’s function is inserting data into and pulling data out of a database. If data is poorly structured, this data access code becomes increasingly complex and error prone.
Another common problem faced by such code bases is that this complex data access code is copied in multiple places. This results in bugs and can make seemingly simple fixes take quite some time. The longer an app has been built on top of a poorly designed database, the worse things get and the more expensive it will be to eventually fix.
As this article will continue to explore bad data can be extremely expensive, but also keep in mind that bad code, caused by bad data, can be even worse.
Bad data is expensive to fix
Have you ever owned a dog that had fleas? To the uninitiated, you have to clean EVERYTHING. Clean 99% of your apartment, but missed a single throw pillow? Guess who is cleaning 99% + 1% of their apartment again. Similarly, you have to eradicate bad data or it will slowly seep its way back into your software.
For this reason, fixing bad data will almost certainly be a large, difficult, and expensive project. Since it is unrealistic to discard existing data, this is typically the process involved with a data clean up project.
A data project typically looks something like this:
- Assess the damage
- Find and document all the existing data.
- Analyze and document the current shortcomings of the data.
- This process begins by designing your ideal database to store this data. Try to avoid making sacrifices to accommodate the existing bad data.
- This typically requires code that will only be used as part of this migration process.
- There is likely some poorly written code that was only written to accommodate the old database. Make sure this code is improved or at the very least identified and documented. The last thing you want is for some other developer to copy this bad code.
This process can easily take weeks or months, depending on the size and state of your data.
Another challenge of data reclamation projects is that they are very difficulty to justify to a project owner, because they do not appear as attractive as building a new feature or fixing known issues. To the unaware observer, nothing in the application changed. The payoff of this sort of project is seen over the longer term.
Bad data makes staff irreplaceable
Bad data can be extremely difficult to understand and work with. For this reason, projects that have a long history of bad data likely have nearly irreplaceable employees. Personally, I have inherited several projects where only one person fully understood how portions of their data worked.
When a company is in this situation, these staff members can be virtually irreplaceable, which is no good for scaling. Additionally, it can take an extremely long time to bring on other resources to a project in this state. If your data was in better shape, you would be able to see contributions from new staff substantially sooner. This is one of the biggest hidden costs of bad data.
Bad data makes reporting error prone and expensive
I have worked on several projects where our development team has designed a database independently, then months later, we learn some other business unit has been generating reports off of our data. Often, these groups are able to build fairly complex reports without even talking to us about the data. They were able to make sense of the schema on their own.
If data is malformed or related in some nonsensical manner, it can be incredibly unclear what is going on. That is why bad data requires code to tell its story, while good data speaks for itself.
Bad data, and specifically bad data that requires code to be intelligible, is especially expensive because more code might have to be written for any reporting to work. In these situations, anywhere the data is accessed, this “cleansing layer” of code will have to be reused or replicated. This can quickly become a nightmare to manage and is extremely error prone.
Bad (or lack of) data leads to uninformed decisions
Not only does bad data cost you money in terms of the efforts involved in fixing or working around it, but it also limits your ability to capitalize on opportunities which you could have, had you been more informed.
One of the most interesting and important parts of developing a mobile app actually happens once the app is out in the world. So often we think an app is going to be used in one way, or by a certain demographic, but it turns out we could not have been more wrong. Without collecting, analyzing, and understanding our data, we would have continued to make decisions based on our false assumptions for years!
Having bad data can you leave you completely in the dark when deciding what portions of your app need to be extended, reevaluated, or potentially removed. It can mean you don’t know what products to list at the top of screens, which labels are harming your conversion rates, and whether anyone is actually using that expensive feature or not.
The important point to take away here is that bad data can increase your costs, but it can also severely hamper your profits.
Bad data is almost everywhere. Nine times out of ten, when we inherit databases there are severe problems. Often this is at large, multi-million dollar companies, but it does still affect smaller businesses.
If your company is looking for a major leg up on the competition, having clean and well organized data might be exactly what you need. If you are starting out now, PLEASE take the few extra days of effort needed to properly structure your data and don’t take shortcuts during development. If your data should be altered to accommodate changes in your application, change them! Your future self, your development team, and your bottom line will thank you.