Does your database make you competitive?

Marius Huysamen, Big Data Technical Team Lead, Obsidian.

Data is essential to the competitiveness of your business, right? After all, we’re told time and time again: data enriches our understanding of our customers and our operations, creating insights we can use for research, development and strategy. Yet this logic may encourage companies to think that more data and better information are mutually exclusive, which is incorrect. You can sit on a lot of data and not get much out of your information stocks, especially if your databases don’t support your data strategies.

Data needs a database, and a database should be fit for purpose, says Marius Huysamen, Big Data technical team leader at software solutions design firm Obsidian: “Your database choices should be discussed as part of a data strategy tied to the business strategy. It’s not like the good old days where you got quotes from three big vendors and picked one. There’s more to it. Depending on your business needs, you need to choose the right database.”

New space for databases

Organizations are often reluctant to disrupt their databases. As these contain valuable data, they are treasures and the loss of a database is often devastating. But as we increasingly rely on data to power our applications and inform business processes, we need responsive databases.

Very often, the performance of an application (and therefore the subsequent impact on customer or user satisfaction) depends on the speed and flexibility of a database. It is therefore essential that business decision makers know why databases are important and how one type of database is not suitable for all situations.

We can generally divide databases into two camps, explains Huysamen: “There are many types of databases. But in most cases, you want to know the difference between relational and non-relational databases. Relational databases are what most people think of: they are structured in tables and ordered by linked relationships with other tables – something that can look a bit like a spreadsheet, although more intense and elaborate.Relational databases are great for organizing data, but they don’t don’t scale well because as your database grows, you need more horsepower to manage the different relationships.

“Non-relational databases don’t use structured tables or a predefined structure and you use something like JSON scripts to create and query entries. They can scale better and work well when you have a lot of unstructured data. But they are not so useful if you are working with defined datasets.”

Choose the best database

Non-relational databases, sometimes also called No-SQL, are very popular because they are well suited for large datasets and cloud-native use cases. This makes relational databases seem obsolete, but Huysamen disagrees: “There is always a need for relational databases. Most businesses have a front office system – perhaps an online retail store – and a back office, which will have sales data analysis and others. You can also use embedded databases on IoT devices. In most cases, companies will need at least two types of databases. It’s not just the centralized system.”

This is the conversation businesses need to have: to get the best performance, databases need to be fit for purpose. If you process too much data for your relational database, it will become heavy and require more server power. But if you need to support structured datasets, non-relational databases will require more effort than necessary.

Some modern databases mix the two models, offering relational features that can evolve further, and hybrid database ecosystems are also gaining ground: for example, using non-relational databases with database languages relational such as SQL, or vice versa. Beyond that, there are other choices such as graph databases or cache databases, each performing specific tasks in a data value chain.

All of these choices can be daunting, which may be why companies don’t want to mess with their databases. Huysamen notes that one must be diligent; your IT infrastructure will become more complicated as you add databases and start duplicating data. He concludes : “Databases are a technical subject, but make the effort to find out where they fit into your data strategy and business needs. If you treat them as an afterthought or a static asset, you can end up with more complexity and worse performance. And also think about control; whether you’re a tenant, partner or owner to get a database, make sure you stay in control of your data and don’t get locked in.”

Maria H. Underwood