Open source or proprietary database management

Sanjay Srivastava has to make a fundamental choice when designing and building data management environments: opt for open source data management technology or buy commercial options?

For Srivastava, who works with clients on such choices, it comes down to the role data plays for the business.

“If you were born in the cloud and data is a fundamental value and a driving force of your business, then go for open source,” said Srivastava, chief digital officer of business transformation services company Genpact. “But not if you are putting something in place in your own environment, and you want to run it for the long haul, and you are using data to augment and support your core business.”

Many enterprise data managers and IT managers find themselves in a similar situation: forced to choose between proprietary and open source software for their data management needs.

This is a choice that will likely be more frequent as the space expands.

Just consider the size of the market already and its expected growth. Grand View Research valued the global enterprise data management market at $ 72.8 billion in 2020, with an expected compound annual growth rate of 13.8% through 2028.

The high dollar value reflects the wide range of capabilities required for an enterprise data management program, as well as the many vendors and open source options on the market.

That, however, is at the heart of the challenge – the plethora of options. And, while Srivastava has a special way to help him decide whether to go proprietary or open source when building an organization’s data management program, he and other experts said every business needs to understand the benefits and challenges of these options and recognize that the choice between the two is not always clear.

On the contrary, there is a series of compromises that must be considered.

“Open source offers cost-effective alternatives to high-cost, off-the-shelf commercial products,” said Sandhya Balakrishnan, US region manager for smart business solutions at Brillio, a consultancy firm. in digital transformation. “However, the most common concerns with open source data management tools include security, lack of support, and often the hidden costs associated with installation and ongoing maintenance.”

On the other hand, “many commercial data management tools successfully protect the complexity of data management for users,” Balakrishnan added.

Pros and Cons of Proprietary Data Management Software

Proprietary software is software whose source code is not available to users; it is sold by business entities as out of the box solutions which may or may not be customizable to any degree.

Although its definition may seem too rigid, IT consultants have pointed out that proprietary data management systems offer several important benefits to organizations.

“The advantage is that even if it’s something you paid more for [than open source], it will work in the business for production, ”said William McKnight, president of McKnight Consulting Group.

Many vendors offer solutions with a full range of additional features. They include integrations so that business teams can build their data management environments faster and easier. And they add more automation.

“They are reliable. They can deliver high performance at scale, security, innovation and automation, ”added Noel Yuhanna, vice president and senior analyst at Forrester Research.

Organizations also benefit from vendor support when choosing proprietary data management technologies, and they typically find it easier to hire the talent needed to implement and maintain data management software. commercial data – especially the most commonly used – versus open source options.

These are important considerations for organizations looking to rapidly advance their use of data, the experts said.

“The ease of development and maintenance, along with the assurance of continuous support and improvements, gives large enterprises the opportunity to evolve by focusing on the right aspects of business architecture. ‘business,’ Balakrishnan said.

However, there are potential drawbacks to proprietary data management software, experts say.

Business teams can’t innovate on proprietary code and instead must rely on vendors to keep pace with the innovations needed to succeed in a rapidly changing digital landscape.

It costs more – especially in upfront costs – than open source options.

And there’s the possibility of getting stuck with one vendor, with the cost and challenges of switching to another vendor outweighing the benefits of the change.

Pros and Cons of Open Source Data Management Software

Unlike proprietary data management software, open source options are released under a license that allows users to deploy the code to develop their own systems and also update, modify, and modify it for their own needs. .

This flexibility allows for the creation of data management solutions that meet the unique needs of each organization, McKnight said.

“With open source, if you want, you can create your own forks in code. For some, that may matter,” he said.

Open source is also cheaper to use.

“It’s obviously good on the budget, so if money is a constraint, then you can go open source,” McKnight added.

Additionally, companies can generally test open source options more easily, allowing them to run a proof of concept or pilot before deploying open source more widely or even moving to paid / higher enterprise versions. expensive.

Leaders in IT and enterprise data may not need to choose between owner and open source – it doesn’t have to be an all-or-nothing approach. Instead, experts say they can use proprietary software for some needs and use open source for others.

In addition, open source allows business teams to innovate code and build on improvements that other users make.

“With the open source community, you have more people contributing to lines of code, so you’re going to get more innovation,” Srivastava said.

These advantages have led many IT and data leaders to turn to open source, Yuhanna added.

“The [COVID-19] The pandemic seemed to elevate the gaze to open source, ”he said. “What we’ve seen is that open source tools are definitely helping you lower your costs. This is one of the driving forces behind its adoption, but open source can also help you avoid vendor blockage and make your architecture sustainable. “

As is the case with proprietary software, open source has some possible drawbacks. Open source is generally more difficult to integrate than proprietary alternatives, Yuhanna said, adding that it usually takes more work to interoperate well.

“It’s a job that needs to be done on the spot,” he said.

Additionally, experts said organizations need technologists with the specialist skills required to create, deploy, maintain, and improve open source code. These technologists must keep abreast of all changes to the code, and they may be required by license to contribute to the open source community. They also need to be able to do all of this work without the 24/7 customer support that typically comes with commercial software products.

The future is a mix

Leaders in IT and enterprise data may not need to choose between owner and open source – it doesn’t have to be an all-or-nothing approach.

Instead, experts say they can use proprietary software for some needs and use open source for others. This, in fact, may be the optimal approach for many organizations.

“In our experience, best of breed is the norm with increasing acceptance of open source,” said Balakrishnan, noting that organizations could turn to commercial vendors for data movement and storage. , but use open source options, such as Apache Kafka or Apache Spark for near real-time data processing and Apache NiFi or Apache Airflow for orchestration or workflow management.

Others said organizations might want to use open source for pilots and proofs of concept and then move to business choices when scaling up.

Business and data IT leaders are also increasingly turning to solutions that essentially combine the two, said Brad Ptasienski, partner of digital consulting firm West Monroe and its market leader in engineering and data analysis.

Noting that West Monroe believes open source “is going to be the new standard and mainstream for large-scale data processing and storage,” Ptasienski said he sees a lot of positives in using solutions. wrapper businesses that have open source at their heart, but with some of the benefits of proprietary software that surround it.

“It’s almost a hybrid approach,” he added. “It works more like a platform, but it is the basis of open source.”

Maria H. Underwood