Complicated and Competitive: How Vertica Navigates Today’s Complex Database Management World

There’s a lot of tension in the world of database management, and it’s not for the faint-hearted.

Vertica Systems Inc. has played in a high-stakes poker game that, over its 15-year history, has involved the world’s three largest public cloud providers, critical enterprise data migration decisions offsite networks and two major US presidential campaigns. Big bets have big consequences, and that’s where the tension comes in.

Vertica, now part of Micro Focus International PLC, launched its technology in 2005 to address scaling and performance issues with traditional enterprise data warehouses, a year before Amazon.com Inc. introduced its public cloud.

As Amazon’s cloud has become a disruptive colossus for businesses, it has also developed its own cloud-native database. Red shift. This competition spawned competition, including from Snowflake Inc., and Vertica was left with a key decision to make. Should it join the cloud-native side or stay firmly rooted in the on-premises world?

The answer came in 2018 when Vertica introduces Eon mode to support running on Amazon EC2 compute and S3 storage. It was the first data store which enabled compute and storage to scale independently for cloud and on-premises workloads.

To make things more interesting, Snowflake and Vertica quickly struck deals with Amazon Web Services Inc., Microsoft Corp. and Google LLC to integrate services, although the three major cloud providers had their own internal data warehousing platforms.

AWS had Redshift, Microsoft ran Azure SQL data warehouseand Google offered BigQuery. This created tension, which the CEO of Snowflake Frank Slootman readily acknowledged.

“It’s inevitable for us to challenge these first-generation cloud products,” Slootman said, during a interview with Diginomica at the company’s Summit event in October. “Does this create tension? Yes. We’re competing against them, and that creates a very complex dynamic, and depending on the day of the week, things can get a little tricky.

Cloud native or hybrid

Vertica must not only compete with Snowflake, but also fight for market share against its own partners’ products. This has put the company at the center of a debate within the enterprise IT community: should data-driven workloads stay on-premises, migrate wholesale to the cloud, or run in both? ?

Recent survey data from the Cloud Native Computing Foundation is showing signs of moving towards a hybrid solution. While 62% preferred public cloud, that percentage was down from 77% the previous year. Now 38% prefer a hybrid model, and the CNCF said it “fully expects this trend to continue”.

Fortunately for Vertica, its decision to launch Eon Mode two years ago has positioned the company to capitalize on the growing interest in hybrid. The technology’s ability to separate compute from storage has proven invaluable to enterprises that depend on rapid elasticity to meet changing workloads.

A good example of this can be found at The trading post, an online ad placement company that bids on hundreds of thousands of ad auctions per second, reaching billions of users worldwide. The trading post broke Rackspace Inc.’s cloud twice and was more than happy to do the same for Eon mode when Vertica was beta testing it.

“Vertica said, ‘Are you ready to break it for us?’ And you’re like, ‘No problem! I can do it!’ Dave Pickleschief technology officer and co-founder of The Trade Desk during a interview Last year. “We broke 25 versions of it. But they kept dialing the number and it went really well in the end, so we’re really happy with how it turned out.

Beyond integration

Breaking systems over and over again can create its own particular form of tension, but there is a method to Vertica’s madness. With Eon Mode, the company recognizes that developers want to go beyond building apps that simply run efficiently on a cloud platform.

The real value comes when solutions are so ingrained in the design process that developers can take full advantage of the range of cloud services on offer. Mere integration is not enough. The database management solution should be an architecture designed from the ground up to operate at maximum efficiency in the cloud.

This critical distinction has been emphasized by Ben Vandiverformer technical director of Vertica and the person mainly responsible for the design and deployment of Eon Mode.

“With Eon Mode, Vertica moves from simple integration with cloud services to introducing a core architecture optimized specifically for the cloud so customers can capitalize on the economics of separating compute and storage,” Vandiver said during a interview in 2018.

The separation of compute and storage was a key breakthrough for Vertica, as it provided enterprise customers with granular scaling. If they ran out of storage space, customers didn’t have to buy unnecessary compute to solve the problem.

Eon Mode was the first data store that enabled independent scaling of compute and storage in both on-premises and cloud environments, but Snowflake delivered it early in cloud-native environments, while like Google with BigQuery and Amazon with its RA3 nodes for Redshift.

Campaign issues

This competitive tension between partners has spilled over into an unlikely but important space: national politics. In 2012, Vertica played a crucial role in President Barack Obama’s successful re-election effort when he was selected as the primary campaign analytics database management system.

However, according to a description published in Wired, the data operation for Hillary Clinton’s 2016 campaign experienced numerous information technology issues, some of which were attributed by DNC staff to Vertica. In 2019, the Democratic National Committee replaced Vertica with a database driven by Google’s BigQuery.

A person familiar with the Democratic campaign operation in 2016 told SiliconANGLE that the party did not significantly patch or upgrade its hardware after Obama’s re-election in 2012. Vertica’s software was expected to run on a system that had not been upgraded for four years, according to this source.

The problems were also compounded by the departure of many key figures from the IT team that ran the database operation for Obama in 2012, according to a reliable source intimately familiar with Project Vertica at the time, who spoke to SiliconANGLE.

Vertica’s experience highlights a simple reality: Businesses aren’t the only ones who need database management technology in a cloud-centric world. AWS Now powers websites for the Federal Election Commission, the Republican National Committee, and the DNC, in addition to providing services to state and country election offices in over 40 states.

There’s a lot of tension around data these days. Is it useful or irrelevant, protected or unprotected, too little or too much? Vertica is in the eye of the hurricane, and there is nowhere else its ruler would rather be.

“If you look at the disruption that’s happening with data, companies that compete with data aren’t just outperforming everyone else in their space, in some cases they’re reinventing and unlocking massive new opportunities,” said Colin Mahonysenior vice president and general manager of Vertica, during a interview in December. “Vertica is really at the heart of this disruption.”

Image: Pixabay

Show your support for our mission by joining our Cube Club and our Cube Event community of experts. Join the community that includes Amazon Web Services and Amazon.com CEO Andy Jassy, ​​Dell Technologies Founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many other luminaries and experts.

Maria H. Underwood