ScyllaDB 5.0 is set to advance NoSQL database capabilities

ScyllaDB held its annual Scylla Summit 2022 event on February 9-10, showcasing a major next release of the open-source NoSQL database.

As was the case with Scylla Summit 2021, the event was virtual. The open-source NoSQL database vendor is now continuing its efforts to add new features to its namesake ScyllaDB database for on-premises and cloud users.

During the conference, ScyllaDB revealed details of its new open source 5.0 update, which is currently in the development stage of release candidate 1. The vendor did not provide a timeline for the general availability of version 5.0, although it is common for open source projects to go through several release candidate stages before a final release.

ScyllaDB positions its platform as a direct replacement for the Apache Cassandra database, which is how many organizations are initially trying out the open source database.

This is the case of global online retailer Rakuten. During a user session on February 9, Hitesh Shah, Head of Engineering at Rakuten, detailed how the company is using ScyllaDB to power its global catalog after transitioning from Cassandra.

How Rakuten Uses ScyllaDB NoSQL Database

Rakuten maintains a global catalog that derives product information from multiple data sources aggregated into the company’s e-commerce platform. When researching a product, Rakuten users navigate to the global catalog to find comparative product information, including pricing.

ScyllaDB is at the heart of [Rakuten] Platform.

Hitech ShahDirector of Engineering, Rakuten

Source data from different vendors is ingested into the Rakuten system’s processing engine, which validates and normalizes the data so that it is all in the same format. The data pipeline uses Kafka as well as Spark. At the end of the data ingestion process, the data all lands in ScyllaDB.

“ScyllaDB is at the heart of the platform,” Shah said.

In terms of data processing volume, Shah said the catalog contains more than 700 million items. The ScyllaDB system now also powers data reads at up to 15,000 requests per second for each database node.

Rakuten didn’t start on ScyllaDB, but ran on Cassandra first, then migrated to ScyllaDB two years ago.

Shah said the main reason for the migration was to enable better performance and easier management of the retailer’s growing data needs. When running Cassandra, Rakuten required 21 database nodes; meanwhile, ScyllaDB could handle the same job, with better performance, using 6 nodes.

“One of the biggest pluses was that it was just an immediate replacement,” Shah said. “So we could basically create a new ScyllaDB cluster, migrate our data, and start pointing our application to ScyllaDB, and boom, we were running in production.”

Hitesh Shah, Head of Engineering at Rakuten, explained how his organization’s global catalog is built with ScyllaDB at its core.

New features coming in open source ScyllaDB 5.0

In the February 9 keynote, ScyllaDB co-founder and CEO Dor Laor outlined the direction the vendor is taking with the new ScyllaDB 5.0 open source database as well as a new cloud service coming this year.

The big change for ScyllaDB in the cloud is a new Database as a Service (DBaaS) that will provide a consumption-based model for database services.

ScyllaDB’s current cloud service runs on AWS and GCP with several different options for workload sizing. Laor said the new DBaaS will be more elastic in terms of scalability as workloads grow or shrink. He did not specify when the ScyllaDB DBaaS would be available.

Laor also discussed a number of features coming in the ScyllaDB update later this year. Laor said ScyllaDB 5.0 will put a heavy emphasis on performance improvements to speed up database read and write operations.

Another advancement comes with support for the Raft consensus algorithm to enable strong data consistency between database nodes. Raft has become increasingly popular in recent years and is also used inside the Kubernetes container orchestration system.

Laor noted that Raft will allow ScyllaDB to be more elastic and scalable while providing immediate data consistency rather than eventual consistency.

“We have an extremely strong foundation for ScyllaDB with Raft,” Laor said.

Maria H. Underwood