Materialize brings streaming SQL database to the cloud

Streaming database startup Materialize today released a public preview of its cloud database-as-a-service offering.

Materialize, based in New York, was founded in 2019 and has spent the last few years developing its database platform, which provides a streaming data capability that allows users to perform SQL requests.

Materialize is able to handle streaming data sources including Apache Kafka. At the base of the platform is the Timely data flow open source stream data processing technology, which allows users to directly query stream data.

With the new Materialize Cloud offering, organizations will benefit from a managed cloud service to operate and manage the streaming SQL database.

Until now, users have deployed and managed Materialize on their own. One such organization is the online alcohol delivery platform drizzle.

We are still self-hosting Materialize, but we are considering potentially migrating to Materialize Cloud in the future.

Denis HumePersonnel Data Engineer, Drizly

“We are really excited about the Materialize Cloud offering,” said Denis Hume, staff data engineer at Drizly. “We still host Materialize as self-hosted, but we are considering potentially migrating to Materialize Cloud in the future.”

Use SQL streaming to improve e-commerce

Emily Hawkins, Head of Data Infrastructure at Drizly, explained that Drizly uses several tools within its data architecture, including a MySQL database, Snowflake for data analytics, Looker for business intelligence, dbt for data transformation and Confluent as a streaming platform.

One challenge has been how to best use the company’s streaming data for the business logic of its applications.

For example, a key issue that many eCommerce vendors face is cart abandonment. Drizly needed a real-time way to better react when a potential customer doesn’t close a sale.

Drizly uses Materialize to ingest incoming Kafka data streams. Hawkins said that with Materialize, Drizly enables event-driven data architecture. When a certain event is identified in the data stream, another series of actions can be triggered.

“If someone ‘adds to cart’ and doesn’t check out within 30 minutes, we call it an abandoned cart,” Hawkins explained. “We can get data from Materialize, and then that triggers our CRM tool to send some kind of notification to that user saying, ‘You didn’t pay, you forgot these items in your cart.'”

Hawkins said the fact that Materialize works with standard SQL, which his team members are already familiar with, is particularly valuable to his organization. As a result, it was faster and easier to get started with Materialize and grow its use for Drizly.

How Materialize works as a streaming SQL database

Although streaming data is a familiar concept to many thanks to Apache Kafka, Materialize CEO Arjun Narayan said that in many cases it is difficult to work with this data.

“Materialize is a database that provides incremental view updates for standard SQL in addition to fast-changing data streams,” Narayan said.

Timely Data Flow provides a materialized view of incoming data. With a materialized view, data is calculated in a format that can be queried with SQL.

For SQL queries, Materialize has built a PostgreSQLName– compatible layer so users can use the same SQL queries they would use with a PostgreSQL database for Materialize streaming data.

“What’s nice about it is that you get all the benefits of the next-gen stream processor, but from a user experience perspective, you can just take queries that run on PostgreSQL and run them on top of Timely Data Flow,” Narayan said.

Materialize Cloud provides users with a dashboard interface to manage multiple streaming SQL database instances.

Maria H. Underwood