Cockroach Labs CEO: Diverse database models are essential as app demands surge

Licensing mixes also needed lest vendors give too much away

Interview On the eve of the Cockroach Labs' annual conference, CEO Spencer Kimball commented about the virtues of a rival database company. "There are real benefits to the way MongoDB does things, and that's why a lot of developers like it," he told The Register from RoachFest in New York.

The annual shindig saw the database company – which counts Netflix, Comcast, and delivery service Shipt as customers – launch its 24.2 release, featuring new vector search capabilities compatible with PostgreSQL's pgvector extension.

Cockroach Labs produces the CockroachDB. The relational system boasts a near PostgreSQL-compatible front end with a distributed back end, in DBaaS and serverless forms, supporting vector search and the JSON document format. The company claims the distributed back end can make it resilient and consistent on a global scale.

Despite recent CockroachDB developments, Kimball said the rapidly growing database market supports a range of approaches, including NoSQL systems such as the document-oriented MongoDB.

"Developers prefer it; it just makes those early stages of application development really breezy, and that's a benefit. If you're building the back end for a mobile application, [MongoDB] is a really good choice," he said.

The comments might be welcomed by MongoDB, whose CEO Dev Ittycheria claimed that its recent success had partly been down to competitive wins against PostgreSQL.

But they come with significant caveats. While in the same database market, MongoDB and CockroachDB rarely compete on use cases, he said.

"We almost never see MongoDB in a real bake-off, because the customers come to the table with a workload or use case, and they'll be very clear what they can do with Mongo, or something that they need to do with Cockroach or another relational database. Typically, that has to do with how transactionally consistent the workload needs to be, and how much complexity it will have over time."

Kimball argued that one of the unheralded benefits of relational databases is that they allow complex use cases to develop over five to ten years.

"Because they have an explicit schema that's denormalized, they're forcing you to think about your data model constantly upfront."

While databases such as MongoDB might be easier to develop applications on to begin with, there is a price to pay in the long term, he said.

"With something like a document store, you're throwing documents in there, and that's super convenient at first, but there is a cumulative tax. The first year, you might have all those documents looking very similar, then you make some changes, and things start to diverge, and you start putting conditionals in your application code… If it's this new version, then I have to look at this sub-document, for example. It starts to become complex."

While debate has raged about whether relational systems can absorb developments in the NoSQL movement, Kimball does not see the two approaches converging.

"Anyone claiming that either there's going to be complete convergence, or that there'll be some dominant modality that just displaces the others without subsuming their functionality... I think both of those views are overwrought. There are extremely strong products that are innovating because applications demand more and more from the databases. It's an arms race, so to try to do everything, well, you can't be at the forefront of the arms race."

Last month, Cockroach Labs changed its forward-looking licensing model. After 24.3, the company is introducing enterprise licensing for customers with more than $10 million in annual revenues. Customers under that get CockroachDB Enterprise Free, while the source code remains available for all to see.

Cockroach had used MariaDB's Business Source License (BSL) 1.1 as the basis for its approach, allowing users to run the Core product on as many nodes as they wanted, so long as they did not commercialize it. Cloud-based database services remain unchanged.

Critics said the "Enterprise Edition" creates a sales-oriented approach in which developers are "squeezed by the vendor." Others criticized that the initial open source approach was a tempting teaser for developers, "and then when investors want revenue growth, the rug gets pulled."

Kimball said he didn't see inconsistencies in Cockroach Labs' licensing. Although it has changed, the aim has always been to "build everything we can to allow a startup to scale and succeed without having the friction of dealing with paying us the way that you'd have to buy Oracle and that sort of thing."

"For quite some time, the open-core model seemed to work. And then things got heated up between Elastic and AWS, which caused us to move to the BSL," he said.

However, as the core database was able to achieve greater scale and put more cores into production, the largest customers were able to use the free product for their applications, and eliminate significant cost, he said.

"That free rider problem looked very threatening on our horizon. In some ways, what we've hewed to is our first principles: we want to make sure that startups, and what we've defined it now as is companies with under $10 million in revenue, don't have to pay us, and they get the full enterprise version.

"We're finding a path that maintains many of the things we felt were most important about open source without being overly zealous about the prerogatives of open source, which have in many ways been challenged just by how the whole ecosystem has evolved, and especially the move to cloud services and huge cloud platforms, able to offer free software that other people have written without really contributing back to it."

But open source remains a vital approach supporting important elements of database engineering and enterprise data infrastructure, he said.

As UC Berkeley students, Kimball and co-founder and CTO Peter Mattis developed the popular open source image editing software GNU Image Manipulation Program (GIMP, since forked as Glimpse, primarily to offer the software under an alternative name).

While cloud-based enterprise-facing services are likely to adhere to the proprietary model to give customers that "aha" moment without having to think about the back end, behind-the-scenes open source projects will have an important role.

"Where open source is unquestionably the dominant model is components, like all the libraries," Kimball said. "Cockroach relies on many open source libraries in order to build our product, and we release a lot of our non-Cockroach [software] as open source."

For example, Cockroach Labs has written a replacement for RocksDB, the key-value database Pebble.

"Pebble is an open source product that we put out there, and many other companies have found it extremely useful," Kimball said. "We benefit from that open source component ecosystem, which is massive and continuing to grow, and we also contribute to it."

No doubt, the debate will continue to rage over where to draw the line between products that remain open source and those that require more restrictive licenses to build a business. Popular database vendors such as Cockroach Labs believe they can succeed in operating on both sides. ®