DevOps Chat: Scaling Data Without Complexity with Cockroach Labs

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, this is Alan Shimel from DevOps.com, and you’re listening to another DevOps Chat. On DevOps Chat today, I’m really happy to be joined by Spencer Kimball. Spencer is CEO/co-founder of Cockroach Labs—hot company in the DB and DevOps space. Spencer, welcome.

Spencer Kimball: Thank you, Alan. It’s a pleasure to be here.

Shimel: Absolutely. So, Spencer, let’s get it out of the way. First, I’m assuming most of our listeners have already heard of Cockroach Labs, but it’s possible they haven’t. Why don’t you give us a little background?

Kimball: Well, sure. I mean, first, we should probably get the obvious question out of the way—why would we name something Cockroach? I think that’ll become clear when I give you the explanation, but Cockroach Labs was founded to build a database, and that database replicates and it’s hard to kill, so that’s where the name came from. Honestly, the name was invented before the product and long before it was a company. So, it was—you know, now I’m in a position of explaining that same thing to Fortune 500 CIOs on a weekly basis. [Laughter] There’s some regret there, but the name makes sense.

So, I can give you a little bit of the background. All three co-founders of Cockroach Labs worked at Google for about 10 years in the office and Google went through Bigtable which started the NoSQL movement, and it quickly moved to Megastore, which introduced some transactionality. Then they created Spanner, and Spanner is something that Google’s been working on for more than a decade now.

When we left Google, we realized Spanner is a solution to many companies’ problems, not just Google’s. And that’s become more and more true in the years since. We did a startup, and even as a startup, we wanted capabilities that Google had built into Spanner, and those weren’t available in open source. So, that’s sort of where the original idea of Cockroach came from. That startup got acquired by Square, the payments processing company, and many other things now as well. And Square needed something like Spanner as well.

So, the idea kind of had gained currency. We started as a GitHub project. That took off, VCs got interested. The company was started, now, four and a half years later, we’re busy selling to the Global 2000, Fortune 500 big companies out there as well as competing on one side with Oracle and on the other side with Amazon.

Shimel: Only on the Internet, man, you know? [Laughter] It’s a crazy story, but true. So, first of all, I absolutely get the Cockroach thing, because they are gonna inherit the earth, you know, or so it seems, especially with the path we’re heading, but let’s not get into politics.

But it is kind of an amazing story, because when—for people who are in the IT business and are somewhat familiar with the history of the dashboard and how Larry Ellison came to be the second richest man in the world or whatever it is he is these days, you know, it’s fascinating that all of everything we do online, almost, is predicated on the ability to store the data and to access that data in ways that allow us to do all of, you know, manipulate the data, analyze the data. I mean, without databases, none of that happens.

And so, when you look at the history of the database and then the, you know, to relational databases and then you mentioned a few other things, SQL database and then NoSQL databases—

Kimball: And back to SQL. [Laughter]

Shimel: Right, and then back to SQL, then you’ve got time stamped databases are a thing now. You know, all revolving around, though, how do we store this data in a way that’s easy to put the data in there and easy for us to access that data, manipulate it, analyze it, you know, and make value out of it, right?

And so, much like a cockroach, data doesn’t die, either, right? [Laughter] It’s eating the world, you know? Where was I recently, I was shopping at Costco or something, and they had eight terabyte, I guess, backup or drives, you know, external drives that were maybe a little bigger than credit cards, you know? Size of a good iPhone, let’s say, a little thicker. And I think it was $169.00 for eight terabytes. [Laughter] And I’m thinking, “My God, you know, when I had those 3.5 inch floppy disks, man, and the bigger ones”—

Kimball: And the first 20 megabyte hard drive, it was—

Shimel: Yeah! I didn’t think there were eight terabytes in the whole world.

Kimball: And there weren’t for a while, right?

Shimel: Yeah, it’s crazy. And, you know, as our data storage and data needs have grown, so has our databases. So, you mentioned the spinoff from Google, you mentioned certain things that the latest, greatest from Google didn’t do and hence, why Cockroach.

But you guys recently also announced a new funding round, if I’m not mistaken, right?

Kimball: Correct, right. We’re going to—that will happen on August 6th.

Shimel: Well, by the time people are listening to this, it’ll be August 6th already, so.

Kimball: Got ya, okay.

Shimel: If you’re listening to this at home, it’s after August 6, they’ve announced their funding round.

Kimball: Yeah, so, we did close that, and it’s a big round. And basically, the additional money equals all the money we’ve raised in the past which, you know, isn’t so unusual, but it’s a lot of money. In aggregate, we’re talking over $100 million.

Shimel: Wow.

Kimball: You know, I think, to put that into perspective, you just have to look at who we’re competing with, right? I mean, certainly, it’s some of the biggest companies in the world, the biggest companies in the world, and we’re also serving the biggest companies in the world, so you do need that money both to create a sales force that’s capable of educating and reaching and selling to those companies, which all—I’ve been humbled and amazed by the process necessary to actually sell to a Fortune 500 company. In some cases, it’s quite involved.

But also, to continue to iterate on the product, and we have to, you just need to hire the best people and create an environment where they can collaborate to move the product forward so that you can compete.

Shimel: Absolutely. But, you know, look—in the Internet and the IT world today, compete means different things to different people, right? We compete by offering great service. You know, I just got done, I was over at DevOps Enterprise Summit in London a month or two ago, and there was this notion there of what folks were calling BVSSH—Better Value Sooner, Safer, Happier, right? And that’s what we try to deliver in DevOps and stuff like that.

So, does competing on that and—but then, in the database world, too, it’s just really, sometimes it’s very, very, very feature driven.

Kimball: Yeah.

Shimel: Right?

Kimball: It’s incremental features, and then there are sort of generational features. And we’re, we’ve been pushing on truly changing the nature of the database, and that is an opportunity that exists because of all the other things happening in the ecosystem—in particular, the global public cloud, which is, I think, one way to describe the combination of AWS and GCP and Alibaba Cloud and Azure. There’s many others, of course, that are far more regional. But in aggregate, all of those can be utilized by new database architectures to allow a company to store data in new ways that ____ their business versus their competitors.

So, just to give you an example, things like GDPR and recent regulations around privacy and Russia and China and Vietnam and South Korea and South America—the list goes on—there’s a lot of attention being placed on how data is localized, where it’s stored in relation to the customer, where it’s processed.

And also—and this is, I think, really exciting—there is an ability, because of this global public cloud, for new services to come online that are able to deliver the end to end user experience that happens instantaneously to the user. So, that means in under 100 milliseconds, which is sort of the user threshold for noticing time intervals. If something feels instantaneous, but it happens in less than 100 milliseconds, you literally, because of physics and the speed of light, you can’t do that for a customer in Australia if all your data is in one data center in Virginia. But that’s the model that’s prevalent right now.

Shimel: Sure is.

Kimball: [Cross talk] on public cloud and with a solution like Cockroach, you’re actually able to have a globally spanning database that’s able to domicile data near the customer and actually deliver both the assurance of some data privacy, that their data is in process and stored close to them in their legal jurisdiction, but also to have an end to end user experience where you click a button on your mobile app, it goes all the way through the application logic, accesses the database, comes all the way back and updates. So, that ability to make that instantaneous, I think, is gonna open a new era of applications.

Shimel: It sure is. You know, so, I remember, Spencer, back—so, one of the companies during the dot com days, one of the companies I helped build was a company called, it came to be known as Interliant, and we were an early ASP application service provider.

But before we changed our name, we were Sage Networks, and we were a large hosting provider, a rollup. We had bought a lot of different hosting companies. And one of the key things for us, like, you know, what really brought us scale versus all these smaller hosting companies we had bought, is that all of these hosting companies’ storage and the data was sort of part and parcel with the same infrastructure, if you will, that was hosting the website and the web server and all of that.

And what we did is, you know, we actually—I don’t wanna say abstracted, because it wasn’t really abstract, but we separated the data and the database from the web server and the applications they were running, right? And now, that freed us up to store that data. This is before there was a cloud, but we had multiple data centers. So, we were at the point, almost, where we could store data, let’s say, in our Atlanta data center and access it or use it on websites where the web servers were stored, let’s say, in Tysons Corner in Virginia at the time. And what was holding us back there was, we didn’t have the connectivity you have now.

Kimball: Yeah.

Shimel: Right? If you had a T3 line, that was hot stuff, you know? That was carrier grade. And so, it was a little different. But, you know, this is the age we live in now where you do have distributed applications. So, of course, why wouldn’t you have distributed data storage and the databases that organize them?

Kimball: Yeah. I mentally have distributed audiences, right? I mean, every time a fledgling startup puts their first mobile app on the app store, the total global population that can have access to that, it’s pretty staggering. I mean, the opportunity there is just tremendous. But, you know, everyone’s—90 some-odd percent of efforts out there are focused on deploying to a single availability zone, right? So, this is, there’s a legacy of how things were built, which is rapidly changing, and you have to do it through the whole stack. At Cockroach Labs and CockroachDB, we’re attacking a fundamental part of that stack, very important, probably one of the harder parts, which is the data storage, the state behind those applications.

Shimel: Yep. No doubt about it. I wanted to talk a little bit about CockroachDB sort of as a project, if you will. Cloud native, right? When you guys were starting out, obviously, you came out of Google, you have a big open source kind of heritage at Google—what were the discussions then about, “Is this something we open source? Is it something we make free?” How do we walk that line, right? Go ahead.

Kimball: So, when I was at Google, from 2002 to 2012 with my co-founders, Google was only starting to do open source towards the end of that tenure. Previously, they were pretty adamant about keeping things closed source. It was just sort of the model that Google evolved with. Now, they’re much more open, and some of their more recent efforts, including TensorFlow and Kubernetes, are examples of where open source in the hands of Google actually is a tremendous advantage both to the company and to the larger ecosystem. But things like Spanner were closed source and many of the things that Spanner is built on are also closed source.

But what Google did do, which is great, is they always publish about these things.

Shimel: Right.

Kimball: When we started Cockroach, we had the benefit of not only working at Google and seeing how these things were built—you know, Peter, one of my co-founders, and myself worked on something called Colossus, which is the exoskeletal distributed file storage system at Google, which runs underneath Spanner.

So, we had a really great opportunity to see why and how Google built the things that they built. But, you know, once we were kinda kicked out of the Google nest—or we left it, I guess, is the better way to say it—

Shimel: Flew the nest. [Laughter]

Kimball: Yeah. We flew the coop, right? We had the benefit of all of the published work that Google had created around things like Spanner, but you know, out in the wider world, you realize, for a database—and when we started this, I guess it was 2014. In 2014, and it’s certainly true today, people aren’t going to pay attention to your database unless it’s open source. And it’s just, it’s sort of a fundamental requirement. And there have been databases that have tried as closed source software, but they haven’t gotten very far, to my knowledge.

So, I just think that—and this is true for me, too, as a potential user of the database, as a potential developer on the database, something that’s not open source feels a little bit icky. [Laughter] Right? Like, there’s not enough surface area that I can latch onto where I feel like I can be the master of my own fate, right?

Shimel: Yeah.

Kimball: Like, if something—that’s what I hated about closed source products. If something goes wrong, it’s just this big black box and, you know, it’s sort of a black hole if you wanna send a support ticket in, what’s gonna happen? Whereas it would be much nicer for everyone involved, in my opinion, if I were able to dig down in there, find out what was wrong, probably my fault, but let’s just say it’s a bug in the product, I could send that into the company and they could take that as a patch.

That’s the beauty of open source and that, as a personal preference and sort of even as an idealistic sort of mindset. I would not have been able to start the database as anything except for open source.

Shimel: Yeah. I mean, and look, quite frankly, there is a bit of a history in the database space of successful open source databases that not only were successful as open source projects, but were equally or more successful as commercial entities or the companies that manage these open source projects—you know, very successful open commercial entities that had great exits and great value for the investors and founders and employees.

So, I mean, in that regard, it’s not like you’re pioneering, right? It’s not like someone says, “Oh, they’re crazy to do this in open source,” right? Well, 15 years ago, they might have said you were crazy, right? [Laughter] You know—

Kimball: I think MySQL kind of, you know—

Shimel: Yep, ____.

Kimball: —blazed the trail, as they say. And Mongo is a very successful—

Shimel: Absolutely.

Kimball: [Cross talk] that uses open source as well. And there’s a number of other ones. But you know what, even when we started this company in 2015, I think the open core model and open source in general seemed like a catalyst, right? Just something that could create business value far more quickly than closed source software and the sort of procurement department and budgets and all this stuff that used to be involved in the cycle of adopting a new technology in order to build your products or services that the engineers or developers at a company needed to—

Shimel: Oh, absolutely. I mean, look, I’ve been in IT 30 years, right? I’ve seen the rise of open source. I was involved in open source security stuff in 2001, ’02, ’03, ’04. The deal is, today, for a lot of developers, if it’s not open, they don’t wanna work on it, number one.

Kimball: Yeah. But—

Shimel: Go ahead.

Kimball: What’s really fascinating is that 2015 might have been sort of the beginning of the end for open source being the true, the most important catalyst of business value. I would argue that, in 2019, the true catalyst to business value, faster business value—business value in sort of relation to your competitors, I think it’s services, right? Cloud-based services. That is—you know, it’s always about, it’s not about the idealism of open source. That’s not why it was successful.

Shimel: No, I think we’ve passed the idealistic stage of open source.

Kimball: It’s basically like, what are the tools out there that a software engineer at company X can use in order to get their project done faster so that it’s more stable, it’s more performant, it costs less, right? All those things—

Shimel: But it’s their preference, and these developers and the Ops and the DevOps folks, they’re the top predator on the food chain at a lot of these companies. They’re the hardest piece of the puzzle. And so, companies are hesitant to tell them, “No, I don’t want you to use that tool” or, “I don’t want you to use this service.” They’re the ones making these choices, where I think, in years past, it was like, “Oh, no, you can’t bring that in here” or, “You can’t use that one” or something like that.

Kimball: Yeah. It kinda [Cross talk].

Shimel: But there’s more, Spencer. Let me tell you this. You know, I’ve been going to a bunch of the Cloud Native cons and Kube cons and those kinds of shows, and I spend my life at IT conferences. I’m telling you that the open source community—and not because of the idealistic ____ and the bizarre and free as in freedom and free as in beer kind of crap.

We’re past that, you know, Dr. Stallman stage, if you will. It is the, that community is the energy driving almost everything we’re seeing in IT today, right? There’s such an energy in there that it’s driving what you’re doing at Cockroach, for sure. It’s driving what we do in DevOps. It’s driving what we see in Cloud Native and Kubernetes and all of these things. It’s where—it’s almost like this well of, like, a nuclear power plant or something, you know what I mean?

Kimball: Yeah, absolutely. And, you know, I think the best model right now that I’m aware of is, you know, it’s called cloud-based services with a very fast time to value, incremental pricing, right? So, you kinda pay for what you use as you go. If you can deliver that with an open source product that’s backing it that people feel that they can get into and they can use if they had to move off that sort of freedom and flexibility, that sort of offers the best of all worlds.

And I think that—and that’s why Amazon’s been so successful in adopting open source technologies, because they’re very good at providing things as a service and they have so many things altogether into one, you know, that can be packaged into one bill, one sort of monitoring platform, one identity access, you know, management platform. It’s very impressive, but, like that, in aggregate, creates so much value for business.

Shimel: It’s crazy. We’re running low on time, so let us bring this back to Cockroach Labs. We seem to have taken off there on a tangent. You guys are offering a managed version.

Kimball: That’s right. So, we have—some of our biggest customers, they wanna run the database themselves, it’s the way they’ve been running it, but over time, that will shift, of course. So, we have a self-service, self-hosted version where you buy subscription licenses if you wanna pay us for it. There’s also, of course, open core, which you can use for whatever purpose.

Then there’s the managed service offering, which we’re calling Cockroach Cloud. And that, we run it for you, and we can run that on AWS or GCP, and the pricing model is much more like something like DynamoDB or Amazon Aurora or MongoDB Atlas, those are sort of similar kinds of offerings.

Shimel: Got it. Excellent, man. Just, again, Spencer, for people who may not be familiar, the website is CockroachLabs.com.

Kimball: That’s right.

Shimel: So, that’s where people can go get more information, there. Any other news that we didn’t hit on, here?

Kimball: Well, you know, I think maybe we didn’t quite cover the funding well enough. I kinda jumped ahead, but we’re raising more than 15 million and the lead investor is Altimeter Capital.

Shimel: Great.

Kimball: So, there’s sort of growth equity added, along with Tiger Global, and then all of our existing investors, Google Ventures—GV is what they’re called now—Benchmark, FirstMark, Redpoint, Index Ventures. So, it’s a great—

Shimel: Impressive list.

Kimball: Yeah, a great group of investors. We’re looking forward to sort of this being an inflection point and really moving forward with our penetration into the Global 2000 and also fundamentally putting a lot of focus on managed services, so, the cloud delivered Cockroach experience, in addition to enabling more and more of this global architecture capability that I was talking about earlier.

Shimel: Absolutely. I mean, it takes resources today to make all of that happen and bring it all to market, obviously, and hiring good people is not—even with the money, it’s not an easy thing to do, as I’m sure you’re finding out. Finding good people is—

Kimball: A lot of these guys have—I liked what you said, at sort of the top of the food chain.

Shimel: Yep, the alpha predator in the thing. It’s crazy.

Kimball: They vote with their feet, right? So, it’s—

Shimel: Yeah, they do. I mean, they don’t even ask for raises, they just go get another job. It’s not—I mean, it’s a crazy, crazy dynamic at play there, but it is what it is.

Well, listen, this was—obviously, we blew past our 15 minute time, but it happens, it’s all good. I want to wish you success. I mean, just a quick digression from the point—you know, there’s a lot of people out here who are gonna listen to this, Spencer, who will say, “You know, I have a great idea for a company. We should go raise a few million bucks in capital, and we can do similar to what Cockroach is doing.” Not necessarily in the DB perhaps, but you know, in startup world, it’s not easy—$100,000,000.00 is a lot of frickin’ money for people to write checks to you for. And it doesn’t come without tremendous responsibility, and having raised a lot of money in my lifetime for venture backed companies, I know what goes into it.

So, congratulations on a successful run with this, and looking forward to seeing what you do with it.

Kimball: I appreciate that, Alan.

Shimel: No problem. Alright. Our guest was Spencer Kimball, CEO, co-founder of Cockroach Labs, makers of the CockroachDB, and this is Alan Shimel for DevOps.com. You’ve just listened to another DevOps Chat.

— Alan Shimel