DevOps Chats: What Is Your Data Really Worth? with Splunk

Transcript

Mitch Ashley: Hi, everyone, this is Mitch Ashley with DevOps.com, and you’re listening to another DevOps Chat podcast. Today, I’m joined by Andi Mann, chief technology advocate with Splunk, and our topic is a report that just recently released called, “What Is Your Data Really Worth?” The ultimate question, you know? It’s kind of, from, do you remember—what’s the, 42? Is that the answer? Okay, anyway, another topic. [Laughter]

So, welcome, Andi—good to have you on DevOps Chat.

Andi Mann: Hi, Mitch. It’s so good to be with you again.

Ashley: Absolutely. Andi and I are friends from way back, so we’ve known each other for quite some time now. [Laughter]

Mann: Yeah, this could get a bit squirrelly, mate. Let’s see how we go. [Laughter]

Ashley: I made sure I didn’t have too much caffeine before we recorded, so hopefully that will kinda keep me in check, anyway, we’ll see. [Laughter]

Mann: [Laughter]

Ashley: So, for folks who don’t know you, tell us a little bit about what you do and what you do at Splunk and a little bit of your background.

Mann: Sure. So, Mitch, I’m chief technology advocate at Splunk for our IT Markets Group. What that means, as an advocate, I spend my time explaining stuff to other people and advocating for technology mostly. So, I talk a lot. A lot of people will see me doing podcasts like this, they’ll see me at conferences and keynotes, writing, and other things. A lot of outbound talking about Splunk and our data delivery platform and all the great solutions we’ve got to bring data to everything.

But a lot of people don’t see the other side of it, which is me advocating for my customers. It’s one of Splunk’s core values that we have two ears and one mouth, and we should try and use them at least in that proportion. So, I try to listen a lot to experts like yourself, like other analysts and pundits, customers, market makers, and try and help Splunk create the best possible products and solutions to make our customers successful.

Ashley: Sounds like when I was a kid, the saying was, “God gave you two ears for a reason, Mitch.” [Laughter]

Mann: [Laughter] Exactly, right? Oh, you weren’t the only one who got told that as a kid, huh?

Ashley: No, I’m guilty as charged. [Laughter] So, let’s just jump to the report. Tell us a little bit about, obviously, you’re in the data aggregation analysis collection, all kinds of interesting parts of the data world and lots of different sources. How did you decide to try to answer the question of what’s your data really worth? What was the genesis of this report?

Mann: Well, the first thing we did was, we decided to find an independent expert to help us. So, we went to Enterprise Strategy Group. Now, you know them, they’re an analyst firm out of Boston.

Ashley: Yeah. Good folks, good folks, yeah.

Mann: Really good folks. Their key areas where they focus on is data and security. And so, for the data part of this survey, that was a very obvious choice. So, they hope to figure out a bunch of questions to ask to see if companies were using their data in advanced ways or not. So, we were looking at things like how much data they use, where do they get their data sources from, which business departments use data in their decision making and a whole bunch of other outcomes.

And we were able to figure out from that with working with ESG, within Enterprise Strategy Group, we were able to figure out a certain percentage, around about 10% of the survey respondents out of 1,300 respondents were what we call data innovators. So, they were taking more data, they were using it in more deliberate ways, they were using it in more business departments and more business decisions.

And so, we were able to figure out, well, if they’re using data in better ways, what are the good things that happen when you do that, and comparatively, for the companies that are using data in sophisticated ways, what are the down sides for them?

So, it’s a really interesting set of questions and answers that we managed to find some really interesting data on.

Ashley: Great question to ask. So, tell us, what was the number one thing that jumped out to you? What was the learning you didn’t expect to get from this study?

Mann: So, I think the, one of the learnings that I think we did expect to get was that using data better helps you in your business, in all sorts of ways. So, the one big area I think I was a little bit surprised at was that using data better is not just gonna help you save money, it’s gonna help you make money. So, for these data innovators, on average, they had a profitability of around 12% gross, across different sizes of businesses as well. That meant about $38 million average total gross profit for these innovative organizations who are collecting, managing, and analyzing data to improve their business.

I think the second thing that surprised me was that this wasn’t even split between top line and bottom line. You know, in IT especially, we often look at ourselves as a cost center. We’re often told to do more with less. We’re often told to find ways to save. What we don’t hear a lot about IT is how important it is to making money, to adding revenue, but that’s what we’ve found from this research. Data innovators added on average over 5% to their annual revenue because they were using data better, and that added to reduction of cost of around 5% as well.

So that’s how we get up to that 10 to 12% on the bottom line, but it’s a combo of making more money and saving money, which is—you know, there’s not a lot of technologies which you can point to for that.

Ashley: Definitely both sides of the line, now. There was 5.32% over 12 months as a result of their data use. Talk about—how do you define a data innovator?

Mann: So, some of the things we looked at for the data innovators—are they more or less sophisticated in their strategy? So, do they have a data strategy, do they have a chief data officer? Do they have specific plans over a 12 to 24 month period of how they’re gonna get more data in and use that data? Do they have analytics programs in place? Do they have a data science program in place? You know, these are some of the signals that we looked at to see what they were able to do with data.

And then in terms of the outcomes, we looked at things like revenue growth, we looked at operational cost reduction, the ability to innovate. How long does it take to get new products or new ideas to market? We looked at outcomes as well, like customer satisfaction, customer retention, ability to make faster decisions.

So, all of this sort of gave us this picture of what does a data innovator look like?

Ashley: Now, just to give folks an idea here, this isn’t like baseball, everybody gets an award and gets to be called a data innovator. This is 11% of global organizations was your measurement. So, it is kinda top 11% kinda cream of the crop of folks.

I found another interesting stat that I read in the report that said one in five data innovators generated more than 20% of their annual revenue from products and services developed in the past 24 months, compared to just 2% of data innovators. So, if you have really invested in analyzing, assessing, using, and applying data, that number is backing up what you said about—yes, it does add to topline growth.

Mann: Yeah. Because, I mean, when you’ve got data to go on, you can make these rapid decisions. If you’re talking about innovation in life, you obviously have written about innovation quite a lot and we’ve talked about it directly a couple of times and, you know, to innovate, you’ve got to be able to make fast decisions. You don’t try things out.

You know, typically, 95% of innovation will fail. That’s okay, as long as you fail fast, fail small, fail cheap, and fail forward. But if you don’t have the right data to make decisions, then all of a sudden, you’re in a data paralysis. You’re in a decision paralysis. You can’t make those rapid decisions. You end up having to get more information. You end up going with the loudest voice in the room, or you end up going with the loudest person in the room which, by the way, tends to work against some of the greatness of diversity and inclusion about getting different opinions, about getting different, diverse opinions and viewpoints.

Ashley: Mm-hmm.

Mann: So, if you—but if you have data, the data speaks for itself, and you can set gates for innovation. There’s a classic innovation theory—you try stuff out, you do it small, you set gates, you pass the gate.

One example, for example, and directly for this audience as well, is thinking about what is a high quality release? Is it a release that has no bugs? Well, that might be too high a bar to get over, but to understand what level testing has it gone through, what is the pass/fail rate? What is the code quality? What is the compliance quality in your code? These sorts of things, they’re data points that you can then make decisions on.

Even more so, you can automate decisions based on data points. If I’ve passed 99.92% of my tests and I’ve run 100% of the tests that I expect, that’s a really good mark and I’m probably gonna go straight into production with it.

Ashley: Mm-hmm.

Mann: So, I can iterate faster, I can do new things in new ways, because I have surety that the decisions I’m making are real and based on substantive information that will matter when I get to prod.

So, innovation is absolutely a strong outcome that we see in this research as well, coming from these data innovators.

Ashley: Now, I kind of threw the trick question at you first. What didn’t you expect to learn from this? I mean, let me toss that one out there again, since we talked about some things you did expect to learn. Were there any small, medium, large surprises that you walked away from some of the answers that came out of the research?

Mann: Yeah, look, I think some of the surprising stuff was just sort of around the vertical side in the industries that were good and industries that were bad. I would’ve expected some other, some industries who are lower on the ability to get data insights, I would’ve expected them to be higher. Specifically, higher ed and public sector. They have a lot of access to data, they don’t necessarily have the same issues with data analytics and aggregation that private sector do. You know, they’ve got things in place around privacy and protection of data.

So, there’s some really positive things. Higher education especially, I would’ve thought, well, they have investigative units. Research is something they do. I was a little surprised by that. What we saw was that technology organizations do really well by using data better. I sort of got that, that makes sense. They’re involved in machine learning programs and AI programs and things like that. They’re on the cutting edge. Somewhere around two-thirds of financial organizations had really good results in terms of higher revenue through better utilization of data assets.

Ashley: Mm-hmm.

Mann: And again, financial organizations are often on the cutting edge of technology and so forth, so that made sense. But for me? Higher education and public sector were only down around 50% at this ability to use data in operational ways. And honestly, that surprised me. I think they could do a lot better. I think they’ve got the fundamentals, the people, the technology, the inquisitiveness and the opportunity, certainly, to be able to use data in better ways. So, yeah, I’d love to see those numbers come out better.

Ashley: Now, I wonder if, do you think there’s a correlation or a connection to this idea that those folks that were data innovators, their culture is, what you termed in this report, quote-unquote data obsessed. It was just data driven company, everything is—you know, decisions are driven by data, you know, collecting and using, it’s not just the loudest voices in your room. It seems like that’s a pretty high correlation there to the folks that are really in a place and are leveraging data in a very successful way.

Mann: Yeah, the cultural aspect is really interesting. You know, we’ve talked about culture in the DevOps community for a decade or more, and how important the cultural change is. And, you know, we know from DevOps you can throw all the tools at the problem. If you don’t have a culture of collaboration and sharing, then you won’t have a collaborative environment to work in regardless of what tools you throw out there.

Data is the same. The data’s there. The big difference between the data innovators and companies that aren’t necessarily as innovative with data, you know, the companies we call the data detractors and so on. One big difference is if they’re inquisitive about the data that exists and go looking for it and look for ways to use it. It’s not necessarily that they have more data, it’s not necessarily that they have different data, it’s not even necessarily that they have data that people don’t understand or understand better or worse. It’s that they have a culture that values data driven decisions.

And so, when the decision comes to the—you know, the meeting comes to a decision factor, they have people in that room who deliberately put their hand up and go, “What is the data saying?” Rather than having the people in that room go, “Okay, I think we’ve got everything we need. What do we all think?”

Ashley: Mm-hmm.

Mann: Right? And that’s a cultural change, Mitch. That’s a cultural difference. Having that data obsession as we’ve termed it in the report means that your culture is looking for data, actively looking to make those data driven decisions. Actively looking to get data from everywhere and bring it to every decision. Not just the important or less important or whatever, every decision. And that absolutely is a cultural difference.

Ashley: So, don’t take this—this is not a trick question at all. I’m interested or curious about your thoughts on data, data, data—extremely important, this report showed the value and the impact that can have. I’m thinking in an analysis standpoint, sometimes you can get into analysis paralysis or maybe the insights aren’t always just in the data, but from other factors and things.

How do you blend both the tools, the experience, the knowledge, the capabilities of the organization and infuse that with data in a really healthy way? What are your thoughts about that?

Mann: Yeah, look, that’s a super question, Mitch, because you know, there are lies, damned lies, and statistics, right?

Ashley: Wow. Yeah, we—

Mann: You can make data say whatever you want. [Laughter]

Ashley: Yes, we can make it say whatever we want—true.

Mann: So, you’ve gotta be careful about stuff, right? You’ve gotta be careful about bias, you know? I talked about diversity and inclusion a little bit. If all your algorithms are written by people that look and sound exactly like you, then they’re certainly gonna reflect who you are and what [Cross talk].

Ashley: Mm-hmm.

Mann: So, having a variety of data, but having a variety of algorithms created by a variety of people from different backgrounds—so, you’ve gotta have diversity in your teams. Again, it’s a cultural thing, right?

Ashley: Mm-hmm.

Mann: Your data will tell you what you want it to if you ask it. So, you’ve gotta have the ability to get more data, you’ve gotta have the ability to ask data, ask continual questions of your data. So, it’s not just the first answer. Typically, when you’re doing data analytics and inquisition, the first answer just pops up more questions for you. So, you’ve gotta be able to go through that iterative cycle of asking more questions. That’s a very fundamental and practical thing to do with how do you structure your data, what tools do you use to inquire after your data. You’ve also gotta have the understanding that some things are not necessarily a data decision. Some things actually don’t have data, and you do need to have personal experience.

I’m a big believer, Mitch, in letting the machines make the right call on stuff they’re good at. Complex data, time series data, high cardinality data, long periodic data—you know, humans are awful at things like pattern matching, we’re awful at looking at long term data about patterns. Machines are really good at that stuff. So, let machines—

Ashley: Also really good at large volumes of data where we, as humans, love a data point of one. You know, my kids—

Mann: Exactly. [Laughter]

Ashley: Those millennials … I have two data points at home, so I’m super set. [Laughter]

Mann: Oh, yeah. anecdata, yeah, it’s the bane of our existence, I think. But it’s important to understand that machines can’t make intuitive leaps, either.

Ashley: Mm-hmm.

Mann: Machines have no imagination whatsoever. Have you ever seen some of the Harry Potter scripts that have been written by machine learning engines?

Ashley: I have not.

Mann: Oh, my goodness—it’s so awful, Mitch. It’s so bad. [Laughter] Because machines have no imagination. So, there is a cutoff point, and it’s a fair question to ask, and I don’t think there’s any definitive answer of where that cut point is. But at some point, you need to have a human to interpret and bring imagination, bring intuition, bring experience to that data driven decision.

But I would certainly posit that you bring that human experience to the data, you don’t just go with a human gut feeling.

Ashley: Mm-hmm.

Mann: The data will tell you what to do.

Ashley: I don’t recall this as looked at or at least talked about in the report—I would imagine those data innovators have figured out where that balance is, it’s not just about having the most data or the data, the answer is always in the data, right? It’s that balance of, you know, it’s the right sources, it’s the negative validation as well as the positive validation, the correlation, the analytics, how statistically valid is the information. So, all kinds of things that data scientists know how to do that help you be really good about how to use that data, and of course, that’s probably a maturity curve that you work up to.

Mann: Yeah, exactly right. I mean, we see—and it’s not necessarily specifically in the data, but we absolutely looked at that maturity curve and what it means to be a data innovator versus a data adopter versus a data deliberator, someone who’s in the early phase, for example. We deliberately looked at what it was like as a company, what patterns. And again, in the DevOps community, they’re very familiar with this concept of patterns and anti-patterns.

Ashley: Mm-hmm.

Mann: And the patterns that the data innovators took gave us a sense for what is a mature business. You know, we’re not necessarily gonna be able to tell you exactly what those gates are that define good innovation or define a good test outcome, or define a good marketing campaign or whatever it is, a product that will be successful.

But what we can do is help you understand what the data told us about data maturity. And so, that’s why we’re actually working on, we actually loaded up onto the Splunk.com website a data maturity calculator. So, it’s a free—it’s a web based assessment tool, it’s free, obviously, so you can actually compare yourself against some of these data innovators.

You know, a really easy way to assess what’s your data use, what tools you need to get the most out of your data, what data you’re missing and how do you compare on this data maturity curve to be able to make those decisions between smart, experienced individuals and definitive or maybe not so definitive data and data driven decisions.

Ashley: Mm-hmm. Interesting. Great. Well, we’ve used our time pretty well here. Let’s certainly find out how do folks get this information? You talked about this assessment tool of kinda where are you on the data innovator curve—how about getting the report, information about it?

Mann: Yeah, absolutely. So, it’s all available up on Splunk.com. In fact, if you go there on the homepage right now, you will see the data-to-everything platform, and you’ll see links there to be able to get that video. You’ll be able to get, also, stories from some of our customers who have used data and turned data into delivery. And, you know, household names like Domino’s and others are really using data to create an impact, to be a data driven organization.

So, yeah, jump onto Splunk.com, you’ll be able to see the report there, you’ll be able to jump on, read the report. You can also take that assessment for yourself. It’s gonna be really fascinating to understand how you line up with those data innovators and where you can go to try and get a slice of that extra money that those innovators are getting.

Ashley: Great. Splunk.com, great place to go for lots of other information as well as this report. Say, by the way, before we end things, I hope you come back. I’d love to have a conversation with you about data in the DevOps world. One of the challenges—data has such a different nature, right, than—

Mann: Yeah.

Ashley: What we can do, you know, more flexibly with software and configuration automation and things around software, oftentimes developers sort of struggle with this amorphous, large piece of data or collections of data and how that’s evolving along with things like DevOps. So, I’d love to have a conversation about that.

Mann: Yeah, that would be a great conversation. Yeah, I would love that, Mitch. That’d be cool.

Ashley: Awesome! Well, hey, thanks a lot, Andi, for joining us today.

Mann: Thank you, Mitch. It’s a pleasure. Always great to talk to you, mate.

Ashley: Always great to talk with you. And I won’t talk about where Australia fell on the list of data innovators, but that’s another topic, so.

Mann: [Laughter] I know! Don’t—please, don’t! Thank you!

Ashley: Okay, alright. [Laughter] Well, you’ve listened to another DevOps Chat podcast. I’d like to thank my friend, colleague, someone I’ve known for quite a while, Andi Mann, chief technology advocate with Splunk for joining us. And thank you, of course—you, our listeners. This is Mitch Ashley with DevOps.com. You’ve listened to another DevOps Chat podcast. Be careful out there.

— Mitchell Ashley