“Accelerate – The State of DevOps Report” by Dr. Nicole Forsgren and the folks at DORA (now part of Google) is far and away the most widely cited research in the DevOps field. Dr. Nicole and her team have brought scientific rigor to the survey over these past six years and the results show it.
Here is your chance to shape the future of the DevOps market by taking 25 minutes of your time and take this year’s state of DevOps survey. These reports are only as good as the info they gather.
So. have a listen to our conversation and then please go take the survey!
As usual, the streaming audio is immediately below, followed by the transcript of our conversation.
Transcript
Alan Shimel: Hey, everyone. This is Alan Shimel, DevOps.com, and you’re listening to another DevOps Chat. I’m really happy to be joined by one of my good friends in the DevOps industry today, Dr. Nicole Forsgren, founder of DORA, DevOps Research and Assessment and also now with Google, right, who acquired DORA. Dr. Nicole.
Nicole Forsgren: Yes. Thank you so much. I’m so excited to be here.
Shimel: I’m excited to have you here. So, you know, I know what – your title at DORA was very easy, CEO and cofounder, but share with us your title at Google now with the audience.
Forsgren: Yes. So now – it’s actually pretty similar to what I was doing at DORA. So now I am doing research and strategy with Google Cloud and I’m still leading all of the dope, fun, amazing DORA work, so DORA is still continuing. We’re still doing all of this exciting external industry amazing work that’s DORA. And actually, that’s what we get to talk about today, right?
Shimel: Absolutely. So, you know, March 14th was five years we’re publishing DevOps.com, and over that time, you know, a lot of water under the bridge, but there are certain foundations that the DevOps community – and I don’t mean just practitioners, I mean the vendors, I mean the evangelists, the leaders, the followers, the laggards, the early adopters – there are just certain foundations that we’ve built this on and one of them is the State of DevOps Survey that you’ve been doing now I’m gonna guess it’s six or seven years, right?
Forsgren: This is – yeah, this is year six.
Shimel: Yes, six years, and –
Forsgren: It’s a kid. [Laughs]
Shimel: It still is, but you know what?
Forsgren: It’s a small child now. [Laughs]
Shimel: Let me see if I remember because I don’t have notes in front of me, Nicole, but over the six years you’ve collected was it 25,000 or 27,000 responses, something like that?
Forsgren: Yeah, yeah, we’re right in there.
Shimel: Yep. And what’s great is over these six years you’ve disseminated so much of the information, and that information and those results of the survey really form the basis for so much of the Dev – of what we think we know anyway about DevOps and Agile and Lean and all of these things that, you know, come together. So much of what the whole thing is premised on comes from the results of this survey, so congratulations on that and thank you.
Forsgren: Thank you. Oh, yes of course. I love it because it helps us understand what we do that’s important and impactful, and it backs up so much that, frankly, some of us know and have done for a decade now, right? So I started so much of this back – you know, I’ve chatted about this, right? – back in my dissertation work about a decade ago, and even so much of what I did as an engineer and as a sysadmin, even before that, but as we know as good little scientists and good little experimenters, right, we need more than just intuition. We need data.
We need to know what’s working and what’s working consistently in part so that we can keep doing it and sometimes so that we can convince people, whether it’s convincing ourselves, whether it’s convincing leaders, whether it’s convincing our teams so that we can say this is what we’re gonna do and this is what we’re gonna keep doing. And so, yeah, sometimes people will read the book Accelerate, you know, that I wrote with Jez and Gene, or they’ll read the State of DevOps reports and they’ll say, “Oh, well yeah, I knew that.” Well that’s great because so often that’s where we get our hypotheses from, right? We’ll reach out to the community, we’ll go to conferences, we’ll speak with engineers, we’ll check Twitter, we’ll read our emails, we’ll speak with so many people in the community and we’ll say, “What are you hearing, what are you seeing, what do you think is working, what’s not working?”
And we’ll look at the things that are like consistently working or maybe there’s mixed signal and we’ll say, “Okay, these are the hypothesis we want to test,” because that’s – like that’s how you do science, right? You form a hypothesis and then you go to test it.
Shimel: So you just said the magic word – science – because, you know, it’s interesting, there’s something – depending who you believe, there’s 400 to 700, 800 vendors in the DevOps space, and I’m gonna bet half of them, maybe more than half, do some sort of – excuse language – but some sort of half-ass survey, right? Where they’ll put out a survey on SurveyMonkey and they ask questions, they get answers, and I am constantly bombarded by PR agencies who say, “You know, our survey found 42 percent of people did this, 36 percent of people did that, blah, blah, blah, blah.” There’s no real science behind these surveys, right? The questions are done in – it’s like political surveys, right?
Forsgren: Right. Yeah.
Shimel: They tell what the answers are – they – what answers they want, and they –
Forsgren: Because they want to support their own product.
Shimel: Of course.
Forsgren: They want to support an agenda, and I understand why. I mean I get it.
Shimel: You know, they’re _____ their own socks, as we say, [laughs] but–
Forsgren: Yeah.
Shimel: – I think one of the things, you know, why you’re sitting here at Google right now, right, and why you’ve have the success you’ve had is that there’s some academic – what’s the word I’m looking for?
Forsgren: Rigor.
Shimel: Rigor. That’s the word I’m looking for. There’s some academic rigor to this survey and I think that is a big reason for its success. I mean, Nicole, I’m going all over the world and people will throw out, you know, stuff to me, and I say, “Where did you get that number?” They say, “Oh, the State of DevOps Report 2018, 2017,” you know, and I’m like, “Oh, okay. Pretty cool.” You know, so it’s a amazing thing. But of course –
Forsgren: Well, and I think it’s because we don’t have anything to sell.
Shimel: Exactly.
Forsgren: Or let me say the thing I have to sell is better outcomes, and I am happy to be convinced at any time with data and with results. If my–
Shimel: And that’s science.
Forsgren: That’s science. If I get my mind changed – and there have been a couple times when my mind is changed and I am reconvinced of another position because we want better outcomes. This work is born of my experience in the early 2000’s. I entered technology in ’98, ’99, 2000 in – and I’ve said this before – in the dot.com bust, and it’s interesting because people are like, “But the dot.com bust didn’t happen, right, the world didn’t end, you know, the Y2K bug didn’t end the world,” but what did happen is some systems failed, some companies failed, so what happened?
Companies fail, people lost their jobs, families were torn apart. We’re seeing some of that trickle down again, so if we can improve outcomes, if we can make companies more successful, deliver value better, if we can make work better, if we can decrease burnout, if we can make work/life balance better, if we can make pushing code easier so we’re not doing seven-day forced marches – ’cause I did that for two years.
Shimel: Right.
Forsgren: If we can make work better we all win.
Shimel: Absolutely.
Forsgren: Everything gets better.
Shimel: Well, and not only those folks who were slinging code and stuff, the world wins because, you know, I was just – right before I got on this podcast with you I was doing a video with another guest and, you know, we were talking about Charlemagne scribes. I don’t know if you know anything about this, right?
Forsgren: Yes. Well, let’s catch everyone up.
Shimel: He was illiterate, Charlemagne, so he counted on his scribes to tell him – well, the Bible was kind of the only book they had, but he counted on his scribes to get him the intelligence, to make him smart, to show him the way, to let other people know what he was doing and all of these things. Today, technologists, developers, and DevOps folks and stuff like this, those of us in the tech world, we’re sort of Charlemagne’s scribes, right? It’s almost within our hands to make the world a better place. And I don’t want to overemphasis it, but it’s true.
Forsgren: But if we can make things more accessible for more people and more places across broader populations –
Shimel: Yeah. No, even just something –
[Crosstalk]
Forsgren: – it really changes the conversation.
Shimel: – as simple as the black hole picture last week, right? I’m sure you looked at it.
Forsgren: Of course.
Shimel: I mean what a great story this is, right? I’m not even gonna get into the idiots who were giving this young woman a hard time, that she really wasn’t the driving force, but what a great – I mean to be able to sit at our desk and peer into what a black hole looks like, the inside of – you know, that’s what it all comes down to, but we can’t do those algorithms, we can’t do that data without the science, right? Science feeds science. But anyway, Nicole, a lot of people have asked me, you know, with DORA now acquired by Google, are they doing the survey? Is it gonna be the same? What is the story? Tell our audience.
Forsgren: Yes. Yes. Okay, so the survey’s still happening and it’s happening now.
Shimel: Okay.
Forsgren: So for anyone who hasn’t taken the survey it’s open right now. It’s about 25 minutes, and right, everyone’s like, “Girl, 25 minutes,” but it’s because it’s that academically rigorous research. So what are we studying?
Shimel: But haven’t they always been about 20, 25 minutes?
Forsgren: It’s always been about 20, 25.
Shimel: Yeah.
Forsgren: And I’ll still love everyone who does it. I’ll come, like, hang out with you and, like, selfie with you and give you a hug or give you like an elbow bump, whatever you want to do.
Shimel: Well, we might be talking thousands and tens of thousands of people, Nicole. You know, _____.
Forsgren: That’s fine, we’ll hang, we’ll hang.
Shimel: Let’s talk about the survey, though. First of all, is there like an easy URL we can talk to people? I’m gonna put the written URL in our show notes, but do you have an easy-to-remember URL that people can _____?
Forsgren: Yes, yes. So it’s bit.ly/2019-ASODR, A-S-O-D-R, ’cause – for accelerate State of DevOps Report.
Shimel: Accelerate State of DevOps Report, ASODR.
Forsgren: That’s – so bit.ly/2019-ASODR.
Shimel: So bit –
Forsgren: nd anyone who’s in tech – well, anyone can go take it, but I would love to hear from our devs, our test, or ops.
Shimel: Security people.
Forsgren: Yeah, yeah.
Shimel: Don’t forget our security DevSecOps people.
Forsgren: Oh sure, Infosec, absolutely.
Shimel: So, Nicole –
Forsgren: And we’re digging into cool stuff this year.
Shimel: That’s what I wanted to ask you. What’s different this year?
Forsgren: We say it every year. I know. I get excited every year. But we’re really digging into cool stuff. So I reached out to some extra experts this year, so automated testing and testing in a slightly different way, right, because testing’s changing. The way we think about testing is changing. We’re also looking at how to build effective deployment pipelines a little bit differently. Disaster recovery practices, right?
Shimel: Yeah.
Forsgren: As our systems get more distributed and more complex, disaster recovery is super important. Things like our technical environment; what does that look like, right? So, so many things. I can’t say all of it. Productivity.
Shimel: So let me – give me a timeframe. Timeframes, Nicole.
Forsgren: Okay. So the survey’s open until May 3rd.
Shimel: We got –
Forsgren: And I know everyone’s like, “I got plenty of time.” You got – I mean kind of.
Shimel: No, go do it now.
Forsgren: A couple weeks. Do it now. And then the survey comes out in August, like mid-August.
Shimel: Oh.
Forsgren: And I know everyone’s like, “That takes a minute,” but it takes a minute to do science. It’s a lot of stats and then writing and then I gotta make it pretty because –
Shimel: Was that always the time? I seem to remember it coming out around DES in US, around like October.
Forsgren: So it always takes about six months. It used to come out June-ish and then –
Shimel: Oh, maybe it was DES London.
Forsgren: Yeah, it was DES London, and then last year we shifted a little bit to fall because I just think it took me a minute to pull together research, design it –
Shimel: Oh, it takes time.
Forsgren: _____ in December, because what I – here’s the thing: I love to hear all of the feedback from the whole fall conference schedule. It gives me a better chance to really get great signal from what everyone’s hearing and what everyone’s saying.
Shimel: Yep.
Forsgren: And then I start pulling together all of my research design in January and February and then we launch in March, so it takes about six months to pull together a really, really good research study.
Shimel: Look, all this is good by me. I was expecting October for DES, so I just – you know, I just picked up two months.
Forsgren: Yeah, yeah.
Shimel: So it’s good by me. Alright. And now Google, of course, is obviously sponsoring it again this year, Google Cloud, with your –
Forsgren: They are – yep. So we’re partnering with them this year like we did last year.
Shimel: Yep.
Forsgren: And then we have lots – we have got a handful of other sponsors just like we do all the time, right?
Shimel: That’s what’s changed in this regard.
Forsgren: Because this is such a great industry report, right?
Shimel: Yeah.
Forsgren: And so we’ve got a handful of wonderful – and it’s still under wraps – but we have amazing, wonderful industry partners and sponsors.
Shimel: Oh, good.
Forsgren: Like we always have, right? So it’s still the DORA Report, it’s still – you know, we still have the DORA team behind it, and I’m super excited to share the results with everybody in August.
Shimel: So we’re excited to read about it. So let me – I don’t know if you can say this or not, but will we see a second Accelerate book?
Forsgren: [Gasps] Probably. Well, because the first book was based on four years of research.
Shimel: Right.
Forsgren: And then we had a 2018 study come out that had some incredible findings like cloud, right? Everyone’s like, “Okay, cloud, but does cloud matter?” And some people are like, “Okay –” science, right? Here’s a great example of science. Some people are like, “Of course cloud matters. Cloud makes a huge difference.” And then there’s some groups that are like, “We made these huge investments in cloud, but I’m not seeing the benefits of cloud.” So here’s what we found: Cloud matters but only if you do it right.
Shimel: Yeah.
Forsgren: Some people are like, “What do you mean do it right?” If you follow the five essential characteristics of cloud computing as defined by NIST, the National Institute of Standards and Technology, we see that you’re 23 times more likely to be an elite performer. See? Stuff like that is super important because that’s really what helps you tease out the differences.
Shimel: I wish I would’ve had this in my – you know, Nicole, I was in Iceland last week at a conference – a small one, 50 – more of a roundtable, 50 people, all high execs at large enterprises, and we – and there was someone giving a presentation there about, you know, advantages, disadvantages, cloud, hybrid cloud, et cetera, and man, I could’ve used some of these metrics there because –
Forsgren: But see that’s what helps you tease out yes it makes a difference but only if you’re actually doing it right. You can’t skip steps. You have to execute. And then you can go back to an exec team who says things like, “We made investments, it doesn’t work.” Okay, but did you execute all the way? Did you execute fully? Are you so early in your transition that you’re just not seeing the benefits yet but they will come; don’t give up.
Shimel: Well, this is – I have to say I’m arguing with people with DevOps who tell me their DevOps failed or whatever.
Forsgren: Oh, absolutely.
Shimel: DevOps only fails when you stop doing DevOps and –
Forsgren: We see the same thing with CI/CD, right? We’ll see teams do a continuous integration initiative and then they’ll redefine it so narrowly to make sure that they hit their target, but then what happens is if you go reevaluate – I think that’s the benefit of the research, is because we have to do it rigorously we end up identifying _____ exactly what it means to do CI. You have to hit four things, which gives you an evaluative criteria. So then you can go back and you can say, “We’re doing CI but I’m not seeing benefits.” Okay, go back to the book. We outline it for you. Are you doing all these four things? And you could say, “Oh, we’ll we’re only doing two.” Okay, that’s why you’re not seeing the benefit.
Shimel: [Inaudible comment]
Forsgren: You have to have all four.
Shimel: But Nicole, this is an excellent reason why people need to buy the book. It makes a big difference.
Forsgren: Yes, and this is also an excellent reason, by the way, please take the survey because –
Shimel: Yes.
Forsgren: – what helps me understand which things are predictive of performance, which things are not, and then you can turn back around. The report is free. It’s free to download. Anyone can go back through and you can see, okay, absolutely these are the things we should continue doing, these are the things that are not helpful and not beneficial. So, by the way, I super – like I love and appreciate anyone who helps contribute to all of this research, but we do pay it forward. We absolutely contribute to the community.
Shimel: Absolutely do, and I think our entire industry should pay it forward by participating. Nicole, as I mentioned to you before we started, this is only a 15, 20-minute interview. We’re close to the half hour, but –
Forsgren: [Laughs] Sorry. Thanks, everyone. I love this community.
Shimel: Absolutely. But let us end with this then. Please go take this survey. It’ll take 20, 25 minutes, and the URL to go take it is bit.ly, which is, of course, B-I-T dot L-Y, slash 2019 slash A for accelerate –
Forsgren: Dash, ASODR. Yep.
Shimel: ASODR.
Forsgren: Sorry, bit.ly/2019-ASODR.
Shimel: ASODR, spell that for us.
Forsgren: A-S-O-D-R.
Shimel: ‘Cause, you know, I had – I’m from New York. We have a tough time with R’s at the end of words.
Forsgren: I know.
Shimel: Like brother, sister, and all that. I can’t help it.
Anyway, Dr. Nicole Forsgren, thanks so much for being our guest. Guys, gals, listening; go take the survey. Nicole, hopefully – I guess I’ll see you in London next perhaps and –
Forsgren: Yes, yes, for sure.
Shimel: Unless maybe at KubeCon in Barcelona is very nice, but I don’t know if you’ll be there. If not, DES London.
Forsgren: Absolutely.
Shimel: Alright. Dr. Nicole Forsgren, CEO and founder of DORA, big-time person at Google Cloud as well, and the – kind of the life and breath of the State of DevOps Survey. Thanks for being our guest on this DevOps Chat.
Forsgren: Thanks so much.
Shimel: Alright. This is Alan Shimel and you’ve just listened to another DevOps Chat.
