DevOps Chat: AppSec and DevSecOps with Contrast Security’s Jeff Williams

I have known of Jeff Williams in the security industry for more than several years. He is a well-respected thought leader in AppSec and OWASP. I finally got a chance to catch up with Jeff and talk with him about Contrast Security, the company he co-founded and how it is helping.

In this DevOps Chat, Jeff and I discuss the state of application security and how AppSec fits into the larger DevOps realm. Jeff’s a great interview with lots to share.

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, and we’re here for another DevOps Chat. Today’s chat has a decidedly security bent. I’m happy to be joined by Jeff Williams, CTO and co-founder of Contrast Security. Jeff, welcome to DevOps Chat.

Jeff Williams: Thanks, Alan. Good to be here.

Shimel: Hey, my pleasure. So, Jeff, you know, as we were talking off mic, this is the first time we’re kinda interviewing anyone at Contrast. Why don’t we—I think it’s only fair we start off with sort of, let’s give our audience a little background. Who and what is Contrast Security?

Williams: Sure. So, I mean, you know that many enterprises struggle with application security. You know, there’s been an application security industry for like 20 years. I helped to start OWASP back in the early 2000s and I wrote the OWASP Top Ten, and organizations are still struggling with the same problems.

So, Contrast targets exactly those problems. We are kind of a radically different approach to doing application security. We’re very compatible with DevOps kinds of processes. And basically, we find vulnerabilities so you can shift left in the development and fix them. We analyze open source code, so you can make sure you’ve got safe versions of the components you’re using, and we protect applications in production. And we do it all through dynamic binary instrumentation. So, very much like a New Relic or an AppDynamics, but for security, not performance.

Shimel: Security prudent security monitoring—excellent. And Jeff, you mentioned, though I don’t know if we’ve spoken like this before, but I’m certainly aware of you for a long time within the security community. This isn’t your first rodeo. [Laughter] You were instrumental in OWASP. You know, we’ve seen application security not—it’s evolved, yes, but it’s still, at the end of the day, it’s about the application, right?

But yet, you know—look, it used to be, application security meant some kind of app firewall, which—

Williams: Yeah.

Shimel: – I’m not saying a WAF isn’t a great tool, it is. It’s changed the game in a lot of ways, I think, in what—

Williams: Not really. [Laughter] I don’t think so.

Shimel: Well, it’s made a lot of people a lot of money and it’s taken a lot of venture capital, let’s put it that way, right?

Williams: Well, that’s not my measure of success. [Laughter]

Shimel: Yeah. But has it made us safer?

Williams: Not really, no. No.

Shimel: But one can argue we’ve done a lot of things and not a lot of them have made us safer, right?

Williams: That’s right.

Shimel: You and I have both been in security for 15, 20–plus years, Jeff. And the latest buzzword is DevSecOps, and being that I’ve transitioned to DevOps five, six years ago, and I run DevOps.com and I’m involved in Security Boulevard, I’m all about DevSecOps, and I’m not here to say it’s snake oil or bull or anything like that. But as we were talking about, it means a lot of different things to a lot of different people.

Williams: That’s right, that’s right.

Shimel: Why [Cross talk], what do you think it means?

Williams: Well, you know, to me, DevSecOps is a way of viewing security the way that folks viewed software development through a DevOps lens.

Let me back up. So, software had a ton of problems 15, 10, 20—I was a developer for a long time back in the early ‘90s and so on. And, you know, software development back then had tons of problems. It was late, it was very error prone, long feedback cycles, slow release times. I worked on one government program for three years, and it never was fielded. [Laughter]

Shimel: Mm-hmm.

Williams: So, that kinda thing plagued software. But, I think, DevOps has really revolutionized that, you know, basically just focusing on getting work flowing, creating short feedback loops and creating a culture of innovation of learning. Like, those things transformed the way that software is produced. I love the factory analogies and all that.

Now, if you look at security, we have exactly the same problems. Security doesn’t really deliver a tangible result. Like, you never know when you’re done. So, the work isn’t defined. That means the work isn’t flowing. You know, we have these tremendously long feedback loops. I did a bunch of work for a major financial institution. They were on a system where some apps got a one year pen test, some apps got a two year pen test, and some apps got a triannual pen test. So, every three years, they would take a look at security. It’s crazy.

And, you know, so we need to make tighter feedback loops, and security has this culture problem. It’s not a culture of innovation and learning, it’s a culture of enforcing rules and compliance and gates and guards and bottlenecks. And that all can change. If we apply the lessons of DevOps to the problem of creating security in an enterprise, we can define the work, we can create shorter feedback loops, and we can create a culture of innovation and learning. That’s really the challenge. In my opinion, that’s why security continues to struggle is, it’s just not delivering what we want it to deliver.

Shimel: Yeah. You know, and—look, truth be told, Jeff, that’s what attracted me to DevOps. I thought it was an opportunity for us to kinda reshape the playing board and maybe change the rules and finally deliver—nothing’s perfect, and I don’t think anything is, I’ve seen enough so-called silver bullets to know none of them—

Williams: Sure.

Shimel: – [Cross talk] full of crap. But I thought, at least, it shifted the game pieces in such a way that we could, it was a more even playing field, or a better situational playing field for us.

Williams: Yeah. Done right, I think DevOps can have a tremendously transformative effect on organizations building software. I’ve seen it in a bunch of large enterprises and small companies, you know, and we’re a DevOps shop here at Contrast as well. You know, we’re deploying six, seven times a day, and I have a huge amount of confidence that that makes us deliver much better software.

And so, my challenge—

Shimel: Much better software, or much—but is it also more secure? Because shouldn’t that be—I mean, that’s something I strive for, right?

Williams: Right.

Shimel: That security becomes synonymous with quality.

Williams: Yeah. That’s right, that’s exactly right.

Shimel: Is it more secure?

Williams: Absolutely, it’s more secure. But you do need some better tools. So, I think, you know, to do security in a DevOps way, you need accurate, real time feedback on the code that you’re building. If you don’t have that, then you have to have experts involved. So, there’s a fundamental relationship. If your tools are producing inaccurate results, then you have to have experts to dig in and resolve the false positives and find the false negatives and, you know, it’s a ton of human expert work.

That’s not compatible with DevOps. So, you need these accurate tools in order to give real time feedback right to developers. But that’s where instrumentation comes in. If you can get inside the code that you want to test for security or protect against attacks, that’s where all the context is. That’s where the information is. And so, using all that information, you can make really accurate decisions about whether something’s a real, exploitable vulnerability, or whether something’s a real, viable attack. And then you can give that instant feedback, you can create tight feedback loops, and you can deliver security.

And I’ve seen it work. Companies have transformed themselves from the way they used to be with a big backlog of security vulnerabilities, they’re running scans every six weeks or six months or whatever. They’re piling up vulnerabilities, they’re not really fixing anything, and that’s a very unhealthy way to be. They transformed from there into an organization where they’ve worked off the backlog and now they’re in what I call the new normal. They’re just dealing with new vulnerabilities as they come in, as part of the normal workflow. And it’s massively more cost effective, because they save all that downstream work of tracking and classifying and risk creating and all this stuff. You know, it’s super expensive and doesn’t produce anybody any value.

Shimel: Yeah. I mean, just, even from the security guy’s point of view to work in that scenario where I don’t have that technical debt, if you wanna call it that—

Williams: It is—security debt.

Shimel: Security debt. That’s a great word for it, right? Without that hanging over my head, how much easier and better is my life? And it’s so true. And the other thing, Jeff, is—honestly, until you’re at that point, I don’t think you can talk about automation and velocity and speeding up this whole thing, because when you’ve got all that dreck hanging in your background, you know, you can’t do it, right? You can’t—

Williams: Well, that’s right. Traditional security is very destructive to modern software development. If you have to involve those bottlenecks and get experts involved, I’ve seen it just wreck DevOps—you know, organizations are trying to do DevOps, but they’re hamstrung by their legacy security teams and processes and technologies. And I wish it wasn’t that way, because security has potential to be a great force for innovation.

Shimel: Yeah.

Williams: If you can give people a fantastically secure platform, they can roll out awesome applications and awesome functionality at super high speed. Do things that normally you wouldn’t think you could do, but if you have the platform, you can really make crazy things secure.

Shimel: Absolutely.

Williams: It’s just—

Shimel: I mean, and, you know, I was talking to my friends, John Willis and Damon Edward, they’ve both presented at Dozen, and the notion is, you can have it all. You can have faster applications, better, more secure applications, higher quality—all three, right? You could have two out of three ________, you could have all three doing DevOps and DevSecOps if you’re doing it right.

But here’s another thing I want to touch on, Jeff, and that is, I don’t know whether it’s shiny trinket syndrome or single-mindedness or we can’t multitask, but just because someone says DevSecOps can do, can make your life better, can do all of the things that you and I are talking about doesn’t mean that we throw everything we learned about security out the door, right?

Williams: No, no—of course not.

Shimel: Well, for too many people, they’re like, “Hey, we don’t—okay, man. You know what? Screw that AppSec stuff, we’re just shifting left,” even though shifting left is, you know, fundamentally AppSec. “We’re just shifting left and we don’t have to worry about all of these other CIA things that we learned about, right, traditionally in security.”

I don’t think anyone’s saying one necessarily replaces the other, or that it—

Williams: No, I think really it’s more about being extremely clear about the outcomes that you want to achieve with security, and then putting in place a highly tuned machine that can produce those results continuously and repeatedly over the course of a project. Shifting left is a weird term, to me.

I love the idea of doing security earlier in the process, but many people see that and say, “Oh, well, developers just do security now,” and they wash their hands of kinda the old ways of doing things. And unfortunately, that’s a recipe for disaster. I think you need to empower those developers with technologies that can let them do their coding and not introduce security vulnerabilities.

And it’s not just about shifting left, because in QA and in staging and CI/CD environments, you want to verify that everything is secure before you push to production. So, you’re really not losing that step. You need to do both, shift left and where it is now. And frankly, I think a lot of organizations should also consider shifting right and doing more protection in production. Because so many organizations that I talk to—I was just talking to a massive technology company here on the West Coast this morning—they’re really pretty blind about what’s going on at the application layer from a security perspective.

So, to me, it’s not really about shifting left, it’s actually about extending left and extending right and using a DevOps process to produce the outcomes that you really want.

Shimel: Yeah. I don’t disagree there. The reason I brought it up, though, is, you know, we could point fingers all around, but sometimes the security industry is its own worst enemy, and the cynics and the cynical within—well, we’re consuming ourselves with other stuff right now, but the cynics and the cynical folks would then say, “Oh, it’s just another buzz word, and it’s just something meant to take your eye off the ball, and we have got to hold this thing in our cold, dead hands and not let go.”

Williams: Yeah, yeah. It’s a fair point. I mean, DevSecOps is a little buzzwordy, and frankly, I’m sympathetic to the folks that say, “No, it’s just DevOps, and security is a part of it, of course.”

Shimel: Now, that’s my—so, that’s where I am.

Williams: Here’s how I think about it, though. Let me try and change your thinking about it—it is just DevOps, but there’s a view of the DevOps process that emphasizes the security pieces. It’s very much like a blueprint for your house. You know, you’ve got the elevations, then you turn the page and it’s got the plumbing plant, and then you turn another page and it’s got the electrical plant and so on.

It’s a little bit like that, for me. DevSecOps is the security view of DevOps. It just emphasizes the security aspects of the DevOps process so that you can see what’s going on and actually understand it in a little different way than if you were just viewing the whole thing all mashed together. It doesn’t really help to just smash it all together and call it all quality. So, that’s how I think about it, it’s just a view of the normal DevOps process.

Shimel: I don’t necessarily disagree. I think there’s only one DevOps, but I think—I know, I mean, so this year, at RSA, for instance, for the fifth year on the Monday of RSA week at the Moscone Center, we’re gonna be putting on DevOps Connect: DevSecOps Days. And over the course of the five years, we’ve called it DevSecOps, we’ve called it Rugged DevOps.

Williams: Yeah.

Shimel: Just don’t call us late for dinner.

Williams: [Laughter]

Shimel: And it’s kind of the security community that has driven the DevSecOps name, because they wanna make security is kinda in the middle, I guess. But—hey, whatever. If that’s what it takes to get people on board and doing things to make this more secure, so much the better.

Jeff, we started talking and I wanted to talk more about Contrast, though. Share with our audience—so, you guys are involved in AppSec, you’ve got a bunch of things going on. Any new news, anything, you know, we are coming into RSA season, everyone always has some sort of news around that. Anything you can share or talk about?

Williams: Well, I think the big thing for us this year is launching what we call run time exploit prevention, which is sort of our next generation attack detection. And the reason that it’s, I think, so compelling is because it leverages our deep experience in instrumenting applications at run time.

And so, instead of just trying to block attacks by looking at HTTP traffic and saying, “Oh, well, that’s got a SQL injection in it” or, “That’s got an XXE in it or something.” Instead, we’re instrumented deep within the code, and using that position, we can access a lot more contextual information. We can make really smart decisions about what an attack really is.

And this is interesting—we’re essentially defusing all forms of SQL injection and deserialization problems and XXE and expression language injection and so on by sandboxing powerful components inside the platforms.

So, I can give you an example. All these struts and spring vulnerabilities from over the last year and a half or so? They all involve sending untrusted data to an expression language and evaluation engine. So, what Contrast does is, we sandbox those evaluation engines to prevent dangerous stuff from happening while they’re being executed. So, you know, when you’re evaluating an expression, you don’t expect to be starting native processes and accessing files and opening socket connections and stuff like that. So, we prevent all those things from happening during expression language evaluation and restrict the program to only evaluating a normal expression.

It’s a very powerful technique for ruling out not just known vulnerabilities like in struts and spring, but we’re ruling out all forms of expression language injection, even ones that nobody’s discovered yet. So, I think that’s very exciting and puts you in a really unique position for dealing with new CVEs that are coming out every week.

Shimel: Yep. Absolutely, and that’s huge. I mean, in spite of everything and in spite of the cynicism, the pessimism, and everything else, it actually is an exciting time to be in development and security, right? We are—

Williams: I love it. [Laughter]

Shimel: Yeah. You know, we’re making progress where good things are happening in spite of, you know, every day we still hear about another breach or another, you know, incident.

Williams: Well, I’ll say this—I think it’s time that every application has the ability to detect if someone’s attacking it, and respond appropriately.

So, forever, we’ve—basically, our policy around AppSec has been, “Well, let’s build a wall around it to try to keep out the bad stuff, and then let’s scan it to see if we can find where any holes are.” And both of those approaches, I think, have sort of gotten less and less useful over the last seven or eight years, to the point where, today, they’re not really very helpful, particularly on modern applications that are built using APIs, they’re deploying containers in cloud and they move around and all sorts of things. Those approaches just don’t work any more.

So, fundamentally, building that protection into the applications itself is very compelling to me. It works at scale, it’s accurate, it’s very fast, it’s much faster than building barriers around stuff. And so, there’s just a number of advantages, and I’ve seen it transform organizations from the kind of organization we were talking about before with the huge security backlog, security debt. And they’ve, over the course of a year or a year and a half, they’ve transformed themselves across thousands of applications, to the point where they’re just dealing with the new stuff coming in. They’re in that new normal.

Shimel: Yep, absolutely. Alright, man. Hey, Jeff, we are—we’re well over our 15 minutes.

Williams: [Laughter] Awesome.

Shimel: Yeah, so, we’re gonna have to call it a break on this one. But you know what? Between now and RSA, maybe we can get you back on here, we can talk a little bit more about this. It’s a hot topic, it’s—

Williams: One quick point, if you want to try Contrast or expose yourself to IAST and RASP, you can use Contrast community edition for free. It’s fully powered, free for one app, and you can just sign up and start using it today.

Shimel: Where do you get it?

Williams: It’s at ContrastSecurity.com/ce.

Shimel: CE. Alright, excellent. Hey, Jeff Williams, CTO of Contrast Security, thanks for being our guest on this episode of DevOps Chat. Until next time, everyone, this is Alan Shimel, have a great day, for DevOps.com, DevOps Chat, and Security Boulevard—take care, everyone.

— Alan Shimel