A great new book on DevOps is coming out called “The Kitty Hawk Venture.” In the vein of “The Phoenix Project,” it is a novel about one fictional company’s journey specifically with continuous testing. Based on a real-life airline company, the book is great case study and learning experience around continuous testing and its importance to the entire CI/CD and DevOps experience.
The book is written by Jeff Schaeffer, general manager of the Continuous Testing business unit at CA Technologies; Alex Martin, CTO of Continuous Testing at CA Technologies (and former CTO of Rally Software); and Aruna Ravichandran, vice president of Gobal Marketing at CA Technologies and author of “DevOps for Digital Leaders.” Beyond the book, there is a companion ebook on continuous testing, as well as other related content.
In this DevOps Chat, we speak with Aruna, who is a frequent guest on our chats, about the book, continuous testing and the overall DevOps market. It is an excellent chat and, as usual, Aruna’s passion for what she does and DevOps shines through.
For those not familiar with our DevOps Chats, the streaming audio of our conversation is immediately below, followed by the transcript of our conversation.
Audio
Transcript
Alan Shimel: Hey everyone, it’s Alan Shimel. DevOps.com. You’re listening to another DevOps chat. This is gonna be a really great chat. I have one of my dear friends in the DevOps industry joining us. Aruna Ravichandran Ravichandran, VP of Marketing DevOps at CA technologies. But more than that, now a multi-time author as well as a code engineer and a bunch of other things. She’s had an amazing, amazing life and career. Aruna Ravichandran, welcome.
Aruna Ravichandran: Thank you, Alan. I am very excited to be here with you today talking about my latest book.
Shimel: Yup. So it’s a new one coming out, hopefully in July I guess is the current timeline. And it’s called, “The Kitty Hawk Venture.”
Ravichandran: That is exactly right. So you know, I think we chatted about a year ago when I wrote my first DevOps book. It was called, “DevOps for Digital Leaders.” And what we decided to do based upon the substance of the book is to basically to take a leaf from the Phoenix Project and write this book in terms of a real life story about one of our customers. I can’t name who the customer is. And the book is actually called, “The Kitty Hawk Venture.”
And it’s a novel about continuous testing in the world of DevOps in order to help customers as they’re going through their digital transformation journey. So it’s not written as a continuous testing handbook.
But it’s basically a real life example which happened with one of our customers and tells the story of the journey they actually went through in terms of how they actually transformed their testing discipline, lines of automation down to continuous testing and how they actually addressed some of the bigger challenges they actually had internally across the board.
Shimel: Got it. So Aruna Ravichandran, I promised you and I promise our audience, no spoilers on this. So I don’t want to go too much into the actual story of the book, let’s keep it at a high level and say that it is about continuous testing and it is based on a real life customer experience. Maybe an amalgamation of customers.
But you know, Aruna Ravichandran, you mentioned your first book, “DevOps for Digital Leaders.” So this is then your second book on DevOps.
Ravichandran: Yes, it is. But this time I’ve actually focused on one of the emerging disciplines within DevOps. the first book was primarily focused around DevOps as a discipline, some of the challenges customers actually face.
Where do they actually start. Because you know, there is no hard and fast rule in terms of DevOps. you can start with continuous integration. You could start with continuous operations. Or you could start with continuous testing.
The book was primarily focused around end to end DevOps and gives some amount of coaching to the end reader on where to start, what’s the maturity model, what are some of the metrics, how do you measure the substance of DevOps.
It also talks about several different customer examples on the journey they actually went through. But and speaking from that, we saw that a whole bunch of our readers are interested in learning more about an emerging discipline around continuous testing. So this book is primarily focused on continuous testing as a discipline and why it’s a very important area within the world of DevOps.
Shimel: Yup. Now, Aruna , though, you hear it in your voice when you’re speaking to our audience here – it comes through loud and clear. this whole area of DevOps – continuous testing specifically with the book. It’s something that you are passionate about. I get it, you’re a VP in the DevOps group. You should feel an affinity. But this seems to go beyond your title and your job. It’s truly something you have a passion for.
The book is almost 200 pages from the early view I saw. You don’t write 200 pages, you know, blah, blah, blah. You’ve gotta have sort of a burning passion. What is the passion that’s driving you?
Ravichandran: So the passion that drives me is my customers. As I talk to my customers in the DevOps space, I am very passionate about solving customer problems. And as I talk to my customers, especially customers who want to make an impact, who are on their digital transformation journey, what I learned was testing is a discipline which has existed for several different decades across the board, the very first time when this actual code was actually implemented.
But what you – what I actually found is that people have testing practices. But it’s mostly around islands of automation. Very, very few customers actually are doing end to end testing, which I call continuous testing.
And even though you have islands of automation, what I actually find is that testing still continues to be manual. I did a poll of a whole bunch of enterprise customers across the board, and I was shocked to find that testing still appears to be 70% manual across the board.
And given all of the – this is my personal reason for why I am very passionate about continuous testing, because I think that DevOps has gained a tremendous amount of precedent, but if you don’t embed continuous testing as a discipline across the software development life cycle, you really cannot go through DevOps.
So continuous delivery has a lot of what do you say affiliation with the market. More and more people are getting educated on how to basically go through the CICD pipeline so that it can go from code to production in minutes. But what they realize is that if you don’t have the testing discipline embedded across the SCLC, going through the pipeline doesn’t buy you anything.
And so that’s why this is a very important topic which I am very passionate about.
Shimel: I mean, Aruna Ravichandran, on some level it’s almost – and I think I’ve heard you say this, continuous testing is almost the missing link. Right? In the continuous delivery, in the software development life cycle delivery chain. Right?
For a lot of – you know what I mean? Companies, they focus on configuration management, on CICD, even on security. But that continuous testing is – it’s right there. It’s a lynchpin. The other thing I wanted to just quickly mention about it is continuous testing isn’t just a pre-delivery —
Shimel: It’s truly continuous.
Ravichandran: You nailed it. I would 100 percent agree with you on everything you stated. It is the missing link within DevOps. continuous testing is a practice of testing across every activity in the software development life cycle with the hope that you can uncover and fix unexpected behavior as soon as it’s ingested.
So it’s about embedding testing as a fundamental and ongoing aspect of every activity through the application life cycle, going all the way from requirements down into production, while facilitating the pre-exact mechanisms what happens in the world of DevOps to where you can actually fix the problem throughout the life cycle.
Shimel: Let me return to “The Kitty Hawk (Venture),” here again if I can for a moment. You mentioned earlier you took a page out of Gene Kim’s Phoenix Project book in that this isn’t similar to other books you’ve written and that I’ve even reviewed. That are your typical sort of business textbooks, right? You follow an outline and a table of contents and it goes. “Kitty Hawk” is more like you say – a novel, I think is the right word for it.
How does that make it different for the reader? Or the listener if it’s an audiobook? For someone reading the book.
Ravichandran: Yup. So for example – this book “[The] Kitty Hawk Venture” is actually based on the main character in the book is Leigh Freemark and she’s a senior director for quality assurance at a company called Runway Air. So the vertical we actually chose was an airline company.
The way this book is actually written is it talks about the journey of what they actually went through from a transformation journey. So they experienced a major glitch in terms of quality and which caused a substantial amount of negative reputation for Runway Air.
So this book actually walks through Leigh’s journey, the journey of her own personal mission of making quality first as a big part of her agenda.
And the way this book is written – we have so many technical books which basically come into the market on DevOps and on testing, so on and so forth. So we wanted to basically – we took a leaf from “The Phoenix Project,” and we also took another leaf from “The DevOps Handbook,” which is the second book which was recently announced by Gene Kim.
So this is basically written as a novel which tells you the journey which Leigh Freemark went through with Runway Air, but at the end of each chapter we give guidance in technical terms, of what was the actual continuous testing journey Leigh actually went through.
Right? So that’s why I said, it’s written as a novel but at the end of each chapter we also have like a handbook around the continuous testing discipline on what changes people actually need to make in order to fully embrace continuous testing as a discipline in the book.
Shimel: And even beyond that, Aruna Ravichandran, from what I understand and correct me if I’m wrong, you guys are planning on coming out with sort of an eBook companion to “Kitty Hawk” on “The Definitive Guide for Continuous Testing.” Is that correct?
Ravichandran: That is exactly right and so we just actually published it and it will be out very soon. It’s called “The Definitive Guide to Continuous Testing.” And here, we walk our users through 11 different disciplines on continuous testing.
Ongoing all the way on small teams [sound drops] _____ to how basically they can go through the transformation journey on continuous testing. And at a very high level, I walk you through eleven disciplines and you will see that completely in the novel because that was the journey which Leigh Freeman actually went through in the book, is the first discipline how do you basically and it will – the – you know, all of your testers.
It could be a developer who is actually doing unit testing or it could be someone who is doing API testing or it could be load testing. Or it could be reproduction testing. All testing in production, you need to be able to have your environment set up. How do you quickly virtualize your environment becomes a very important part of the mix.
How do you basically break on your environment, how do you actually create your environments very, very quickly is one of the first disciplines. The second discipline talks about testing management right? So regardless of what testing you’re doing having the right data at the right time is very critical. So how quickly can you generate your data?
And given the world of GDPR right now, you know, which basically went into a test in May, people have to be even more critical about generating test data. You want to make sure that your GDP are compliant. And so having the test data at the right time is critical but also being compliant with a whole bunch of mandates out there is even more critical. The third discipline talks about test automation. How do you basically fully orchestrate the application pipeline.
So that if the scripts actually fail, you can orchestrate how you can actually achieve continuous testing. The fourth one is around pipeline orchestration. This is the backbone. Everything is tied to it. everything must be integrated with your entire automation stream. You need to understand how it works, how to interpret the results and how do you make quick decisions in order to find out which part of testing is actually a bottleneck?
And this is what we actually call pipeline authorization. The fifth one focuses on the various different competences, right? If you are testing, the world is going more towards an API economy. So how do you actually bring API testing discipline so that you reduce the reliance on UI testing? Especially for business logic testing.
The sixth one is around performance and load testing. Right? So you can’t think about performance and load testing toward the end of the testing life cycle. You need to think about shifting left performance and load testing so that you can enable development teams to actually think about this much more earlier on.
The seventh one is the around the new emerging practice around security testing. So this is where everybody talks about DevSecOps, it’s all about shift left security. How do you enable developers to actually embed static code analysis as a part of their unit testing?
How do you basically get them to think about security testing by making it easy for them? By having it integrated as a part of their ID environments, whether it’s intelliga or it’s xless or visual studios, so on and so forth. So security testing is the seventh one.
The eighth one is accidents of test driven development and behavior-driven development. So you take the accidents criteria for each user and create that to ensure that those were actually met. Right? So this keeps testing focused within sprints and ensures developers develop what the business actually expect.
The ninth one is around automated test generation. And this is the activity of manually designing and writing manual tests of automated tests which is the biggest bottleneck.
They are the computing magazine data surveyed very recently and they asked, which is the biggest bottleneck across your software development life cycle? 63% of the practitioners actually said that testing was a bottleneck. And why was testing a bottleneck? Because it takes time.
Think about it, you have actually created 200 test cases for your regression suite for a vertical feature you are going to launch. Let’s assume you’ve caught a major defect and you fix the vertical defect, it means that you have to go back and recreate all of your regression text suites.
And if there was an automated way that you could take your requirements and as things changed in production you can recreate your test suite, that will be a tremendous amount of savings for anybody to actually not have to create the test cases manually which then eliminates the bottleneck which we actually have.
The tenth one is around requirement engineering. You must include all of SDLC stakeholders into your continuous testing journey. This means finding a better way to communicate and collaborate right from the requirements phase. The earlier one includes requirements engineering into continuous testing initiatives, the smoother that continuous testing journey will be. Because team members will be working with the same understanding right from the beginning.
And the last but not least – the eleventh discipline talks about feedback loops. This is the heart and soul of DevOps, it still continues to be the heart and soul of continuous testing. These are critical to ensure that you can have successful continuous testing.
Shimel: Excellent. You know, Aruna Ravichandran, sitting here looking – I want you to talk about quality, I don’t know if we have time. Maybe we’ll do it on another show. But the 11 disciplines, you did a great job laying them out here. I’m just wondering if someone listening says to themselves, oh my goodness, where do I begin? This is daunting. There is eleven of them. What is your advice to someone just starting out, listening in here – do I do eleven at all at the same time? What do I start with? What – do I take the path of least resistance? How do I get started with this?
Ravichandran: So I break it up into six different areas, so this is a question you get asked for DevOps and you get asked the same question around continuous testing. So if I have to give guidance on techniques on how to actually adopt – it’s also a cultural transformation too.
So the first one is, you know, you need to figure out a way to improve the relationship between the tester and each developer. So keeping the team small and creating inter-team collaboration makes it very easy to actually report and actually share the results between the tester and developer.
So that would be one important tactic which needs to be adopted from a cultural transformation perspective. Number two, is like I said, people actually have automation within testing, which is what we call test automation. But they really haven’t automated the whole thing. So I would say continue to make automation a priority.
So figure out where people are spending a lot of their time – it could be around building a regression test suite, because they tend to be manual. So how do you automate? So keeping that automation mindset is important. Figuring out where you’re spending cycles of your time in terms of doing manual tasks is important. So making automation a priority is important.
Map out within your SDLC and identify your automation opportunity. Number three is make it small – right? Like get small. Break it down into smaller increments so that it’s easier to test after every aspect of the SDLC. So break down increments after coding. Break it after design. This makes it easier to automate the test and it also makes it much more easy to deploy and integrate as a part of continuous delivery.
The fourth one is you want to keep track of everything. Use metrics. Start-up criteria. Right? So your defect density rate is high, figure out how can you reduce the defect density rate. Continuous testing is all about immediately identifying if things are working or not. How do you basically rinse and repeat if it’s working, or how do you actually pivot and change if it’s actually not working.
The fifth one is getting the right continuous testing tools. Tools play a big role. So we talked about also transformation, we talked about processes and practices, and also having the right tools is important. Figure out the tools that help you develop tests and analyze continuously.
Pick the best of breed tools that work together in a way that incorporates very easily into your continuous delivery environment. Choose tools which have community based support, that have open source plans that everyone shares the challenges and solutions and have interesting and lead to interesting use cases.
And last but not least, the number sixth one, is develop a system to display your results. So do deep dives into the results, because that shows you that if you have your code not working, where the gaps are. define your KPI and your acceptance criteria and make them quantifiable. Create dashboards to track those KPIs, including a baseline so that you can actually track with the sequence changes. So these are some of the disciplines I always share with my customers as well as my clients.
Shimel: Excellent. And now with everyone listening on here. Aruna Ravichandran we’re almost out of time, one more question if you have a second.
Ravichandran: Absolutely. I do.
Shimel: So I don’t know if we’re letting the cat out of the bag. If you can’t talk about it, politely say so. But you’re about to roll out something we hear called continuous testing academy? Launching this summer?
Ravichandran: Yup.
Shimel: So what’s that about?
Ravichandran: So the key thing – going back to the question you asked, this continuous testing is like an emerging discipline. People want to know what do I need to do, how do I need to start? How do I get there. And so last year I launched the – I saw that there was a tremendous amount of need in the market around JMeter.
And so I launched JMeter academy in order to basically to provide input to the community so they could actually train themselves on JMeter. But this year, knowing that continuous testing is gaining tremendous amount of momentum, I’m going to launch a continuous testing academy.
This is going to have a basic module, it will also have an advanced module. And people will actually have the opportunity to go through the entire module and get themselves certified. We’re also going to launch a certification and you will have the opportunity to become continuous testing certified by DevOps institute. So this is our way to give back to the community in order to basically get people understanding on how they need to evolve their continuous testing practice in order to make a bigger impact in the DevOps world.
Shimel: Excellent, again, Aruna Ravichandran – great job. You are just, I mean, literally, what a dynamo. All over the place. Doing so many different projects. All with DevOps and continuous testing as a theme there. Congratulations and hats off to you. One of my heroes. Anyway, we’re about out of time. Thank you so much, Aruna Ravichandran, for giving us a preview. Once Kitty Hawk is available to the public, I’d love to have you back on and perhaps we can chat more about it, sound good?
Ravichandran: Yep. That sounds awesome. As always, Alan, it was excellent talking to you. I love your energy, I love your passion. And looking forward to partnering with you more as we basically enable our customers and prospects to continue on marching further along in the world of DevOps.
Shimel: Well stay tuned for some announcements, we’re working on it. Aruna Ravichandran, I will be in touch. Aruna Ravichandran Ravichandran, a VP product marketing, marketing DevOps, CA technology, multiple times author, and really DevOps enthusiast. Thank you for being our guest on DevOps chats today. We’ll speak to you soon. This is Alan Shimel for DevOps.com. You’ve just listened to another DevOps chat. Have a great day everyone.
— Alan Shimel
