Review Category : Blogs

A how to guide on modern monitoring and alerting

syslog-ng, riemann, collectd-notifications, elasticsearch: putting it all together Context At our organization (CCIN2P3) we are building an event-based infrastructure to push structured messages to different subsystems for alerting, reporting and storage. Using syslog-ng, each message is normalized into a structured event, optionally correlated with other messages, and conditionally routed to the next systems, including: a synchronous web-dashboard, different asynchronous alerting systems, and a searchable storage backend. The events which are collected are essentially system and application logs. Here’s a few examples of interesting messages: puppet-agent[16528]: Finished catalog run in 44.06 seconds kernel: Killed process 29959, UID 42046, (hadd) total-vm:202363492kB, anon-rss:13069860kB, file-rss:60kB ata2.00: exception Emask 0x0 SAct 0xffff SErr 0x0 action 0x0 EXT3-fs error (device dm-1): ext3_journal_start_sb: Detected aborted journal The unified nature of this pipeline makes it possible for a human to easily identify an event in all the available back- and frontends. In this post you’ll learn a way ...

Read More →

Provisioning versus Configuration Example

A few weeks back I wrote about the difference between configuration and provisioning noting, primarily, that there are differences between the two tasks. It remains an important distinction to make because it’s really where the rubber meets the road (or the app meets the network) where it becomes important. As the infrastructure/network side of the house is where we’re concentrating our efforts to apply DevOps, it’s necessary to dig a bit deeper on this topic. So today we’ll examine  couple of concrete examples of the difference – and why it’s important. Your application is being deployed and it needs, wonder of wonders, a load balancer. Cause, elasticity. Scale. You know, cloudy type stuffs. So you grab that golden image and launch that virtual machine with Whatever Your Favorite Load Balancing Thing might be (HA Proxy, Nginx, F5, etc…). Voila! You’re ready to go, right? Not by a long shot. You’ve provisioned a load balancing ...

Read More →

A Culture of Trust

Trust is by far the most critical element of a DevOps culture.  It is also the most difficult to achieve.  IT was not built for trust.  The silo structure that is common in most IT organizations was built for specialization and territorial ownership.  The silo walls are thick. While DevOps may not be able to break down the silos, but it can encourage and groom a culture of trust. Trust is difficult to define.  We know when we feel it, we know when we don’t.   In 1993, Dr. Duane C. Tway, Jr. published a dissertation called “A Construct of Trust”.  In it, Dr. Tway defines trust as “The state of readiness for unguarded interaction with someone or something.” Dr. Tway believes that trust is actually “constructed” from three basic elements: The capacity for trusting The perception of competence The perception of intentions The capacity for trusting is how your total life experiences have developed your ...

Read More →

Being Agile is all about controlling the flow

For businesses, Agile is the new fashionable initiative as was E-business was in the 2000’s, Six Sigma was in the 90’s and Total Quality Management (TQM) was in the 80’s. If we look back across the spectrum of these initiatives, we need to examine how many of these initiatives met the goals the business was looking for and, more importantly, if they were fully adopted, remembering that many were derailed, did they deliver the value the business was seeking? Of course we cannot answer this question with generalities—it’s a spectrum. For some businesses, these initiatives were red herrings, some did not follow through to completion, but certain changes adopted as part of these initiatives had a very positive impact, while others succeeded wildly. It’s a Harvard Business Review case study as to why a business ended up at the point on the spectrum that they did. Clearly, one fundamental aspect ...

Read More →

It’s About What Works.

Enterprises have been working through production delays and operational inefficiencies for decades. Don’t get sucked in too deeply to the ever-present mantra of “this time is different”. There is a lot of source material out there from people in a different kind of operations that have already been through the same wringer as DevOps presents. You just have to know where to look and remember in the end “it’s about what works.” I was talking to Lori about my concept for this blog, and she sent me a link to a pretty good (dated, but relevant in light of DevOps) PDF that introduces quite a few of the concepts. But let me pull from personal experience to try and show how little uniqueness is enshrouded in DevOps. Like DevOps… In the past. I was hired into a utility company a long time ago to take them from a run-of-the-mill utility ...

Read More →

Testing in the Agile Age

The Agile Age For a business to survive or, dare I say, thrive in the “Agile Age”, it must adopt agile methodologies. The term agile may ring some bells with those of you from the product lifecycle field. But when I say agile, I’m not referring to product lifecycle terms like Scrum or Kanban. I’m talking about agility throughout the organization, starting from the CEO, through marketing, sales, R&D and, of course, to IT operations. To compete in today’s environment, you must act and react fast! If you don’t, your competition will beat you to it. Today, the barriers to compete are minimal and the only way to defend one’s stature is by innovating in short iterations. Agility facilitates survival in a highly competitive business environment. And it dictates a different DNA. One that can move fast, innovate and work in ridiculously short iterations. This new DNA demands an IT infrastructure that supports agility. What was so right in ...

Read More →

First speakers announced for Camp DevOps Houston, October 28th

We are very happy to announce the first group of speakers for Camp DevOps Houston on October 28, 2014 at the Norris Conference Centers.  There are still a few days left to submit your abstract and join this stellar line up. Email campdevops@devops.com if you would like to join us deep in the heart of Texas at Camp DevOps.  You can also get more information about attending or sponsoring Camp DevOps Houston at http://www.campdevops.com. Ariel Tseitlin Ariel Tseitlin is a Partner at Scale Venture Partners where he focuses on enterprise software investments in cloud, big data, security, and mobile. Ariel joined ScaleVP from Netflix, where he was Director of Cloud Solutions and responsible for creating and operating one of the most modern cloud infrastructures in the industry, accounting for a full third of all US downstream internet traffic at peak. Ariel’s team built many of the Netflix OSS components like ...

Read More →

Is Docker leaving Immutable Infrastructure?

According to Chad Fowler’s definition, Immutable Infrastructure can be defined as: “[S]ervers (or whatever) are deployed once and not changed. If they are changed for some reason, they are marked for garbage collection. Software is never upgraded on an existing server. Instead, the server is replaced with a new functionally equivalent server.” Though the idea is powerful implementing it is actually very difficult, especially with hypervisors. Thus, the DevOps community has advocated Docker from its launch to market because of its capability of building the Immutable Infrastructure, due to its layered image approach and super fast “booting” performance. However, last week Docker introduced us all to a new feature to allow “Writable /etc/hosts, /etc/hostname and /etc/resolv.conf”: “You can now edit /etc/hosts, /etc/hostname and /etc/resolve.conf in a running container. This is useful if you need to install bind or other services that might override one of those files. Note, however, that ...

Read More →

Specialists vs. Generalists for Enterprise DevOps

Welcome back to the DevOps.com series, ‘Enterprise DevOps Q&A.’ I have a backlog of Q&A in store, but don’t let that stop you sending in more! If I can’t answer them myself, I will try to find someone who can. For this post, I was recently asked about the challenge that large enterprises are facing over whether to use generalists or specialists to drive their DevOps transformation… Q. Does a DevOps environment prefer generalists or specialists to succeed? This question has ricocheted for a while. Google CIO Ben Fried – who spent 14 years in IT at a large enterprise (Morgan Stanley) – spoke as long ago as 2011 about how ‘Generalists, Not Specialists, Will Scale the Web.’ Some webscale businesses like Netflix even espouse a singular ‘NoOps‘ skillset, although they continued to have separate specialists … and maybe still do. However, in large enterprises, I have seen DevOps work ...

Read More →

Daddy DevOps

I found out this week I’m going to become a Dad. It’s a strange mix of emotions from excited, to terrified, to panicked and happiness. Not too dissimilar to undertaking a big new project, scoping out how complex it is, getting close to go-live and then delivering it [if any one you say I should use Agile processes for childbirth go to the back of the class - having a baby is definitely Waterfall] At the moment I have time on my hands, which is probably one of the reasons I blog.  Time also gives you the room to think and ponder.  And because I spend all day, every day talking about DevOps I tend to think about it even in my downtime (and sleep) which is fairly plentiful despite a busy day job. With my partner away this weekend, bored of playing on the Xbox and all my friends busy/married/parents, I’ve ...

Read More →
Directory powered by Business Directory Plugin