Review Category : Blogs

A how to guide on modern monitoring and alerting

 syslog-ng, riemann, collectd-notifications, elasticsearch Monitoring IT systems, collecting log messages and system metrics, and turning them into searchable and actionable data is a constant challenge in system management and maintenance. This hands-on tutorial shows you how to create such an integrated and scalable solution using current open-source tools like Riemann, Elasticsearch, Kibana, Nagios, and syslog-ng. In this post you’ll learn a way to implement this model, and achieve the following: Collect system metrics Monitor events for outliers Normalize and Correlate these events Route the events to a real-time stream processor and to a searchable storage backend Context At our organization (CCIN2P3) we are building an event-based infrastructure to push structured messages to different subsystems for alerting, reporting and storage. Using syslog-ng, each message is normalized into a structured event, optionally correlated with other messages, and conditionally routed to the next systems, including: a synchronous web-dashboard, different asynchronous alerting systems, and ...

Read More →

Daddy DevOps

I found out this week I’m going to become a Dad. It’s a strange mix of emotions from excited, to terrified, to panicked and happiness. Not too dissimilar to undertaking a big new project, scoping out how complex it is, getting close to go-live and then delivering it [if any one you say I should use Agile processes for childbirth go to the back of the class - having a baby is definitely Waterfall] At the moment I have time on my hands, which is probably one of the reasons I blog.  Time also gives you the room to think and ponder.  And because I spend all day, every day talking about DevOps I tend to think about it even in my downtime (and sleep) which is fairly plentiful despite a busy day job. With my partner away this weekend, bored of playing on the Xbox and all my friends busy/married/parents, I’ve ...

Read More →

Making DevOps Work in Complex Enterprise Environments

Welcome back to the DevOps.com series, ‘Enterprise DevOps Q&A’. Remember, if you have specific questions about Enterprise DevOps, let me know and I will try to answer them for you. Today’s question is: Q. In large enterprises, there are always environmental differences. From public cloud to internal hosting, from virtual desktops to legacy and mainframe – it is not possible to have full consistency between environments. Do you have any suggestions for approaching this gap? The Challenge of Enterprise Complexity This is a huge problem in larger enterprises, especially as DevOps practices grow beyond just isolated departmental applications. Architects have one system for prototyping; developers have a different system to code on; test and QA have their own unique ‘standard’ systems; while operations have untouchable systems for production. Add to this the complexity of backending into CICS or IMS applications, accessing data from DB2 or ADABAS on iSeries, or automating ...

Read More →

Me and My Kanban

There has been a growing interest in the Kanban board, a tool that helps teams visualize their workflows, limit work in progress and get more “done”.   Kanban Boards are mostly associated with Agile Scrum teams as a collaborative means of seeing the burndown of a sprint.   In fact, Scrum and Kanban are considered to be so interdependent that a new framework called Scrumban is starting to emerge.   However, Kanban is not limited to Agile.  A Kanban board can be very useful for any project or scope of work that would benefit from visualization. The premise behind the Kanban is almost deceptively simple.  A board is built with three columns:  “To Do”, “In Progress” and “Done”.   At the start of each sprint, all of the work that needs to be completed is captured on individual sticky notes and placed in the “To Do” column.  When the task or work is actually ...

Read More →

Keep Calm and Refactor Operations

Developers have been refactoring forever to gain efficiencies and avoid mistakes. Let’s apply more than just theory to ops to do the same. There’s about a million articles, blogs, tweets and infographics available today that bespeak of DevOps as this or that or the other thing. But one core truth remains: DevOps in part attempts to apply methodologies like agile to operations in order to gain efficiencies and avoid mistakes. We need the efficiencies because the number of apps that need to be deployed, updated, migrated, and maintained is increasing, not decreasing, and there simply isn’t enough time in the day (or during the monthly change window) to get it all done. We need to eliminate mistakes because they are, according to data from initialstate, very costly. It takes 30x longer to fix one bug (an error) than it does to write one line of code. Anyone who has ever ...

Read More →

It’s web performance, let’s not call it WebDevPerfOps

Velocity was great this year. It was only my 2nd time out to Santa Clara, but from what I understand every Velocity conference is great. Few conferences can provide such rapid payback. Velocity does a good job setting the right vibe. The talks are often as entertaining as they are informative. This video of Scott Hanselman is a great example of that. He did an artful job of telling an open source audience they can run their code in the Azure cloud too. I happen to know a few companies that are doing just that. It’s good to see Microsoft continue to embrace open source, as I am a fan of running C# on Linux myself. Getting a room of passionate experts together and making them laugh over plenty of food and drinks is a setup for awesome collaboration. When this crowd is having a good time the conversation drifts ...

Read More →

Culture Debt

There has been a lot of talk lately about IT’s accumulation of “technical debt’.  Technical debt describes necessary work from a prior deployment that was deferred in favor of other tasks.   As technical debt grows, the ability to make future changes is hindered because of a backlog that was not addressed.   I have recently come to recognize that IT suffers from another type of debt – culture debt.  IT is a young industry that grew rapidly  while facing constant pressure to bring technical innovation to the market. Culture considerations were deferred  in favor of building and deploying products and services.  IT’s silo culture grew organically out of the need for diversifed sets of specialized experience and expertise.   While cultural debt was accumulating,  IT’s complexities were increasing.  The single platform mainframe vanished in favor of multi-platform servers.   Production applications grew exponentially. The IT supply chain went from a single ...

Read More →

Network Engineers as Programmers? Not so fast.

It is interesting, as the triplet new waves of hype evolve – cloud, SDN, and DevOps – that we are repeatedly hearing The Voice of Reason say things like “All network engineers need to be programmers”. While this is certainly not the only voice that demonstrates what I’m talking about, an example of it is Kirk Byers’ commentary on Network Computing’s site. (http://www.networkcomputing.com/data-centers/programming-an-essential-skill-for-network-engineers/a/d-id/1297898) I do not know Mr. Byers, and want to be clear, he’s not the only one portraying this wisdom, so the reference is merely that – a reference to the syndrome I discuss here. In DevOps, certainly having staff members that understand both application development and network engineering is a bonus. Being one of those fabled unicorns, I also know that they’re few and far between. I’ve worked with some great ones over the years that “just get it”, and they certainly add value to an automation ...

Read More →

Panamax, Docker management for humans

Panamax, a new open source tool from CenturyLink brings the promise of building complex Docker based environments like a child using Legos. The tool is a management interface for managing and building multiple Docker environments. You can search the official Docker repository as well as a number of others to find best of breed, Docker pre-built containers. Panamax is available on GitHub and is released under an Apache 2.0 license. I spoke to Lucas Carlson, chief innovation officer at CenturyLink yesterday about Panamax. Lucas also heads up the CenturyLink Labs and this is one of the first projects that they have developed and released. Lucas gave me a demo of Panamax. In just a few moments we had searched for and found several WordPress containers. We chose one from the official Docker repository and quickly installed it. After that it was simple to choose other services such as Ruby and add ...

Read More →

Rackspace DevOps automation service broadly available

Rackspace is the #1 player in managed cloud. While they recently announced they would be withdrawing from the pure IaaS market, they have now double-downed their bet on managed cloud with the general availability of the Rackspace DevOps automation service around managed cloud. Rackspace had previously announced their DevOps automation service a few months back, but that was a limited release. Now DevOps automation is one of the two choices Rackspace customers will have in setting up a managed cloud service. With this general release Rackspace is offering DevOps automation on both Linux based clouds utilizing Chef among other tools and on Windows using native Windows tools. In addition Rackspace offers enhanced environment stack support. According to the release: Rackspace now supports the following environment stacks, which can be deployed in hours on the Rackspace Hybrid Cloud: Chef, Rails, Node.js, PHP, Tomcat and Python. Customers can use recommended stack architectures ...

Read More →
Directory powered by Business Directory Plugin