DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • Azure Migration Strategy: Tools, Costs and Best Practices
  • Blameless Integrates Incident Management Platform With Opsgenie
  • OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
  • Red Hat Brings Ansible Automation to Google Cloud
  • Three Trends That Will Transform DevOps in 2023

Home » Blogs » DevOps Toolbox » Google Looks to Simplify Building of Workflows

Google Looks to Simplify Building of Workflows

Avatar photoBy: Mike Vizard on May 3, 2018 Leave a Comment

Google this week launched the beta of Cloud Composer, a managed Apache Airflow service for building workflows that promises to simplify management of integrated DevOps processes. Apache Airflow is an incubation project being developed under the auspices of the Apache Software Foundation that enables IT teams to programmatically author, schedule and monitor workflows using directed acyclic graphs (DAGs) that visualize pipelines running in production, monitor their progress and can be used to troubleshoot issues.

Recent Posts By Mike Vizard
  • Blameless Integrates Incident Management Platform With Opsgenie
  • Red Hat Brings Ansible Automation to Google Cloud
  • Automation Challenges Holding DevOps Back
Avatar photo More from Mike Vizard
Related Posts
  • Google Looks to Simplify Building of Workflows
  • Apache Druid’s Role in Modern Data Analytics
  • Pepperdata® Code Analyzer for Apache Spark Highlights Performance Bottlenecks for Developers
    Related Categories
  • Blogs
  • DevOps Toolbox
    Related Topics
  • apache
  • Cloud Composer
  • google
  • open source
Show more
Show less

James Malone, a product manager for Google Cloud, said Cloud Composer leverages Apache Airflow to not only make it simpler to manage workflows, but also port them between various public and private clouds, including on-premises IT environments.

TechStrong Con 2023Sponsorships Available

Cloud Composer also includes the Google Developer Console and Cloud software development kit (SDK), a cloud identity-aware proxy, Stackdriver logging and monitoring, identity access management (IAM) and Python support, along with simplified DAG management and streamlined Airflow runtime.

Google plans to add support for autoscaling and availability in additional Google Cloud regions over time, Malone said, as well as integrate Google Composer with Kubernetes to extend the reach of Apache Airflow workflows to containers.

Pricing for Cloud Composer is consumption-based as measured by vCPU/hour, GB/month and GB transferred/month.

Malone said Cloud Composer will play a critical role in enabling DevOps teams to construct pipelines spanning hybrid cloud computing environments. Today IT administrators regularly make use of text editors or scripts they write to automate a process. But it’s difficult for those scripts to be shared between multiple IT administrators. Cloud Composer and Apache Airflow make it easier to share workflows that automate processes across DevOps teams. Just as importantly, those workflows will be accessible to IT administrators who don’t have the programming skills required to develop a script.

Those workflows will also play a critical role in unifying DevOps and DataOps processes around a common set of continuous integration/continuous deployment (CI/CD) processes, Malone added.

Most organizations today are managing multiple clouds in isolation from one another. But as organizations look to build applications faster, they want to be able to deploy those applications on the most optimal platform available without having to develop a unique set of pipelines for each cloud environment. Having an application locked into a specific cloud platform not only is economically unacceptable, it’s also fundamentally inefficient from an application development and deployment perspective.

Apache Airflow is not the only tool that can be employed to construct pipelines. But Malone noted that because it is an open source Apache project, organizations can be assured that workflows they develop can run anywhere now and into the future. Google chose Apache Airflow to build Cloud Composer largely because an active development community is emerging around this open source project, he said.

The way IT platforms are managed is transforming rapidly. The days when servers and applications where managed in isolation are over. IT administrators are being asked to manage fleets of server and storage systems running more applications than ever. The tools required to achieve that goal are thankfully now becoming a lot more accessible to everyone in IT.

— Mike Vizard

Filed Under: Blogs, DevOps Toolbox Tagged With: apache, Cloud Composer, google, open source

« Stack Overflow Launches Stack Overflow for Teams
AlertSite Raises API Monitoring Bar by Providing Teams Unparalleled End-to-End Visibility into API Health »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Automating Day 2 Operations: Best Practices and Outcomes
Tuesday, February 7, 2023 - 3:00 pm EST
Shipping Applications Faster With Kubernetes: Myth or Reality?
Wednesday, February 8, 2023 - 1:00 pm EST
Why Current Approaches To "Shift-Left" Are A DevOps Antipattern
Thursday, February 9, 2023 - 1:00 pm EST

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

Azure Migration Strategy: Tools, Costs and Best Practices
February 3, 2023 | Gilad David Maayan
Blameless Integrates Incident Management Platform With Opsgenie
February 3, 2023 | Mike Vizard
OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings
Red Hat Brings Ansible Automation to Google Cloud
February 2, 2023 | Mike Vizard
Three Trends That Will Transform DevOps in 2023
February 2, 2023 | Dan Belcher

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

New Relic Bolsters Observability Platform
January 30, 2023 | Mike Vizard
Jellyfish Adds Tool to Visualize Software Development Workflows
January 31, 2023 | Mike Vizard
OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings
Cisco AppDynamics Survey Surfaces DevSecOps Challenges
January 31, 2023 | Mike Vizard
Let the Machines Do It: AI-Directed Mobile App Testing
January 30, 2023 | Syed Hamid
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.