During a Snowday 2022 event, Snowflake this week moved to make it simpler to build applications on top of its cloud platform for managing data in addition to enhancing the overall security of the platform.
The company has made generally available Snowpark, a framework for building Java, Scala, SQL or Python applications. Snowflake also committed to making generally available in early 2023 an open source Streamlit framework for building Python applications that Snowflake acquired earlier this year.
Snowflake also revealed it is adding observability tools to the platform to make it easier to build, test, debug, deploy and monitor data pipelines, including alerting tools (private preview), logging capabilities (private preview), event tracing (private preview), task graphs and history (public preview).
Torsten Grabs, director of product management for Snowflake, said those capabilities will make it easier to alert DevOps teams to issues that might arise with applications deployed on the Snowflake platform.
At the same time, Snowflake is also making it simpler to build data pipelines by making a Serverless Tasks capability generally available. In addition, the company is previewing a Schema Inference engine that makes it simpler to ingest data and a Dynamic Tables capability to declaratively manage data pipelines.
Other capabilities being added to the platform include a public preview of a Query Acceleration Service to make additional compute resources available on demand. Snowflake has also improved query efficiency with join elimination in addition to providing search optimizations.
There are also now cost optimization tools that enable IT teams to run a cost-benefit analysis of Snowflake services in addition to tools that make it easier to replicate data across multiple cloud computing environments running the Snowflake platform.
IT teams can now also replicate Streams and Tasks via a capability in public preview that enables data pipelines to seamlessly work on secondary Snowflake accounts and configure failovers using a graphical tool that is available in private beta.
Finally, Snowflake has also enhanced its cross-cloud Snowgrid Capability to make it easier to share both public and private listings of data in addition to applying tags to and searching masked data.
The latest editions to the Snowflake platform come at a time when more organizations are starting to adopt DataOps best practices to manage data. Snowflake is now making a case for building and deploying applications on top of a platform that automates the management of the data pipelines used by applications to access data.
It’s not clear to what degree the Snowflake platform will be employed to host applications, but the company is clearly betting the volume of data residing in its cloud platform will attract a wide range of application developers that, in turn, will need to be supported by DevOps teams.
Of course, Snowflake is not the only provider of a data warehouse or data lake that has similar ambitions. Regardless of platform, however, the one thing that is clear is that a convergence of DevOps and DataOps workflows is now underway as IT organizations look to streamline the management of the massive amounts of data that now reside in the cloud.
