Elyra reaches 1.0.0

By Luciano Resende, Open Source AI Platform Architect, CODAIT, IBM

Elyra is an open source project that extends the JupyterLab user interface to simplify the development of data science and AI models.

Elyra is proud to announce its 1.0.0 release. If this is the first time you are hearing about Elyra, check out our announcement blog for more details about the project.

The 1.0.0 release of Elyra includes:

• Notebook Pipelines visual editor
• Ability to run notebooks as batch jobs
• Reusable Code Snippets  (new)
• Hybrid runtime support (based on Jupyter Enterprise Gateway)
• Python script execution capabilities within the editor
• Python script navigation using auto-generated Table of Contents
• Notebook navigation using auto-generated Table of Contents
• Notebook versioning based on Git integration
• Reusable configuration and editor for runtimes (new)
• Support for JupyterLab 2.x (new)
• JupyterHub Support (new)
• Ability to try Elyra from Binder (new)
• Support for JupyterLab Dark Theme

Notebook Pipelines

Elyra’s Notebook Pipeline Editor simplifies the conversion of multiple notebooks into batch jobs or workflows. By leveraging cloud-based resources to run their experiments faster, the data scientists, machine learning engineers, and AI developers are then more productive, and therefore able to spend more of their time focusing on their technical skills.

Based on the great feedback from the Elyra user base, this release brings lots of bug fixes usability enhancements such as:

• Enhanced inline user documentation
• Validation capabilities to Pipeline Editor that notifies users of missing or invalid configuration values
• Optimized dependency handling providing a much faster submission of pipelines
• Easier access to previous submitted experiments from the Pipeline Editor
• Support for “bring your own image” to be used as the environment to execute Notebooks on the external runtime

Reusable Code Snippets

Code snippets give you the ability to save time and reuse task-oriented blocks of code. Elyra’s new code snippets extension enables easy discovery, creation and insertion of reusable snippets of code into your Notebooks, Python Scripts or even Markdown files used for documentation directly from the JupyterLab workspace. This makes the process of writing code more efficient and accessible.

The list of available code snippets is found in the left side pane and includes a preview of each snippet, and an option to either copy a snippet or insert it directly inline.

Code snippets can also be conveniently created and edited from within JupyterLab.

Leveraging  Table of Contents for Notebooks and Python Scripts

Navigating large files to find specific sections in Notebooks or function definitions in Python Scripts can be difficult tasks. The Table of Contents extension, which was enhanced to support navigating Python Scripts provides an easy outline of your contents, enabling easy navigation.

To allow for streamlined python development, Elyra’s python editor is now accompanied by an auto-generated Table of Contents, which allows for efficient navigation within large python scripts.

Reusable configuration and editor for runtimes

Elyra introduced a ‘shared configuration service’ that simplifies workspace configuration management, enabling things like information around accessing external runtimes to be configured once and shared across multiple components.

With Elyra 1.0, this service is now used by multiple components and have been enhanced with schema based validation capabilities and a full set of REST APIs.  With this release of Elyra, users can also easily browse, create and edit these configurations from within the JupyterLab user interface.

JupyterHub support

With Elyra 1.0.0 we have also created a docker image and provided necessary [configuration steps](https://elyra.readthedocs.io/en/latest/recipes/deploying-elyra-in-a-jupyterhub-endvironment.html) to integrate Elyra with JupyterHub.

Try Elyra from Binder

To experiment with Elyra without installing it locally, just click on the binder link:

https://mybinder.org/v2/gh/elyra-ai/elyra/v1.0.0?urlpath=lab/tree/binder-demo

Using Elyra in real Analytics and AI scenarios

While building Elyra, we work very close with data scientists, machine learning engineers and AI developers, and we have been building a few scenarios to validate the user experience when developing models and other applications using Elyra.

Analyzing COVID-19 time series data

One of the examples creates a pipeline to analyze COVID-19 time series data sets from both USA and Europe which is available as open-source in the [covid-notebook github repository](https://github.com/CODAIT/covid-notebooks)

 Analyzing NOAA weather time series data set and explore forecasting

Another example utilizes [DAX – Data Asset Exchange](https://developer.ibm.com/exchanges/data/) NOAA dataset and produces a pipeline that consumes and applies ETL into the data set, and then goes about analyzing and experimenting different forecasting capabilities.

Elyra community adoption 

The Elyra community is working very hard to promote adoption and create a heath community around the project. In the past few months, we are starting to see some momentum in the past several weeks and below are some details :

Github Stars : Elyra main repository is reaching close to 500 as of August 2020, please continue to show your support for the project and giving us more stars.

Projects depending on Elyra: Other then the two examples scenarios mentioned above, we are starting to see other communities adopting elyra into their projects, CalPoly has been using the Elyra code snippets extension in their summer-intern projects and other communities are also experimenting with Elyra.

Downloads: We have also seen an increase of downloads of the Elyra npm packages in the past several weeks, and we are floating between 60k – 70k weekly downloads.

Elyra in the Enterprise: Components of Elyra are integrated and available in IBM Cloud Pak for Data and Watson Studio offerings.

Summary

Building on a Jupyter Notebooks foundation, the de facto tool for data scientists, machine learning engineers and AI developers, Elyra 1.0.0 provides a set of AI centric extensions to JupyterLab aiming to help users through the model development life cycle complexities, making JupyterLab even better for AI practitioners.

We would appreciate for you to get involved with the [Elyra project] (https://github.com/elyra-ai/elyra/). Read our [contributing guidelines] (https://elyra.readthedocs.io/en/latest/index.html), create new issues if you have questions, suggestions for new features or to report any bugs. We also welcome contributions via GitHub pull requests.