About the App Workbench

The BlueData App Workbench (also called bdwb) is a Python-based CLI framework that provides a rich set of APIs, macros, and a shell to:

• Create Docker images from Dockerfiles created using BlueData base images.
• Orchestrate the run time environment for single and multi-node deployments.
• Package and load images into the BlueData EPIC App Store (i.e. new Catalog entries).

This functionality allows EPIC Platform Administrators to manage, create, and update the preferred applications and versions of applications available to their end users in the BlueData App Store.

The App Workbench focuses mainly on the following three use cases:

1. Modify or upgrade an existing Hadoop or Spark distribution in the App Store. For example:
   • Modify Cloudera CDH version 5.4.3 and add a security patch to the base image.
   • Create a new CDH version 5.5.1 image starting from existing CDH 5.4.3 image.
2. Add a new application as an edge node for Hadoop or Spark with auto-provisioning​. For example:
   • Splunk/Hunk could be one team’s preferred tool for operational analytics; those end users may want “Splunk on Hadoop (Hunk)” as an edge node for new Hadoop deployments.
   • Users with a cluster dedicated for ETL may want Talend as an edge node, pre-wired for immediate use.
3. Create new images for Big Data applications and frameworks. For example:
   • Data science teams often need for tools beyond Hadoop and Spark – they’ll want to add Kafka, Cassandra, and other applications like H2O for their testing, development, prototyping, and experimentation.
   • A user running Spark on YARN in a Hadoop cluster may be interested in trying Spark 1.6 or Spark 2.0 standalone, and they may want to add tools such as Zeppelin or Jupyter notebooks and Spark Job Server to their Spark clusters. 

The BlueData EPIC platform and App Workbench provide seamless support for all of the scenarios described above. Organizations can easily maintain and run multiple applications and tools (and multiple versions) in parallel, to support a wide variety of Big Data use cases.

How the App Workbench Works

As shown in the command line screenshot below, bdwb supports the following:

• Interactive shell
• Inline help for commands and subcommands
• Autocomplete and contextual help
• Commands to build images, configure Docker instances, define new catalog entries and options, add a logo, and much more.

To add a new application or modify an existing one, users can either run these commands interactively or create a .wb file containing a series of commands. Much like a Python script, running the .wb runs all of the commands in that file.

Below is a sample file. Note the command #!/usr/bin/env bdwb in the first line:

The following screenshot shows the App Store in the BlueData EPIC web interface, along with corresponding binaries from a specific BlueData installation: