first commit

CHANGELOG.md

+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
+and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
+
+## Unreleased
+### Added
+- Initial release!

LICENSE.txt

+Copyright (c) 2016-2017 Ohio Supercomputer Center
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

+# Batch Connect - Example Jupyter Notebook Server
+
+![GitHub Release](https://img.shields.io/github/release/osc/bc_example_jupyter.svg)
+![GitHub License](https://img.shields.io/github/license/osc/bc_example_jupyter.svg)
+
+An example Batch Connect app that launches a Jupyter Notebook server within a
+batch job.
+
+## Prerequisites
+
+This Batch Connect app requires the following software be installed on the
+**compute nodes** that the batch job is intended to run on (**NOT** the
+OnDemand node):
+
+- [Jupyter Notebook](http://jupyter.readthedocs.io/en/latest/) 4.2.3+ (earlier
+  versions are untested but may work for you)
+- [OpenSSL](https://www.openssl.org/) 1.0.1+ (used to hash the Jupyter Notebook
+  server password)
+
+**Optional** software:
+
+- [Anaconda](https://www.continuum.io/anaconda-overview) 4.3.13+ and its
+  [Jupyter Notebook extensions](https://docs.continuum.io/anaconda/user-guide/tasks/use-jupyter-notebook-extensions)
+  that allow users to define custom environment-based kernels from within the
+  Jupyter notebook dashboard.
+- [Lmod](https://www.tacc.utexas.edu/research-development/tacc-projects/lmod)
+  6.0.1+ or any other `module restore` and `module load <modules>` based CLI
+  used to load appropriate environments within the batch job before launching
+  the Jupyter Notebook server.
+
+## Install
+
+These are command line only installation directions.
+
+We start by downloading a zipped package of this code. This allows us to start
+with a fresh directory that has no git history as we will be building off of
+this.
+
+```sh
+# Download the zip from the GitHub page
+wget https://github.com/OSC/bc_example_jupyter/archive/master.tar.gz
+
+# Create a catchy directory
+mkdir my_jupyter_app
+
+# Unzip the downloaded file into this directory
+tar xzvf master.tar.gz -C my_jupyter_app --strip-components=1
+
+# Change the working directory to this new directory
+cd my_jupyter_app
+```
+
+From here you will make any modifications to the code that you would like and
+version your changes in your own repository:
+
+```sh
+# Version our app by making a new Git repository
+git init
+
+#
+# Make all your code changes while testing them in the OnDemand Dashboard
+#
+# ...
+#
+
+# Add the files to the Git repository
+git add --all
+
+# Commit the staged files to the Git repository
+git commit -m "my first commit"
+```
+
+## Contributing
+
+1. Fork it ( https://github.com/OSC/bc_example_jupyter/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

form.yml

+# Batch Connect app configuration file
+#
+# @note Used to define the submitted cluster, title, description, and
+#   hard-coded/user-defined attributes that make up this Batch Connect app.
+---
+
+# **MUST** set cluster id here that matches cluster configuration file located
+# under /etc/ood/config/clusters.d/*.yml
+# @example Use the Owens cluster at Ohio Supercomputer Center
+#     cluster: "owens"
+cluster: ""
+
+# Title of the app displayed in the Dashboard
+title: "Jupyter Notebook"
+
+# Description of the app displayed in the Dashboard (can use multi-line string
+# and Markdown syntax)
+description: |
+  This app will launch a Jupyter Notebook server on one or more nodes.
+
+# Define attribute values that aren't meant to be modified by the user within
+# the Dashboard form
+attributes:
+  # Set the corresponding modules that need to be loaded for Jupyter to run
+  #
+  # @note It is called within the batch job as `module load <modules>` if
+  #   defined
+  # @example Do not load any modules
+  #     modules: ""
+  # @example Using default python module
+  #     modules: "python"
+  # @example Using specific python module
+  #     modules: "python/3.5"
+  # @example Using combination of modules
+  #     modules: "python/3.5 cuda/8.0.44"
+  modules: "python"
+
+  # Whether Conda extensions will be available within the Jupyter notebook
+  # server
+  #
+  # @note Allows the user to use conda environment-based kernels from their
+  #   Jupyter notebook dashboard
+  # @example Do NOT allow Conda extensions to be used
+  #   conda_extensions: "0"
+  # @example Do allow Conda extensions to be used
+  #   conda_extensions: "1"
+  # @see https://docs.continuum.io/anaconda/user-guide/tasks/use-jupyter-notebook-extensions
+  conda_extensions: "1"
+
+  # Any extra command line arguments to feed to the `jupyter notebook ...`
+  # command that launches the Jupyter notebook within the batch job
+  extra_jupyter_args: ""
+
+# All of the attributes that make up the Dashboard form (in respective order),
+# and made available to the submit configuration file and the template ERB
+# files
+#
+# @note You typically do not need to modify this unless you want to add a new
+#   configurable value
+# @note If an attribute listed below is hard-coded above in the `attributes`
+#   option, then it will not appear in the form page that the user sees in the
+#   Dashboard
+form:
+  - modules
+  - conda_extensions
+  - extra_jupyter_args
+  - bc_num_hours
+  - bc_num_slots
+  - bc_account
+  - bc_queue
+  - bc_email_on_started

manifest.yml

+---
+name: Jupyter
+category: Interactive Apps
+subcategory: Servers
+role: batch_connect

submit.yml.erb

+# Job submission configuration file
+#
+# @note Used to override job script batch parameters
+# @see http://www.rubydoc.info/gems/ood_core/OodCore/Job/Script
+---
+
+# PBS/Torque specific, override the node resource by specifying the ppn
+#script:
+#  native:
+#    resources:
+#      nodes: "<%= bc_num_slots.blank? ? 1 : bc_num_slots.to_i %>:ppn=28"

template/before.sh.erb

+# This script (`before.sh.erb`) is sourced directly into the main Bash script
+# that is run when the batch job starts up. It is called before the `script.sh`
+# is forked off into a separate process.
+#
+# There are some helpful Bash functions that are made available to this script
+# that encapsulate commonly used routines when initializing a web server:
+#
+#   - find_port
+#       Find available port in range [$1..$2]
+#       Default: 2000 65535
+#
+#   - create_passwd
+#       Generate random alphanumeric password with $1 characters
+#       Default: 32
+#
+# We **MUST** supply the following environment variables in this
+# `before.sh.erb` script so that a user can connect back to the web server when
+# it is running (case-sensitive variable names):
+#
+#   - $host (already defined earlier, so no need to define again)
+#       The host that the web server is listening on
+#
+#   - $port
+#       The port that the web server is listening on
+#
+#   - $password
+#       The plain text password used to authenticate to the web server with
+
+# Find available port to run server on
+port=$(find_port)
+
+# Generate SHA1 encrypted password (requires OpenSSL installed)
+SALT="$(create_passwd 16)"
+password="$(create_passwd 16)"
+PASSWORD_SHA1="$(echo -n "${password}${SALT}" | openssl dgst -sha1 | awk '{print $NF}')"
+
+# The `$CONFIG_FILE` environment variable is exported as it is used in the main
+# `script.sh.erb` file when launching the Jupyter Notebook server.
+export CONFIG_FILE="${PWD}/config.py"
+
+# Generate Jupyter configuration file with secure file permissions
+(
+umask 077
+cat > "${CONFIG_FILE}" << EOL
+c.NotebookApp.ip = '*'
+c.NotebookApp.port = ${port}
+c.NotebookApp.port_retries = 0
+c.NotebookApp.password = u'sha1:${SALT}:${PASSWORD_SHA1}'
+c.NotebookApp.base_url = '/node/${host}/${port}/'
+c.NotebookApp.open_browser = False
+c.NotebookApp.allow_origin = '*'
+c.NotebookApp.notebook_dir = '${HOME}'
+c.NotebookApp.disable_check_xsrf = True
+<%- if context.conda_extensions == "1" -%>
+c.NotebookApp.kernel_spec_manager_class = "nb_conda_kernels.CondaKernelSpecManager"
+c.NotebookApp.nbserver_extensions = {
+  "nb_conda": True,
+  "nb_anacondacloud": True,
+  "nbpresent": True
+}
+<%- end -%>
+EOL
+)

template/script.sh.erb

+#!/bin/bash -l
+
+# Set working directory to home directory
+cd "${HOME}"
+
+#
+# Start Jupyter Notebook Server
+#
+
+<%- unless context.modules.blank? -%>
+# Restore the module environment to avoid conflicts
+module restore
+
+# Load the require modules
+module load <%= context.modules %>
+<%- end -%>
+
+# Launch the Jupyter Notebook Server
+jupyter notebook --config="${CONFIG_FILE}" <%= context.extra_jupyter_args %>

view.html.erb

+<p>
+  If you see <code>Failed to connect to ...</code>, then wait a few seconds
+  before trying the <strong>Connect to Jupyter</strong> button again. This
+  warning appeared because the Jupyter Notebook is still starting up.
+</p>
+
+<form action="/node/<%= host %>/<%= port %>/login" method="post" target="_blank">
+  <input type="hidden" name="password" value="<%= password %>">
+  <button class="btn btn-primary" type="submit">
+    <i class="fa fa-eye"></i> Connect to Jupyter
+  </button>
+</form>