What is the Jupyter Notebooks service?
Jupyter Notebooks are available to all users at notebooks.msi.umn.edu. This interactive computing environment requires only a web browser, and enables data analysis and visualization on our HPC resources in a shareable, reproducible notebook format. Notebooks currently supports both the JupyterLab and Classic Notebook interfaces, computation with the Python (version 2 and 3) and R languages, and can also interoperate with user-installed Jupyter environments. MSI Python tutorial materials provide a useful overview.
There are many excellent tutorials and videos online that explain how to use Jupyter Notebooks.
How do I connect to Notebooks at MSI?
You can connect to the Jupyter Notebooks service using any modern web browser. You do not need to install any special software. If you are off-campus you will need to connect through the University VPN.
- To begin, visit notebooks.msi.umn.edu in your browser. You will be prompted to log in with your University InternetID. You will be asked to select your group if you belong to more than one group.
- If you do not already have a running Jupyter session, you will be prompted to "Start My Server".
If your Jupyter Notebooks server is already running, you will be redirected directly to your running session. - On the "Server Options" page, select a job type (see below for additional information) and click "Start" to start your session.
- Please be patient. Usually your Jupyter Notebooks server will start in under a minute, but during busy periods you may have to wait up to several minutes, especially if you have selected one of the larger job types.
- Once your session starts, you will be redirected to the JupyterLab interface. You can start new notebooks, command line terminals, and other resources by clicking icons in the launcher. You can browse, open, and manage files via the directory view in the left sidebar.
- Your session is hosted by a running job, and will continue running if you close your web browser or even log out. You can reconnect to your running session by logging in to notebooks.msi.umn.edu (including from another computer or a different location).
- When you are done working, you should end your session to release the compute resources for other users. From the "File" menu at the top of the work area, select "Hub Control Panel" and press "Stop My Server". Unless you end your session, your server will continue running and consuming resources until the requested time runs out, even if you log out or close your web browser.
What do the "Server Options" mean?
Your session is provided by a Jupyter Notebook server running on your behalf on a scheduled HPC resource. Behind the scenes a job is created and scheduled to run for you. The "Server Options" form allows you to select the resources requested for that job. The currently supported options include:
- Basic Interactive - 3 cores, 8 GB, 24 hours, 48 GB local scratch
This is the default session type; it runs on a standard Mesabi node and will generally start up quickly. With a long time limit and small resource request, it is appropriate for all-day general computing tasks. - Agate Interactive - 4 cores, 32 GB, 12 hours, 64 GB local scratch
A basic interactive session on an Agate node; also appropriate for all-day computing, but requests a shorter time limit as this newer hardware is in higher demand. - Mesabi High-mem - 12 cores, 128 GB, 4 hours, 180 GB local scratch
This reserves one half of a ram256g node. This session and the ones below are limited to 4 hours to help the scheduler quickly meet these larger resource requests. - Mangi AMD wide - 24 cores, 45 GB, 4 hours, 75 GB local scratch
- Agate A40 GPU session - 16 cores, 60 GB, 4 hour, 80 GB local scratch, 1 A40 GPU
- K40 GPU session - 12 cores, 60 GB, 4 hour, 100 GB local scratch, 1 K40 GPU
The first session options in the list are appropriate for starting a long-lived session that you will use all day for any type of computational work. The lower options request more scarce resources - larger amounts of memory or CPU cores, or GPU accelerators - and as such request a shorter time limit of only four hours. A good approach is to use a general interactive session to develop and test your workflow, and then start a more resource-heavy session if needed to run your workflow. Note that with the new Named Servers feature (see below), you no longer need to exit your interactive session to start the second shorter-lived session. It is still good practice to shut down your session(s) (see step 7 above) when you are done to make those resources available to other users.
What are Named Servers?
We have recently deployed an upgrade that enables Jupyterhub Named Servers. This feature enables you to have multiple sessions running at the same time. For example, you can keep an interactive session running throughout the work day, and start a second session that requests large memory or a GPU only when needed.
To use this feature, visit the Jupyterhub home page: load notebook.msi.umn.edu, click the "Home" navigation link if the Hub is already open, or select File -> Hub Control Panel if you have Jupyterlab open. Your primary session is started and stopped using the large buttons at the top. To use additional sessions, type a short descriptive name in the text box and click the "Add New Server" link. You will be taken to the session type selection screen, and can launch a new session from there.
After you have done this, if you return to the Hub home page you will see your named server has a line in the Named Servers table. This will show you the server name. If the server is currently running, it will also show you a link that will take you back to that session and a button to stop the job. If the server is not running, you will instead see buttons to start or delete the server entry.
What happened to the old notebook interface?
MSI recently made JupyterLab the default interface for the Notebooks service. The previous interface is now called the Classic Notebook. We hope this modern interface allows a smoother and more productive experience. Some important differences include:
- Classic Notebook opened notebooks, terminals, etc in new browser tabs. JupyterLab opens them in a unified workspace in a single tab.
- Classic Notebook created new notebooks and other resources via a "New..." menu. JupyterLab uses a launcher panel with prominent icons.
Please see the JupyterLab documentation for more details, including a detailed tour of the interface and features. If you prefer to use the Classic Notebook interface, it is still available via the "Help" menu in the menu bar.