If you have an installation of X11 on your local system, you can access Oscar with X forwarding enabled, so that the windows, menus, cursor, etc. of any X applications running on Oscar are all forwarded to your local X11 server. Here are some resources for setting up X11:

• Mac OS - https://www.xquartz.org

• Windows - https://sourceforge.net/projects/xming

Mac/Linux

Once your X11 server is running locally, open a terminal and use

$ ssh -X <user>@ssh.ccv.brown.edu

to establish the X forwarding connection. Then, you can launch GUI applications from Oscar and they will be displayed locally on your X11 server.

Windows (PuTTY)

For Windows users using PuTTY, enable X forwarding under Connections->SSH->X11:

To log in to Oscar you need Secure Shell (SSH) on your computer.

There are two options for signing into Oscar: with or without VPN.

Summary of SSH Hosts

• ssh.ccv.brown.edu You can connect from anywhere. You will need Two Factor Authentication

• sshcampus.ccv.brown.edu You can connect when whithin Brown Wifi, Network or VPN. You will need to set up passwordless authentication.

• desktop.ccv.brown.edu This is the host to be used when connecting from a remote IDE, i.e., Visual Studio Code.

• transfer.ccv.brown.edu This host is used to transfer files to/from oscar using SFTP protocol

macOS and Linux

To log in to Oscar, open a terminal and

• If you are not connected to the Brown VPN, use the following command:

ssh -X username@ssh.ccv.brown.edu

• If you are connected to the Brown VPN, use the following command:

ssh -X username@sshcampus.ccv.brown.edu

The -X allows Oscar to display windows on your machine. This allows you to open and use GUI-based applications, such as the text editor gedit.

Windows

Windows users need to install an SSH client. We recommend PuTTY, a free SSH client for Windows.

• If you are not connected to the Brown VPN, use username@ssh.ccv.brown.edu as the Host Name and click Open.

• If you are connected to the Brown VPN, use username@sshvpn.ccv.brown.edu as the Host Name and click Open.

Connecting to Oscar for the First Time

The first time you connect to Oscar you will see a message about the authenticity of the host:

The authenticity of host 'ssh.ccv.brown.edu (138.16.172.8)' can't be established.
RSA key fingerprint is SHA256:Nt***************vL3cH7A.
Are you sure you want to continue connecting (yes/no)?

You can type yes and press return. On subsequent logins you should not see this message.

You will then be prompted for your password.

You will now be in your home directory on Oscar. In your terminal you will see a prompt like this:

[username@login004 ~]$

Congratulations, you are now on one of the Oscar login nodes! The login nodes are for administrative tasks such as editing files and compiling code. To use Oscar for computation you will need to use the compute nodes. To get to the compute nodes from the login nodes you can either start an interactive session on a compute node, or submit a batch job.

When connecting from a campus network to sshcampus.ccv.brown.edu you can set up SSH keys as a form of authentication instead of having to enter your password interactively. Follow the insctructions below that correspond to your operating system/connection method.

Step 1 : Check for existing SSH key pair

Before generating new SSH key pair first check if you have an SSH key on your local machine.

If there are existing keys, please move to Step 3

Step 2 : Generate a new SSH Keypair

Press Enter to accept the default file location and file name.

The ssh-keygen will ask you to type a secure passphrase. This is optional. If you don't want to use a passphrase just press Enter

Verify the SSH keys are generated correctly, you should see two files id_rsa and id_rsa.pub under ~/.ssh directory.

Step 3 : Copy the public key to Oscar

You will now need to copy your public key to Oscar. There are two ways to acomplish this.

With ssh-copy-id

If your OS comes with the ssh-copy-id utility, then you'll be able to copy your public key into Oscar as follows:

You will be prompted for a Password. The public key will be appended to the authorized_keys file on Oscar.

If you used a custom name for your key instead of the default id_rsa then you'll need pass the name of your key to ssh-copy-id i.e.,

Without ssh-copy-id

If your system does not come with the ssh-copy-id utility installed, then you'll need to copy your public key by hand.

1. Get the contents of id_rsa.pub file. One option is to use cat in your teminal cat id_rsa.pub.

2. Copy the contents of this file to your clipboard, as we need to upload it to Oscar.

3. Login into Oscar via regular ssh ssh <username>@ssh.ccv.brown.edu. Once you are on the login node, open the authorized_keys file with your text editor of choice e.g., vim ~/.ssh/authorized_keys or nano ~/.ssh/authorized_keys Add your public keys to end of this file. Save and exit.

Step 4 : Login to Oscar using your SSH keys

If everything went well, you will be logged in immediately withouth prompting you for a password.

When regularly connecting to multiple remote systems over SSH, you’ll find that remembering all the hosts and various command-line options becomes tedious. OpenSSH allows setting up a configuration file to store different SSH options for each remote machine you connect t.

SSH Config File Location

OpenSSH client-side (in this case your personal computer) configuration file is named config, and it is stored in the hidded .sshdirectory under your user’s home directory (i.e., ~/.ssh)

When you use the command for the first time. The ~/.ssh directory is automatically created. If the directory doesn’t exist on your system, create it using the command below:

By default, the SSH configuration file may not exist, so you may need to create it using the :

This file must be readable and writable only by the user and not accessible by others:

SSH Config File Structure Basics

The SSH Config File takes the following structure:

The contents of the SSH config file is organized into sections. Each section starts with the Host directive and contains specific SSH options used when establishing a connection with the remote SSH server.

Oscar Hosts

Here we peovide a list of Oscar hosts and typical SSH configuration options. You have two options

1. Copy the list of hosts below directly into your SSH Config File (i.e., ~/.ssh/config)

2. Keep this content in a spearate file for Oscar hosts, lets say ~/.ssh/config.oscar and include that file in your main configuration file. In this case, the first line of ~/.ssh/config will be Include "~/.ssh/config.oscar"

Connecting to your preconfigured host

You may now connect using the shortchut notation provided by your configuration file. That is, all you need to type is:

According to the configuration above, this is equivalent to

Much shorter. Enjoy!

SSH provides a method of sharing the ssh keys on your local machine with Oscar. This feature is called Agent Forwarding and can be useful for instance when working with version control or other services that authenticate via ssh keys. Below are instructions on how to configure your SSH connection to forward ssh-agent for diffeent operating systems

Key Generation & Setup

1. Open PuTTYgen (this comes as part of the package), change the 'Number of bits in a generated key:' to 4096 (recommended), then click 'Generate'

2. Move your cursor around randomly in order to "salt" your key, while the key is being generated. Once the key is generated, you should see something like this:

3. Replace the text in the 'Key comment:' field with something recognizable and enter a passphrase in the two fields below.

4. Copy the text in the 'Public key for pasting...' field (the text continues past what is displayed) and paste it wherever the public key is needed. If you are using GitHub, you can now create a new SSH key in your Personal Settings and paste this text into the 'Key' field.

5. Click on 'Save private key' and select a logical/recognizable name and directory for the file. Your private key is saved in the selected file.

6. Open Pageant (also part of the PuTTY package). If a message saying "Pageant is already running" is displayed, open your system tray and double click on the Pageant icon.

7. Click on 'Add Key' and select the file you saved when generating your key earlier (Step 5). If it is requested, enter the passphrase you created at Step 3 to complete the process.

Arbiter2 is a cgroups-based mechanism that is designed to prevent the misuse of login nodes and VSCode node, which are scarce, shared resources. It is installed on shared nodes listed below:

• login005

• login006

• node1103 (VSCode)

Status and Limits

Arbiter2 applies different limits to a user's processes depending on the user's status: normal, penalty1, and penalty2.

Normal Status and Limits

Upon first log in, the user is in the normal status. These normal limits apply to all the user's processes on the node:

Penalty1 Status and Limits

When a user's processes consume CPU time more than the default CPU time limit for a period of time, the user's status is changed to the penalty1 status. These penalty1 limits are applied:

While a user is in penalty1 status, their processes are throttled if they consume more CPU time than penalty1 limit. However, if a user's processes exceed penalty1 memory limit, the processes (PIDs) will be terminated by cgroups.

The user's status returns to the normal status after a user's processes consume CPU time less than the penalty1 limit for 30 minutes.

Penalty2 Status and Limits

When a user's processes consume more CPU time than the penalty1 limit for a period of time, the user is put in the penalty2 status, and the penalty2 limits apply to the user's processes.

In penalty2 status, the user's processes will be throttled if they consume more CPU time than penalty2 limit. However, if a user's processes exceed penalty2 memory limit, the processes (PIDs) will be terminated by cgroups.

The user's status returns to the normal status after a user's processes consume CPU time less than the penalty2 limit for one hour.

Penalty3 Status and Limits

When a user's processes consume more CPU time than the penalty2 limit for a period of time, the user is put in the penalty3 status. These penalty3 limits apply to the user's processes.

In penalty3 status, the user's processes will be throttled if they consume more CPU time than penalty3 limit. If a user's processes exceed penalty3 memory limit, the processes (PIDs) will be terminated by cgroups

The user's status returns to the normal status after a user's processes consume CPU time less than the penalty3 limit for two hours.

Email Notification

A user receives an email notification upon each violation. Below is a example email:

Violation of usage policy

A violation of the usage policy by ccvdemo (CCV Demo,,,,ccvdemo@brown.edu) on login006 was automatically detected starting at 08:53 on 04/25.

This may indicate that you are running computationally-intensive work on the interactive/login node (when it should be run on compute nodes instead). Please utilize the 'interact' command to initiate a SLURM session on a compute node and run your workloads there.

You now have the status penalty1 because your usage has exceeded the thresholds for appropriate usage on the node. Your CPU usage is now limited to 80% of your original limit (8.0 cores) for the next 30 minutes. In addition, your memory limit is 80% of your original limit (40.0 GB) for the same period of time.

These limits will apply on login006.

High-impact processes

Usage values are recent averages. Instantaneous usage metrics may differ. The processes listed are probable suspects, but there may be some variation in the processes responsible for your impact on the node. Memory usage is expressed in GB and CPU usage is relative to one core (and may exceed 100% as a result).

Recent system usage

*This process is generally permitted on interactive nodes and is only counted against you when considering memory usage (regardless of the process, too much memory usage is still considered bad; it cannot be throttled like CPU). The process is included in this report to show usage holistically.

**This accounts for the difference between the overall usage and the collected PID usage (which can be less accurate). This may be large if there are a lot of short-lived processes (such as compilers or quick commands) that account for a significant fraction of the total usage. These processes are whitelisted as defined above.

Required User Actions

When a user receives an alert email that the user is put in a penalty status, the user should

• kill the processes that use too much resources on the shared node listed in the alert email, and/or reduce the resources used by the processes

• submit an interactive job, a batch job, or an interactive Open OnDemand app to run computational intensive programs including but not limited to Python, R and Matlab

• consider attending CCV workshops or tutorials to learn more about correctly using Oscar.

Exempt Processes

Essential Linux utilities, such as rsync, cp, scp, SLURM commands, creating Singularity images, and code compilation, are exempt. To obtain a comprehensive list, please get in touch with us

Agent Forwarding with PuTTY

1. Once adding your private key to Pageant, open PuTTY and navigate to the Auth menu.

2. Check the 'Allow agent forwarding' checkbox, and return to the Session menu.

3. Enter the Host Name you usually use to connect to Oscar, and click 'Open'.

4. Entering your password. If you have ssh keys setup on your local computer to connect to GitHub, you can confirm your ssh-agent was properly forwarded by checking GitHub . If the ssh command fails, your agent has not been properly forwarded.

Start the SSH-Agent

First, start your ssh-agent with the command below.

You should see an output similar to this:

Add Key(s)

Next, add your ssh private keys to the running agent (using the ssh-add command on line 1). This step may be repeated for every key pair you use to connect to different git servers. For most, this file is called id_rsa and will live in ~/.ssh/id_rsa. If you set a password for your ssh keys, the agent will prompt you to enter them.

Confirm the ssh keys have been loaded into the agent with ssh-add -L:

Connect to Oscar

Now ssh into Oscar with the -A option as shown on the first line below (replace username with your Oscar username). -A will forward your ssh-agent to Oscar, enabling you to use the ssh keys on your laptop while logged into Oscar.

If you have ssh keys setup on your local computer to connect to GitHub, you can confirm your ssh-agent was properly forwarded by checking GitHub . If the ssh command fails, your agent has not been properly forwarded.

Always connecting with Agent Forwarding

To make these changes permanent, you can add the ForwardAgent yes option to your ssh configuration file. To learn more about configuring your ssh connections, visit