Oscar's Filesystem

CCV uses IBM's General Parallel File System (GPFS). Users have a home, data, and scratch space.

home ~

• 20GB of space

• Optimized for many small files

• Nightly backups (7 days)

• Quota is per individual user

• Grace period of 14 days

data ~/data

• Each PI gets 256GB for free

• Optimized for reading large files

• Nightly backups (7 days)

• Quota is by group

• Grace period of 14 days

scratch ~/scratch

• 512GB of space - contact [email protected] increase on a temporary basis

• Optimized for reading/writing large files

• NO BACKUPS

• Purging: files not accessed for 30 days may be deleted

• Quota is per individual user

• Grace period of 21 days

Files not accessed for 30 days will be deleted from your scratch directory. This is because scratch is high performance space. The fuller scratch is the worse the read/write performance. Use ~/data for files you need to keep long term.

The scratch purge is on individual files. It is by 'atime' which is when the file was last read. You can use 'find' to find files that are at risk of being purged, e.g. to find files in the current directory that have not been accessed in the last 25 days:

find . -atime +25

A good practice is to configure your application to read any initial input data from ~/data and write all output into ~/scratch. Then, when the application has finished, move or copy data you would like to save from ~/scratch to ~/data.

Note: class or temporary accounts may not have a ~/data directory!

To see how much space you have on Oscar you can use the command myquota. Below is an example output

                       Block Limits                              |           File Limits              
    Type    Filesystem           Used    Quota   HLIMIT    Grace |    Files    Quota   HLIMIT    Grace
    -------------------------------------------------------------|--------------------------------------
    USR     home               8.401G      20G      30G        - |    61832   524288  1048576        -
    USR     scratch              332G     512G      12T        - |    14523   323539  4194304        -
    FILESET data+apollo        11.05T      20T      24T        - |   459764  4194304  8388608        -

You can go over your quota up to the hard limit for a grace period. This grace period is to give you time to manage your files. When the grace period expires you will be unable to write any files until you are back under quota.

There is a quota for space used and for number of files. If you hit the hard limit on either of these you will be unable to write any more files until you are back under quota.