NSC's Centre Storage system is a large disk storage system that is connected to all our academic compute systems (Triolith, Kappa and Matter).
All user and project data on Triolith, Kappa and Matter is stored on Centre Storage. It also replaces the "Vagn" system.
Previously, we have sometimes referred to the Vagn replacement ("Vagn2", "DCS") and the Centre Storage replacement as two separate systems. In the end, it was decided to procure a single large storage system to serve both user communities.
Storage will be allocated to each project based on their needs, and the available space, regardless of project type. The process for applying for storage space will differ somewhat depending on how much space you ask for (see below).
The available space will initially be approximately 2800 TiB (more than three times the combined size of the systems being replaced).
Most of this space will be available for project storage for new and existing projects. However, due to the need to save some space for future new projects, not all space will be allocated initially.
A small part (3-4%) of the space will be used for home directories and software installations, and some will be used for "snapshots" (enables users to restore deleted files for a limited time on all file systems).
All computing projects on Triolith, Kappa and Matter will automatically be allocated a small (currently 500 GiB) storage area for that project. This area will automatically be writable by all project members.
Please note that the initial limit of 500 GiB per project is set somewhat arbitrarily to a level which we believe will be enough for most small projects1.
We are well aware that 500 GiB will not be enough for many projects. If your project needs to store more data, you are welcome to contact NSC to apply for more space (see below).
Send an email to email@example.com. Include the following information:
The project PI should send the application to NSC (we will accept an application from a senior member of the project if it is clear that the PI is aware of the application, e.g by Cc:ing the PI).
Small allocations (no fixed limit, but up to a few TiB) will usually be processed within 1-2 working days, and usually granted.
Medium allocations (up to 50 TiB) will usually be processed monthly. These will have to fit into our long-term storage plan (to avoid running out of space before the system can be expanded).
In some cases we might get back to you and request additional details, or suggest improvements that can be made to how you store data.
Only groups that have already been allocated computing time on Triolith, Kappa or Matter can apply for Small or Medium storage allocations.
In late 2014, NSC will start accepting proposals for large storage allocations (these have sometimes been known as "DCS" - "Data-intensive Compute Service")
Applications for large storage allocations will be evaluated at least twice per year. Usually to coincide with the processing of SNAC large compute allocations.
To qualify, you must have a need for large-scale storage (>50TiB per project).
Since Centre Storage is not an archive but a place to store data that is actively being used, allocations will primarily be granted to projects who will use NSC compute resources to produce or analyze the data they will be storing.
To apply for a large allocation, use SUPR and apply in the round "DCS 2014"
By default, each project storage area is limited to one million files.
As storing many small files cause certain performance problems (e.g when checking/repairing damage to the file system), we will only raise this limit if you can show that you have a genuine need to store very many (i.e small) files.
If you need the limit to be raised, please email firstname.lastname@example.org and explain why, and how many files you need to store.
When your project ends, your project directory remains writable for 30 days, to give running jobs time to finish and to let you collect your data, do post-processing and transfer data away from NSC. If you have nowhere to store your data, we recommend National Storage.
When your 30 days are up, the project directory remains read-only for another 30 days.
After that time (i.e 60 days after your project ends), your data is deleted! It is the responsibility of the project PI to plan for what to do with the data when the project ends.
NSC Centre Storage is not a long-term archive for your research data!
If you are granted an new project with a storage allocation at NSC before the 60 days are up, you can choose to retain your old project directory connected to the new project. You can do this from NSC Express or by contacting NSC.
Unfortunately SNAC currently does not handle the storage part of the project application form fully.
SNAC knows how many core hours it may give out on each system, but does not yet know how much storage it can allocate, so this is left up to the centers to decide.
NSC currently considers the storage numbers on your SNAC application as a guideline.
If you requested a certain amount of storage on your SNAC application and it was approved by SNAC without any comments, it's very likely that NSC will approve a request for the same amount of space, but you need to ask us first, it will not happen automatically.
In May 2014, 75% of all active projects on Triolith used less than 500 GiB of data on /nobackup↩