Proposed F2F meeting agenda 29 Aug 2018
08:30 Maria and Martin arrive at MAX IV
09:00 - 09:40 PReSTO STATUS AND DEMO (Martin Moche)
Demo the HPC-MX setup at the MAX IV cluster:
- PReSTO menu
- launch XDSAPP/XDSGUI at one compute node
- Match JOBS and PROCESSORS with total number of compute node cores allocated
- native.script and anomalous.script for convenient data processing with XDSAPP/DIALS/autoPROC
- MR-SAD via autoSHARP, CCP4i2 vs mr_rosetta in PHENIX using 3GET
- Run PHENIX-GUI at login node submit jobs to a single compute node with many cores
- Demo Phenix-GUI settings
- Launching PHENIX jobs from GUI vs sbatch script and parameter.eff file
- Trello for internal communication
- PyMOL is parallel when creating movies for PowerPoint
- Launch ccp4mg and chimera from PReSTO menu
- Arcimboldo_Lite and shelxl is very parallel
- hkl2map is very parallel except shelxe
- Use 3G1U to demo Coot graphics performance at laptop PC vs Triolith/Aurora/MAX IV
09:40 - 10:00 DEMO CrystFEL data processing for XFEL (Oskar Aurelius)
- Show Cryst-FEL version 0.7.0 performing data processing
10:00 – 10:30 Coffee
The NSC participants that travel in the morning join at this time - Torben, Torbjörn, and Sebastian.
10:30 – 11:30 FEEDBACK AND PERSPECTIVES FROM PRESTO USERS
- Martin will prepare questions and organise with a few users (2-3) to provide feedback regarding using PReSTO.
- 10:30 - 10:50 Martin Hällberg on Zoom
- 10:50 - 11:10 Joseph Brock? on Zoom
- 11:10 - 11:30 Oskar Aurelius
11:30 – 12:00 PReSTO: context, people, aim (Maria)
- relation to SNIC (Swedish National Infrastructure of Compute)
- national and international context, long-term vision
- project working model and organization
- staff and roles (round-table presentation)
12:00 - 13:00 Lunch
13:00 – 14:30 KNOWLEDGE TRANSFER
From PReSTO to MAXIV (Sebastian/Torbjörn and Torben)
- What advantages do EasyBuild offer compared to standard HPC installation?
- Easyconfigs for build specifications
- Easyblocks python modules for software build and installation procedures
- Easyconfigs vs. Easyblocks
- Human readable dependencies, demo CCP4 update including its dependencies
- Highlights from PReSTO installation with EasyBuild
- Unusual or costly dependencies
- Some MX software lack version control
- Differences between Triolith/Aurora/MAXIV (if any)
- Home-made hacks required in PReSTO
- Manipulation of forkxds script for multi-node XDS runs
- MX-HPC adaptions suggested to software developers
- Make ccp4i2 working with SLURM
- Make multi-node jobs runnable in PHENIX when using SLURM
From MAX IV to PReSTO (Johan/Jie/Ana/Magnus/Gustavo)
Processing of BioMAX data today during MAX IV visit
- Automatic. What is used today: XDS, EDNA, autoPROC.
- .mtz-files, normal, anomalous
- Manual.
- Access to Presto and the MAX IV cluster through thinlinc, shell that automatically brings you to PReSTO environment cluster.
- Practices.
- Data collection session is fast and intensive. Autoprocessing during beamtime mainly fastDP (Diamond Light Source) or XDS output.
- What will users need for data reduction, phasing, structure determination etc.
- Reduction (XDS) absolutely necessary. Structure determination should be the goal; Dimple, Bessy pipeline.
- Future. How to work. Bottle necks. Challenges.
- Pipelines need to be more realible? Need to know on the fly if they got data to some quality.
- Would be nice to challenge user to analyse data as it comes. Can thinlinc/presto be more easily presented?
- Implement for structure solution. Fragment screening.
After beamtime/from home.
- Users might reprocess. Fragment screening can be many dataset.
- Easy access and seamless transfer between infrastructures?
- Access to data via Lunarc, how is it expected to work? How long after data collection? When is this service scheduled to be implemented? MAX IV user ID and Lunarc ID.
- Remote access.
- Access to MAX IV cluster shift? Cluster at MAX IV? What makes it different from using Lunarc?
Who gets access today?
- Academic.
- Presto access. Access to Lunarc if Swedish. License situation...
- Industrial.
- Access to automatic processing using XDS and CCP4 using purchased licenses. Access to cluster@MAXIV? access to PReSTO? Access to Lunarc/SNIC?
How to make priorities for the remaining four months of this funding of PReSTO - Tentative discussion points
- Users and PReSTO usage: How to get users more involved in analysing data online?
- Can thinlinc/presto be more easily presented?
- How can we promote users to take advantage of PReSTO and HPCs offline?
- How can users be adviced – training, youtube channels..?
- What more SW shold be included? EDNA? Pipelines? FS-pipeline (Gustavo), Python3? DIALS in cross node mode.
- Can PReSTO installation be used for automated scripts for data processing BioMAX?
- Can we support the same software?
- Pipelines need to be more reliable; need to know data quality on the fly.
- Can NSC assist in developing automated data processing scripts at BioMAX or for analysing user data beyond data processing? Support parallel computing?
- How can we promote industrial involvment and the access they need?
- Seamless transfer between infrastructures from user perspective? What is missing?
- How can maintenance of PReSTO be operated with no aditional funding?
14:30 – 15:00 BIOMAX AND LUNARC COLLABORATION (Artur Barczyk)
- Data transfer from BioMAX to LUNARC during beam time
- For academics
- For industry
- BioMAX data storage - current status and future?
- Short term during beam time
- Medium term 1-2 months after beam time
- Long term after 6-12 months
15:00 – 15:30 Coffee
15:30 – 16:30 MAX IV and PReSTO for industry (Johan Unge)
- What services would MAX IV like to provide for industry?
- What services are industry looking for?
- Comment by Derek Logan and Marie Håkansson from Saromics Biostructures
- Beam time and support for using some Cloud service
- Comment by Erik Lindahl on Zoom
16:30 - 16:45 International users
- If possible, we will try to get an update from SNIC/MAX IV discussions regarding this.
User Area
