howto-make-dlcs.md

Auto-generation of Design Load Cases

WARNING: these notes contain configuration settings that are specif to the DTU Wind Energy cluster Gorm. Only follow this guide in another environment if you know what you are doing!

Introduction

For the auto generation of load cases and the corresponding execution on the cluster, the following events will take place:

• Create an htc master file, and define the various tags in the exchange files (spreadsheets).
• Generate the htc files for all the corresponding load cases based on the master file and the tags defined in the exchange files. Besides the HAWC2 htc input file, a corresponding pbs script is created that includes the instructions to execute the relevant HAWC2 simulation on a cluster node. This includes copying the model to the node scratch disc, executing HAWC2, copying the results from the node scratch disc back to the network drive.
• Submit all the load cases (or the pbs launch scripts) to the cluster queueing system. This is also referred to as launching the jobs.

Important note regarding file names. On Linux, file names and paths are case sensitive, but on Windows they are not. Additionally, HAWC2 will always generate result and log files with lower case file names, regardless of the user input. Hence, in order to avoid possible ambiguities at all times, make sure that there are no upper case symbols defined in the value of the following tags (as defined in the Excel spreadsheets): [Case folder], [Case id.], and [Turb base name].

The system will always force the values of the tags to be lower case anyway, and when working on Windows, this might cause some confusing and unexpected behaviour. The tags themselves can have lower and upper case characters as can be seen in the example above.

Notice that throughout the document $USER refers the your user name. You can either let the system fill that in for you (by using the variable $USER), or explicitly user your user name instead. This user name is the same as your DTU account name (or student account/number).

This document refers to commands to be entered in the terminal on Gorm when the line starts with g-000 $. The command that needs to be entered starts after the $.

Connecting to the cluster

You connect to the cluster via an SSH terminal. SSH is supported out of the box for Linux and Mac OSX terminals (such as bash), but requires a separate terminal client under Windows. Windows users are advised to use PuTTY and can be downloaded at: http://www.chiark.greenend.org.uk/~sgtatham/putty/. Here's a random tutorial, you can use your favourite search engine if you need more or different instructions. More answers regarding PuTTY can also be found in the online documentation.

The cluster that is setup for using the pre- and post-processing tools for HAWC2 has the following address: gorm.risoe.dk.

On Linux/Mac connecting to the cluster is as simple as running the following command in the terminal:

ssh $USER@gorm.risoe.dk

Use your DTU password when asked. This will give you terminal access to the cluster called Gorm.

The cluster can only be reached when on the DTU network (wired, or only from a DTU computer when using a wireless connection), when connected to the DTU VPN, or from one of the DTU databars.

More information about the cluster can be found on the Gorm-wiki

Mounting the cluster discs

You need to be connected to the DTU network in order for this to work. You can also connect to the DTU network over VPN.

When doing the HAWC2 simulations, you will interact regularly with the cluster file system and discs. It is convenient to map these discs as network drives (in Windows terms). Map the following network drives (replace $USER with your user name):

\\mimer\hawc2sim
\\gorm\$USER # this is your Gorm home directory

Alternatively, on Windows you can use WinSCP to interact with the cluster discs.

Note that by default Windows Explorer will hide some of the files you will need edit. In order to show all files on your Gorm home drive, you need to un-hide system files: Explorer > Organize > Folder and search options > select tab "view" > select the option to show hidden files and folders.

From Linux/Mac, you should be able to mount using either of the following addresses:

//mimer.risoe.dk/hawc2sim
//mimer.risoe.dk/well/hawc2sim
//gorm.risoe.dk/$USER

You can use either sshfs or mount -t cifs to mount the discs.

Preparation

Add the cluster-tools script to your system's PATH of you Gorm environment, by editing the file .bash_profile file in your Gorm’s home directory (/home/$USER/.bash_profile), and add the following lines (add at the end, or create a new file with this file name in case it doesn't exist):

export PATH=$PATH:/home/MET/repositories/toolbox/pbsutils/

(The corresponding open repository is on the DTU Wind Energy Gitlab server: pbsutils. Please considering reporting bugs and/or suggest improvements there. You're contributions are much appreciated!)

If you have been using an old version of this how-to, you might be pointing to an earlier version of these tools/utils and any references containing cluster-tools or prepost should be removed from your .bash_profile file.

After modifying .bash_profile, save and close it. Then, in the terminal, run the command:

g-000 $ source ~/.bash_profile

You will also need to configure wine and place the HAWC2 executables in your local wine directory, which by default is assumed to be ~/.wine32, and pbsutils contains and automatic configuration script you can run:

g-000 $ config-wine-hawc2.sh

If you need more information on what is going on, you can read a more detailed description [here] (https://gitlab.windenergy.dtu.dk/toolbox/WindEnergyToolbox/blob/master/docs/configure-wine.md).

All your HAWC2 executables and DLL's are now located at /home/$USER/.wine32/drive_c/bin.

Notice that the HAWC2 executable names are hawc2-latest.exe, hawc2-118.exe, etc. By default the latest version will be used and the user does not need to specify this. However, when you need to compare different version you can easily do so by specifying which case should be run with which executable.

Log out and in again from the cluster (close and restart PuTTY).

At this stage you can run HAWC2 as follows:

g-000 $ wine32 hawc2-latest htc/some-intput-file.htc

Updating local HAWC2 executables

When there is a new version of HAWC2, or when a new license manager is released, you can update your local wine directory as follows:

g-000 $ cp /home/MET/hawc2exe/* /home/$USER/.wine32/drive_c/bin/

The file hawc2-latest.exe will always be the latest HAWC2 version at /home/MET/hawc2exe/. When a new HAWC2 is released you can simply copy all the files from there again to update.

Method A: Generating htc input files on the cluster

Use ssh (Linux, Mac) or putty (MS Windows) to connect to the cluster.

With qsub-wrap.py will be used to create a job for the cluster that will run the htc file generator within the correct environment Python environment. A emplate for such a file, which works for standard DLBs, can be found here:

/home/MET/repositories/toolbox/WindEnergyToolbox/wetb/prepost/dlctemplate.py

For example, in order to generate the default IEC DLCs:

g-000 $ cd path/to/HAWC2/model # folder where the hawc2 model is located
g-000 $ qsub-wrap.py -f /home/MET/repositories/toolbox/WindEnergyToolbox/wetb/prepost/dlctemplate.py --prep

You could consider copying the template into the HAWC2 model folder to avoid typing such a long command over and over again.

Note that the following folder structure for the HAWC2 model is assumed:

|-- control
|   |-- ...
|-- data
|   |-- ...
|-- htc
|   |-- DLCs
|   |   |-- dlc12_iec61400-1ed3.xlsx
|   |   |-- dlc13_iec61400-1ed3.xlsx
|   |   |-- ...
|   |-- _master
|   |   `-- dtu10mw_master_C0013.htc

The load case definitions should be placed in Excel spreadsheets with a *.xlsx extension. The above example shows one possible scenario whereby all the load case definitions are placed in htc/DLCs (all folder names are case sensitive). Alternatively, one can also place the spreadsheets in separate sub folders, for example:

|-- control
|   |-- ...
|-- data
|   |-- ...
|-- htc
|   |-- dlc12_iec61400-1ed3
|   |   |-- dlc12_iec61400-1ed3.xlsx
|   |-- dlc13_iec61400-1ed3
|   |   |-- dlc13_iec61400-1ed3.xlsx

In order to use this auto-configuration mode, there can only be one master file in _master that contains _master_ in its file name.

For the NREL5MW and the DTU10MW HAWC2 models, you can find their respective master files and DLC definition spreadsheet files on Mimer. When connected to Gorm over SSH/PuTTY, you will find these files at:

/mnt/mimer/hawc2sim # (when on Gorm)

Method B: Generating htc input files interactively on the cluster