developer-guide.md

Developer guide

Thank you for your interest in developing wetb. This guide details how to contribute to wetb in a way that is efficient for everyone.

Contents

• Fork
• Requirements
• Install Python
• Install/build dependencies
• Get wetb
• Install wetb
• Contributions
• Upload contributions
• Make and upload wheels

Fork project

We prefer that you make your contributions in your own fork of the project, make your changes and make a merge request.

The project can be forked to your own user account via the <Fork> button on the frontpage

Requirements

Command line

This guide will use the command line (aka command prompt) frequently. You can launch a Windows terminal as follows: press Start> and type "cmd" + <Enter>. A link to the command prompt should be visible now.

In case you want an alternative, more capable windows terminal, you could consider using ConEmu (this is optional).

ConEmu-Maximus5 is a Windows console emulator with tabs, which presents multiple consoles and simple GUI applications as one customizable GUI window with various features.

Git

• Download and install Git version control system for Windows 64-bit here. Only select the Windows Portable option if you know what you are doing or if you do not have administrative rights on your computer.

• Git comes with a simple GUI, but there are more and different options available if you are not happy with it, see here.

• On windows we highly recommend tortoisegit. It is a gui integrated into the windows explorer.

Install Python

For all platforms we recommend that you download and install the Anaconda - a professional grade, full blown scientific Python distribution.

Installing Anaconda, activate root environment

• Download and install Anaconda (Python 3.5 version, 64 bit installer is recommended) from https://www.continuum.io/downloads

Note: The Python 2.7 or Python 3.5 choice of Anaconda only affects the root environment. You can always create additional environments using other Python versions, see below.

• Update the root Anaconda environment (type in a terminal):

>> conda update --all

• Activate the Anaconda root environment in a terminal as follows:

>> activate

and your terminal will do something like:

C:\Users\> activate
(root) C:\Users\>

note that the name of the environment is now a prefix before the current path.

use deactivate to deactivate the environment.

Optionally, create other independent Anaconda environments

By using environments you can manage different Python installations with different versions on your system. Creating environments is as easy as:

>> conda create -n py27 python=2.7
>> conda create -n py34 python=3.4
>> conda create -n py35 python=3.5

These environments can be activated as follows:

>> activate py27
>> activate py34
>> activate py35

The Python distribution in use will now be located in e.g. <path_to_anaconda>/env/py35/

use deactivate to deactivate the environment.

Install/build dependencies

• Compiler (wetb contains cython extensions that require a compiler):
  • Linux: gcc (should be installed by default)
  • Windows:
    • Python 2.7: Microsoft Visual C++ Compiler for Python 2.7, or the direct link.
    • Python 3.4: MS Visual Studio 2010
    • Python 3.5: MS Visual Studio 2015 or Visual C++ Build Tools
    • Only one MS Visual Studio version can be installed, but you can for example install MS Visual Studio 2010 alongside the Visual C++ Build Tools.
• numpy
• cython
• scipy
• pandas
• xlrd and xlwt from python-excel
• openpyxl
• h5py
• matplotlib
• pytables
• pyscaffold
• pytest
• pytest-cov
• six, future
• nose, sphinx, blosc, pbr, psutil, coverage, setuptools_scm
• parimeko
• sshtunnel
• pandoc , pypandoc: convert markdown formatted readme file to rst for PyPi compatibility. See also issue #22 (closed). pandoc is available in Anaconda. When installing pypandoc via pip, you have to install pandoc via your package manager (Linux/Mac).
• twine: upload package to PyPi

Install the necessary Python dependencies using the conda package manager:

>> conda install setuptools_scm future h5py pytables pytest pytest-cov nose sphinx blosc pbr paramiko
>> conda install scipy pandas matplotlib cython xlrd coverage xlwt openpyxl psutil pandoc
>> conda install -c conda-forge pyscaffold sshtunnel twine pypandoc --no-deps

Note that --no-deps avoids that newer packages from the channel conda-forge will be used instead of those from the default anaconda channel. Depending on which packages get overwritten, this might brake your Anaconda root environment. As such, using --no-deps should be used for safety (especially when operating from the root environment).

Note that:

• With Python 2.7, blosc fails to install.
• With Python 3.6, twine, pypandoc fails to install.

Get wetb

Copy the https - link on the front page of your fork of wetb

>> git clone <https-link>

or via tortoise-git:

• Right-click in your working folder
• "Git Clone..."
• <Ok>

Install wetb

>> cd WindEnergyToolbox
>> pip install -e . --no-deps

Note that the --no-deps option here is used for the same reason as explained above for the conda-forge channel: it is to avoid that pip will replace newer packages compared to the ones as available in the Anaconda channel.

Update wetb

>> cd WindEnergyToolbox
>> git pull
>> pip install -e . --no-deps

Run tests

Note that the test should be executed from a clean repository and which is not used as a development installation with pip install -e .. For example, create a clone of your local git repository in which your development takes place, but name the top level folder to something else:

>> git clone WindEnergyToolbox/ wetb_tests
>> cd wetb_tests

In order to make sure your git repository is clean, this will remove all untracked files, and undo all untracked changes. WARNING: you will loose all untracked files and changes!!

>> git clean -df & git checkout .

Now we have clean repository that is not used as a development installation directory, and we simply track our own local development git repository. Use git pull to get the latest local commits.

>> python -m pytest --cov=wetb

Contributions

If you make a change in the toolbox, that others can benefit from please make a merge request.

If you can, please submit a merge request with the fix or improvements including tests.

The workflow to make a merge request is as follows:

• Create a feature branch, branch away from master
• Write tests and code
• Push the commit(s) to your fork
• Submit a merge request (MR) to the master branch of
• Link any relevant issues in the merge request description and leave a comment on them with a link back to the MR
• Your tests should run as fast as possible, and if it uses test files, these files should be as small as possible.
• Please keep the change in a single MR as small as possible. Split the functionality if you can

Upload contributions

To be written

Make and upload wheels to PyPi

Workflow for creating and uploading wheels is as follows:

• Make tag: git tag "vX.Y.Z", and push tag to remote: git push --tags
• In order to have a clean version number (which is determined automagically) make sure your git working directory is clean (no uncommitted changes etc).
• pip install -e . --upgrade
• python setup.py bdist_wheel -d dist (wheel includes compiled extensions)
• On Linux you will have to rename the binary wheel file (see PEP 513 for a background discussion):
  • from: wetb-0.0.5-cp35-cp35m-linux_x86_64.whl
  • to: wetb-0.0.5-cp35-cp35m-manylinux1_x86_64.whl
• python setup.py sdist -d dist (for general source distribution installs)
• twine upload dist/*

In case of problems:

• Make sure the version tag is compliant with PEP 440, otherwise twine upload will fail. This means commit hashes can not be part of the version number. Note that when your git working directory is not clean, the scheme for automatic versioning number will add dirty to the version number.