Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • python-at-risoe/scientific-python-workshops/2-getting-started
  • lapm/2-getting-started
2 results
Show changes
Commits on Source (3)
# python notebook checkpoints
materials/.ipynb_checkpoints/
\ No newline at end of file
%% Cell type:markdown id: tags:
# NumPy Exercise: Correlated Gaussian Random Variables
%% Cell type:markdown id: tags:
In many situations, we need to have a series of correlated Gaussian random variables, which can then be transformed into other distributions of interest (uniform, lognormal, etc.). Let's see how to do that with NumPy in Python.
### Given:
|Variable | Value | Description |
| ---: | :---: | :--- |
|`n_real` | `1E6` | number of realizations|
|`n_vars` | 3 | number of variables to correlate|
|`cov` | `[[ 1. , 0.2, 0.4], [ 0.2, 0.8, 0.3], [ 0.4, 0.3, 1.1]]` | covariance matrix|
### Theory
The procedure for generating correlated Gaussian is as follows:
1. Sample `[n_vars x n_real]` (uncorrelated) normal random variables
2. Calculate `chol_mat`, the Cholesky decomposition of the covariance matrix
3. Matrix-multiply your random variables with `chol_mat` to produce a `[n_vars x n_real]` array of correlated Gaussian variables
### Exercise
Do the following:
1. Fill in the blank cells below so that the code follows the theory outlined above.
2. Calculate the variances of the three samples of random variables. Does it match the diagonal of the covariance matrix?
3. Calculate the correlation coefficient between the first and second random samples. Does it match `cov[0, 1]`?
### Hints
- In the arrays of random variables, each row `i` corresponds to a *sample* of random variable `i` (just FYI).
- Google is your friend :)
%% Cell type:code id: tags:
``` python
import numpy as np # import any needed modules here
```
%% Cell type:code id: tags:
``` python
n_real = int(1E6) # number of realizations
n_vars = 3 # number of random variables we want to correlate
cov = np.array([[ 1. , 0.2, 0.4], [ 0.2, 0.8, 0.3], [ 0.4, 0.3, 1.1]]) # covariance matrix
```
%% Cell type:code id: tags:
``` python
unc_vars = np.random.randn(n_vars, n_real) # create [n_vars x n_real] array of uncorrelated (unc) normal random variables
```
%% Cell type:code id: tags:
``` python
chol_mat = np.linalg.cholesky(cov) # calculate the cholesky decomposition of the covariance matrix
```
%% Cell type:code id: tags:
``` python
cor_vars = chol_mat @ unc_vars # [n_vars x n_real] array of correlated (cor) random variables
```
%% Cell type:code id: tags:
``` python
cor_vars.var(axis=1) # calculate variances of each sample of random variables
```
%% Cell type:code id: tags:
``` python
np.corrcoef(cor_vars[0, :], cor_vars[1, :]) # calculate the correlation coefficient between the first and second random samples
```
%% Cell type:markdown id: tags:
# Matplotlib Exercise: Visualizing Correlated Gaussian Random Variables
%% Cell type:markdown id: tags:
Now that we know how to generate correlated random variables, let's visualize them.
### Exercise
Make the following plots. All plots must have x and y labels, titles, and legends if there is more than one dataset in the same axes.
1. Overlaid histograms of your samples of uncorrelated random variables with 30 bins (use `histtype='step'`)
2. A scatterplot of $X_2$ vs $X_1$ with marker size equal to 2. Overlay the the theoretical line ($y=x$) in a black, dashed line.
3. Overlaid histograms of your samples of correlated random variables with 30 bins (use `histtype='step'`)
### Hints
- In the arrays of random variables, each row `i` corresponds to a *sample* of random variable `i` (just FYI).
- Google is your friend :)
%% Cell type:code id: tags:
``` python
import matplotlib.pyplot as plt # need to import matplotlib, of course
import numpy as np # import any needed modules here
```
%% Cell type:code id: tags:
``` python
n_real = int(1E6) # number of realizations
n_vars = 3 # number of random variables we want to correlate
cov = np.array([[ 1. , 0.2, 0.4], [ 0.2, 0.8, 0.3], [ 0.4, 0.3, 1.1]]) # covariance matrix
```
%% Cell type:code id: tags:
``` python
unc_vars = np.random.randn(n_vars, n_real) # create [n_vars x n_real] array of uncorrelated (unc) normal random variables
```
%% Cell type:code id: tags:
``` python
chol_mat = np.linalg.cholesky(cov) # calculate the cholesky decomposition of the covariance matrix
```
%% Cell type:code id: tags:
``` python
cor_vars = chol_mat @ unc_vars # [n_vars x n_real] array of correlated (cor) random variables
```
%% Cell type:code id: tags:
``` python
cor_vars.var(axis=1) # calculate variances of each sample of random variables
```
%% Cell type:code id: tags:
``` python
np.corrcoef(cor_vars[0, :], cor_vars[1, :]) # calculate the correlation coefficient between the first and second random samples
```
%% Cell type:markdown id: tags:
## Plot 1: Histogram of Uncorrelated Variables
%% Cell type:markdown id: tags:
Make a plot with overlaid histograms of your samples of uncorrelated random variables with 30 bins (use histtype='step').
%% Cell type:code id: tags:
``` python
# insert code here
```
%% Cell type:markdown id: tags:
## Plot 2: Scatterplot of X2 vs. X1
%% Cell type:markdown id: tags:
Make a scatterplot of $X_2$ vs $X_1$ with marker size equal to 2. Overlay the the theoretical line ($y=x$) in a black, dashed line.
%% Cell type:code id: tags:
``` python
# insert code here
```
%% Cell type:markdown id: tags:
## Plot 3: Histogram of Correlated Variables
%% Cell type:markdown id: tags:
Make a plot with overlaid histograms of your samples of uncorrelated random variables with 30 bins (use histtype='step').
%% Cell type:code id: tags:
``` python
# insert code here
```
Source diff could not be displayed: it is too large. Options to address this: view the blob.
%% Cell type:markdown id: tags:
# Testing You in Pandas
%% Cell type:markdown id: tags:
Given the Risø V52 met mast data, let's select and make some plots.
### Exercises
Make the following plots:
1. For a wind speed sector from 250 to 340 degrees, plot a histogram of the mean cup anemometer mean wind speed at 70 m.
2. Scatter plot the turbulence intensity versus the mean wind speed for two heights (your choice). Overlaid plots are preferred, but if you can't figure out how to do it, then separate plots are fine.
Be sure to add axis labels, legends, titles, etc., where applicable.
%% Cell type:markdown id: tags:
## Preliminaries
%% Cell type:markdown id: tags:
As always, we must first import the modules we want to use before we can write any code. I'm also setting the jupyter matploblib option to be interactive, as we can do in notebooks.
%% Cell type:code id: tags:
``` python
% matplotlib notebook
import matplotlib.pyplot as plt
import pandas as pd
```
%% Cell type:markdown id: tags:
Load the mean and std dev `.csv` files to dataframes and concatenate them.
%% Cell type:code id: tags:
``` python
dfs = []
for csv_name, suffix in zip(['demo_risoe_data_means.csv', 'demo_risoe_data_stdvs.csv'], ['_mean', '_stdv']):
df_path = f'data/{csv_name}'
df = pd.read_csv(df_path) # load csv to dataframe
df['name'] = pd.to_datetime(df['name'].astype(str), format='%Y%m%d%H%M') # convert name to datetime
df.set_index('name', inplace=True) # set name as index
df = df.add_suffix(suffix)
dfs.append(df)
met_df = pd.concat(dfs, axis=1); # suppress jupyter output with semicolon
```
%% Cell type:markdown id: tags:
## Exercise 1: Wind Speed Histogram for Sector
For a wind speed sector from 250 to 340 degrees, plot a histogram of the mean cup anemometer mean wind speed at 70 m.
%% Cell type:code id: tags:
``` python
ax = met_df[(met_df.Wdir_41m_mean >= 250) & (met_df.Wdir_41m_mean <= 340)].plot(y='Wsp_70m_mean', kind='hist')
ax.set_xlabel('Cup Anem. Wind Speed @ 70 m')
ax.legend().set_visible(False) # hide the legend since we have an x axis
```
%% Output
%% Cell type:markdown id: tags:
## Exercise 2: TI vs U
Scatter plot the turbulence intensity versus the mean wind speed for two heights (your choice).
%% Cell type:code id: tags:
``` python
heights = [44, 57, 70] # I'll do three because it's fun!
colors = ['r', 'g', 'b']
ax = None
for i_ht, ht in enumerate(heights):
met_df[f'TI_{ht}m'] = met_df[f'Wsp_{ht}m_stdv'] / met_df[f'Wsp_{ht}m_mean']
ax = met_df.plot(x=f'Wsp_{ht}m_mean', y=f'TI_{ht}m', kind='scatter',
c=colors[i_ht], s=2, label=f'{ht} m', ax=ax)
ax.legend()
ax.set_xlabel('Mean Wind Speed')
ax.set_ylabel('Turbulence Intensity');
```
%% Output
%% Cell type:code id: tags:
``` python
```