Skip to content

Commit fbb7793

Browse files
committed
added test directory in examples
broken code for now changed directory name necessary fixes might work improve kappa model by reducing noise better formatting intermediate changes additional changes improvement to tutorial docs build working fix build fixed make and added another jl file removed data from make.jl and updated alt_kappa added pre-rendered plots undid changes to Project.toml in docs small change to comments added surface flux example
1 parent 55ea3fa commit fbb7793

File tree

8 files changed

+564
-1
lines changed

8 files changed

+564
-1
lines changed

docs/Project.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,4 @@ Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
99
Thermodynamics = "b60c26fb-14c3-4610-9d3e-2d17fe7ff00c"
1010

1111
[compat]
12-
CloudMicrophysics = "0.5"
12+
CloudMicrophysics = "0.5"

docs/make.jl

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@ using EnsembleKalmanProcesses,
55
Documenter,
66
Plots, # so that Literate.jl does not capture precompilation output
77
Literate
8+
using Downloads
9+
using DelimitedFiles
810

911
# Gotta set this environment variable when using the GR run-time on CI machines.
1012
# This happens as examples will use Plots.jl to make plots and movies.
@@ -19,6 +21,7 @@ examples_for_literation = [
1921
"LossMinimization/loss_minimization.jl",
2022
"SparseLossMinimization/loss_minimization_sparse_eki.jl",
2123
"AerosolActivation/aerosol_activation.jl",
24+
"SurfaceFluxExample/kappa_calibration.jl",
2225
]
2326

2427
if isempty(get(ENV, "CI", ""))
@@ -42,6 +45,31 @@ for example in examples_for_literation
4245
)
4346
end
4447

48+
# The purpose of the code below is to remove the occurrences of the Documenter @example block
49+
# created by Literate in order to prevent Documenter from evaluating the code blocks from
50+
# kappa_calibration.jl. The reason the code blocks do not work is due to a package compatibility
51+
# mismatch, i.e. aerosol_activation is only compatible with CloudMicrophysics v0.5, and CloudMicrophysics v0.5
52+
# is only compatible with SurfaceFluxes v0.3. However, kappa_calibration.jl requires a new version of SurfaceFluxes,
53+
# version 0.6, making it impossible to evaluate the code blocks in the markdown file kappa_calibration.md without
54+
# running into errors. Thus, we filter out the @example blocks and merely display the code on the docs.
55+
56+
# Another reason we cannot evaluate the code blocks in kappa_calibration is that kappa_calibration depends
57+
# on locally downloaded data. Because we cannot download data to the remote repository, it is never plausible
58+
# to run kappa_calibration remotely.
59+
60+
# read file and copy over modified
61+
kappa_md_file = open("docs/src/literated/kappa_calibration.md", "r")
62+
all_lines = string("")
63+
while (!eof(kappa_md_file))
64+
line = readline(kappa_md_file) * "\n"
65+
line = replace(line, "@example kappa_calibration" => "")
66+
global all_lines *= line
67+
end
68+
69+
# write to file
70+
kappa_md_file = open("docs/src/literated/kappa_calibration.md", "w")
71+
write(kappa_md_file, all_lines)
72+
close(kappa_md_file)
4573
#----------
4674

4775
api = [
@@ -66,6 +94,7 @@ examples = [
6694
"Aerosol activation" => "literated/aerosol_activation.md",
6795
"TOML interface" => "examples/sinusoid_example_toml.md",
6896
"HPC interfacing example: ClimateMachine" => "examples/ClimateMachine_example.md",
97+
"Surface Fluxes" => "literated/kappa_calibration.md",
6998
"Template" => "examples/template_example.md",
7099
]
71100

36 KB
Loading
11.9 KB
Loading
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
[deps]
2+
CLIMAParameters = "6eacf6c3-8458-43b9-ae03-caf5306d3d53"
3+
Distributions = "31c24e10-a181-5473-b8eb-7969acd0382f"
4+
EnsembleKalmanProcesses = "aa8a2aa5-91d8-4396-bcef-d4f2ec43552d"
5+
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
6+
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
7+
SurfaceFluxes = "49b00bb7-8bd4-4f2b-b78c-51cd0450215f"
8+
Thermodynamics = "b60c26fb-14c3-4610-9d3e-2d17fe7ff00c"
Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# # Alternative Kappa Calibration Example
2+
# ## Overview
3+
#=
4+
In this example, just like in kappa_calibration.jl, we use the inverse problem to calibrate the von-karman constant, κ in
5+
the equation: u(z) = u^* / κ log (z / z0),
6+
which represents the wind profile in Monin-Obukhov
7+
Similarity Theory (MOST) formulations. We use the same dataset: https://turbulence.pha.jhu.edu/Channel_Flow.aspx
8+
9+
Instead of using u^* as an observable, we use the dataset's u, and each ensemble member will estimate u
10+
through the profile equation u(z) = u^* / κ log (z / z0).
11+
=#
12+
13+
# ## Prerequisites
14+
#=
15+
[EnsembleKalmanProcess.jl](https://github.com/CliMA/EnsembleKalmanProcesses.jl),
16+
=#
17+
18+
# ## Example
19+
20+
# First, we import relevant modules.
21+
using LinearAlgebra, Random
22+
using Distributions, Plots
23+
using EnsembleKalmanProcesses
24+
using EnsembleKalmanProcesses.ParameterDistributions
25+
const EKP = EnsembleKalmanProcesses
26+
27+
using Downloads
28+
using DelimitedFiles
29+
30+
FT = Float64
31+
32+
mkpath(joinpath(@__DIR__, "data")) # create data folder if not exists
33+
web_datafile_path = "https://turbulence.oden.utexas.edu/channel2015/data/LM_Channel_5200_mean_prof.dat"
34+
localfile = "data/profiles.dat"
35+
Downloads.download(web_datafile_path, localfile)
36+
data_mean_velocity = readdlm("data/profiles.dat", skipstart = 112) ## We skip 72 lines (header) and 40(laminar layer)
37+
38+
web_datafile_path = "https://turbulence.oden.utexas.edu/channel2015/data/LM_Channel_5200_mean_stdev.dat"
39+
localfile = "data/vel_stdev.dat"
40+
Downloads.download(web_datafile_path, localfile)
41+
# We skip 72 lines (header) and 40(laminar layer)
42+
data_stdev_velocity = readdlm("data/vel_stdev.dat", skipstart = 112)
43+
44+
# We extract the required info for this problem
45+
u_star_obs = 4.14872e-02 # add noise later
46+
z0 = FT(0.0001)
47+
κ = 0.4
48+
49+
# turn u into distributions
50+
u = data_mean_velocity[:, 3] * u_star_obs
51+
z = data_mean_velocity[:, 1]
52+
u = u[1:(length(u) - 1)] # filter out last element because σᵤ is only of length 727, not 728
53+
z = z[1:(length(z) - 1)]
54+
55+
σᵤ = data_stdev_velocity[:, 4] * u_star_obs
56+
dist_u = Array{Normal{Float64}}(undef, length(u))
57+
for i in 1:length(u)
58+
dist_u[i] = Normal(u[i], σᵤ[i])
59+
end
60+
61+
# u(z) = u^* / κ log (z / z0)
62+
function physical_model(parameters, inputs)
63+
κ = parameters[1] # this is being updated by the EKP iterator
64+
(; u_star_obs, z, z0) = inputs
65+
u_profile = u_star_obs ./ κ .* log.(z ./ z0)
66+
return u_profile
67+
end
68+
69+
function G(parameters, inputs, u_profile = nothing)
70+
if (isnothing(u_profile))
71+
u_profile = physical_model(parameters, inputs)
72+
end
73+
return [maximum(u_profile) - minimum(u_profile), mean(u_profile)]
74+
end
75+
76+
Γ = 0.0001 * I
77+
η_dist = MvNormal(zeros(2), Γ)
78+
noisy_u_profile = [rand(dist_u[i]) for i in 1:length(u)]
79+
y = G(nothing, nothing, noisy_u_profile)
80+
81+
parameters = (; κ)
82+
inputs = (; u_star_obs, z, z0)
83+
# y = G(parameters, inputs) .+ rand(η_dist)
84+
85+
# Assume that users have prior knowledge of approximate truth.
86+
# (e.g. via physical models / subset of obs / physical laws.)
87+
prior_u1 = constrained_gaussian("κ", 0.35, 0.25, 0, Inf);
88+
prior = combine_distributions([prior_u1])
89+
90+
# Set up the initial ensembles
91+
N_ensemble = 5;
92+
N_iterations = 10;
93+
94+
rng_seed = 41
95+
rng = Random.MersenneTwister(rng_seed)
96+
initial_ensemble = EKP.construct_initial_ensemble(rng, prior, N_ensemble);
97+
98+
# Define EKP and run iterative solver for defined number of iterations
99+
ensemble_kalman_process = EKP.EnsembleKalmanProcess(initial_ensemble, y, Γ, Inversion(); rng = rng)
100+
101+
for n in 1:N_iterations
102+
params_i = get_ϕ_final(prior, ensemble_kalman_process)
103+
G_ens = hcat([G(params_i[:, m], inputs) for m in 1:N_ensemble]...)
104+
EKP.update_ensemble!(ensemble_kalman_process, G_ens)
105+
end
106+
107+
# Mean values in final ensemble for the two parameters of interest reflect the "truth" within some degree of
108+
# uncertainty that we can quantify from the elements of `final_ensemble`.
109+
final_ensemble = get_ϕ_final(prior, ensemble_kalman_process)
110+
mean(final_ensemble[1, :]) # [param, ens_no]
111+
112+
113+
ENV["GKSwstype"] = "nul"
114+
zrange = z
115+
plot(zrange, noisy_u_profile, c = :black, label = "Truth", linewidth = 2, legend = :bottomright)
116+
plot!(zrange, physical_model(parameters, inputs), c = :green, label = "Model truth", linewidth = 2)# reshape to convert from vector to matrix)
117+
plot!(
118+
zrange,
119+
[physical_model(get_ϕ(prior, ensemble_kalman_process, 1)[:, i]..., inputs) for i in 1:N_ensemble],
120+
c = :red,
121+
label = reshape(vcat(["Initial ensemble"], ["" for i in 1:(N_ensemble - 1)]), 1, N_ensemble), # reshape to convert from vector to matrix
122+
)
123+
plot!(
124+
zrange,
125+
[physical_model(final_ensemble[:, i]..., inputs) for i in 1:N_ensemble],
126+
c = :blue,
127+
label = reshape(vcat(["Final ensemble"], ["" for i in 1:(N_ensemble - 1)]), 1, N_ensemble),
128+
)
129+
xlabel!("Z")
130+
ylabel!("U")
131+
png("profile_plot")

0 commit comments

Comments
 (0)