### A Pluto.jl notebook ###
# v0.15.1

using Markdown
using InteractiveUtils

# ╔═╡ 0a95c7a9-959a-4d91-9c76-266d352c8ec6
begin
	using PlutoUI
	PlutoUI.TableOfContents(title = "Contents")
end

# ╔═╡ 0b614b64-d774-4be2-b6fc-c6586d6ce46b
using Plots

# ╔═╡ ce37949c-fa81-11eb-2c5f-5356851bed48
md"""
# The Statistics of Card Shuffles
"""

# ╔═╡ 2137cb5a-cf1c-4d31-91a7-edf88c4c3f79
md"""
## Objectives
- Realistically **simulate** the two most common card shuffles
- Lay out a method for **measuring randomness** in a deck of cards
- Compare the two shuffle techniques in regard to their effectiveness
"""

# ╔═╡ 96838eb9-d7ce-4ff2-a0f5-9d9d13d292fc
md"""
## Introduction
In this case study we will give a practical approach to comparing different card shuffles. Simulating card shuffles in a realistic way is not as trivial as it sounds, deciding when a deck is well shuffled is even trickier. For the purpose of this case study we will always work with a deck of 52 distinct cards.
"""

# ╔═╡ 39301101-8962-4db1-a746-c865b0d59db3
md"""
## The Deck of Cards
The first question we have to ask ourselves is how we are going to simulate shuffles on a computer. We will represent the deck of 52 cards with an array containing the integers 1 to 52, initially in order, with 1 being the top card by convention:
"""

# ╔═╡ 4d5e1966-aa3e-4690-9957-33d672ae353f
deck = collect(1:52)

# ╔═╡ 95f00951-0d5c-4757-8826-56ebc3aca32b
md"""
We also need a way to neatly visualise our shuffles. The Julia colour functionality (from the 'Colors' Package) comes in very handy here. The following function displays the deck as a horizontal strip where each card is encoded by a unique colour in the range from blue to red.
"""

# ╔═╡ e54f1fa4-c040-4d9f-99ea-59bfeccdcb3f
displayDeck(deck) = [RGB{Float32}(x/52,0.3,1-x/52) for x in deck]

# ╔═╡ 9d80001b-b967-4cab-b785-978a8c4055cb
displayDeck(deck)

# ╔═╡ 9a596318-209c-4eef-b8a9-50cbb11b2cc0
md"""
## Simulating Card Shuffles
"""

# ╔═╡ 9fb354b9-a7ca-475d-ac3f-1c88d1972d58
md"""
### The Riffle Shuffle

The riffle shuffle is probably the most famous shuffling technique. To perform a riffle shuffle, approximately half of the deck is held in each hand with the thumbs pointing inwards. Then cards are released simultaneous by both thumbs so that they interleave while falling to the table. A perfect riffle shuffle is one where the cards alternate perfectly. There are two types of perfect riffle shuffles, one where the top card moves to be second from the top (called an in-shuffle), and one where the top and bottom cards are preserved (called an out-shuffle).

To simulate a realistic riffle shuffle we will make use of the _**Gilbert–Shannon–Reeds Model**_. To generate a random permutation according to this model, we flip a fair coin 52 times, to determine for each position of the shuffled deck whether it came from the first packet or the second packet. Then we split the initial deck into two packets whose sizes are the number of tails and the number of heads flipped, and use the same coin flip sequence to determine from the bottom of which packet to pull each card of the shuffled deck. This algorithm has been reported to be a good match for experimentally observed outcomes of human riffle shuffling.
"""

# ╔═╡ fa788983-e7e3-48ce-b874-8378563869a3
function riffle(deck, n = 1)
	if n == 0 return copy(deck) end
	bits = rand(Bool, 52)
	newDeck = zeros(Int, 52)
	(l, r) = (1, sum(bits)+1)
	for i in 1:52
		if bits[i] == 1
			newDeck[i] = deck[l]
			l += 1
		else
			newDeck[i] = deck[r]
			r += 1
		end
	end
	return riffle(newDeck, n-1)
end

# ╔═╡ 3f4e23d9-3102-4a1b-b763-ebb6e9264b9a
md"""
In this procedure, $n$ denotes the number of riffles performed sequentially.
"""

# ╔═╡ 61674df8-b398-4ba7-9913-f32822617828
displayDeck(riffle(deck,1))

# ╔═╡ 063aaaa4-1225-4b4b-89f3-7ff3bc205c74
displayDeck(riffle(deck,2))

# ╔═╡ 449066ba-0e05-4c0c-9083-ac1298dc50d0
displayDeck(riffle(deck,3))

# ╔═╡ e21b7bd3-5df7-4eec-8563-dafe24076c4f
md"""
As we increase $n$, the order of the deck looks more and more random, just as expected. After only 3 shuffles however, there is still some clearly visible structure present.
"""

# ╔═╡ e17dba39-871d-4c65-8d66-6fc0ef708c17
md"""
### The Overhand Shuffle

The overhand shuffle is the easiest and most widespread shuffling technique. To perform the shuffle, the deck is gradually transferred from one hand to the other by sliding off small packets from the top of the deck with the thumb of the receiving hand. The process is repeated several times. The randomness of the whole shuffle is increased by the number of small packets in each iteration as well as the number of iterations performed.

To simulate an overhand shuffle, we will pick each of the $51$ positions between adjacient cards as a 'cutting point' with some probability $p$, independent of each other. We then just reverse the order of the resulting packets definied by these cutting points. While the optimal value of $p$ is $1/2$, practically the deck is usually only split into approximately 10 packets. We will therefore use a generous value of $p = 1/5$ to get an expected number of $10.2$ cutting points corresponding to $11.2$ packets.
"""

# ╔═╡ 672a8b21-9b9b-4223-81ea-abed8fad9d0e
function overhand(deck, n = 1, p = 0.2)
	if n == 0 return copy(deck) end
	cuts = push!([i for i in 51:-1:1 if rand() < p], 0)
	newDeck = []
	lastcut = 52
	for c in cuts
		append!(newDeck, deck[c+1:lastcut])
		lastcut = c
	end
	return overhand(newDeck, n-1, p)
end

# ╔═╡ c65b75e0-b974-4d02-b68a-94533310ee32
md"""
Again, $n$ denotes the number of iterations of the shuffle performed.
"""

# ╔═╡ 95c25530-33f6-4de4-acc8-439f039a61a1
displayDeck(overhand(1:52,1))

# ╔═╡ f3e87927-5568-47bc-abcc-fc30afcc129f
displayDeck(overhand(1:52,2))

# ╔═╡ ed9d5dbe-64cd-462f-b937-4fc7880f2dc3
displayDeck(overhand(1:52,3))

# ╔═╡ 1a304723-2f76-4ae9-aa99-638f6e405db4
md"""
Compared to the riffle shuffle, it seems like the overhand shuffle doesn't randomize the deck as quickly. After 3 overhand shuffles there are still quite a few blocks of consecutive cards that have not been seperated. Let's see how we can make the 'effectiveness' of these shuffles more concrete.
"""

# ╔═╡ 626314cb-a77b-4a6a-ac08-132276887030
md"""
## Meassuring Randomness
"""

# ╔═╡ c571311e-5946-4f06-8e9d-d5f8d8ed3430
md"""
### The Kullback–Leibler divergence
Given a reference probability distribution $Q$, the Kullback–Leibler divergence is a measure of how much a second probability distribution $P$ differs from $Q$:

$D_\text{KL}(P\parallel Q) = \sum _x P(x)\log \left(\frac {P(x)}{Q(x)}\right)$

where the sum is over all elements of the probability space in question. Note that the expression is not symmetric in $P$ and $Q$. This is because similarly to a hypothesis test, the distribution $Q$ should be taken as the predicted distribution whereas $P$ is the observed distribution.
"""

# ╔═╡ 62e5ddbf-f908-4b47-a9f8-fee28d82beea
md"""
Unfortunately, we cannot apply this measure to the entire probability space of all permutations since $52! \approx 8.069\times 10^{67}$ is way too large to iterate through. Instead, we will analyse the distribution of final positions of a single card in the deck. If this distribution is close to uniform we can safely assume that the deck is well shuffled. Also note that since the shuffle techniques in consideration treat all cards about equally, we will focus on the card that is initially on top of the deck. If anything, this card will be shuffled least well, as in the beginning it can only 'move' in one direction.
"""

# ╔═╡ d5f0b623-22d3-4476-9d8d-4045fb03650c
function topCardDist(shuffle, n, m)
	dist = zeros(52)
	deck = collect(1:52)
	for i in 1:m
		dist[argmin(shuffle(deck, n))] += 1/m
	end
	return dist
end

# ╔═╡ 257972d2-fcef-48ff-81d3-25d5661a4151
md"""
In this procedure, $n$ denotes the number of sequential interations of the shuffle and $m$ is the number of data points collected for the distribution. We can now use the kullback–leibler divergence to get a metric of how 'far away' a distribution is from the desired, uniform distribution:
"""

# ╔═╡ 11263177-37bb-45de-8310-a3e612d8e652
function KL(dist)
	res = 0.0
	for p in dist
		if p == 0 continue end
		res += p*log(52 * p)
	end
	return res
end

# ╔═╡ 4804d7ac-15ff-4614-8065-43aefecfb56f
md"""
Let us use this measure of randomness to compare the two shuffles:
"""

# ╔═╡ c5c392e8-12ec-438b-9e37-58e36227eaf4
md"""
##### Top Card Distribution of the Riffle Shuffle
"""

# ╔═╡ 2c73ded2-216a-44e0-8c21-c816c289bec3
begin
plot(xlims = (1,52), xlabel = "Position", ylabel = "Frequency")
plot!(topCardDist(riffle, 1, 1000000), label = "1 Iteration")
plot!(topCardDist(riffle, 2, 1000000), label = "2 Iterations")
plot!(topCardDist(riffle, 3, 1000000), label = "3 Iterations")
plot!(topCardDist(riffle, 4, 1000000), label = "4 Iterations")
plot!(topCardDist(riffle, 5, 1000000), label = "5 Iterations")
plot!(topCardDist(riffle, 6, 1000000), label = "6 Iterations")
plot!(topCardDist(riffle, 7, 1000000), label = "7 Iterations")
end

# ╔═╡ 637e1c56-feb2-42e0-9c39-f7ad33616d31
begin
plot(legend = false, xticks = 1:10, ylims = (-0.05,1))
plot!(ylabel = "Kullback–Leibler Divergence", xlabel = "Iterations")
plot!([KL(topCardDist(riffle, n, 100000)) for n in 1:10])
end

# ╔═╡ 96e6723a-0fa7-4cd3-a9b4-fb1fa0dffa40
md"""
As expected, with increased number of iteration the final position of the top card tends to be uniformly distributed. From this data it seems like after 7 iterations of the riffle shuffle there is not much to be gained from doing more iterations.
"""

# ╔═╡ 16077fbd-689f-45bb-b063-1b4aeefe8c59
md"""
##### Top Card Distribution of the Overhand Shuffle
"""

# ╔═╡ eae5a4eb-cc4e-402a-b96c-e036c6838b64
begin
plot(xlims = (1,52), xlabel = "Position", ylabel = "Frequency")
plot!(topCardDist(overhand, 1, 1000000), label = "1 Iteration")
plot!(topCardDist(overhand, 2, 1000000), label = "2 Iterations")
plot!(topCardDist(overhand, 3, 1000000), label = "3 Iterations")
plot!(topCardDist(overhand, 4, 1000000), label = "4 Iterations")
plot!(topCardDist(overhand, 5, 1000000), label = "5 Iterations")
plot!(topCardDist(overhand, 6, 1000000), label = "6 Iterations")
plot!(topCardDist(overhand, 7, 1000000), label = "7 Iterations")
end

# ╔═╡ 4de522e2-8aba-4465-b9d5-4d77d3451687
md"""
Other than for the riffle shuffle, the shape of the distribution alternates between having its peak at the very bottom or the very top of the deck. This is to be expected, since one iteration of the overhand shuffle reverses the deck, roughly speaking. We also note that the convergence to the uniform distribution seems to be much slower than for the riffle shuffle.
"""

# ╔═╡ b8bdb3c3-a57f-4f4e-a20e-2600602d5b81
begin
plot(xticks = 1:2:41, ylims = (-.05,1))
plot!(ylabel = "Kullback–Leibler Divergence", xlabel = "Iterations")
plot!([KL(topCardDist(overhand, n, 10000)) for n in 1:41], label = "Overhand Shuffle")
plot!([KL(topCardDist(riffle, n, 10000)) for n in 1:41], label = "Riffle Shuffle")
end

# ╔═╡ d462dbb3-790b-43b7-98ef-39c2427a0512
md"""
We conclude that the riffle shuffle is the supervior shuffling technique, at least under this metric of effectiveness. The data shows that it takes more than 40 iterations of the overhand shuffle to achieve the same degree of uniformity as riffling the deck 8 times.
"""

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
PlutoUI = "7f904dfe-b85e-4ff6-b463-dae2292396a8"

[compat]
Plots = "~1.20.1"
PlutoUI = "~0.7.9"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "84918055d15b3114ede17ac6a7182f68870c16f7"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.1"

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c3598e525718abcc440f69cc6d5f60dda0a1b61e"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.6+5"

[[Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "e2f47f6d8337369411569fd45ae5753ca10394c6"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.0+6"

[[ColorSchemes]]
deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"]
git-tree-sha1 = "9995eb3977fbf67b86d0a0a0508e83017ded03f2"
uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
version = "3.14.0"

[[ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "344f143fa0ec67e47917848795ab19c6a455f32c"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.32.0"

[[CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[Contour]]
deps = ["StaticArrays"]
git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7"
uuid = "d38c429a-6771-53c6-b99e-75d170b6e991"
version = "0.5.7"

[[DataAPI]]
git-tree-sha1 = "ee400abb2298bd13bfc3df1c412ed228061a2385"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.7.0"

[[DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "7d9d316f04214f7efdbb6398d545446e246eff02"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.10"

[[DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[EarCut_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "92d8f9f208637e8d2d28c664051a00569c01493d"
uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5"
version = "2.1.5+1"

[[Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b3bfd02e98aedfa5cf885665493c5598c350cd2f"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.2.10+0"

[[FFMPEG]]
deps = ["FFMPEG_jll"]
git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8"
uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a"
version = "0.4.1"

[[FFMPEG_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "LibVPX_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"]
git-tree-sha1 = "3cc57ad0a213808473eafef4845a74766242e05f"
uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5"
version = "4.3.1+4"

[[FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "35895cf184ceaab11fd778b4590144034a167a2f"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.1+14"

[[Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "cbd58c9deb1d304f5a245a0b7eb841a2560cfec6"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.1+5"

[[FriBidi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91"
uuid = "559328eb-81f9-559d-9380-de523a88c83c"
version = "1.0.10+0"

[[GLFW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"]
git-tree-sha1 = "dba1e8614e98949abfa60480b13653813d8f0157"
uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89"
version = "3.3.5+0"

[[GR]]
deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "182da592436e287758ded5be6e32c406de3a2e47"
uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71"
version = "0.58.1"

[[GR_jll]]
deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "d59e8320c2747553788e4fc42231489cc602fa50"
uuid = "d2c73de3-f751-5644-a686-071e5b155ba9"
version = "0.58.1+0"

[[GeometryBasics]]
deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"]
git-tree-sha1 = "58bcdf5ebc057b085e58d95c138725628dd7453c"
uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
version = "0.4.1"

[[Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "7bf67e9a481712b3dbe9cb3dac852dc4b1162e02"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+0"

[[Grisu]]
git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2"
uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe"
version = "1.0.2"

[[HTTP]]
deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"]
git-tree-sha1 = "44e3b40da000eab4ccb1aecdc4801c040026aeb5"
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
version = "0.9.13"

[[IniFile]]
deps = ["Test"]
git-tree-sha1 = "098e4d2c533924c921f9f9847274f2ad89e018b8"
uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f"
version = "0.5.0"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[IterTools]]
git-tree-sha1 = "05110a2ab1fc5f932622ffea2a003221f4782c18"
uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
version = "1.3.0"

[[IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.3.0"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "8076680b162ada2a031f707ac7b4953e30667a37"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.2"

[[JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "d735490ac75c5cb9f1b00d8b5509c11984dc6943"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.0+0"

[[LAME_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c"
uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d"
version = "3.100.1+0"

[[LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[LaTeXStrings]]
git-tree-sha1 = "c7f1c695e06c01b95a67f0cd1d34994f3e7db104"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.2.1"

[[Latexify]]
deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"]
git-tree-sha1 = "a4b12a1bd2ebade87891ab7e36fdbce582301a92"
uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316"
version = "0.15.6"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[LibVPX_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "12ee7e23fa4d18361e7c2cde8f8337d4c3101bc7"
uuid = "dd192d2f-8180-539f-9fb4-cc70b1dcf69a"
version = "1.10.0+0"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "761a393aeccd6aa92ec3515e428c26bf99575b3b"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+0"

[[Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[Libglvnd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"]
git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf"
uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29"
version = "1.3.0+3"

[[Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "340e257aada13f95f98ee352d316c3bed37c8ab9"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+0"

[[Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "0fb723cd8c45858c22169b2e42269e53271a6df7"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.7"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MbedTLS]]
deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"]
git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe"
uuid = "739be429-bea8-5141-9913-cc70e7f3736d"
version = "1.0.3"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Measures]]
git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f"
uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e"
version = "0.3.1"

[[Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "4ea90bd5d3985ae1f9a908bd4500ae88921c5ce7"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.0"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[NaNMath]]
git-tree-sha1 = "bfe47e760d60b82b66b61d2d44128b62e3a369fb"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "0.3.5"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[Ogg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7937eda4681660b4d6aeeecc2f7e1c81c8ee4e2f"
uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051"
version = "1.3.5+0"

[[OpenSSL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "15003dcb7d8db3c6c857fda14891a539a8f2705a"
uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
version = "1.1.10+0"

[[Opus_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720"
uuid = "91d4177d-7536-5919-b921-800302f37372"
version = "1.3.2+0"

[[OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "477bf42b4d1496b454c10cce46645bb5b8a0cf2c"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.0.2"

[[Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[PlotThemes]]
deps = ["PlotUtils", "Requires", "Statistics"]
git-tree-sha1 = "a3a964ce9dc7898193536002a6dd892b1b5a6f1d"
uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a"
version = "2.0.1"

[[PlotUtils]]
deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"]
git-tree-sha1 = "501c20a63a34ac1d015d5304da0e645f42d91c9f"
uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043"
version = "1.0.11"

[[Plots]]
deps = ["Base64", "Contour", "Dates", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs"]
git-tree-sha1 = "8365fa7758e2e8e4443ce866d6106d8ecbb4474e"
uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
version = "1.20.1"

[[PlutoUI]]
deps = ["Base64", "Dates", "InteractiveUtils", "JSON", "Logging", "Markdown", "Random", "Reexport", "Suppressor"]
git-tree-sha1 = "44e225d5837e2a2345e69a1d1e01ac2443ff9fcb"
uuid = "7f904dfe-b85e-4ff6-b463-dae2292396a8"
version = "0.7.9"

[[Preferences]]
deps = ["TOML"]
git-tree-sha1 = "00cfd92944ca9c760982747e9a1d0d5d86ab1e5a"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.2"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[Qt5Base_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"]
git-tree-sha1 = "ad368663a5e20dbb8d6dc2fddeefe4dae0781ae8"
uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1"
version = "5.15.3+0"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[RecipesBase]]
git-tree-sha1 = "b3fb709f3c97bfc6e948be68beeecb55a0b340ae"
uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
version = "1.1.1"

[[RecipesPipeline]]
deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"]
git-tree-sha1 = "2a7a2469ed5d94a98dea0e85c46fa653d76be0cd"
uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c"
version = "0.3.4"

[[Reexport]]
git-tree-sha1 = "5f6c21241f0f655da3952fd60aa18477cf96c220"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.1.0"

[[Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "4036a3bd08ac7e968e27c203d45f5fff15020621"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.1.3"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Scratch]]
deps = ["Dates"]
git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.1.0"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[Showoff]]
deps = ["Dates", "Grisu"]
git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de"
uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f"
version = "1.0.3"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "3240808c6d463ac46f1c1cd7638375cd22abbccb"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.2.12"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[StatsAPI]]
git-tree-sha1 = "1958272568dc176a1d881acb797beb909c785510"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.0.0"

[[StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "fed1ec1e65749c4d96fc20dd13bea72b55457e62"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.9"

[[StructArrays]]
deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"]
git-tree-sha1 = "000e168f5cc9aded17b6999a560b7c11dda69095"
uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
version = "0.6.0"

[[Suppressor]]
git-tree-sha1 = "a819d77f31f83e5792a76081eee1ea6342ab8787"
uuid = "fd094767-a336-5f1f-9728-57cf17d0bbfb"
version = "0.2.0"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "TableTraits", "Test"]
git-tree-sha1 = "d0c690d37c73aeb5ca063056283fde5585a41710"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.5.0"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[URIs]]
git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355"
uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
version = "1.3.0"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[Wayland_jll]]
deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23"
uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89"
version = "1.19.0+0"

[[Wayland_protocols_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll"]
git-tree-sha1 = "2839f1c1296940218e35df0bbb220f2a79686670"
uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91"
version = "1.18.0+4"

[[XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[Xorg_libXcursor_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd"
uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724"
version = "1.2.0+4"

[[Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[Xorg_libXfixes_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4"
uuid = "d091e8ba-531a-589c-9de9-94069b037ed8"
version = "5.0.3+4"

[[Xorg_libXi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"]
git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246"
uuid = "a51aa0fd-4e3c-5386-b890-e753decda492"
version = "1.7.10+4"

[[Xorg_libXinerama_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"]
git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123"
uuid = "d1454406-59df-5ea1-beac-c340f2130bc3"
version = "1.1.4+4"

[[Xorg_libXrandr_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631"
uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484"
version = "1.5.2+4"

[[Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[Xorg_libxkbfile_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2"
uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a"
version = "1.1.0+4"

[[Xorg_xcb_util_image_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97"
uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b"
version = "0.4.0+1"

[[Xorg_xcb_util_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"]
git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1"
uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5"
version = "0.4.0+1"

[[Xorg_xcb_util_keysyms_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00"
uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7"
version = "0.4.0+1"

[[Xorg_xcb_util_renderutil_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e"
uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e"
version = "0.3.9+1"

[[Xorg_xcb_util_wm_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67"
uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361"
version = "0.4.1+1"

[[Xorg_xkbcomp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"]
git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b"
uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4"
version = "1.4.2+4"

[[Xorg_xkeyboard_config_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"]
git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d"
uuid = "33bec58e-1273-512f-9401-5d533626f822"
version = "2.27.0+4"

[[Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "cc4bf3fdde8b7e3e9fa0351bdeedba1cf3b7f6e6"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.0+0"

[[libass_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "acc685bcf777b2202a904cdcb49ad34c2fa1880c"
uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0"
version = "0.14.0+4"

[[libfdk_aac_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7a5780a0d9c6864184b3a2eeeb833a0c871f00ab"
uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280"
version = "0.1.6+4"

[[libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[libvorbis_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"]
git-tree-sha1 = "c45f4e40e7aafe9d086379e5578947ec8b95a8fb"
uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a"
version = "1.3.7+0"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

[[x264_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "d713c1ce4deac133e3334ee12f4adff07f81778f"
uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a"
version = "2020.7.14+2"

[[x265_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "487da2f8f2f0c8ee0e83f39d13037d6bbf0a45ab"
uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76"
version = "3.0.0+3"

[[xkbcommon_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"]
git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6"
uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd"
version = "0.9.1+5"
"""

# ╔═╡ Cell order:
# ╟─ce37949c-fa81-11eb-2c5f-5356851bed48
# ╟─2137cb5a-cf1c-4d31-91a7-edf88c4c3f79
# ╟─96838eb9-d7ce-4ff2-a0f5-9d9d13d292fc
# ╟─39301101-8962-4db1-a746-c865b0d59db3
# ╠═4d5e1966-aa3e-4690-9957-33d672ae353f
# ╟─95f00951-0d5c-4757-8826-56ebc3aca32b
# ╠═e54f1fa4-c040-4d9f-99ea-59bfeccdcb3f
# ╠═9d80001b-b967-4cab-b785-978a8c4055cb
# ╟─9a596318-209c-4eef-b8a9-50cbb11b2cc0
# ╟─9fb354b9-a7ca-475d-ac3f-1c88d1972d58
# ╠═fa788983-e7e3-48ce-b874-8378563869a3
# ╟─3f4e23d9-3102-4a1b-b763-ebb6e9264b9a
# ╠═61674df8-b398-4ba7-9913-f32822617828
# ╠═063aaaa4-1225-4b4b-89f3-7ff3bc205c74
# ╠═449066ba-0e05-4c0c-9083-ac1298dc50d0
# ╟─e21b7bd3-5df7-4eec-8563-dafe24076c4f
# ╟─e17dba39-871d-4c65-8d66-6fc0ef708c17
# ╠═672a8b21-9b9b-4223-81ea-abed8fad9d0e
# ╟─c65b75e0-b974-4d02-b68a-94533310ee32
# ╠═95c25530-33f6-4de4-acc8-439f039a61a1
# ╠═f3e87927-5568-47bc-abcc-fc30afcc129f
# ╠═ed9d5dbe-64cd-462f-b937-4fc7880f2dc3
# ╟─1a304723-2f76-4ae9-aa99-638f6e405db4
# ╟─626314cb-a77b-4a6a-ac08-132276887030
# ╟─c571311e-5946-4f06-8e9d-d5f8d8ed3430
# ╟─62e5ddbf-f908-4b47-a9f8-fee28d82beea
# ╠═d5f0b623-22d3-4476-9d8d-4045fb03650c
# ╟─257972d2-fcef-48ff-81d3-25d5661a4151
# ╠═11263177-37bb-45de-8310-a3e612d8e652
# ╟─4804d7ac-15ff-4614-8065-43aefecfb56f
# ╟─c5c392e8-12ec-438b-9e37-58e36227eaf4
# ╟─2c73ded2-216a-44e0-8c21-c816c289bec3
# ╟─637e1c56-feb2-42e0-9c39-f7ad33616d31
# ╟─96e6723a-0fa7-4cd3-a9b4-fb1fa0dffa40
# ╟─16077fbd-689f-45bb-b063-1b4aeefe8c59
# ╟─eae5a4eb-cc4e-402a-b96c-e036c6838b64
# ╟─4de522e2-8aba-4465-b9d5-4d77d3451687
# ╟─b8bdb3c3-a57f-4f4e-a20e-2600602d5b81
# ╟─d462dbb3-790b-43b7-98ef-39c2427a0512
# ╟─0a95c7a9-959a-4d91-9c76-266d352c8ec6
# ╟─0b614b64-d774-4be2-b6fc-c6586d6ce46b
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


"""
prime_check(number)

Checks to see if a number is a prime or not

A number is prime if it has exactly two factors: 1 and itself.

# Example

```julia
prime_check(2) # returns true
prime_check(3) # returns true
prime_check(5) # returns true
prime_check(7) # returns true
prime_check(11) # returns true
prime_check(13) # returns true
prime_check(17) # returns true
prime_check(19) # returns true
prime_check(23) # returns true
prime_check(29) # returns true
prime_check(30) # returns false
```

Contributed By:- [Ashwani Rathee](https://github.com/ashwani-rathee) and [Rratic](https://github.com/Rratic)
"""
function prime_check(number::N) where {N<:Integer}
    if 1 < number < 4
        # 2, 3 are primes
        return true
    elseif number < 2 || number % 2 == 0
        # Negative ,0,1, and even numbers except 2 are not primes
        return false
    end
    for i in 3:2:(isqrt(number)+1)
        if number % i == 0
            return false
        end
    end
    return true
end


function secant(f::Function, xi::Number, dx::Number, ea::Number, maxIterations::Integer)
  # Estimate the root of functions using the secant method.
  #
  # Keyword arguments:
  #  f -- Anonymous function to find the root of
  #  xi -- Initial estimate for the root
  #  dx -- Pertubation to use for estimating df/dx
  #  ea -- Minimum relative error for a solution
  #  maxIterations -- Maximum number of iterations to carry out
  #
  # Returns:
  #  An estimate of the root of the function

  xo = xi; # Current estimate of root

  for i = 1:maxIterations
    xoLast = xo;
    xo = xo - (f(xo) * dx) / (f(xo+dx) - f(xo));
    relativeError = abs((xo - xoLast) / xo) * 100;

    (relativeError < ea || isnan(relativeError)) && break;
  end

  return xo;
end


module Geometry

export centroid, squaredradius
"""
Computes the centroid of a simplex defined by vertices in 'X',
which is a matrix of dimension nx(n+1) and each column is a vertex.

Input arguments
---------------
simplex::Array{Float64, 2}  Simplex represented by an n-by-(n+1) matrix, where each column is a vertex.

Returns
-------
centroid::Array{Float64, 1}   Row vector. The centroid of the simplex.
"""
function centroid(simplex::AbstractArray{Float64, 2})
    n = size(simplex)[1]
    centroid = simplex * ones(n + 1, 1)/(n + 1)
    return(centroid)
end

"""
Computes the radius of a simplex defined by vertices in 'X',
which is a matrix of dimension nx(n+1) and each column is a vertex.

Input arguments
---------------
simplex::Array{Float64, 2}  Simplex represented by an n-by-(n+1) matrix, where each column is a vertex.

Returns
-------
radius::Float64  The radius of the simplex.

"""
function squaredradius(simplex)
    n = size(simplex)[1]
    C1 = repeat(centroid(simplex), 1, (n + 1))
    radius = maximum(ones(1, n) * ((simplex - C1)))^2
    return(radius::Float64)
end

end


"""
    abs_val(num)

Program to find the absolute value of a number

# Example
```julia
abs_val(-100) # returns 100
abs_val(0) # returns 0
abs(123.1) # returns 123.1
-1000 == abs_val(-1000) #returns false
1000 == abs_val(1000) #returns true
```

Contributed By:- [Ashwani Rathee](https://github.com/ashwani-rathee)
"""
function abs_val(num)
    return num < 0 ? -num : num
end

"""
    abs_max(x)

Program to find the max absolute value in a vector

# Example
```julia
abs_max([1,3,4]) # returns 4
abs_max([-3,1,2]) # returns -3
abs_max([-7,-3,6]) #returns -7
```

Contributed By:- [Ashwani Rathee](https://github.com/ashwani-rathee)
"""
function abs_max(x)
    result = sort(x, by = abs; alg = QuickSort)
    return result[length(result)]
end

"""
    abs_min(num)

Program to find the min absolute value in a vector

# Example
```julia
abs_min([1,3,4]) # returns 1
abs_min([-3,1,2]) # returns 1
abs_min([-7,-3,6]) #returns -3
```

Contributed By:- [Ashwani Rathee](https://github.com/ashwani-rathee)
"""
function abs_min(x)
    result = sort(x, by = abs; alg = QuickSort)
    return result[1]
end


"""
    RSOCtoSOCBridge{T, F, G}

The `RotatedSecondOrderCone` is `SecondOrderCone` representable; see [BN01, p. 104].
Indeed, we have ``2tu = (t/√2 + u/√2)^2 - (t/√2 - u/√2)^2`` hence
```math
2tu \\ge \\lVert x \\rVert_2^2
```
is equivalent to
```math
(t/√2 + u/√2)^2 \\ge \\lVert x \\rVert_2^2 + (t/√2 - u/√2)^2.
```
We can therefore use the transformation ``(t, u, x) \\mapsto (t/√2+u/√2, t/√2-u/√2, x)``.
Note that the linear transformation is a symmetric involution (i.e. it is its own transpose and its own inverse).
That means in particular that the norm of constraint primal and dual values are preserved by the tranformation.

[BN01] Ben-Tal, Aharon, and Nemirovski, Arkadi. *Lectures on modern convex optimization: analysis, algorithms, and engineering applications*. Society for Industrial and Applied Mathematics, 2001.
"""
struct RSOCtoSOCBridge{T,F,G} <:
       SetMapBridge{T,MOI.SecondOrderCone,MOI.RotatedSecondOrderCone,F,G}
    constraint::CI{F,MOI.SecondOrderCone}
end

function rotate_function_type(G::Type{<:MOI.AbstractVectorFunction}, T::Type)
    S = MOIU.promote_operation(/, T, MOIU.scalar_type(G), T)
    Y = MOIU.promote_operation(-, T, S, S)
    Z = MOIU.promote_operation(+, T, S, S)
    return MOIU.promote_operation(vcat, T, Z, Y, G)
end

function concrete_bridge_type(
    ::Type{<:RSOCtoSOCBridge{T}},
    G::Type{<:MOI.AbstractVectorFunction},
    ::Type{MOI.RotatedSecondOrderCone},
) where {T}
    return RSOCtoSOCBridge{T,rotate_function_type(G, T),G}
end

"""
    SOCtoRSOCBridge{T, F, G}

We simply do the inverse transformation of [`RSOCtoSOCBridge`](@ref). In fact, as the
transformation is an involution, we do the same transformation.
"""
struct SOCtoRSOCBridge{T,F,G} <:
       SetMapBridge{T,MOI.RotatedSecondOrderCone,MOI.SecondOrderCone,F,G}
    constraint::CI{F,MOI.RotatedSecondOrderCone}
end

function concrete_bridge_type(
    ::Type{<:SOCtoRSOCBridge{T}},
    G::Type{<:MOI.AbstractVectorFunction},
    ::Type{MOI.SecondOrderCone},
) where {T}
    return SOCtoRSOCBridge{T,rotate_function_type(G, T),G}
end


#===============================================================================

The number 512 is interesting because it is equal to the sum of its digits
raised to some power: 5 + 1 + 2 = 8, and 8^3 = 512. Another example of a number
with this property is 614656 = 28^4.

We shall define an to be the nth term of this sequence and insist that a number
must contain at least two digits to have a sum.

You are given that a2 = 512 and a10 = 614656.

Find a30.

--------------------------------------------------------------------------------

The solution is to build iteratively a table whose row and column indices
correspond respectively to the exponents and digit sums. The table is
semi-infinite in each direction (i.e. indices starts at 1 but are unlimited
otherwise).

At iteration n, all numbers up to 2^n (at least) are processed. Those that
fulfil the condition (col = digit_sum(col^row)) are inserted in order in a
queue.

The elements of the queue are returned in order. Once the queue is empty or if
the first element is greater than 2^n, then a new iteration is restarted.

===============================================================================#

import Base
using Base.Test

struct Item
    value :: Int64
    dsum  :: Int
    exp   :: Int
end

Item(dsum::Int, exp::Int) = Item(dsum^exp, dsum, exp)

index(item::Item) = item.dsum + item.exp

isvalid(item::Item) = sum(digits(item.value)) == item.dsum && item.value > 9

Base.isless(lhs::Item, rhs::Item) =  lhs.value < rhs.value

function insertSorted!(list, item)
    index = searchsortedfirst(list, item)
    insert!(list, index, item)
end

struct NumberIter end

mutable struct NumberIterState
    target :: Int
    borders :: Array{Item}
    queue :: Array{Item}
end

Base.start(::NumberIter) = NumberIterState(1, Item[Item(1, 1)], Item[])

function Base.next(::NumberIter, state)
    function populate()
        state.target *= 2

        # Extend existing rows
        for e in 2:length(state.borders) # e is the exponent
            item = state.borders[e]
            while item.value < state.target
                item = Item(item.dsum+1, e)
                if isvalid(item)
                    insertSorted!(state.queue, item)
                end
            end
            state.borders[e] = item
        end

        # Add a border
        push!(state.borders, Item(2, length(state.borders)+1))
    end

    while true
        if isempty(state.queue)
            populate()
        else
            item = shift!(state.queue)

            if item.value <= state.target
                return (item, state)
            else
                unshift!(state.queue, item)
                populate()
            end
        end
    end
end

Base.done(::NumberIter, state) = false

function Euler(limit)
    result = Int64[]
    for item in NumberIter()
        push!(result, item.value)
        if length(result) == limit
            return result
        end
    end
end

euler = Euler(30)

@test euler[2] == 512
@test euler[10] == 614656

println("Euler 0119: $(euler[30])")


using CSV, Distributions, HypothesisTests

data1 = CSV.read("../data/machine1.csv", header=false)[:,1]
data2 = CSV.read("../data/machine2.csv", header=false)[:,1]
xBar1, xBar2 = mean(data1), mean(data2)
n1, n2 = length(data1), length(data2)
alpha = 0.05
t = quantile(TDist(n1+n2-2),1-alpha/2)

s1, s2 = std(data1), std(data2)
sP = sqrt(((n1-1)*s1^2 + (n2-1)*s2^2) / (n1+n2-2))

println("Calculating formula: ", (xBar1 - xBar2 - t*sP* sqrt(1/n1 + 1/n2),
				  xBar1 - xBar2 + t*sP* sqrt(1/n1 + 1/n2)))
println("Using confint(): ", confint(EqualVarianceTTest(data1,data2),alpha))


using Plots
using DifferentialEquations

function seirmodel(du, u, p, t)
    S = u[1]
    E = u[2]
    I = u[3]
    R = u[4]
    dS = -p[1]*S*I
    dE = p[1]*S*I-p[2]*E
    dI = p[2]*E-p[3]*I
    dR = p[3]*I
    du[1] = dS
    du[2] = dE
    du[3] = dI
    du[4] = dR
end

function solveSEIR(u0, β, γ, δ, time_span)

    prob = ODEProblem(seirmodel, u0, time_span, [β, δ, γ])

    sol = solve(prob)

    return sol

end

R0=2.0
N=10000
γ=0.1
β=γ*R0/N
δ=0.25

sol = solveSEIR(Array{Float64}([N-1,0,1,0]), β, γ, δ, (0.0,360.0))

plot_graph = plot(sol, vars=(0,1), title="SEIR model/Runge-Kutta R0=$R0 γ=$γ δ=$δ", label="Suscepible",
                  color=:blue, seriestype=:line, legend=:left, size=(1200,900))
plot!(plot_graph, sol, vars=(0,2), label="Exposed", color=:darkmagenta)
plot!(plot_graph, sol, vars=(0,3), label="Infectious", color=:red)
plot!(plot_graph, sol, vars=(0,4), label="Removed", color=:green)

png(plot_graph, "seir_ode.png")


using LinearAlgebra
import LinearAlgebra:checksquare

"""
    function sweep_op_internal!(A::AbstractMatrix{Float64}, k::Int64)

    Implement the naive sweep operator described by Goodnight, J. (1979). "A Tutorial on the SWEEP Operator." The American Statistician.
"""
function sweep_op_internal!(A::AbstractMatrix{Float64}, k::Int64)
    p = checksquare(A)
    if k >= p
        throw(ArgumentError("Incorrect k value"))
    end

    @inbounds D = A[k,k] # step 1 D = Aₖₖ

    if D == zero(eltype(A))
        throw(error("the element $k,$k is zero"))
    end

    @inbounds for i in 1:p
        A[k,i] = A[k,i] / D   # step 2 divide row k by D
    end

    @inbounds for i in 1:(k - 1) # step 3: for every other row  i != k
        B = A[i,k]   # let B = Aᵢₖ
        for j in 1:p
            A[i,j] = A[i,j] - B * A[k, j]  # Subtract B * row k from row i
        end
        A[i,k] = - B / D # set Aᵢₖ = -B/D
    end

    @inbounds for i in k + 1:p # step 3: for every other row  i != k
        B = A[i,k]   # let B = Aᵢₖ
        for j in 1:p
            A[i,j] = A[i,j] - B * A[k, j]  # Subtract B * row k from row i
        end
        A[i,k] = - B / D # set Aᵢₖ = -B/D
    end

    @inbounds A[k,k] = 1 / D # step 4 Set Aₖₖ = 1/D

    return A
end

function sweep_op!(A::AbstractMatrix{Float64}, k::Int64)
    sweep_op_internal!(A, k)
end

function sweep_op!(A::AbstractMatrix{Float64}, ks::AbstractVector{Int64})
    for k in ks
        sweep_op_internal!(A, k)
    end
    return A
end

"""
    function sweep_op_full!(A::AbstractMatrix{Float64})

    (internal) Get SSE, error sum of squares, for all p-1 models. The last one is the SSE for the full model.
"""
function sweep_op_full!(A::AbstractMatrix{Float64})
    n , p = size(A)

    SSEv = Vector{Float64}(undef, p-1)

    for k in 1:p-1
        sweep_op_internal!(A, k)
        SSEv[k] = A[p,p]
    end
    return SSEv
end


using Test
using StaticArrays, IntervalSets
import ClimaCore.DataLayouts: IJFH
import ClimaCore:
    Fields, Domains, Meshes, Topologies, Spaces, Operators, Geometry
using StaticArrays, IntervalSets, LinearAlgebra

FT = Float64

radius = FT(1)
domain = Domains.SphereDomain(radius)

sdiv = Operators.Divergence()
wdiv = Operators.WeakDivergence()

@testset "divergence" begin
    Ne = 7
    Nq = 6

    mesh = Meshes.EquiangularCubedSphere(domain, Ne)
    grid_topology = Topologies.Topology2D(mesh)

    quad = Spaces.Quadratures.GLL{Nq}()
    space = Spaces.SpectralElementSpace2D(grid_topology, quad)
    coords = Fields.coordinate_field(space)

    u_local = @. Geometry.UVVector(
        cosd(coords.long),
        -sind(coords.long) * sind(coords.lat),
    )

    u = Geometry.transform.(Ref(Geometry.Contravariant12Axis()), u_local)

    div_us = sdiv.(u)
    Spaces.weighted_dss!(div_us)

    div_uw = wdiv.(u)
    Spaces.weighted_dss!(div_uw)

    div_exact = @. -2 * sind(coords.long) * cosd(coords.lat) / radius

    @test div_us ≈ div_exact rtol = 1e-2
    @test div_uw ≈ div_exact rtol = 1e-2
end

"""
    convergence_rate(err, Δh)
Estimate convergence rate given vectors `err` and `Δh`
    err = C Δh^p+ H.O.T
    err_k ≈ C Δh_k^p
    err_k/err_m ≈ Δh_k^p/Δh_m^p
    log(err_k/err_m) ≈ log((Δh_k/Δh_m)^p)
    log(err_k/err_m) ≈ p*log(Δh_k/Δh_m)
    log(err_k/err_m)/log(Δh_k/Δh_m) ≈ p
"""
convergence_rate(err, Δh) =
    [log(err[i] / err[i - 1]) / log(Δh[i] / Δh[i - 1]) for i in 2:length(Δh)]

@testset "convergence tests for the divergence operator on the sphere" begin
    Nes = [3, 9, 27]
    Nqs = [4, 6]

    for (Iq, Nq) in enumerate(Nqs)
        err_sdiv = Array{FT}(undef, length(Nes))
        err_wdiv = Array{FT}(undef, length(Nes))
        Δh = Array{FT}(undef, length(Nes))

        for (Ie, Ne) in enumerate(Nes)
            mesh = Meshes.EquiangularCubedSphere(domain, Ne)
            grid_topology = Topologies.Topology2D(mesh)

            quad = Spaces.Quadratures.GLL{Nq}()
            space = Spaces.SpectralElementSpace2D(grid_topology, quad)
            coords = Fields.coordinate_field(space)

            u_local = @. Geometry.UVVector(
                cosd(coords.long),
                -sind(coords.long) * sind(coords.lat),
            )

            u =
                Geometry.transform.(
                    Ref(Geometry.Contravariant12Axis()),
                    u_local,
                )

            div_us = sdiv.(u)
            Spaces.weighted_dss!(div_us)

            div_uw = wdiv.(u)
            Spaces.weighted_dss!(div_uw)

            div_exact = @. -2 * sind(coords.long) * cosd(coords.lat) / radius

            err_sdiv[Ie] = norm(div_us .- div_exact)
            err_wdiv[Ie] = norm(div_uw .- div_exact)
            Δh[Ie] = 1 / Ne

        end

        convergence_rate_sdiv = convergence_rate(err_sdiv, Δh)
        convergence_rate_wdiv = convergence_rate(err_wdiv, Δh)

        for Ie in range(1, length = length(Nes) - 1)
            @test convergence_rate_sdiv[Ie] ≈ (Nq - 1) atol = 0.1
            @test convergence_rate_wdiv[Ie] ≈ (Nq - 1) atol = 0.1
        end
    end
end


using LinearAlgebra, Plots, Statistics, TaylorSeries, Printf

path1=realpath(dirname(@__FILE__)*"/..")
include(string(path1,"/src/TaylorBigF.jl"))

# 3.2 Function Approximation

p=2000;setprecision(p); setrounding(BigFloat, RoundUp);
precision(BigFloat)==p
L=BigFloat("1"); n=BigInt(201); iN=convert(Int64,n); dx=BigFloat("2")*L/(n-BigFloat("1"));
x=convert(Array{BigFloat},-L:dx:L)
length(x)==n
# x=-L:dx:L
# x=convert(Array{BigFloat},(rand(n).-1/2)/10)
I=zeros(BigFloat, n, n);for i=1:n I[i,i]=BigFloat("1") end

V=TaylorBigF.calc_Vandermonde(x,n,n)

function func(x) sin(x) end

f=func.(x)
a=V\f
# plot(a)
# inv_V_comp = inv(V)
# a=inv_V_comp*f

# inv_V_analyt,det_V_analyt=TaylorBigF.calc_Vandermonde_inv_formula(x,n);
# a=inv_V_analyt*f

# inv_V_comp = V\I
# a=inv_V_comp*f

# a=convert(Array{BigFloat},convert(Array{Float64},V)\convert(Array{Float64},f))

xi=(x.+dx/BigFloat("2"))[1:end-1];
Vi=TaylorBigF.calc_Vandermonde(xi,n-1,n);

fi_comp=Vi*a;
fi_analyt=func.(xi)
diff_f=maximum(abs.(f-V*a));@sprintf("diff_f=%1.3e",diff_f)
diff_fi=maximum(abs.(fi_analyt-fi_comp));@sprintf("diff_fi=%1.3e",diff_fi)

# l1=30;fF=convert(Array{Float64},fi_comp);xF=convert(Array{Float64},x);xiF=convert(Array{Float64},xi);
# ticksY = round.(range(minimum(fF[1:l1]),stop=maximum(fF[1:l1]),length = 5),digits=2);
# ticksX = round.(range(minimum(xiF[1:l1]),stop=maximum(xiF[1:l1]),length = 5),digits=2);
# scatter(xF[1:l1],f[1:l1],color=:black,markersize=3,xlabel="x",
# ylabel="f(x)",yticks=ticksY,xticks=ticksX,label="f(x) analytical",
# legend=:bottomright,markershape=:rtriangle)
# scatter!(xiF[1:l1],fi_comp[1:l1],color=:black,markersize=3,
# label=string("f(x) numerical p=",p),markershape=:circle)# circle,rect,cross

# savefig(string("f__.pdf"))


using SDELab2 # SDELab2.jl should be findable via LOAD_PATH
println("run.jl")

function drift_f(u,r)
  return (r*u)
end
#
function diff_g_mat(u,dw,alpha)
  return alpha*(u.*dw)
end
#
function diff_g_vecs(alpha)
  m=2
  A=Array(Function,m)
  g1(u)=[u[1]*alpha,0]
  g2(u)=[0,u[2]*alpha]
  A[1]=g1
  A[2]=g2
  return A
end
#
function gbm_exact(y0,r,sig,t,w,ito_flag)
  if ito_flag==true
    out=exp((r-0.5*sig^2)*t+sig*w).*y0
  else
    out=exp(r*t+sig*w).*y0
  end
end
#
function get_error(dt,solver,test_case)
  r=test_case["r"]
  sig=test_case["sig"]
  noise_type=test_case["noise_type"]
  d=2
  m=2
  tf=1
  u=test_case["initial_data"]
  fcn=set_fcn(d,m,x->drift_f(x,r),(x,dw)->diff_g_mat(x,dw,sig),diff_g_vecs(sig), noise_type)
  opt=set_opt(dt,solver)
  t,y,W=sde_strong_solution(fcn, [0,tf], u, opt)
  exact=gbm_exact(u,r,sig,t[end],W,opt["is_ito"])
  yf=reshape(y[end,:],fcn.d)
  return norm(exact-yf)/norm(exact)# relative error
end
test_case1=Dict("r"=>-0.1, "sig"=>0.5, "noise_type"=>2, "initial_data"=>[1.,1.])
#

#fcn=set_fcn(2,2,x->drift_f(x,r),(x,dw)->diff_g_mat(x,dw,sig),diff_g_vecs(1), 2)
#opt=set_opt(0.01,"SIBDF")
#opt["nlsolve_param"].xtol=1e-5

#t,y,W=sde_strong_solution(fcn, linspace(0.0,1.0,10000), u, opt)
#exact=gbm_exact(u,r,sig,t[end],W,opt["is_ito"])
#
#plot(t,y)
#print(W,"u=",u)
dt_vals=[0.01,0.005,0.002,0.001,0.0005,1e-4]
error=similar(dt_vals)
all_error=Dict()
M=10
for key in keys(SDELab2.method_code)
  print("Integrator:", method_code[key])
  for i in 1:length(dt_vals)
    sum=0
    for m in 1:M
      err=get_error(dt_vals[i],key,test_case1)
      sum+=err^2
   end
  error[i]=sqrt(sum/M)
  end
  all_error[key]=deepcopy(error)
end
using PyPlot
for key in keys(method_code)
  loglog(dt_vals,all_error[key],label=method_code[key])
  println(method_code[key],all_error[key])
end
legend()


"""
    TestFunctions.jl

A collection of test functions for evaluating optimization algorithms.  Excerpted from Appendix B of:
Kochenderfer and Wheeler, "Algorithms for Optimization", MIT Press, 2019 (forthcoming)
"""
module TestFunctions

export
    ackley,
    booth,
    branin,
    flower,
    michalewicz,
    rosenbrock,
    wheeler,
    circle

"""
    ackley(x, a=20, b=0.2, c=2π)

Ackley's function.  N-D.
"""
function ackley(x, a=20, b=0.2, c=2π)
    d = length(x)
    return -a*exp(-b*sqrt(sum(x.^2)/d)) -
        exp(sum(cos.(c*xi) for xi in x)/d) + a + e
end

"""
    booth(x)

Booth's function. 2D.
"""
booth(x) = (x[1]+2x[2]-7)^2 + (2x[1]+x[2]-5)^2

"""
    branin(x; a=1, b=5.1/(4π^2), c=5/π, r=6, s=10, t=1/(8π))

Branin Function. 2D.
"""
function branin(x; a=1, b=5.1/(4π^2), c=5/π, r=6, s=10, t=1/(8π))
    return a*(x[2]-b*x[1]^2+c*x[1]-r)^2 + s*(1-t)*cos(x[1]) + s
end

"""
    flower(x; a=1, b=1, c=4)

Flower function. 2D.
"""
function flower(x; a=1, b=1, c=4)
    return a*norm(x) + b*sin(c*atan2(x[2],x[1]))
end

"""
    michalewicz(x; m=10)

Michaelewicz function.  N-D.
"""
function michalewicz(x; m=10)
    return -sum(sin(v)*sin(i*v^2/π)^(2m) for
                (i,v) in enumerate(x))
end

"""
    rosenbrock(x; a=1, b=5)

Rosenbrock's banana function. 2-D.
"""
rosenbrock(x; a=1, b=5) = (a-x[1])^2 + b*(x[2] - x[1]^2)^2

"""
    wheeler(x, a=1.5)

Wheeler's ridge. 2-D.
"""
wheeler(x, a=1.5) = -exp(-(x[1]*x[2] - a)^2 - (x[2]-a)^2)

"""
    circle(x)

Circle function. Multiobjective.  2-D.
"""
function circle(x)
    θ = x[1]
    r = 0.5 + 0.5*(2x[2]/(1+x[2]^2))
    y1 = 1 - r*cos(θ)
    y2 = 1 - r*sin(θ)
    return [y1, y2]
end

end # module


using Test

import LinearAlgebra.I
import LinearAlgebra.pinv
import LinearAlgebra.lu

include("../../../lib/KTBC.jl")
import .KTBC.CreateKTBC

# 8
K₅, T₅ = CreateKTBC(5)

6 * (inv(T₅) - inv(K₅))

6 * inv(K₅)

# 9
K₄, T₄, B₄, C₄  = CreateKTBC(4)
f = [
  1
  -1
  1
  -1
]
e = [
  1
  1
  1
  1
]

u = pinv(C₄) * f

function roundWithLosts(value, lost=10^(-5))
    round.(Int, lost * value)
end

# left division operator: \ - action its just like inv(a) * b
@test roundWithLosts(C₄ \ e) == roundWithLosts(inv(C₄) * e)
@test roundWithLosts(C₄ \ f) == roundWithLosts(inv(C₄) * f)

# 10
H₃ = [
  2 -1 0
  -1 2 -1
  0 -1 1
]

invH₃ = round.(Int, inv(H₃))

I₃ = convert(Array{Int}, I(3))

J₃ = reverse(I₃; dims=2)

@test J₃ * T₃ * J₃ == H₃

@test invH₃ == J₃ * round.(Int, inv(T₃)) * J₃

U₃ = createU(3)
S₃ = createS(3)

@test U₃ * U₃' == H₃

@test S₃' * S₃ == invH₃

# 11

U₃ = createS(3)

SE = [
 0.0  0.0  1.0
 0.0  1.0  1.0
 1.0  1.0  1.0
]

NW = [
 1.0  1.0  1.0
 1.0  1.0  0.0
 1.0  0.0  0.0
]

L = [
 1.0  0.0  0.0
 1.0  1.0  0.0
 1.0  1.0  1.0
]

@test J₃ * U₃ == SE

@test U₃ * J₃ == NW

@test J₃ * U₃ * J₃ == L

@test SE * NW == [
 1.0  0.0  0.0
 2.0  1.0  0.0
 3.0  2.0  1.0
]

# 12, 13
L, U = lu(C₄)

@test round.(Int, U * 6) == [
  12  -6   0  -6
  0   9  -6  -3
  0   0   8  -8
  0   0   0   0
]

U = 6 \ round.(Int, U * 6)

# Only non-zero rows
U = U[vec(mapslices(row -> any(row .!= 0), U', dims=1)), :]
Uᵀ = U'

UUᵀ = U * Uᵀ

UUᵀ = 36 \ round.(Int, 36 * U * Uᵀ)
U⁺ = Uᵀ * inv(UUᵀ)

L = C₄ * U⁺

# Proof
@test C₄ == round.(Int, L * U)

# Or
Lₐ = C₄ * pinv(U)
@test C₄ == round.(Int, Lₐ * U)


module NewtonsMethod

using LinearAlgebra, ForwardDiff

function newtonroot(f, f_prime; x_0,  tol = 1E-7, maxiter = 1000)
    x_old=x_0
    normdiff=Inf
    iter=1

    if(abs(f_prime(x_old))<=tol)
        return (value=x_old,normdiff=f(x_old),iter=0)
    end

    while normdiff>tol && iter<=maxiter
        x_new= x_old - f(x_old)/f_prime(x_old)
        if isnan(x_new)
            return (nothing, nothing, nothing)
        end
        normdiff=norm(x_new-x_old)
        x_old=x_new
        iter = iter + 1

    end
    if(normdiff>tol)
        return (nothing, nothing, nothing)
    end

    return (value=x_old,normdiff=normdiff,iter=iter)
end

function newtonroot(f; x_0, tol = 1E-7, maxiter = 1000)
    #find derivate of f

    x_old=x_0
    normdiff=Inf
    iter=1

    D(f) = x -> ForwardDiff.derivative(f, x)
    f_prime=D(f)

    if(abs(f_prime(x_old))<=tol)
        return (value=x_old,normdiff=f(x_old),iter=0)
    end

    while normdiff>tol && iter<=maxiter
        x_new= x_old - f(x_old)/f_prime(x_old)
        if isnan(x_new)
            return (nothing, nothing, nothing)
        end
        normdiff=norm(x_new-x_old)
        x_old=x_new

        iter += 1
    end
    if(normdiff>tol)
        return (nothing, nothing, nothing)
    end
    return (value=x_old,normdiff=normdiff,iter=iter)
end

end


### A Pluto.jl notebook ###
# v0.12.6

using Markdown
using InteractiveUtils

# ╔═╡ af957aaa-1cb9-11eb-1eb9-6bf4907a1628
using LinearAlgebra

# ╔═╡ 2572e992-1cba-11eb-3315-1f132089db17
A = [10 2;3 4]

# ╔═╡ 35f6a224-1cba-11eb-0e07-c9c4b43619b8
det(A)

# ╔═╡ 5ef57da4-1cba-11eb-12ae-8bd131d32935
tr(A)

# ╔═╡ 9bc6985a-1cba-11eb-3836-c314153cc1cd
inv(A)

# ╔═╡ ad81fcd8-1cba-11eb-2a91-af2cfbc9cc73
eigvals(A)

# ╔═╡ db9c32d2-1cba-11eb-36fd-4db5d1ed8a9c
eigvecs(A)

# ╔═╡ 97dfd928-1cbb-11eb-2858-2f5a079ee685
factorize(A)

# ╔═╡ d36a84b4-1cbb-11eb-1884-b7165e7741b3
Symmetric(A)

# ╔═╡ eba461d0-1cbb-11eb-042a-6f6d85b8f6e6
Diagonal(A)

# ╔═╡ f59c977a-1cbb-11eb-1a8a-17f77b281e40
Tridiagonal(A)

# ╔═╡ 1387ed2a-1cbc-11eb-1d9b-ed159167d81f
svd(A)

# ╔═╡ 6ffbd562-1cbc-11eb-37ab-d368cb70fe31
qr(A)

# ╔═╡ 86c0ffa2-1cbc-11eb-3f90-8fe517e4a514
eigen(A)

# ╔═╡ f3f9a022-1cbe-11eb-37b9-392a6bd21c8d
rank(A)

# ╔═╡ fd489688-1cbe-11eb-19f0-83313a3c4f0d
A*A

# ╔═╡ 3e1faae8-1cbf-11eb-11b7-a7f3ce3b43c1
A.*A

# ╔═╡ Cell order:
# ╠═af957aaa-1cb9-11eb-1eb9-6bf4907a1628
# ╠═2572e992-1cba-11eb-3315-1f132089db17
# ╠═35f6a224-1cba-11eb-0e07-c9c4b43619b8
# ╠═5ef57da4-1cba-11eb-12ae-8bd131d32935
# ╠═9bc6985a-1cba-11eb-3836-c314153cc1cd
# ╠═ad81fcd8-1cba-11eb-2a91-af2cfbc9cc73
# ╠═db9c32d2-1cba-11eb-36fd-4db5d1ed8a9c
# ╠═97dfd928-1cbb-11eb-2858-2f5a079ee685
# ╠═d36a84b4-1cbb-11eb-1884-b7165e7741b3
# ╠═eba461d0-1cbb-11eb-042a-6f6d85b8f6e6
# ╠═f59c977a-1cbb-11eb-1a8a-17f77b281e40
# ╠═1387ed2a-1cbc-11eb-1d9b-ed159167d81f
# ╠═6ffbd562-1cbc-11eb-37ab-d368cb70fe31
# ╠═86c0ffa2-1cbc-11eb-3f90-8fe517e4a514
# ╠═f3f9a022-1cbe-11eb-37b9-392a6bd21c8d
# ╠═fd489688-1cbe-11eb-19f0-83313a3c4f0d
# ╠═3e1faae8-1cbf-11eb-11b7-a7f3ce3b43c1


module GMG
export V_cycle!
function jacobi_relaxation!(level::Int,nx::Int,ny::Int,u::Array{Float64,2},f::Array{Float64,2},iters::Int,pre::Bool)

    dx=1.0/nx
    dy=1.0/ny
    Ax=1/dx^2;Ay=1/dy^2
    Ap=1.0/(2.0*(Ax+Ay))

    #Dirichlet BC
    u[  1,:]=-u[    2,:]
    u[end,:]=-u[end-1,:]
    u[  :,1]=-u[    :,2]
    u[:,end]=-u[:,end-1]

  #if it is a pre-sweep, u is fully zero (on the finest grid depends on BC, always true on coarse grids)
  # we can save some calculation, if doing only one iteration, which is typically the case.
    if ((pre) && (level>1))
        u[2:nx+1,2:ny+1] = -Ap*f[2:nx+1,2:ny+1]

        #Dirichlet BC
        u[  1,:]=-u[    2,:]
        u[end,:]=-u[end-1,:]
        u[  :,1]=-u[    :,2]
        u[:,end]=-u[:,end-1]
        iters=iters-1
    end

    for it = 1:iters
        u[2:nx+1,2:ny+1] = Ap*(Ax*(u[3:nx+2,2:ny+1] + u[1:nx,2:ny+1])
                             + Ay*(u[2:nx+1,3:ny+2] + u[2:nx+1,1:ny])
                             - f[2:nx+1,2:ny+1])
        #Dirichlet BC
        u[  1,:]=-u[    2,:]
        u[end,:]=-u[end-1,:]
        u[  :,1]=-u[    :,2]
        u[:,end]=-u[:,end-1]
    end
    #  if(not pre):
    #    return u,None

    res=zeros(Float64,nx+2,ny+2)
    res[2:nx+1,2:ny+1]=f[2:nx+1,2:ny+1]-(( Ax*(u[3:nx+2,2:ny+1]+u[1:nx,2:ny+1])
                                         + Ay*(u[2:nx+1,3:ny+2]+u[2:nx+1,1:ny])
                                         - 2.0*(Ax+Ay)*u[2:nx+1,2:ny+1]))
    return u,res
end

function restrict(nx,ny,v)
  """
  restrict 'v' to the coarser grid
  """
  v_c=zeros(Float64,nx+2,ny+2)

  v_c[2:nx+1,2:ny+1]=0.25*(v[2:2:2*nx,2:2:2*ny]+v[2:2:2*nx,3:2:2*ny+1]+v[3:2:2*nx+1,2:2:2*ny]+v[3:2:2*nx+1,3:2:2*ny+1])

  return v_c
end

function prolong(nx,ny,v)
  """
  interpolate 'v' to the fine grid
  """
  v_f=zeros(Float64,2*nx+2,2*ny+2)

  a=0.5625; b=0.1875; c= 0.0625

  v_f[2:2:2*nx  ,2:2:2*ny  ] = a*v[2:nx+1,2:ny+1]+b*(v[1:nx  ,2:ny+1]+v[2:nx+1,1:ny]  )+c*v[1:nx  ,1:ny  ]
  v_f[3:2:2*nx+1,2:2:2*ny  ] = a*v[2:nx+1,2:ny+1]+b*(v[3:nx+2,2:ny+1]+v[2:nx+1,1:ny]  )+c*v[3:nx+2,1:ny  ]
  v_f[2:2:2*nx  ,3:2:2*ny+1] = a*v[2:nx+1,2:ny+1]+b*(v[1:nx  ,2:ny+1]+v[2:nx+1,3:ny+2])+c*v[1:nx  ,3:ny+2]
  v_f[3:2:2*nx+1,3:2:2*ny+1] = a*v[2:nx+1,2:ny+1]+b*(v[3:nx+2,2:ny+1]+v[2:nx+1,3:ny+2])+c*v[3:nx+2,3:ny+2]

  return v_f
end

function V_cycle!(nx::Int,ny::Int,num_levels::Int,u::Array{Float64,2},f::Array{Float64,2},level::Int)
  """
  V cycle
  """
  if (level==num_levels)#bottom solve
    u,res=jacobi_relaxation!(level,nx,ny,u,f,50,true)
    return u,res
  end
  #Step 1: Relax Au=f on this grid
  u,res=jacobi_relaxation!(level,nx,ny,u,f,2,true)

  #Step 2: Restrict residual to coarse grid
  res_c=restrict(nx÷2,ny÷2,res)

  #Step 3:Solve A e_c=res_c on the coarse grid. (Recursively)
  e_c=zero(res_c)
  e_c,res_c=V_cycle!(nx÷2,ny÷2,num_levels,e_c,res_c,level+1)

  #Step 4: Interpolate(prolong) e_c to fine grid and add to u
  u=u+prolong(nx÷2,ny÷2,e_c)

  #Step 5: Relax Au=f on this grid
  u,res=jacobi_relaxation!(level,nx,ny,u,f,1,false)
  return u,res
end

end


#=
Let d(n) be defined as the sum of proper divisors of n (numbers less than n which divide evenly into n).
If d(a) = b and d(b) = a, where a ≠ b, then a and b are an amicable pair and each of a and b are called amicable numbers.

For example, the proper divisors of 220 are 1, 2, 4, 5, 10, 11, 20, 22, 44, 55 and 110; therefore d(220) = 284. The proper divisors of 284 are 1, 2, 4, 71 and 142; so d(284) = 220.

Evaluate the sum of all the amicable numbers under 10000.
=#

function sum_proper_divisors(n)
  i = 1
  sum = 0
  while (i < n)
    if n%i==0
      sum += i
    end
    i += 1
  end
  sum
end

function calc()
  sum = 0
  for i in 1:9_999
    a = sum_proper_divisors(i)
    b = sum_proper_divisors(a)
    if i == b && a != b
      sum += i
    end
  end
  sum
end
@time println(calc())


using Plots; gr()
using DifferentialEquations

# Lorenz example from documentation
function g(du,u,p,t)
 du[1] = p[1]*(u[2]-u[1])
 du[2] = u[1]*(p[2]-u[3]) - u[2]
 du[3] = u[1]*u[2] - p[3]*u[3]
end
u0 = [1.0;0.0;0.0]
tspan = (0.0,180.0)
p = [10.0,28.0,8/3]
prob = ODEProblem(g,u0,tspan,p)

sollor = solve(prob,reltol=1e-14,abstol=1e-14)

plot(sollor,title="Lorenz System",label=false, xaxis="Time (x)")

# Get the results
u1 = [s[1] for s in sollor.u];
u2 = [s[2] for s in sollor.u];
u3 = [s[3] for s in sollor.u];
t = sollor.t

plot(t,u1, xaxis=false, yaxis=false, legend=false)
plot!(t,u2, xaxis=false, yaxis=false, legend=false)
plot!(t,u3, xaxis=false, yaxis=false, legend=false)
plot(u1,u2, xaxis=false, yaxis=false, legend=false)
plot(u1,u3, xaxis=false, yaxis=false, legend=false)

# Logistic model as system of 2 ODEs
# Cell = u[1]
# Agar = u[2]

function g(du,u,p,t)
 du[1] = p[1]*u[1]*u[2]
 du[2] = -p[1]*u[1]*u[2]
end
u0 = [0.1;1.0]
tspan = (0.0,10.0)
p = [5.0]
prob = ODEProblem(g,u0,tspan,p)

sollogistic = solve(prob,reltol=1e-8,abstol=1e-8)
plot(sollogistic)

using Plots
using DifferentialEquations

function g(du,u,p,t)
 du[1] = p[1]*(u[2]-u[1])
 du[2] = u[1]*(p[2]-u[3]) - u[2]
 du[3] = u[1]*u[2] - p[3]*u[3]
end
u0 = [1.0;0.0;0.0]
tspan = (0.0,180.0)
p = [10.0,28.0,8/3]
prob = ODEProblem(g,u0,tspan,p)

sollor = solve(prob,reltol=1e-14,abstol=1e-14)

t = sollor.t
u1 = [s[1] for s in sollor.u];
u2 = [s[2] for s in sollor.u];
u3 = [s[3] for s in sollor.u];
# Get the results
plotlzsys=plot(t,u1, xaxis=false, yaxis=false, legend=false)
plot!(t,u2, xaxis=false, yaxis=false, legend=false)
plot!(t,u3, xaxis=false, yaxis=false, legend=false)

plotlz = plot(u1,u2, xaxis=false, yaxis=false, legend=false)
#plot(u1,u3, xaxis=false, yaxis=false, legend=false)

plot(plotlzsys,plotlz,layout=(1,2),legend=false,linewidth=3 )


# Arvind Prasadan
# 2019 January
# DMF Package
# Generates a realization of an cosine sequence

"""
	gen_cos_sequence(n = 100, f = 1.0, fs = 1.0, phase = 0.0)

Generates a realization of an cosine sequence

# Arguments
- `n`: Length of sequence
- `f`: Frequency
- `fs`: Sampling frequency
- `phase`: Phase shift

# Outputs
- `x`: vector of cosine process realization
- `t`: Time vector
"""
function gen_cos_sequence(n = 100, f = 1.0, fs = 1.0, phase = 0.0)

    n = round(abs(n[1])) # Samples
    n = (1 <= n) ? n : 1

    f = abs(f[1]) # Frequency
    fs = abs(fs[1]) # Sampling Frequency

    phase = phase[1] # Phase shift

    t = collect(0.0:(1.0 / fs):((n - 1) / fs)) # Time vector

    x = cos.(f * t .+ phase) # Cosines: time signature

    return x, t
end


# Exercise "Histogram of the English alphabets"
# Julia code generate the histogram
using Plots,DelimitedFiles
f = readdlm("ch3_data_english.txt")
bar(f/100,xticks = (1:26,'a':'z'),yticks = 0.0:0.02:0.1,label = false)

# Exercise "Histogram of the throwing a die"
# Julia code generate the histogram
using Plots,Random
q = rand(1:6,100)
histogram(q,bins = 6,normalize = true,label = "N = 100")

# Exercise "Histogram of an exponential random variable"
# Julia code used to generate the plots
using Plots,Random,Distributions
λ = 1
X = rand(Exponential(1/λ),1_000)
histogram(X,bins = 200,label = "K = 200")

# Exercise "Cross validation loss"
using Plots,Random,Distributions,StatsBase
λ = 1
n = 1_000
X = rand(Exponential(1/λ),n)
m = 6:200
J = zeros(195)
for i = 1:195
    hist = fit(Histogram,X,range(minimum(X),maximum(X),length = m[i]+1))
    h = n/m[i]
    J[i] = 2/((n-1)*h)-((n+1)/((n-1)*h))*sum( (hist.weights/n).^2 )
end

plot(m,J,label = false)

# Exercise "Expectation E[X] where X is uniform from [0,1]"
using Random,Statistics
X = rand(10_000)
mean(X)

# Exercise "Mean from PMF"
# Julia code to compute the expectation
p = [0.25,0.5,0.25]
x = [0,1,2]
EX = sum(p .* x)

# Exercise "Mean of geometric random variable"
k = 1:100
p = 0.5.^k
EX = sum(p .* k)

# Exercise "Bernoulli Random Variables"
using Plots,Distributions
p = 0.5
n = 1
X = rand(Binomial(n,p),1_000)
histogram(X,bins = 2,label = false)

# Julia code to generate Erdos Renyi Graph
using Plots,Random,LinearAlgebra,GraphRecipes
A = rand(40,40) .< 0.1
A = triu(A,1)
A = A + A'
graphplot(A)

# Exercise "Binomial Random Variables" 2
using Plots,Distributions
p = 0.5
n = 10
X = rand(Binomial(n,p),5_000)
histogram(X,label = false)

# Exercise "Binomial CDF"
using Plots,Distributions
x = 0:10
p = 0.5
n = 10
F = cdf.(Binomial(n,p),x)
plot(x,F,linetype = :steppost,label = false,markershape = :circle)

# Exercise "Poisson CDF"
## Code for the Poisson PMF
using Plots,Distributions
λ_set = [1,4,10]
p = zeros(21,3)
k = 0:1:20
for i = 1:3
    p[:,i] = pdf.(Poisson(λ_set[i]),k)
end
plot(k,p[:,1],line = :stem,markershape = :circle,label = "λ = 1")
plot!(k,p[:,2],line = :stem,markershape = :circle,label = "λ = 4")
plot!(k,p[:,3],line = :stem,markershape = :circle,label = "λ = 10")

## Code for Poisson CDF
using Plots,Distributions
λ_set = [1,4,10]
p = zeros(21,3)
k = 0:1:20
for i = 1:3
    p[:,i] = cdf.(Poisson(λ_set[i]),k)
end
plot(k,p[:,1],line = :steppost,markershape = :circle,label = "λ = 1")
plot!(k,p[:,2],line = :steppost,markershape = :circle,label = "λ = 4")
plot!(k,p[:,3],line = :steppost,markershape = :circle,label = "λ = 10",legend = :bottomright)

# Exercise "Poisson-Binomial approximation"
using Plots,Distributions
n = 5_000
p = 0.01
λ = n*p
x = 0:120
y = pdf.(Binomial(n,p),x)
z = pdf.(Poisson(λ),x)
plot(x,y,line = :stem,markershape = :circle,label = "Binomial,n = 5000,p = 0.01")
plot!(x,z,label = "Poisson,λ = 50")

# Photon Shot Noise
using Distributions,Images,ImageView,Random
x = Float64.(load("cameraman.tif"))

α = 10
λ = α*x
X1 = rand.(Poisson.(λ))
imshow(X1)

α = 100
λ = α*x
X1 = rand.(Poisson.(λ))
imshow(X1)

α = 1_000
λ = α*x
X1 = rand.(Poisson.(λ))
imshow(X1)


using Statistics: mean

struct LinearModel
    X::Array
    Y::Vector
    beta::Vector #FIMXE: of Real
end

# OLS estimation using (\)
# https://github.com/giob1994/Alistair.jl/blob/3a11c19150169695581b46e4d1895f0641a4c29d/src/linregress.jl#L38-L41
function ols(X::Array, Y::Array)::Array
    return (X' * X) \ (X' * Y)
end

"""Fit ordinary linear regression for observations."""
function ols(sample::Sample; intercept::Bool=false)::LinearModel
    X = (intercept ? add_intercept : identity)(sample.X)
    beta_hat = ols(X, sample.Y)
    return LinearModel(X, sample.Y, beta_hat)
end

# another interface
function ols(x, y; intercept::Bool=false)::LinearModel
    return ols(Sample(x,y), intercept=intercept)
end

"""Return fitted dependent variable Y."""
function yhat(lm::LinearModel)
    return lm.X * lm.beta
end

sum_of_squares(x::Vector)::Real = sum(x .^ 2)

"""Residual sum of squares."""
rss(lm::LinearModel) = sum_of_squares(yhat(lm) - lm.Y)

"""Total sum of squares for Y - mean.
   Also equals var(Y)*n.
"""
tss(lm::LinearModel) = sum_of_squares(lm.Y .- mean(lm.Y)) #

"""R2 (unadjusted) = 1-(RSS/TSS)"""
r2(lm::LinearModel) = 1 - rss(lm)/tss(lm)

betas(lm::LinearModel) = lm.beta

# display LinearModel
function has_intercept(lm::LinearModel)
    return all(lm.X[:,1] .== 1.0)
end

function equation(lm::LinearModel):: String
    equation(has_intercept(lm), lm.beta)
end

function pretty_round(beta, precision::Int=3)
    map(x -> round(x, digits=precision), beta)
end

"""Make string like 3 + 0.5*X1 - 0.1*X2"""
function equation(has_intercept, betas)
    fmt = b -> (b >= 0) ? (" + $b") : (" - $(abs(b))")
    vars = ["*X$i" for i in 1:length(betas)]
    if has_intercept
        vars = pushfirst!(vars[1:end-1], "")
    end
    gen = zip(map(fmt, pretty_round(betas)), vars)
    return "Y =" * join(["$k$x" for (k, x) in gen], "")
end

function desc(lm::LinearModel)::String
    by_line(args...) = join(args, "\n")
    quack_ = has_intercept(lm) ? " " : " no "
    return by_line("Linear model with$(quack_)intercept: $(equation(lm))",
                   "Coefficients: $(lm.beta)",
                   "Sorry I do not know p-values yet.",
                   "R-squared: $(pretty_round(r2(lm)))")
end

show(lm::LinearModel) = println(desc(lm))


using Distributions

# Gamma distribution
k = 3
θ = 2
ρ(x) = x > 0 ? 1/(2*θ^k)*x^(k-1)*exp(-x/θ) : 0

#=
Implementation of random walk sampler Metropolis-Hastings.
Inputs:
- n: The number of states to return
- b: The standard deviation of the normal distribution used in the proposal
     distribution
Outputs:
- Returns an array of length n denoting states S_i in Metropolis-Hastings alg.
=#
function metropolis_hastings(n, b)

    # Samples
    samples = zeros(n)
    samples[1] = rand()

    # Conditional Q distribution
    d = Normal(0,b)

    acceptances = 0

    for i=2:n
        # Propose Y
        ξ = rand(d)
        Y = samples[i-1] + ξ

        # Acceptance probability
        α = min(1, ρ(Y)/ρ(samples[i-1]))

        # Next state
        if rand() < α
            samples[i] = Y
            acceptances += 1
        else
            samples[i] = samples[i-1]
        end
    end

    println(acceptances)

    return samples
end

# Obtain 1000 samples with b = 2
S = metropolis_hastings(1000,16)
density_values = [ρ(x) for x = 0:0.01:30]

histogram(S, normalize = true, legend = false,
    bg = RGB(247/255, 236/255, 226/255), fill = RGB(0,0,20/255))
hist = plot!(0:0.01:30, density_values, lc = :lightgreen)

mixing = plot(S, legend = false, bg = RGB(247/255, 236/255, 226/255), lc = RGB(0,0,20/255),xlabel = "k")

plot(mixing, hist)

savefig("../images/mh_sampling_1_2.svg")


"""
    knapsack_solver(array::Array{Int64,1}, weight::Array{Int64,1}, capacity::Int64)

The knapsack problem is linked to the combinatorial optimization. In more detail all
array-elements are characterized by a given weight. The excersive is to maximize the the
value of subsequence of array-elements by not ooutbiding the `capacity`. For this purpose a
recurisve comparsion between old and new added items is designed wich continously update the
maximum reacahbele value sum. For more information see: [https://en.wikipedia.org/wiki/Knapsack_problem](https://en.wikipedia.org/wiki/Knapsack_problem)

# Arguments
- `array::Array{Int64,1}`: Array with the current values of each items.
- `weight::Array{Int64,1}`: Array with the current weoght of each items.
- `capacity::Int64`: Maximum weight to carry.

# Examples
```julia-repl
julia> import ClassicAlgorithmsCollections
julia> val = [60, 100, 120, 230, 220]
julia> wt = [10, 20, 25, 30, 25]
julia> W = 50
julia> ClassicAlgorithmsCollections.knapsack_solver(val, wt, W)
340
```
"""
function knapsack_solver(array::Array{Int64,1}, weight::Array{Int64,1}, capacity::Int64)
    n = length(array)

    if n == 1 || capacity == 0
        return 0
    end

    if (weight[n] > capacity)
        return knapsack_solver(array[1:n-1], weight[1:n], capacity)

    else
        item_new =
            array[n] + knapsack_solver(array[1:n-1], weight[1:n], capacity - weight[n])
        item_old = knapsack_solver(array[1:n-1], weight[1:n], capacity)
        if item_new > item_old
            return item_new
        end
        return item_old
    end
end


__precompile__()

module RowEchelon
using LinearAlgebra

export rref, rref!

function rref!(A::Matrix{T}, ɛ=T <: Union{Rational,Integer} ? 0 : eps(norm(A,Inf))) where T
    nr, nc = size(A)
    i = j = 1
    while i <= nr && j <= nc
        (m, mi) = findmax(abs.(A[i:nr,j]))
        mi = mi+i - 1
        if m <= ɛ
            if ɛ > 0
                A[i:nr,j] .= zero(T)
            end
            j += 1
        else
            for k=j:nc
                A[i, k], A[mi, k] = A[mi, k], A[i, k]
            end
            d = A[i,j]
            for k = j:nc
                A[i,k] /= d
            end
            for k = 1:nr
                if k != i
                    d = A[k,j]
                    for l = j:nc
                        A[k,l] -= d*A[i,l]
                    end
                end
            end
            i += 1
            j += 1
        end
    end
    A
end

rrefconv(::Type{T}, A::Matrix) where {T} = rref!(copyto!(similar(A, T), A))

"""
    rref(A)
Compute the reduced row echelon form of the matrix A.
Since this algorithm is sensitive to numerical imprecision,
* Complex numbers are converted to ComplexF64
* Integer, Float16 and Float32 numbers are converted to Float64
* Rational are kept unchanged

```jldoctest
julia> rref([ 1  2 -1  -4;
              2  3 -1 -11;
             -2  0 -3  22])
3×4 Array{Float64,2}:
 1.0  0.0  0.0  -8.0
 0.0  1.0  0.0   1.0
 0.0  0.0  1.0  -2.0

julia> rref([16  2  3  13;
              5 11 10   8;
              9  7  6  12;
              4 14 15   1])
4×4 Array{Float64,2}:
 1.0  0.0  0.0   1.0
 0.0  1.0  0.0   3.0
 0.0  0.0  1.0  -3.0
 0.0  0.0  0.0   0.0

julia> rref([ 1  2  0   3;
              2  4  0   7])
2×4 Array{Float64,2}:
 1.0  2.0  0.0  0.0
 0.0  0.0  0.0  1.0
```
"""
rref(A::Matrix{T}) where {T} = rref!(copy(A))
rref(A::Matrix{T}) where {T <: Complex} = rrefconv(ComplexF64, A)
rref(A::Matrix{ComplexF64}) = rref!(copy(A))
rref(A::Matrix{T}) where {T <: Union{Integer, Float16, Float32}} = rrefconv(Float64, A)
rref(A::AbstractMatrix) = rref(Matrix(A))
rref(A::Adjoint) = rref(Matrix(A))

include("RowEchelon_with_pivots.jl")

end # module


# p9.jl - polynomial interpolation in equispaced and Chebyshev pts

N = 16;
xx = -1.01:.005:1.01; clf();
for i = 1:2
    i==1 && ( (s,x) = ("equispaced points", -1 + 2*(0:N)/N) );
    i==2 && ( (s,x) = ("Chebyshev points", cos.(pi*(0:N)/N)) );
    subplot(2,2,i)
    u = 1./(1+16*x.^2);
    uu = 1./(1+16*xx.^2);
    p = polyfit(x,u);              # interpolation
    pp = p(xx);                    # evaluation of interpolant
    plot(x,u,".",markersize=6)
    plot(xx,pp)
    axis([-1.1,1.1,-1,1.5]); title(s);
    error = signif( norm(uu-pp,Inf), 5);
    text(-.5,-.5,"max error = $error",fontsize=8);
end


using TuringModels

# ### snippet 4.43

df = CSV.read(joinpath(@__DIR__, "..", "..", "data", "Howell1.csv"))

# Use only adults and center the weight observations

df2 = filter(row -> row.age >= 18, df)
mean_weight = mean(df2.weight)
df2.weight_c = df2.weight .- mean_weight
first(df2, 5)

# Extract variables for Turing model

x = df2.weight_c
y = df2.height

# Define the regression model

@model line(x, y) = begin
    #priors
    alpha ~ Normal(178.0, 100.0)
    beta ~ Normal(0.0, 10.0)
    s ~ Uniform(0, 50)

    #model
    mu = alpha .+ beta*x
    y .~ Normal.(mu, s)
end

# Draw the samples

chns = sample(line(x, y), NUTS(0.65), 1000)

# Compare with a previous result

clip_43s_example_output = "
Iterations = 1:1000
Thinning interval = 1
Chains = 1,2,3,4
Samples per chain = 1000

Empirical Posterior Estimates:
         Mean        SD       Naive SE       MCSE      ESS
alpha 154.597086 0.27326431 0.0043206882 0.0036304132 1000
 beta   0.906380 0.04143488 0.0006551430 0.0006994720 1000
sigma   5.106643 0.19345409 0.0030587777 0.0032035103 1000

Quantiles:
          2.5%       25.0%       50.0%       75.0%       97.5%
alpha 154.0610000 154.4150000 154.5980000 154.7812500 155.1260000
 beta   0.8255494   0.8790695   0.9057435   0.9336445   0.9882981
sigma   4.7524368   4.9683400   5.0994450   5.2353100   5.5090128
";

# Look at the proper draws (in corrected chn2)

describe(chns) |> display

# End of `04/m4.2t.jl`


#   Unit examples of the FourierAnalysis Package for julia language
#   v 0.0.1 - last update 24th of September 2019
#
#   MIT License
#   Copyright (c) 2019, Marco Congedo, CNRS, Grenobe, France:
#   https://sites.google.com/site/marcocongedo/home

# ? CONTENTS :
#   This example show how to compute the analytic signal via
#   Hilbert transform and how to extract inforation from them.

#   ~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~  #
#                                                                             #
#   ~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~~¤~  #

using FourierAnalysis, FFTW, LinearAlgebra, Statistics, Plots

# add module for reading the two EEG text files to be used ater
push!(LOAD_PATH, @__DIR__)
using IOtxt

# Get EEG file names with complete path (they have extension .txt)
S=getFilesInDir(@__DIR__; ext=(".txt",))

# read the two EEG data files and put them in a Matrix object
X1=readEEG(S[1])
X2=readEEG(S[2])

######################################################################
# Test Hilbert Transform Analytic Signal

using FourierAnalysis, FFTW, LinearAlgebra, Statistics, Plots, DSP
t=128; lab=["x", "real(y)", "imag(y)"]

# Analytic signal of one vector
x=sinusoidal(10, 2, 128, t, π/2; DC=10) # cosine
y=analyticsignal(x)
# note that real(y) is x without the DC level, i.e., x=real(y)+DC
plot([x, real(y), imag(y)]; labels=lab)

# make a check
s=sinusoidal(10, 2, 128, t, 0) # sine
norm(s-imag(y)) # should be zero

# Analytic Signal by DSP.jl
y2=hilbert(x)
norm(s-imag(y2)) # should be zero
# DSP.jl does not remove the DC level
# thus x=real(y2) in this case
plot([x, real(y2), imag(y2)]; labels=lab)

# Analytic signal of multiple vectors
x=hcat(x, sinusoidal(10, 3, 128, t, π/2; DC=10))
y=analyticsignal(x)

# welch-like analytic signal of one vector
# (note edge effects)
x=sinusoidal(10, 2, 128, t*4, π/2; DC=0)
y=analyticsignal(x, t)
plot([x, real(y), imag(y)]; labels=lab)

# Welch-like analytic signal of multiple vectors
x=hcat(x, sinusoidal(10, 3, 128, t*4, π/2; DC=0))
y=analyticsignal(x, t)

t=256
lim=512
Y=analyticsignal(X1[1:lim, :], lim)
plot([X1[1:lim, 1], real(Y[:, 1]), imag(Y[:, 1])])

# compute Welch-like Analytis Signal altogether for X1 and X2
𝒀 = analyticsignal([X1[1:lim, :], X2[1:lim, :]], lim)

## test amplitude, phase and polar functions

x=sinusoidal(10, 2, 128, t*4, 0).*sinusoidal(10, 1, 128, t*4, 0)

# amplitude and phase of a vector using analytic signal standard method
y=analyticsignal(x)
a=amplitude(y)
ϕ=phase(y, func=x->(x+π)/2π*50)
plot(x; labels="signal")
plot!(a; labels="amplitude")
plot!(ϕ; labels="phase")

# see what happen if `x` contains energy in frequencies below sr/wl Hz
# (see documentation of `analyticSignal` function)
y=analyticsignal(x, 64)
a=amplitude(y)
ϕ=phase(y, func=x->(x+π)/2π*50)
plot(x; labels="signal")
plot!(a; labels="amplitude")
plot!(ϕ; labels="phase")

# get Amplitude from analytic Signal of a data matrix holding multiple time-series
A=amplitude(Y)
plot(A[:, 1:2])

# get Phase from analytic Signal
𝛷=phase(Y)
plot(𝛷[:, 1:1])

# get Phase from analytic Signal and transform it in [-1, 1]
𝛷=phase(Y, func=x->(x+π)/2π)
plot(𝛷[:, 1:1])

# get Phase from analytic Signal and transform it to the sine of the phase
𝛷=phase(Y, func=x->sin(x))
plot(𝛷[:, 1:1])

# get Amplitude and phase from analytic Signal
A, 𝛷=polar(Y)


# Julia code for Chapter 9

# Chapter 9.1 Confidence Interval

####################################################################
## Histogram of the sample average
using Distributions, Plots, Printf
gmm = MixtureModel(Normal[Normal(), Normal(5., 1.),], [0.3, 0.7])
x = range(-5., 10., length=1000)
# the population distribution
plot(x, pdf(gmm, x), linewidth=3, label=false, title="Population distribution")
# histograms of 4 random samples of size 50
plot_array = []
for i=1:4
    bin = -5:10
    Y = rand(gmm, 50)
    tt = @sprintf("Mean = %3.2f", mean(Y))
    push!(plot_array, histogram(Y, bins=bin, label = tt, legend=:topleft))
end
plot(plot_array...)
# histogram of sample average
M = zeros(10000)
bin = range(2., 5., step=0.1)
for i = 1:10000
    M[i] = mean(rand(gmm, 50))
end
histogram(M, bins=bin, label=false, title="Histogram of the sample average")

####################################################################
## Compute confidence interval
# Julia code to compute the width of the confidence interval
using Distributions
α = 0.05
μ = 0.
σ = 1.
ϵ = quantile(Normal(μ, σ ), 1. - α/2.) # CI is estimator ± ϵ

####################################################################
## Visualize the t-distribution
# Julia code to plot the t-distribution
using Distributions, Plots
x = range(-5.,5.,length=100)
p1 = pdf(Normal(),x)
p2 = pdf(TDist(11-1),x)
p3 = pdf(TDist(3-1),x)
p4 = pdf(TDist(2-1),x)
plot([p1,p2,p3,p4], label = ["Gaussian(0,1)" "t-dist, N = 11" "t-dist, N = 3" "t-dist, N = 2"], legend=:topright)

####################################################################
## Construct a confidence interval from data
# Julia code to generate a confidence interval
using Statistics, Distributions, Printf
x = [72., 69., 75., 58., 67., 70., 60., 71., 59., 65.]
θhat = mean(x)  # Sample mean
σhat = std(x)   # Sample standard deviation
N = size(x,1)
ν = N-1 # degrees of freedom
α = 0.05        # confidence level
z = quantile(TDist(ν), 1. - α/2.)
CI_L = θhat - z*σhat/sqrt(N)
CI_U = θhat + z*σhat/sqrt(N)
@printf("Confidence interval: [%5.3f - %5.3f]", CI_L, CI_U)

####################################################################

# Chapter 9.2 Bootstrap

####################################################################
## Visualize bootstrap
# Julia code to illustrate bootstrap
using Plots, Distributions, Statistics
gmm = MixtureModel(Normal[Normal(), Normal(5., 1.),], [0.3, 0.7])
# visualize the population
x = range(-5., 10., length=1000);
f = pdf(gmm, x);
plot(x, pdf(gmm,x), linewidth=3, label="Population", legend=:topleft)
# Visualize bootstrap dataset, the real sample data
N = 100
X = rand(gmm, N) # the real sample data
p1 = histogram(X, bins = range(-5,10,length=30), label=false)
# visualize 3 bootstrap samples
plot_array = []
for i=1:3
    # Bootstrap dataset
    idx = rand(1:N, N)
    Xb1 = X[idx]
    push!(plot_array, histogram(Xb1, bins = range(-5.,10.,length=30)))
end
plot(plot_array..., label=false)

# Bootstrap the median
K = 1000 # number of bootstrap re-samples
θhat = zeros(K)
for i=1:K               # repeat K times
    idx = rand(1:N, N)  # sampling w/ replacement
    Y = X[idx]
    θhat[i] = median(Y) # estimator
end
M = mean(θhat)          # bootstrap mean
V = var(θhat)           # bootstrap variance
# histogram of bootstrap replications of estimator
histogram(θhat, bins=range(3.5,5.5,length=30), legend=false)

####################################################################
## Bootstrap median using real data
# Julia code to estimate a bootstrap variance
using Statistics
X = [72., 69., 75., 58., 67., 70., 60., 71., 59., 65.]
N = size(X,1)
K = 1000
θhat = zeros(K)
for i=1:K                       # repeat K times
    idx = rand(1:N, N)          # sampling w/ replacement
    Y = X[idx]
    θhat[i] = median(Y)         # estimator
end
M = mean(θhat)                  # bootstrap mean
V = var(θhat)                   # bootstrap variance

####################################################################

# Chapter 9.3 Hypothesis Testing

####################################################################
## Estimate Z-value
θhat = 0.29                 # Your estimate
θ = 0.25                    # Your hypothesis
σ = sqrt(θ*(1. - θ))        # Known standard deviation
N = 1000                    # Number of samples
Z_hat = (θhat - θ)/(σ/sqrt(N))

####################################################################
## Compute critical value
using Distributions
α = 0.05
z_α = quantile(Normal(), 1-α)

####################################################################
## Compute p-value
using Distributions
p = cdf(Normal(), -1.92)

####################################################################

# Chapter 9.5 ROC Curve

####################################################################
## Plot an ROC curve
# Julia code to plot ROC curve
using Distributions, Plots, Statistics
σ = 2.
μ = 3.
PF1 = zeros(1000); PD1 = zeros(1000)
PF2 = zeros(1000); PD2 = zeros(1000)
αset = range(0.,1.,length=1000)
d = Normal()
for i=1:1000
    α = αset[i]
    PF1[i] = α
    PD1[i] = α
    PF2[i] = α
    PD2[i] = 1. - cdf(d, quantile(d, 1-α)-μ/σ)
end
plot(PF1, PD1, linewidth=3, label = "Blind guess")
plot!(PF2, PD2, linewidth=3, label = "Neyman-Pearson", legend=:bottomright)
# Computer area under curve
auc1 = sum(PD1.*[0.; diff(PF1)])
auc2 = sum(PD2.*[0.; diff(PF2)])

####################################################################
## Another ROC curve
# Julia code to generate the ROC curve.
using Distributions, Plots
σ = 2.
μ = 3.
PF1 = zeros(1000); PD1 = zeros(1000)
PF2 = zeros(1000); PD2 = zeros(1000)
αset = range(0.,0.5,length=1000)
d = Normal()
for i=1:1000
    α = αset[i]
    PF1[i] = 2. *α
    PD1[i] = 1. - (cdf(d, quantile(d,1. - α) - μ/σ) -
        cdf(d, -quantile(d, 1. -α) - μ/σ))
end
αset = range(0.,1.,length=1000)
for i=1:1000
    α = αset[i]
    PF2[i] = α
    PD2[i] = 1. - cdf(d, quantile(d, 1-α)-μ/σ)
end
args = (linewidth=3, xlabel = "p_F", ylabel="p_D")
plot(PF1, PD1; args..., label = "Proposed decision")
plot!(PF2, PD2; args..., label = "Neyman-Pearson", legend=:bottomright)

####################################################################
## ROC on real data
using DelimitedFiles, Plots
scores = readdlm("ch9_ROC_example_data.txt")
scores = scores[:] # a vector
tau = range(0.,1.,length=1000)
labels = [ones(50); zeros(50)]
PF = zeros(1000)
PD = zeros(1000)
for i=1:1000
    idx   = (scores .<= tau[i])
    predict = zeros(100)
    predict[idx]   .= 1.
    true_positive  = 0; true_negative  = 0;
    false_positive = 0; false_negative = 0;
    for j=1:100
        if (predict[j]==1) && (labels[j]==1)
            true_positive += 1   end
        if (predict[j]==1) && (labels[j]==0)
            false_positive += 1 end
        if (predict[j]==0) && (labels[j]==1)
            false_negative += 1 end
        if (predict[j]==0) && (labels[j]==0)
            true_negative += 1   end
    end
    PF[i] = false_positive/50
    PD[i] = true_positive/50
end
plot(PF, PD, linewidth=3, legend=false)


@doc raw"""
    ExponentialKernel([α=1])

The exponential kernel is given by the formula:

```math
\kappa(\mathbf{x},\mathbf{y}) = \exp\left(-\alpha ||\mathbf{x} - \mathbf{y}||\right)
\qquad \alpha > 0
```

where ``\alpha`` is a scaling parameter of the Euclidean distance. The exponential kernel,
also known as the Laplacian kernel, is an isotropic Mercer kernel. The constructor is
aliased by `LaplacianKernel`, so both names may be used:
"""
struct ExponentialKernel{T<:AbstractFloat} <: MercerKernel{T}
    alpha::HyperParameter{T}
    ExponentialKernel{T}(α::Real) where {T<:AbstractFloat} = new{T}(
        HyperParameter(convert(T,α), interval(OpenBound(zero(T)), nothing))
    )
end
ExponentialKernel(α::T=1.0) where {T<:Real} = ExponentialKernel{floattype(T)}(α)
LaplacianKernel = ExponentialKernel

@inline exponentialkernel(z::T, α::T) where {T<:AbstractFloat} = exp(-α*sqrt(z))

@inline basefunction(::ExponentialKernel) = SquaredEuclidean()
@inline function kappa(κ::ExponentialKernel{T}, z::T) where {T<:AbstractFloat}
    exponentialkernel(z, getvalue(κ.alpha))
end


using Plots
pyplot()
#using Random
using Distributions
using Optim

taskNr = 3;

####### Code used for several tasks #######
function myNormal(x,m,s)
    # PDF of Gaussian (watch out for elementwise . !)
    y = 1/sqrt(2*pi*s^2) * exp.(-(x.-m).^2 ./ (2*s^2))

    return y
end

# Parameters:
N = 10000;
mu = 3.4;
sigma = sqrt(3);

# Draw samples from built-in normal distribution
r = rand(Normal(mu,sigma), N);
###################################

if taskNr == 1
    # Draw a histogram, normalised
    histogram(r, normalize = :pdf,
        label = string(N, " Samples"))

    # Draw the PDF
    lo = minimum(r);
    hi = maximum(r);
    symRange = maximum([abs(lo)-mu, abs(hi)-mu]);
    x = range(mu-symRange, mu+symRange, length = 100);
    plot!(x, myNormal(x,mu,sigma),
        w = 2,
        label = "PDF")

    savefig("TaskA-2-a.eps")

elseif taskNr == 2
    # Determine granularity and create x and y:
    dx = .1;
    x = collect(-10:dx:20);
    p = myNormal(x,mu,sigma);

    # Riemann sums of
    # Expected value (weigh elementwise by x):
    E_Riemann = sum(p .* x) * dx;
    # Variance (weigh square-distance elementwise by x):
    variance_Riemann = sum(p .* (x.-mu).^2) * dx;

    # Check closeness:
    println(string("Distance of calculated E from true value: ",
        mu - E_Riemann))
    println(string("Distance of calculated E from sample value: ",
        mean(r) - E_Riemann))
    println(string("Distance of calculated Variance from true value ",
        sigma^2 - variance_Riemann))
    println(string("Distance of calculated Variance from sample value ",
        var(r) - variance_Riemann))
    # Note: var(r) automatically scales with n-1 instead of n

elseif taskNr == 3
    # Parameters:
    N = 10000;
    mu = 0;
    sigma = 1;

    # Draw samples from built-in normal distribution
    x = rand(Normal(mu,sigma), N);

    # Make a log normally distributed vector
    y = exp.(x);

    # Draw a histogram, normalised
    histogram(y,
        normalize = :pdf,
        bins = 0:.1:maximum(y),
        w = .1,     # line width of bin edges
        label = string(N, " Samples"))

    # transform the normal pdf to log-normal pdf:
    pLog(x,m,s) = myNormal(log.(x),m,s) ./ x

    # Draw the PDF
    lo = minimum(y);
    hi = maximum(y);
    y_pdf = collect(lo:.1:hi);
    z = pLog(y_pdf,mu,sigma)
    plot!(y_pdf, z,
        w = 1.5,
        label = "PDF")

    savefig("TaskA-2-d-i.eps")

    # Modes using the points in the PDF:
    @time begin
        pdfMax = maximum(z);
        maxIdx = findfirst(z .== pdfMax);
    end

    modeManual = y_pdf[maxIdx]
    println(string("The manual mode is @ y = ", modeManual))

    # Modes using optimisation:
    myObjective(a) = -pLog(a[1],mu,sigma)    # fix the mean and variance
    @time begin
        optim = optimize(myObjective,[0.0, 0.0],NelderMead());
    end

    modeOptim = optim.minimizer[1];
    println(string("The optimised mode is @ y = ", modeOptim))

end