### A Pluto.jl notebook ###
# v0.16.0

using Markdown
using InteractiveUtils

# ╔═╡ bf54eece-fc02-11ea-3dbc-b37a5644eab1
using Plots

# ╔═╡ cd9cc940-fc02-11ea-2a40-19bc6f79a6d3
using LaTeXStrings

# ╔═╡ dad114fe-fbfb-11ea-32dd-a302838af30f
md"# Linear Programming: Introduction"

# ╔═╡ fdab0c70-fbfb-11ea-116d-9f1b613f9d0d
md"""## Definition

Formally, a linear program is an optimization problem of the form:

```math
\begin{aligned}
\min \vec{c}^\mathsf{T}\vec x&\\
\textrm{ subject to }&\begin{cases}
\mathbf{A}\vec x=\vec b\\
\vec x\ge\vec 0
\end{cases}
\end{aligned}
```

where $\vec c\in\mathbb R^n$, $\vec b\in\mathbb R^m$ and $\mathbf A \in \mathbb R^{m\times n}$. The vector inequality $\vec x\ge\vec 0$ means that each component of $\vec x$ is nonnegative. Several variations of this problem are possible; eg. instead of minimizing, we can maximize, or the constraints may be in the form of inequalities, such as $\mathbf A\vec x\ge \vec b$ or $\mathbf A\vec x\le\vec b$. We shall see later, these variations can all be rewritten into the standard form."""

# ╔═╡ 0d88a212-fbfc-11ea-37ae-c5e0c6a894e8
md"""## Example

A manufacturer produces  four different  products  $X_1$, $X_2$, $X_3$ and $X_4$. There are three inputs to this production process:

- labor in man weeks,
- kilograms of raw material A, and
- boxes  of raw  material  B.

Each product has different input requirements. In determining each  week's production schedule, the manufacturer cannot use more than the available amounts of  manpower and the two raw  materials:

|Inputs|$X_1$|$X_2$|$X_3$|$X_4$|Availabilities|
|------|-----|-----|-----|-----|--------------|
|Person-weeks|1|2|1|2|20|
|Kilograms of material A|6|5|3|2|100|
|Boxes of material B|3|4|9|12|75|
|Production level|$x_1$|$x_2$|$x_3$|$x_4$| |

These constraints can be written in mathematical form

```math
\begin{aligned}
x_1+2x_2+x_3+2x_4\le&20\\
6x_1+5x_2+3x_3+2x_4\le&100\\
3x_1+4x_2+9x_3+12x_4\le&75
\end{aligned}
```

Because negative production levels are not meaningful, we must impose the following nonnegativity constraints on the production levels:

```math
x_i\ge0,\qquad i=1,2,3,4
```

Now suppose that one unit of product $X_1$ sells for €6 and $X_2$, $X_3$ and $X_4$ sell for €4, €7 and €5, respectively. Then, the total revenue for any production decision $\left(x_1,x_2,x_3,x_4\right)$ is

```math
f\left(x_1,x_2,x_3,x_4\right)=6x_1+4x_2+7x_3+5x_4
```

The problem is then to maximize $f$ subject to the given constraints."""

# ╔═╡ 67071830-fbfc-11ea-3925-798d3d288500
md"""## Vector Notation

Using vector notation with

```math
\vec x = \begin{pmatrix}
x_1\\x_2\\x_3\\x_4
\end{pmatrix}
```

the problem can be written in the compact form

```math
\max
\begin{pmatrix}6&4&7&5\end{pmatrix}
\begin{pmatrix}
x_1\\x_2\\x_3\\x_4
\end{pmatrix}\\
\textrm{ subject to }
\begin{cases}
\begin{pmatrix}
1&2&1&2\\
6&5&3&2\\
3&4&9&12
\end{pmatrix}
\begin{pmatrix}
x_1\\x_2\\x_3\\x_4
\end{pmatrix}\le
\begin{pmatrix}
20\\100\\75
\end{pmatrix}\\
\begin{pmatrix}
x_1\\x_2\\x_3\\x_4
\end{pmatrix}\ge
\begin{pmatrix}
0\\0\\0\\0
\end{pmatrix}
\end{cases}
```
 """

# ╔═╡ 3bb281f0-fc02-11ea-34f0-fd00648c27bb
md"""## Two-dimensional Linear Program

Many fundamental concepts of linear programming are easily illustrated in two-dimensional space.

Consider the following linear program:

```math
\begin{aligned}
\max&
\begin{pmatrix}
1&5
\end{pmatrix}
\begin{pmatrix}
x_1\\
x_2
\end{pmatrix}&\\
\textrm{ subject to }
&\begin{cases}
\begin{pmatrix}
5&6\\
3&2
\end{pmatrix}
\begin{pmatrix}
x_1\\
x_2
\end{pmatrix}\le
\begin{pmatrix}
30\\
12
\end{pmatrix}\\
\begin{pmatrix}
x_1\\
x_2
\end{pmatrix}\ge
\begin{pmatrix}
0\\
0
\end{pmatrix}
\end{cases}
\end{aligned}
```
"""

# ╔═╡ e5488250-fc02-11ea-0bc1-4365461fb202
let
	x = -2:6
	plot(x, (30 .- 5 .* x) ./ 6, linestyle=:dash, label=L"5x_1+6x_2=30")
	plot!(x, (12 .- 3 .* x) ./ 2, linestyle=:dash, label=L"3x_1+2x_2=12")
	plot!([0,4,1.5,0,0],[0,0,3.75,5,0], linewidth=2, label="constraints")
	plot!(x, -x ./ 5, label=L"f\left(x_1,x_2\right)=x_1+5x_2=0")
	plot!(x, (25 .- x) ./ 5, label=L"f\left(x_1,x_2\right)=x_1+5x_2=25")
end

# ╔═╡ 08de4fb0-fc03-11ea-08fb-616f792f7c7f
md"""## Slack Variables

Theorems and solution techniques are usually stated for problems in standard form. other forms of linear programs can be converted as the standard form. If a linear program is in the form

```math
\begin{aligned}
\min &\vec{c}^\mathsf{T}\vec x\\
\textrm{ subject to }& \begin{cases}
\mathbf{A}\vec x\ge \vec b\\
\vec x\ge\vec 0
\end{cases}
\end{aligned}
```

then by introducing _surplus variables_, we can convert the orginal problem into the standard form

```math
\begin{aligned}
\min& \vec{c}^\mathsf{T}\vec x\\
\textrm{ subject to }& \begin{cases}
\mathbf{A}\vec x-\mathbf I\vec y = \vec b\\
\vec x\ge\vec 0\\
\vec y\ge\vec 0
\end{cases}
\end{aligned}
```

where $\mathbf I$ is the $m\times m$ identity matrix.

If, on the other hand, the constraints have the form

```math
\begin{cases}
\mathbf{A}\vec x\le b\\
\vec x\ge\vec 0
\end{cases}
```

then we introduce the _slack variables_ to convert the constraints into the form

```math
\begin{cases}
\mathbf{A}\vec x+\mathbf I\vec y = \vec b\\
\vec x\ge\vec 0\\
\vec y\ge\vec 0
\end{cases}
```

Consider the following optimization problem

```math
\begin{aligned}
\max& x_2-x_1\\
\textrm{ subject to }&
\begin{cases}
3x_1=x_2-5\\
\left|x_2\right|\le2\\
x_1\le0
\end{cases}
\end{aligned}
```

To convert the problem into a standard form, we perform the following steps:

1. Change the objective function to:

```math
\min x_1 - x_2
```

2. Substitute $x_1=-x_1^\prime$.

3. Write $\left|x_2\right|\le2$ as $x_2\le 2$ and $-x_2\le 2$.

4. Introduce slack variables $y_1$ and $y_2$, and convert the inequalities above to

```math
\begin{cases}
\hphantom{-}x_2 + y_1 =2\\
-x_2+y_2 =2
\end{cases}
```

5. Write $x_2=u-v$ with $u,v\ge0$.

Hence, we obtain

```math
\begin{aligned}
\min& -x_1^\prime-u+v\\
\textrm{ subject to }&
\begin{cases}
3x_1^\prime+u-v=5\\
u-v+y_1=2\\
v-u+y_2=2\\
x_1^\prime,u,v,y_1,y_2\ge0
\end{cases}
\end{aligned}
```
"""

# ╔═╡ cfd86bf0-fc03-11ea-22ef-b99d3e3fbfce
md"""## Fundamental Theorem of Linear Programming

We consider the system of equalities

```math
\mathbf{A}\vec x=\vec b
```

where $\mathrm{rank}\,\mathbf A=m$.

Let $\mathbf B$ a square matrix whose columns are $m$ linearly independent columns of $\mathbf A$. If necessary, we reorder the columns of $\mathbf A$ so that the columns in $\mathbf B$ appear first: $\mathbf A$ has the form $\left(\mathbf B |\mathbf N\right)$.

The matrix is nonsingular, and thus we can solve the equation

```math
\mathbf B\vec x_\mathbf B = \vec b
```

The solution is $\vec x_\mathbf B = \mathbf B^{-1}\vec b$.

Let $\vec x$ be the vector whose first $m$ components are equal to $\vec x_\mathbf B$ and the remaining components are equal to zero. Then $\vec x$ is a solution to $\mathbf A\vec x=\vec b$. We call $\vec x$  a _basic solution_. Its components refering to the the components of $\vec x_\mathbf B$ are called _basic variables_.

- If some of the basic variables are zero, then the basic solution is _degenerate_.
- A vector $\vec x$ satisfying $\mathbf A\vec x=\vec b$, $\vec x \ge \vec 0$, is said to be a _feasible solution_.
- A feasible solution that is also basic is called a _basic feasible solution_.

The fundamental theorem of linear programming states that when solving a linear programming problem, we need only consider basic feasible solutions. This is because the optimal value (if it exists) is always achieved at a basic solution."""

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"

[compat]
LaTeXStrings = "~1.2.1"
Plots = "~1.22.1"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[Adapt]]
deps = ["LinearAlgebra"]
git-tree-sha1 = "84918055d15b3114ede17ac6a7182f68870c16f7"
uuid = "79e6a3ab-5dfb-504d-930d-738a2a938a0e"
version = "3.3.1"

[[ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[Bzip2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "19a35467a82e236ff51bc17a3a44b69ef35185a2"
uuid = "6e34b625-4abd-537c-b88f-471c36dfa7a0"
version = "1.0.8+0"

[[Cairo_jll]]
deps = ["Artifacts", "Bzip2_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "JLLWrappers", "LZO_jll", "Libdl", "Pixman_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "f2202b55d816427cd385a9a4f3ffb226bee80f99"
uuid = "83423d85-b0ee-5818-9007-b63ccbeb887a"
version = "1.16.1+0"

[[ColorSchemes]]
deps = ["ColorTypes", "Colors", "FixedPointNumbers", "Random"]
git-tree-sha1 = "9995eb3977fbf67b86d0a0a0508e83017ded03f2"
uuid = "35d6a980-a343-548e-a6ea-1d62b119f2f4"
version = "3.14.0"

[[ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "024fe24d83e4a5bf5fc80501a314ce0d1aa35597"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.11.0"

[[Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "Reexport"]
git-tree-sha1 = "417b0ed7b8b838aa6ca0a87aadf1bb9eb111ce40"
uuid = "5ae59095-9a9b-59fe-a467-6f913c188581"
version = "0.12.8"

[[Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "4866e381721b30fac8dda4c8cb1d9db45c8d2994"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.37.0"

[[CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[Contour]]
deps = ["StaticArrays"]
git-tree-sha1 = "9f02045d934dc030edad45944ea80dbd1f0ebea7"
uuid = "d38c429a-6771-53c6-b99e-75d170b6e991"
version = "0.5.7"

[[DataAPI]]
git-tree-sha1 = "bec2532f8adb82005476c141ec23e921fc20971b"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.8.0"

[[DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "7d9d316f04214f7efdbb6398d545446e246eff02"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.10"

[[DataValueInterfaces]]
git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6"
uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464"
version = "1.0.0"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[EarCut_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "3f3a2501fa7236e9b911e0f7a588c657e822bb6d"
uuid = "5ae413db-bbd1-5e63-b57d-d24a61df00f5"
version = "2.2.3+0"

[[Expat_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b3bfd02e98aedfa5cf885665493c5598c350cd2f"
uuid = "2e619515-83b5-522b-bb60-26c02a35a201"
version = "2.2.10+0"

[[FFMPEG]]
deps = ["FFMPEG_jll"]
git-tree-sha1 = "b57e3acbe22f8484b4b5ff66a7499717fe1a9cc8"
uuid = "c87230d0-a227-11e9-1b43-d7ebe4e7570a"
version = "0.4.1"

[[FFMPEG_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "JLLWrappers", "LAME_jll", "Libdl", "Ogg_jll", "OpenSSL_jll", "Opus_jll", "Pkg", "Zlib_jll", "libass_jll", "libfdk_aac_jll", "libvorbis_jll", "x264_jll", "x265_jll"]
git-tree-sha1 = "d8a578692e3077ac998b50c0217dfd67f21d1e5f"
uuid = "b22a6f82-2f65-5046-a5b2-351ab43fb4e5"
version = "4.4.0+0"

[[FixedPointNumbers]]
deps = ["Statistics"]
git-tree-sha1 = "335bfdceacc84c5cdf16aadc768aa5ddfc5383cc"
uuid = "53c48c17-4a7d-5ca2-90c5-79b7896eea93"
version = "0.8.4"

[[Fontconfig_jll]]
deps = ["Artifacts", "Bzip2_jll", "Expat_jll", "FreeType2_jll", "JLLWrappers", "Libdl", "Libuuid_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "21efd19106a55620a188615da6d3d06cd7f6ee03"
uuid = "a3f928ae-7b40-5064-980b-68af3947d34b"
version = "2.13.93+0"

[[Formatting]]
deps = ["Printf"]
git-tree-sha1 = "8339d61043228fdd3eb658d86c926cb282ae72a8"
uuid = "59287772-0a20-5a39-b81b-1366585eb4c0"
version = "0.4.2"

[[FreeType2_jll]]
deps = ["Artifacts", "Bzip2_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "87eb71354d8ec1a96d4a7636bd57a7347dde3ef9"
uuid = "d7e528f0-a631-5988-bf34-fe36492bcfd7"
version = "2.10.4+0"

[[FriBidi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "aa31987c2ba8704e23c6c8ba8a4f769d5d7e4f91"
uuid = "559328eb-81f9-559d-9380-de523a88c83c"
version = "1.0.10+0"

[[GLFW_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libglvnd_jll", "Pkg", "Xorg_libXcursor_jll", "Xorg_libXi_jll", "Xorg_libXinerama_jll", "Xorg_libXrandr_jll"]
git-tree-sha1 = "dba1e8614e98949abfa60480b13653813d8f0157"
uuid = "0656b61e-2033-5cc2-a64a-77c0f6c09b89"
version = "3.3.5+0"

[[GR]]
deps = ["Base64", "DelimitedFiles", "GR_jll", "HTTP", "JSON", "Libdl", "LinearAlgebra", "Pkg", "Printf", "Random", "Serialization", "Sockets", "Test", "UUIDs"]
git-tree-sha1 = "c2178cfbc0a5a552e16d097fae508f2024de61a3"
uuid = "28b8d3ca-fb5f-59d9-8090-bfdbd6d07a71"
version = "0.59.0"

[[GR_jll]]
deps = ["Artifacts", "Bzip2_jll", "Cairo_jll", "FFMPEG_jll", "Fontconfig_jll", "GLFW_jll", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Libtiff_jll", "Pixman_jll", "Pkg", "Qt5Base_jll", "Zlib_jll", "libpng_jll"]
git-tree-sha1 = "ef49a187604f865f4708c90e3f431890724e9012"
uuid = "d2c73de3-f751-5644-a686-071e5b155ba9"
version = "0.59.0+0"

[[GeometryBasics]]
deps = ["EarCut_jll", "IterTools", "LinearAlgebra", "StaticArrays", "StructArrays", "Tables"]
git-tree-sha1 = "58bcdf5ebc057b085e58d95c138725628dd7453c"
uuid = "5c1252a2-5f33-56bf-86c9-59e7332b4326"
version = "0.4.1"

[[Gettext_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "9b02998aba7bf074d14de89f9d37ca24a1a0b046"
uuid = "78b55507-aeef-58d4-861c-77aaff3498b1"
version = "0.21.0+0"

[[Glib_jll]]
deps = ["Artifacts", "Gettext_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Libiconv_jll", "Libmount_jll", "PCRE_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "7bf67e9a481712b3dbe9cb3dac852dc4b1162e02"
uuid = "7746bdde-850d-59dc-9ae8-88ece973131d"
version = "2.68.3+0"

[[Graphite2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "344bf40dcab1073aca04aa0df4fb092f920e4011"
uuid = "3b182d85-2403-5c21-9c21-1e1f0cc25472"
version = "1.3.14+0"

[[Grisu]]
git-tree-sha1 = "53bb909d1151e57e2484c3d1b53e19552b887fb2"
uuid = "42e2da0e-8278-4e71-bc24-59509adca0fe"
version = "1.0.2"

[[HTTP]]
deps = ["Base64", "Dates", "IniFile", "Logging", "MbedTLS", "NetworkOptions", "Sockets", "URIs"]
git-tree-sha1 = "60ed5f1643927479f845b0135bb369b031b541fa"
uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3"
version = "0.9.14"

[[HarfBuzz_jll]]
deps = ["Artifacts", "Cairo_jll", "Fontconfig_jll", "FreeType2_jll", "Glib_jll", "Graphite2_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg"]
git-tree-sha1 = "8a954fed8ac097d5be04921d595f741115c1b2ad"
uuid = "2e76f6c2-a576-52d4-95c1-20adfe4de566"
version = "2.8.1+0"

[[IniFile]]
deps = ["Test"]
git-tree-sha1 = "098e4d2c533924c921f9f9847274f2ad89e018b8"
uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f"
version = "0.5.0"

[[InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[IterTools]]
git-tree-sha1 = "05110a2ab1fc5f932622ffea2a003221f4782c18"
uuid = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
version = "1.3.0"

[[IteratorInterfaceExtensions]]
git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856"
uuid = "82899510-4779-5014-852e-03e436cf321d"
version = "1.0.0"

[[JLLWrappers]]
deps = ["Preferences"]
git-tree-sha1 = "642a199af8b68253517b80bd3bfd17eb4e84df6e"
uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210"
version = "1.3.0"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "8076680b162ada2a031f707ac7b4953e30667a37"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.2"

[[JpegTurbo_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "d735490ac75c5cb9f1b00d8b5509c11984dc6943"
uuid = "aacddb02-875f-59d6-b918-886e6ef4fbf8"
version = "2.1.0+0"

[[LAME_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "f6250b16881adf048549549fba48b1161acdac8c"
uuid = "c1c5ebd0-6772-5130-a774-d5fcae4a789d"
version = "3.100.1+0"

[[LZO_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "e5b909bcf985c5e2605737d2ce278ed791b89be6"
uuid = "dd4b983a-f0e5-5f8d-a1b7-129d4a5fb1ac"
version = "2.10.1+0"

[[LaTeXStrings]]
git-tree-sha1 = "c7f1c695e06c01b95a67f0cd1d34994f3e7db104"
uuid = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
version = "1.2.1"

[[Latexify]]
deps = ["Formatting", "InteractiveUtils", "LaTeXStrings", "MacroTools", "Markdown", "Printf", "Requires"]
git-tree-sha1 = "a4b12a1bd2ebade87891ab7e36fdbce582301a92"
uuid = "23fbe1c1-3f47-55db-b15f-69d7ec21a316"
version = "0.15.6"

[[LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[Libffi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "761a393aeccd6aa92ec3515e428c26bf99575b3b"
uuid = "e9f186c6-92d2-5b65-8a66-fee21dc1b490"
version = "3.2.2+0"

[[Libgcrypt_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgpg_error_jll", "Pkg"]
git-tree-sha1 = "64613c82a59c120435c067c2b809fc61cf5166ae"
uuid = "d4300ac3-e22c-5743-9152-c294e39db1e4"
version = "1.8.7+0"

[[Libglvnd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll", "Xorg_libXext_jll"]
git-tree-sha1 = "7739f837d6447403596a75d19ed01fd08d6f56bf"
uuid = "7e76a0d4-f3c7-5321-8279-8d96eeed0f29"
version = "1.3.0+3"

[[Libgpg_error_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "c333716e46366857753e273ce6a69ee0945a6db9"
uuid = "7add5ba3-2f88-524e-9cd5-f83b8a55f7b8"
version = "1.42.0+0"

[[Libiconv_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "42b62845d70a619f063a7da093d995ec8e15e778"
uuid = "94ce4f54-9a6c-5748-9c1c-f9c7231a4531"
version = "1.16.1+1"

[[Libmount_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "9c30530bf0effd46e15e0fdcf2b8636e78cbbd73"
uuid = "4b2f31a3-9ecc-558c-b454-b3730dcb73e9"
version = "2.35.0+0"

[[Libtiff_jll]]
deps = ["Artifacts", "JLLWrappers", "JpegTurbo_jll", "Libdl", "Pkg", "Zlib_jll", "Zstd_jll"]
git-tree-sha1 = "340e257aada13f95f98ee352d316c3bed37c8ab9"
uuid = "89763e89-9b03-5906-acba-b20f662cd828"
version = "4.3.0+0"

[[Libuuid_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7f3efec06033682db852f8b3bc3c1d2b0a0ab066"
uuid = "38a345b3-de98-5d2b-a5d3-14cd9215e700"
version = "2.36.0+0"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[MacroTools]]
deps = ["Markdown", "Random"]
git-tree-sha1 = "5a5bc6bf062f0f95e62d0fe0a2d99699fed82dd9"
uuid = "1914dd2f-81c6-5fcd-8719-6d5c9610ff09"
version = "0.5.8"

[[Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[MbedTLS]]
deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"]
git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe"
uuid = "739be429-bea8-5141-9913-cc70e7f3736d"
version = "1.0.3"

[[MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[Measures]]
git-tree-sha1 = "e498ddeee6f9fdb4551ce855a46f54dbd900245f"
uuid = "442fdcdd-2543-5da2-b0f3-8c86c306513e"
version = "0.3.1"

[[Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "bf210ce90b6c9eed32d25dbcae1ebc565df2687f"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.2"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[NaNMath]]
git-tree-sha1 = "bfe47e760d60b82b66b61d2d44128b62e3a369fb"
uuid = "77ba4419-2d1f-58cd-9bb1-8ffee604a2e3"
version = "0.3.5"

[[NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[Ogg_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "7937eda4681660b4d6aeeecc2f7e1c81c8ee4e2f"
uuid = "e7412a2a-1a6e-54c0-be00-318e2571c051"
version = "1.3.5+0"

[[OpenSSL_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "15003dcb7d8db3c6c857fda14891a539a8f2705a"
uuid = "458c3c95-2e84-50aa-8efc-19380b2a3a95"
version = "1.1.10+0"

[[Opus_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "51a08fb14ec28da2ec7a927c4337e4332c2a4720"
uuid = "91d4177d-7536-5919-b921-800302f37372"
version = "1.3.2+0"

[[OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[PCRE_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b2a7af664e098055a7529ad1a900ded962bca488"
uuid = "2f80f16e-611a-54ab-bc61-aa92de5b98fc"
version = "8.44.0+0"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "438d35d2d95ae2c5e8780b330592b6de8494e779"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.0.3"

[[Pixman_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "b4f5d02549a10e20780a24fce72bea96b6329e29"
uuid = "30392449-352a-5448-841d-b1acce4e97dc"
version = "0.40.1+0"

[[Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[PlotThemes]]
deps = ["PlotUtils", "Requires", "Statistics"]
git-tree-sha1 = "a3a964ce9dc7898193536002a6dd892b1b5a6f1d"
uuid = "ccf2f8ad-2431-5c83-bf29-c5338b663b6a"
version = "2.0.1"

[[PlotUtils]]
deps = ["ColorSchemes", "Colors", "Dates", "Printf", "Random", "Reexport", "Statistics"]
git-tree-sha1 = "2537ed3c0ed5e03896927187f5f2ee6a4ab342db"
uuid = "995b91a9-d308-5afd-9ec6-746e21dbc043"
version = "1.0.14"

[[Plots]]
deps = ["Base64", "Contour", "Dates", "Downloads", "FFMPEG", "FixedPointNumbers", "GR", "GeometryBasics", "JSON", "Latexify", "LinearAlgebra", "Measures", "NaNMath", "PlotThemes", "PlotUtils", "Printf", "REPL", "Random", "RecipesBase", "RecipesPipeline", "Reexport", "Requires", "Scratch", "Showoff", "SparseArrays", "Statistics", "StatsBase", "UUIDs"]
git-tree-sha1 = "4c2637482176b1c2fb99af4d83cb2ff0328fc33c"
uuid = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
version = "1.22.1"

[[Preferences]]
deps = ["TOML"]
git-tree-sha1 = "00cfd92944ca9c760982747e9a1d0d5d86ab1e5a"
uuid = "21216c6a-2e73-6563-6e65-726566657250"
version = "1.2.2"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[Qt5Base_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Fontconfig_jll", "Glib_jll", "JLLWrappers", "Libdl", "Libglvnd_jll", "OpenSSL_jll", "Pkg", "Xorg_libXext_jll", "Xorg_libxcb_jll", "Xorg_xcb_util_image_jll", "Xorg_xcb_util_keysyms_jll", "Xorg_xcb_util_renderutil_jll", "Xorg_xcb_util_wm_jll", "Zlib_jll", "xkbcommon_jll"]
git-tree-sha1 = "ad368663a5e20dbb8d6dc2fddeefe4dae0781ae8"
uuid = "ea2cea3b-5b76-57ae-a6ef-0a8af62496e1"
version = "5.15.3+0"

[[REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[RecipesBase]]
git-tree-sha1 = "44a75aa7a527910ee3d1751d1f0e4148698add9e"
uuid = "3cdcf5f2-1ef4-517c-9805-6587b60abb01"
version = "1.1.2"

[[RecipesPipeline]]
deps = ["Dates", "NaNMath", "PlotUtils", "RecipesBase"]
git-tree-sha1 = "7ad0dfa8d03b7bcf8c597f59f5292801730c55b8"
uuid = "01d81517-befc-4cb6-b9ec-a95719d0359c"
version = "0.4.1"

[[Reexport]]
git-tree-sha1 = "45e428421666073eab6f2da5c9d310d99bb12f9b"
uuid = "189a3867-3050-52da-a836-e630ba90ab69"
version = "1.2.2"

[[Requires]]
deps = ["UUIDs"]
git-tree-sha1 = "4036a3bd08ac7e968e27c203d45f5fff15020621"
uuid = "ae029012-a4dd-5104-9daa-d747884805df"
version = "1.1.3"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Scratch]]
deps = ["Dates"]
git-tree-sha1 = "0b4b7f1393cff97c33891da2a0bf69c6ed241fda"
uuid = "6c6a2e73-6563-6170-7368-637461726353"
version = "1.1.0"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[Showoff]]
deps = ["Dates", "Grisu"]
git-tree-sha1 = "91eddf657aca81df9ae6ceb20b959ae5653ad1de"
uuid = "992d4aef-0814-514b-bc4d-f2e9a6c4116f"
version = "1.0.3"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[StaticArrays]]
deps = ["LinearAlgebra", "Random", "Statistics"]
git-tree-sha1 = "3240808c6d463ac46f1c1cd7638375cd22abbccb"
uuid = "90137ffa-7385-5640-81b9-e52037218182"
version = "1.2.12"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[StatsAPI]]
git-tree-sha1 = "1958272568dc176a1d881acb797beb909c785510"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.0.0"

[[StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "8cbbc098554648c84f79a463c9ff0fd277144b6c"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.10"

[[StructArrays]]
deps = ["Adapt", "DataAPI", "StaticArrays", "Tables"]
git-tree-sha1 = "2ce41e0d042c60ecd131e9fb7154a3bfadbf50d3"
uuid = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
version = "0.6.3"

[[TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[TableTraits]]
deps = ["IteratorInterfaceExtensions"]
git-tree-sha1 = "c06b2f539df1c6efa794486abfb6ed2022561a39"
uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c"
version = "1.0.1"

[[Tables]]
deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "TableTraits", "Test"]
git-tree-sha1 = "1162ce4a6c4b7e31e0e6b14486a6986951c73be9"
uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
version = "1.5.2"

[[Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[URIs]]
git-tree-sha1 = "97bbe755a53fe859669cd907f2d96aee8d2c1355"
uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
version = "1.3.0"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[Wayland_jll]]
deps = ["Artifacts", "Expat_jll", "JLLWrappers", "Libdl", "Libffi_jll", "Pkg", "XML2_jll"]
git-tree-sha1 = "3e61f0b86f90dacb0bc0e73a0c5a83f6a8636e23"
uuid = "a2964d1f-97da-50d4-b82a-358c7fce9d89"
version = "1.19.0+0"

[[Wayland_protocols_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll"]
git-tree-sha1 = "2839f1c1296940218e35df0bbb220f2a79686670"
uuid = "2381bf8a-dfd0-557d-9999-79630e7b1b91"
version = "1.18.0+4"

[[XML2_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libiconv_jll", "Pkg", "Zlib_jll"]
git-tree-sha1 = "1acf5bdf07aa0907e0a37d3718bb88d4b687b74a"
uuid = "02c8fc9c-b97f-50b9-bbe4-9be30ff0a78a"
version = "2.9.12+0"

[[XSLT_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Libgcrypt_jll", "Libgpg_error_jll", "Libiconv_jll", "Pkg", "XML2_jll", "Zlib_jll"]
git-tree-sha1 = "91844873c4085240b95e795f692c4cec4d805f8a"
uuid = "aed1982a-8fda-507f-9586-7b0439959a61"
version = "1.1.34+0"

[[Xorg_libX11_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll", "Xorg_xtrans_jll"]
git-tree-sha1 = "5be649d550f3f4b95308bf0183b82e2582876527"
uuid = "4f6342f7-b3d2-589e-9d20-edeb45f2b2bc"
version = "1.6.9+4"

[[Xorg_libXau_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4e490d5c960c314f33885790ed410ff3a94ce67e"
uuid = "0c0b7dd1-d40b-584c-a123-a41640f87eec"
version = "1.0.9+4"

[[Xorg_libXcursor_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXfixes_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "12e0eb3bc634fa2080c1c37fccf56f7c22989afd"
uuid = "935fb764-8cf2-53bf-bb30-45bb1f8bf724"
version = "1.2.0+4"

[[Xorg_libXdmcp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fe47bd2247248125c428978740e18a681372dd4"
uuid = "a3789734-cfe1-5b06-b2d0-1dd0d9d62d05"
version = "1.1.3+4"

[[Xorg_libXext_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "b7c0aa8c376b31e4852b360222848637f481f8c3"
uuid = "1082639a-0dae-5f34-9b06-72781eeb8cb3"
version = "1.3.4+4"

[[Xorg_libXfixes_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "0e0dc7431e7a0587559f9294aeec269471c991a4"
uuid = "d091e8ba-531a-589c-9de9-94069b037ed8"
version = "5.0.3+4"

[[Xorg_libXi_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXfixes_jll"]
git-tree-sha1 = "89b52bc2160aadc84d707093930ef0bffa641246"
uuid = "a51aa0fd-4e3c-5386-b890-e753decda492"
version = "1.7.10+4"

[[Xorg_libXinerama_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll"]
git-tree-sha1 = "26be8b1c342929259317d8b9f7b53bf2bb73b123"
uuid = "d1454406-59df-5ea1-beac-c340f2130bc3"
version = "1.1.4+4"

[[Xorg_libXrandr_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libXext_jll", "Xorg_libXrender_jll"]
git-tree-sha1 = "34cea83cb726fb58f325887bf0612c6b3fb17631"
uuid = "ec84b674-ba8e-5d96-8ba1-2a689ba10484"
version = "1.5.2+4"

[[Xorg_libXrender_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "19560f30fd49f4d4efbe7002a1037f8c43d43b96"
uuid = "ea2f1a96-1ddc-540d-b46f-429655e07cfa"
version = "0.9.10+4"

[[Xorg_libpthread_stubs_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "6783737e45d3c59a4a4c4091f5f88cdcf0908cbb"
uuid = "14d82f49-176c-5ed1-bb49-ad3f5cbd8c74"
version = "0.1.0+3"

[[Xorg_libxcb_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "XSLT_jll", "Xorg_libXau_jll", "Xorg_libXdmcp_jll", "Xorg_libpthread_stubs_jll"]
git-tree-sha1 = "daf17f441228e7a3833846cd048892861cff16d6"
uuid = "c7cfdc94-dc32-55de-ac96-5a1b8d977c5b"
version = "1.13.0+3"

[[Xorg_libxkbfile_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libX11_jll"]
git-tree-sha1 = "926af861744212db0eb001d9e40b5d16292080b2"
uuid = "cc61e674-0454-545c-8b26-ed2c68acab7a"
version = "1.1.0+4"

[[Xorg_xcb_util_image_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "0fab0a40349ba1cba2c1da699243396ff8e94b97"
uuid = "12413925-8142-5f55-bb0e-6d7ca50bb09b"
version = "0.4.0+1"

[[Xorg_xcb_util_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxcb_jll"]
git-tree-sha1 = "e7fd7b2881fa2eaa72717420894d3938177862d1"
uuid = "2def613f-5ad1-5310-b15b-b15d46f528f5"
version = "0.4.0+1"

[[Xorg_xcb_util_keysyms_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "d1151e2c45a544f32441a567d1690e701ec89b00"
uuid = "975044d2-76e6-5fbe-bf08-97ce7c6574c7"
version = "0.4.0+1"

[[Xorg_xcb_util_renderutil_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "dfd7a8f38d4613b6a575253b3174dd991ca6183e"
uuid = "0d47668e-0667-5a69-a72c-f761630bfb7e"
version = "0.3.9+1"

[[Xorg_xcb_util_wm_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xcb_util_jll"]
git-tree-sha1 = "e78d10aab01a4a154142c5006ed44fd9e8e31b67"
uuid = "c22f9ab0-d5fe-5066-847c-f4bb1cd4e361"
version = "0.4.1+1"

[[Xorg_xkbcomp_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_libxkbfile_jll"]
git-tree-sha1 = "4bcbf660f6c2e714f87e960a171b119d06ee163b"
uuid = "35661453-b289-5fab-8a00-3d9160c6a3a4"
version = "1.4.2+4"

[[Xorg_xkeyboard_config_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Xorg_xkbcomp_jll"]
git-tree-sha1 = "5c8424f8a67c3f2209646d4425f3d415fee5931d"
uuid = "33bec58e-1273-512f-9401-5d533626f822"
version = "2.27.0+4"

[[Xorg_xtrans_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "79c31e7844f6ecf779705fbc12146eb190b7d845"
uuid = "c5fb5394-a638-5e4d-96e5-b29de1b5cf10"
version = "1.4.0+3"

[[Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[Zstd_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "cc4bf3fdde8b7e3e9fa0351bdeedba1cf3b7f6e6"
uuid = "3161d3a3-bdf6-5164-811a-617609db77b4"
version = "1.5.0+0"

[[libass_jll]]
deps = ["Artifacts", "Bzip2_jll", "FreeType2_jll", "FriBidi_jll", "HarfBuzz_jll", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "5982a94fcba20f02f42ace44b9894ee2b140fe47"
uuid = "0ac62f75-1d6f-5e53-bd7c-93b484bb37c0"
version = "0.15.1+0"

[[libfdk_aac_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "daacc84a041563f965be61859a36e17c4e4fcd55"
uuid = "f638f0a6-7fb0-5443-88ba-1cc74229b280"
version = "2.0.2+0"

[[libpng_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Zlib_jll"]
git-tree-sha1 = "94d180a6d2b5e55e447e2d27a29ed04fe79eb30c"
uuid = "b53b4c65-9356-5827-b1ea-8c7a1a84506f"
version = "1.6.38+0"

[[libvorbis_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Ogg_jll", "Pkg"]
git-tree-sha1 = "c45f4e40e7aafe9d086379e5578947ec8b95a8fb"
uuid = "f27f6e37-5d2b-51aa-960f-b287f2bc3b7a"
version = "1.3.7+0"

[[nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

[[x264_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "4fea590b89e6ec504593146bf8b988b2c00922b2"
uuid = "1270edf5-f2f9-52d2-97e9-ab00b5d0237a"
version = "2021.5.5+0"

[[x265_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"]
git-tree-sha1 = "ee567a171cce03570d77ad3a43e90218e38937a9"
uuid = "dfaa095f-4041-5dcd-9319-2fabd8486b76"
version = "3.5.0+0"

[[xkbcommon_jll]]
deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg", "Wayland_jll", "Wayland_protocols_jll", "Xorg_libxcb_jll", "Xorg_xkeyboard_config_jll"]
git-tree-sha1 = "ece2350174195bb31de1a63bea3a41ae1aa593b6"
uuid = "d8fb68d0-12a3-5cfd-a85a-d49703b185fd"
version = "0.9.1+5"
"""

# ╔═╡ Cell order:
# ╟─dad114fe-fbfb-11ea-32dd-a302838af30f
# ╟─fdab0c70-fbfb-11ea-116d-9f1b613f9d0d
# ╟─0d88a212-fbfc-11ea-37ae-c5e0c6a894e8
# ╟─67071830-fbfc-11ea-3925-798d3d288500
# ╟─3bb281f0-fc02-11ea-34f0-fd00648c27bb
# ╠═bf54eece-fc02-11ea-3dbc-b37a5644eab1
# ╠═cd9cc940-fc02-11ea-2a40-19bc6f79a6d3
# ╠═e5488250-fc02-11ea-0bc1-4365461fb202
# ╟─08de4fb0-fc03-11ea-08fb-616f792f7c7f
# ╟─cfd86bf0-fc03-11ea-22ef-b99d3e3fbfce
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


using DifferentialEquations, Plots

#Solving the simple pendulum with a traditional ODE method
#==========================================================#
function pendulum(du, u, params, t)
#reference: http://www.pgccphy.net/ref/advmech.pdf page 6

    g = params[1] #gravitational acceleration
    m = params[2] #mass
    l = params[3] #length

    θ = u[1]
    ℒ = u[2]

    d_θ = ℒ/(m*l^2)
    d_ℒ = -m*g*l*sin(θ)

    du .= [d_θ, d_ℒ]
    return nothing
end

g = 9.81
m = 2.
l = 1.
params = [g,m,l]
u0 = [1.0,1.0]

prob = ODEProblem(pendulum, u0, (0., 100.), params)
sol1 = solve(prob, AutoVern7(Rodas5()), dt = .05)

#==========================================================#

#Solving the simple pendulum with the DiffEqPhysics.jl HamiltonianProblem()
#==========================================================#
function H(θ, ℒ, params)
    g = params[1] #gravitational acceleration
    m = params[2] #mass
    l = params[3] #length

    return ℒ^2/(2*m*l^2) + m*g*l*(1-cos(θ))
end

g = 9.81
m = 2.
l = 1.
params = [g,m,l]
θ₀ = 1.
ℒ₀ = 1.

prob = HamiltonianProblem(H, θ₀, ℒ₀, (0., 100.), p=params)
sol2 = solve(prob, SofSpa10(), dt = .05);
#==========================================================#

#Plotting
#==========================================================#
plot(sol1, vars=1, tspan=(0,20), label="ODE Method")
plot!(sol2.t, sol2[1,:], xlim=(0,20), label="Hamiltonian Method")
#They produce the same solution!


using JuMP

using DataStructures
using Parameters
using Printf
using SparseArrays, LinearAlgebra

import Xpress

root_path = dirname(@__DIR__)
push!(LOAD_PATH, root_path);
cd(root_path)
include(joinpath(root_path, "src", "PTDF.jl"));

## programme d'optimisation pour calculer theta
function compute_theta(B,P)
    nb_buses = length(P);
    model = Model(Xpress.Optimizer);
    set_optimizer_attribute(model, "OUTPUTLOG", 0);

    @variable(model, theta[1:nb_buses]);
    @objective(model,Min,0);

    for i in range(1,nb_buses)
        @constraint(model,sum(B[i,j] * theta[j] for j in range(1,nb_buses)) == P[i]);
    end

    # print(model);

    optimize!(model);

    return [value(theta[i]) for i in range(1,nb_buses)];
end;

function compute_ptdf_by_optim(network::PTDF.Network,ref_bus_num::Int, EPS_DIAG::Float64)
    nb_buses = length(network.buses);
    nb_branches = length(network.branches);

    # calcul de B
    B=PTDF.get_B(network,EPS_DIAG,false);

    # calcul de la colonne de la PTDF pour chaque noeud
    PTDF_matrix = zeros(nb_branches,nb_buses);
    # println(network.buses)
    for bus in network.buses
        bus_num = bus[1]
        # println("$bus, id is $bus_num")
        #définition de la distribution des injections
        P = zeros(nb_buses);
        if bus_num != ref_bus_num
            P[bus_num]=1;
            P[ref_bus_num]+=-1;
        end

        #calcul de theta tel que B*theta = P
        theta = compute_theta(B,P);

        #calcul de la PTDF
        for branch in collect(network.branches)
            branch_num = branch[1];
            x_inv = PTDF.get_b(branch[2]);

            ior = branch[2].from;
            iex = branch[2].to;

            PTDF_matrix[branch_num,bus_num] = x_inv*(theta[ior]-theta[iex]);
        end
    end
    return PTDF_matrix;
end;

function compare_ptdfs(network::PTDF.Network,ref_bus_num::Int, EPS_DIAG::Float64)

    ptdf_optim = compute_ptdf_by_optim(network,ref_bus_num, EPS_DIAG);
    ptdf_with_inv = PTDF.compute_ptdf(network,ref_bus_num, EPS_DIAG);

    return ptdf_optim-ptdf_with_inv;
end;

##### Example
EPS_DIAG = 0e-6;
ref_bus_num = 1;
data_path_sparse = joinpath(@__DIR__, "..", "data_matpower", "case1354pegase");
data_path = data_path_sparse;

network = PTDF.read_network(data_path);

# ptdf_optim = compute_ptdf_by_optim(network,ref_bus_num);
ptdf_difference = norm(compare_ptdfs(network, ref_bus_num, EPS_DIAG));
println("error_max = $ptdf_difference");


using Plots, Statistics

plotly()

NUM = 1000

function is_single(kindness::Float64, looks::Float64, LIM::Float64=9.0)::Bool
    d = √(kindness^2 + looks^2)
    if d > LIM
        single = false
    else
        single = true
    end
    return single
end

function assign_color(single::Bool)::String
    if single
        output = "red"
    else
        output = "blue"
    end
    return output
end

kindness = rand(1:.1:10, NUM);
looks = rand(1:.1:10, NUM);
singles = is_single.(kindness, looks);

p = scatter(looks, kindness, color=assign_color.(singles), legend = false)
xlabel!("Looks")
ylabel!("Kindness")

println("Correlation of kindness vs looks for $NUM people is: $(cor(kindness, looks))")
println("Correlation of kindness vs looks conditioned on single status $(sum(singles)) is $(cor(kindness[singles], looks[singles]))")


module NewtonsMethod

using LinearAlgebra, Statistics, Compat, ForwardDiff

function newtonroot(f, f′, x₀; tol=1E-7, maxiter=1000)
    x_old = x₀
    normdiff = Inf
    iter = 1
    while normdiff > tol && iter <= maxiter
        x_new = x_old - f(x_old)/f′(x_old)
        normdiff = norm(x_new - x_old)
        x_old = x_new
        iter += 1
    end
    normdiff > tol ? (nothing, normdiff, iter) : (x_old, normdiff, iter)
end

function newtonroot(f, x₀; tol=1E-7, maxiter=1000)
    Diff(f) = x -> ForwardDiff.derivative(f, x)
    f′ = Diff(f)
    newtonroot(f, f′, x₀, tol=tol, maxiter=maxiter)
end

export newtonroot

end


using ModelingToolkit
using Plots
using DifferentialEquations
using LinearAlgebra

@parameters t σ ρ β  # add metadata to variables, i.e. this is a parameter!
@variables x(t) y(t) z(t)  # time-dependent variables
D = Differential(t)

D(x)  # dx/dt

eqs = [ D(D(x)) ~ σ * (y-x),
        D(y) ~ x*(ρ-z) - y,
        D(z) ~ x*y - β*z]

# generate an ode system from the equationsj
sys = ODESystem(eqs)

# what are the equations?
equations(sys)

# but ODE solver want's things in first order form...
# So just lower the order!
sys = ode_order_lowering(sys)
equations(sys)
# now we have 4 - first order ODEs

# define the initial conditions
u0 = [D(x) => 2.0,
      x => 1.0,
      y => 0.0,
      z => 0.0]

# define parameter values
p = [σ => 28.0,
     ρ => 10.0,
     β => 8/3]

# build the ODE problem, i.e. translate from Symbolic to Numeric
tspan = (0.0, 100.0)

prob = ODEProblem(sys, u0, tspan, p, jac=true, sparse=true)  # build analytic jacobian function and use sparse arrays

sol = solve(prob, Tsit5())

states(sys)  # what is the ordering of the solution vector

# HYPE: we can use variables to index solution object!!!!!
sol[x]  # get time series for x
sol[D(x)]

# what if you want to interpolate
sol(50.0, idxs=y)  # i.e. what is y(t) for t=50.0

plot(sol, vars=(x, y))

# ----- Lorenz equations --------------

eqs = [D(x) ~ σ*(y-x),
       D(y) ~ x*(ρ-z) - y,
       D(z) ~ x*y - β*z]

# same symbolic equations
@named lorenz1 = ODESystem(eqs)
@named lorenz2 = ODESystem(eqs)

# define a mixture between the two

@variables a(t)
@parameters γ

# define the connection by an algebraic equation
connections = [0 ~ lorenz1.x + lorenz2.y + a*γ]

# indep var, array of state variables, array of parameters and the systems of odes these relate
connected = ODESystem(connections, t, [a], [γ], systems=[lorenz1, lorenz2])
equations(connected)

# define initial conditions, parameters, timespan
u0 = [lorenz1.x => 1.0, lorenz1.y => 0.0, lorenz1.z => 0.0, lorenz2.x => 0.0, lorenz2.y => 1.0, lorenz2.z => 0.0, a=> 2.0]
p = [lorenz1.σ => 10.0, lorenz1.ρ => 28.0, lorenz1.β => 8/2, lorenz2.σ => 10.0, lorenz2.ρ => 28.0, lorenz2.β => 8/3, γ => 2.0]
tspan = (0.0, 100.0)

prob = ODEProblem(connected, u0, tspan, p)

sol = solve(prob)

plot(sol, vars=(a, lorenz1.x, lorenz2.z))

# --------------- Using MTK without using MTK ------------------------
# naive version that allocates array instead of using inplace f(du, u, p, t)
function rober(u, p, t)
    y₁, y₂, y₃ = u
    k₁, k₂, k₃ = p
    [-k₁*y₁+k₃*y₂*y₃, k₁*y₁-k₂*y₂^2-k₃*y₂*y₃, k₂*y₂^2]
end

prob = ODEProblem(rober, [1.0, 0.0, 0.0], (0.0, 1e5), (0.04, 3e7, 1e4))
# turn it into an internal MTK problem defined symbolically
sys = modelingtoolkitize(prob)
equations(sys)

# redefine the problem to use the symbolic jacobian
prob_jac = ODEProblem(sys, [1.0, 0.0, 0.0], (0.0, 1e5), (0.04, 3e7, 1e4), jac=true)

using BenchmarkTools

@btime sol = solve(prob)
@btime sol = solve(prob_jac)

#----------- problems with Algebraic Constraints-------------------
function pendulum!(du, u, p, t)
    x, dx, y, dy, T = u
    g, L = p
    du[1] = dx
    du[2] = T*x
    du[3] = dy
    du[4] = T*y -g
    du[5] = x^2 + y^2 - L^2 # the length
    return nothing
end

# Mu' = f
pendulum_fun! = ODEFunction(pendulum!, mass_matrix=Diagonal([1,1,1,1,0]))
u0 = [1.0, 0, 0, 0, 0]
p = [9.8, 1]
tspan = (0, 10.0)

pendulum_prob = ODEProblem(pendulum_fun!, u0, tspan, p)

# this doesn't work for technical reasons about DAE's. We need to refactor it into a nicer form
sol = solve(pendulum_prob, Rodas4(), abstol=1e-8, reltol=1e-8)

# convert to symbolic representation
traced_sys = modelingtoolkitize(pendulum_prob)
equations(traced_sys)
# lower dae index and simplify
pendulum_sys = structural_simplify(dae_index_lowering(traced_sys)) # this is just being explicit, we could have just done structural simplify
equations(pendulum_sys)

prob = ODAEProblem(pendulum_sys, Pair[], tspan)
sol = solve(prob, Tsit5(), abstol=1e-8, reltol=1e-8)
plot(sol, vars=states(traced_sys))


## Let's see how finite difference approximations work.

using Printf
myfun = x -> sin(x)
myderiv = x -> cos(x) # so we can get error
finitediff(f,x,h) = (f(x+h) - f(x))/h
x = pi/6
truederiv = myderiv(x)
for h = 10.0.^-(1:16)
  fdderiv = finitediff(myfun,x,h)
  @printf("h=%.1e approx=%.8e err=%.8e\n",
    h, fdderiv, abs(fdderiv-truederiv))
end

## Now try a multidim version
function finitediff_i(f,x,h,i)
  fx = f(x)
  xi = x[i] # make a copy
  x[i] += h
  fxh = f(x) # eval f(x+h)
  x[i] = xi # restore
  return (fxh-fx)/h
end
finitediffgrad(f,x,h) = map( i -> finitediff_i(f,x,h,i), 1:length(x) )
## validate our work on gradient of quadratics
using LinearAlgebra
Q = [1 0.5;0.5 1]
s = [-1,-1.5]
quadratic2(A,b) = x -> x'*A*x + x'*b
myfun = quadratic2(Q,s)
myderiv = x -> 2*Q*x + s
x = [3.5;-1.5]
truederiv = myderiv(x)
for h = 10.0.^-(1:16)
  fdderiv = finitediffgrad(myfun,x,h)
  @printf("h=%.1e  err=%.8e\n",
    h, norm(fdderiv-truederiv))
end


p = [ randn(2) for i = 1:10 ] # 10 random points - note array-of-array

# Plot the points as dots
# The functions first and last simply collects the corresponding indices
using PyPlot
plot(first.(p), last.(p), ".")
axis("equal"); grid(true);

function clockwise_oriented(p1, p2, p3)
    # Return true if the line-segment between points p1,p2 is clockwise
    # oriented to the line-segment between points p1,p3
    cross = (p3[2] - p1[2]) * (p2[1] - p1[1]) - (p3[1] - p1[1]) * (p2[2] - p1[2])
    return cross > 0
end

# Tests
println(clockwise_oriented([0,0], [1,1], [2,3]))
println(clockwise_oriented([0,0], [1,1], [3,2]))

function convex_hull(p)
    # Find the nodes on the convex hull of the point array p using
    # the Jarvis march (gift wrapping) algorithm

    _, pointOnHull = findmin(first.(p)) # Start at left-most point
    hull = [pointOnHull] # Output: Vector of node indices on the convex hull

    while length(hull) ≤ 1 || hull[1] != hull[end] # Loop until closed polygon
        nextPoint = hull[end] % length(p) + 1 # First candidate, any point except current
        for j = 1:length(p) # Consider all other points
            if clockwise_oriented(p[hull[end]], p[nextPoint], p[j]) # If "more to the left", switch
                nextPoint = j
            end
        end
        push!(hull, nextPoint) # Update current point
    end
    return hull
end

# Example: 100 random points, compute and draw the convex hull
p = [ randn(2) for i = 1:100 ]
hull = convex_hull(p)
plot(first.(p), last.(p), ".")
plot(first.(p[hull]), last.(p[hull]))
axis("equal"); grid(true);

using LinearAlgebra
function linesegment_intersect(p1, p2, q1, q2, δ)
    A = [p2-p1 q1-q2]
    b = q1-p1
    if rank(A) == 2
        st = A \ b
        if all(-δ .≤ st .≤ 1 + δ)
            pintersect = p1 + st[1] * (p2 - p1)
            return true, pintersect
        else
            return false, nothing
        end
    else
        # Warning: parallel lines, should check if they actually overlap
        return false, nothing
    end
end

# Example: n random lines, find and plot all intersections
n = 10
lines = [ [rand(2), rand(2)] for i = 1:n ]

# Plot all lines
for line in lines
    plot([line[1][1], line[2][1]], [line[1][2], line[2][2]])
end

# Find all intersections - note that we only consider j>i to avoid checking
# the same 2 line-segments twice
for i = 1:n
    for j = i+1:n
        isect, pq = linesegment_intersect(lines[i][1], lines[i][2], lines[j][1], lines[j][2], 0)
        if isect
            plot(pq[1], pq[2], "o")
        end
    end
end

axis("equal"); grid(true);

function tplot(p, t)
    # Plot triangular mesh with nodes `p` and triangles `t`
    tris = convert(Array{Int64}, hcat(t...)')
    tripcolor(first.(p), last.(p), tris .- 1, 0*tris[:,1],
              cmap="Set3", edgecolors="k", linewidth=1)
    axis("equal")
    return
end

p = [[0,0], [1,0], [0,1], [1,2], [2,1]]
t = [[1,2,5], [1,5,3], [3,5,4]]
tplot(p,t)

using Triangle

function delaunay(p)
    # Delaunay triangulation `t` of array of nodes `p`
    vertex_map = Array{Int64,1}(1:size(p,1))
    t = Triangle.basic_triangulation(convert(Array{Float64}, hcat(p...)'), vertex_map)
end

t = delaunay(p)
tplot(p,t)

p = [ randn(2) for i = 1:50 ]
t = delaunay(p)
tplot(p,t)


# Part of submodule Utils of BetaML
# Function of a single argument (including scalars and vectors), like activation functions but also gini, entropy,...)

# ------------------------------------------------------------------------------
# Various neural network activation functions as well their derivatives

#identity(x)          = x already in Julia base
didentity(x)          = one(x)
""" relu(x) \n\n Rectified Linear Unit \n\n https://www.cs.toronto.edu/~hinton/absps/reluICML.pdf"""
relu(x)               = max(zero(x), x)
""" drelu(x) \n\n Rectified Linear Unit \n\n https://www.cs.toronto.edu/~hinton/absps/reluICML.pdf"""
drelu(x)              = x <= zero(x) ? zero(x) : one(x)
"""elu(x; α=1) with α > 0 \n\n https://arxiv.org/pdf/1511.07289.pdf"""
elu(x; α=one(x))      = x > zero(x) ? x : α *(exp(x) - one(x))
"""delu(x; α=1) with α > 0 \n\n https://arxiv.org/pdf/1511.07289.pdf"""
delu(x; α=one(x))      = x > zero(x) ? one(x) : elu(x, α=α) + α
"""celu(x; α=1) \n\n https://arxiv.org/pdf/1704.07483.pdf"""
celu(x; α=one(x))     = max(zero(x),x)+ min(zero(x), α *(exp(x / α) - one(x) ))
#celu(x; α=one(x))    = if x >= zero(x) x/α else exp(x/α)-one(x) end
"""dcelu(x; α=1) \n\n https://arxiv.org/pdf/1704.07483.pdf"""
dcelu(x; α=one(x))    = x >= zero(x) ? one(x) : exp(x/α)
"""plu(x;α=0.1,c=1) \n\n Piecewise Linear Unit \n\n https://arxiv.org/pdf/1809.09534.pdf"""
plu(x;α=0.1,c=one(x)) = max(α*(x+c)-c,min(α*(x-c)+c,x)) # convert(eltype(x), α)
"""dplu(x;α=0.1,c=1) \n\n Piecewise Linear Unit derivative \n\n https://arxiv.org/pdf/1809.09534.pdf"""
dplu(x;α=0.1,c=one(x)) = ( ( x >= (α*(x+c)-c)  &&  x <= (α*(x+c)+c) ) ? one(x) : α ) # convert(eltype(x), α)

"""
    pool1d(x,poolSize=2;f=mean)

Apply funtion `f` to a rolling poolSize contiguous (in 1d) neurons.

Applicable to `VectorFunctionLayer`, e.g. `layer2  = VectorFunctionLayer(nₗ,f=(x->pool1d(x,4,f=mean))`
**Attention**: to apply this funciton as activation function in a neural network you will need Julia version >= 1.6, otherwise you may experience a segmentation fault (see [this bug report](https://github.com/FluxML/Zygote.jl/issues/943))
"""
pool1d(x,poolSize=3;f=mean) = [f(x[i:i+poolSize-1]) for i in 1:length(x)-poolSize+1] # we may try to use CartesianIndices/LinearIndices for a n-dimensional generalisation

#tanh(x) already in Julia base
"""dtanh(x)"""
dtanh(x)              = sech(x)^2  # = 1-tanh(x)^2
"""sigmoid(x)"""
sigmoid(x)            = one(x)/(one(x)+exp(-x))
"""dsigmoid(x)"""
dsigmoid(x)           = exp(-x)*sigmoid(x)^2
"""softmax (x; β=1) \n\n The input x is a vector. Return a PMF"""
softmax(x; β=one.(x)) = exp.((β .* x) .- lse(β .* x))  # efficient implementation of softmax(x)  = exp.(x) ./  sum(exp.(x))
softmax(x, β) = softmax(x, β=β)
""" dsoftmax(x; β=1) \n\n Derivative of the softmax function \n\n https://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/"""
function dsoftmax(x; β=one(x[1]))
    x = makeColVector(x)
    d = length(x)
    out = zeros(d,d)
    y = softmax(x,β=β)
    for i in 1:d
        smi = y[i]
        for j in 1:d
            if j == i
                out[i,j] = β*(smi-smi^2)
            else
                out[i,j] = - β*y[j]*smi
            end
        end
    end
    return out
end

"""softplus(x) \n\n https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Softplus"""
softplus(x)           = log(one(x) + exp(x))
"""dsoftplus(x) \n\n https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Softplus"""
dsoftplus(x)          = 1/(1+exp(-x))
""" mish(x) \n\n https://arxiv.org/pdf/1908.08681v1.pdf"""
mish(x)               = x*tanh(softplus(x))
""" dmish(x) \n\n https://arxiv.org/pdf/1908.08681v1.pdf"""
dmish(x) = x*(1 - tanh(log(exp(x) + 1))^2)*exp(x)/(exp(x) + 1) + tanh(log(exp(x) + 1))

"""
   autoJacobian(f,x;nY)

Evaluate the Jacobian using AD in the form of a (nY,nX) matrix of first derivatives

# Parameters:
- `f`: The function to compute the Jacobian
- `x`: The input to the function where the jacobian has to be computed
- `nY`: The number of outputs of the function `f` [def: `length(f(x))`]

# Return values:
- An `Array{Float64,2}` of the locally evaluated Jacobian

# Notes:
- The `nY` parameter is optional. If provided it avoids having to compute `f(x)`
"""
function autoJacobian(f,x;nY=length(f(x)))
    x = convert(Array{Float64,1},x)
    #j = Array{Float64, 2}(undef, size(x,1), nY)
    #for i in 1:nY
    #    j[:, i] .= gradient(x -> f(x)[i], x)[1]
    #end
    #return j'
    j = Array{Float64, 2}(undef, nY, size(x,1))
    for i in 1:nY
        j[i,:] = gradient(x -> f(x)[i], x)[1]'
    end
    return j
end

# ------------------------------------------------------------------------------
# Partition tasks..

"""
   gini(x)

Calculate the Gini Impurity for a list of items (or rows).

See: https://en.wikipedia.org/wiki/Decision_tree_learning#Information_gain
"""
function gini(x)

    counts = classCounts(x)
    N = size(x,1)
    impurity = 1.0
    for c in counts
        probₖ     = c / N
        impurity -= probₖ^2
    end
    return impurity
  #=
  counts = classCountsWithLabels(x)
  N = size(x,1)
  impurity = 1.0
  for k in keys(counts)
    probₖ = counts[k] / N
    impurity -= probₖ^2
  end
  return impurity
  =#
end

"""
   entropy(x)

Calculate the entropy for a list of items (or rows).

See: https://en.wikipedia.org/wiki/Decision_tree_learning#Gini_impurity
"""
function entropy(x)
    counts = classCounts(x)
    N = size(x,1)
    entr = 0.0
    for c in counts
        probₖ = c / N
        entr -= probₖ * log2(probₖ)
    end
    return entr
end

"""variance(x) - population variance"""
variance(x) = var(x,corrected=false)

# ------------------------------------------------------------------------------
# ------------------------------------------------------------------------------

"""bic(lL,k,n) -  Bayesian information criterion (lower is better)"""
bic(lL,k,n) = k*log(n)-2*lL
"""aic(lL,k) -  Akaike information criterion (lower is better)"""
aic(lL,k)   = 2*k-2*lL

# ------------------------------------------------------------------------------
# Various kernel functions (e.g. for Perceptron)
"""Radial Kernel (aka _RBF kernel_) parametrised with γ=1/2. For other gammas γᵢ use
`K = (x,y) -> radialKernel(x,y,γ=γᵢ)` as kernel function in the supporting algorithms"""
radialKernel(x,y;γ=1/2) = exp(-γ*norm(x-y)^2)
"""Polynomial kernel parametrised with `c=0` and `d=2` (i.e. a quadratic kernel).
For other `cᵢ` and `dᵢ` use `K = (x,y) -> polynomialKernel(x,y,c=cᵢ,d=dᵢ)` as
kernel function in the supporting algorithms"""
polynomialKernel(x,y;c=0,d=2) = (dot(x,y)+c)^d


using GLM, Distributions, DataFrames

data1 = parse.(Float64, readlines("../data/machine1.csv"))
data2 = parse.(Float64, readlines("../data/machine2.csv"))
data3 = parse.(Float64, readlines("../data/machine3.csv"))

function manualANOVA(allData)
    nArray = length.(allData)
    d = length(nArray)

    xBarTotal = mean(vcat(allData...))
    xBarArray = mean.(allData)

    ssBetween = sum( [nArray[i]*(xBarArray[i] - xBarTotal)^2 for i in 1:d] )
    ssWithin = sum([sum([(ob - xBarArray[i])^2 for ob in allData[i]])
				for i in 1:d])
    dfBetween = d-1
    dfError = sum(nArray)-d

    msBetween = ssBetween/dfBetween
    msError = ssWithin/dfError
    fStat = msBetween/msError
    pval = ccdf(FDist(dfBetween,dfError),fStat)
    return (fStat,pval)
end

function glmANOVA(allData)
    nArray = length.(allData)
    d = length(nArray)

    treatment = vcat([fill(k,nArray[k]) for k in 1:d]...)
    response = vcat(allData...)
    dataFrame = DataFrame(Response=response, Treatment=categorical(treatment))
    modelH0  = lm(@formula(Response ~ 1), dataFrame)
    modelH1a = lm(@formula(Response ~ 1 + Treatment), dataFrame)
    res = ftest(modelH1a.model, modelH0.model)
    (res.fstat[1],res.pval[1])
end

println("Manual ANOVA: ", manualANOVA([data1, data2, data3]))
println("GLM ANOVA: ", glmANOVA([data1, data2, data3]))


#----------------------------#
# see page 57 in Sarkka book #
#----------------------------#

function filteringrecursion(y; A = A, H = H, Q = Q, R = R, m₀ = m₀, P₀ = P₀)

    N = length(y)

    # Store here state distribution

    μstate, Σstate = Array{Vector{Float64}}(undef, N), Array{Matrix{Float64}}(undef, N)

    # Start recursion with prior mean and prior covariance

    mₖ₋₁, Pₖ₋₁ = m₀, P₀

    for k in 1:N

        # prediction step

        mₖ⁻ = A * mₖ₋₁

        Pₖ⁻ = A * Pₖ₋₁ * A' + Q     # (4.20)

        # update step

        𝐯ₖ = y[k] - H * mₖ⁻

        Sₖ = H * Pₖ⁻ * H' + R

        Kₖ = (Pₖ⁻ * H') / Sₖ

        mₖ = mₖ⁻ + Kₖ * 𝐯ₖ

        Pₖ = Pₖ⁻ - Kₖ * Sₖ * (Kₖ)'  # (4.21)

        # store mean and covariance of Gaussian distribution N(xₖ|mₖ, Pₖ)

        Pₖ = (Pₖ+(Pₖ)') / 2

        μstate[k], Σstate[k] = mₖ, Pₖ

        # next state

        mₖ₋₁, Pₖ₋₁ = mₖ, Pₖ

    end

    return μstate, Σstate

end

function testfiltering() # Works ✅

    yclean, y, x, A, H, Q, R, m₀, P₀ = simulatedata()

    μ, Σ = filteringrecursion(y; A = A, H = H, Q = Q, R = R, m₀ = m₀, P₀ = P₀)

    figure()

    subplot(211)
    plot([yᵢ[1]    for yᵢ in yclean], label = "simulated 1 clean")
    plot([yᵢ[1]    for yᵢ in y], label = "simulated 1")
    plot([(H*m)[1] for m  in μ], label = "mean 1")
    legend()

    subplot(212)
    plot([yᵢ[2]    for yᵢ in y], label = "simulated 2")
    plot([(H*m)[2] for m  in μ], label = "mean 2")

    legend()

end


# Julia code to compute the correlation coefficient
using Distributions
using Plots

x = rand(MvNormal([0, 0], [3 1; 1 1]), 1000)
# x = rand(MvNormal([0, 0], [3 0; 0 3]), 1000)
# x = rand(MvNormal([0, 0], [3 2.9; 2.9 3]), 1000)
scatter(x[1, :], x[2, :])

σ₁ = std(x[1, :])
σ₂ = std(x[2, :])
μ₁ = mean(x[1, :])
μ₂ = mean(x[2, :])
Exy = mean(x[1, :] .* x[2, :])
ρ = (Exy - μ₁ * μ₂) / (σ₁ * σ₂)

####################################################################

# Julia code to compute a mean vector.
using Statistics

X = randn(100, 2)
mean(X, dims=1)

####################################################################

# Julia code to compute covariance matrix.
using Statistics

X = randn(100, 2)
cov(X)

####################################################################

# Julia code: Overlay random numbers with the Gaussian contour.
using Distributions
using Plots

p = MvNormal([0, 0], [0.25 0.3; 0.3 1])

X = rand(p, 1000)
x₁ = -2.5:0.01:2.5
x₂ = -3.5:0.01:3.5

f(x₁, x₂) = pdf(p, [x₁, x₂])

scatter(X[1, :], X[2, :], legend=false)
contour!(x₁, x₂, f, linewidth=2)

####################################################################

# Julia code: Gaussian(0,1) --> Gaussian(mu,sigma)
using Distributions

x = rand(MvNormal([0, 0], [1 0; 0 1]), 1000)
Σ = [3 -0.5; -0.5 1]
μ = [1, -2]
y = Σ^(1/2) * x .+ μ

####################################################################

# Julia code: Gaussian(mu,sigma) --> Gaussian(0,1)
using Distributions

y = rand(MvNormal([1, -2], [3 -0.5; -0.5 1]), 100)
μ = mean(y, dims=2)
Σ = cov(y')
x = Σ^(-1/2) * (y .- μ)

####################################################################

# Julia code to perform the principal component analysis
using LinearAlgebra
using Distributions

x = rand(MvNormal([0, 0], [2 -1.9; -1.9 2]), 1000)
Σ = cov(x')
S, U = eigen(Σ)
U[:, 1]
U[:, 2]


_primes = [2, 3]

function is_prime(n, known_primes)
    for prime in known_primes
        if n == prime || prime > n
            return true
        end
        if n % prime == 0
            return false
        end
    end
    return true
end

function next_prime(n)
    inx = findfirst(x -> x > n, _primes)
    if inx == 0
        # Compute prime
        x = n + 1
        while !is_prime(x, _primes)
            x+=1
        end
        push!(_primes, x)
        return x
    end
    return _primes[inx]
end

function find_highest_prime_factor(n)

    current_prime = 2
    while (n % current_prime) != 0
        current_prime = next_prime(current_prime)
    end

    if is_prime(n, _primes)
        return n
    end
    return find_highest_prime_factor(n / current_prime)
end
@assert is_prime(2, _primes) == true
@assert is_prime(6, _primes) == false
@assert next_prime(2) == 3
@assert next_prime(3) == 5
@assert find_highest_prime_factor(6) == 3
@assert find_highest_prime_factor(11) == 11
@assert find_highest_prime_factor(15) == 5
@assert find_highest_prime_factor(21) == 7
@assert find_highest_prime_factor(26) == 13
println(find_highest_prime_factor(13195))
@assert find_highest_prime_factor(13195) == 29
println(find_highest_prime_factor(600851475143))


using Distributions, LinearAlgebra

function rowenhorst(mean, uncond_sd, rho, num_states)
    """
    Rowenhort's method to approximate AR(1) process with Markov chain (ln y_t = mu + rho y_{t-1} + e_t)
    Note: this function also normalizes effective labour to one.

    #### Fields

    - 'mean': unconditional mean of income process
    - 'uncond_sd': unconditional standard deviation
    - 'rho': autocorrelation coefficient
    - 'num_states': number of states we want discretized

    #### Returns

    - 'transition_matrix': num_states x num_states array where
    transition_matrx[i,j] is prob. of going from i to j
    - 'ygrid': vector of income state space of length num_states

    """

    # construct grids
    step_r = uncond_sd*sqrt(num_states-1)
    ygrid = -1:2/(num_states-1):1
    ygrid = mean .+ step_r*ygrid

    # initialize transition probabilities
    p = (rho+1)/2
    q = p

    transition_matrix = [p 1-p; 1-q q]

    # rowenhort's method
    for i = 2:num_states-1
        a1 = [transition_matrix zeros(i, 1); zeros(1, i+1)]
        a2 = [zeros(i,1) transition_matrix; zeros(1, i+1)]
        a3 = [zeros(1, i+1); transition_matrix zeros(i,1)]
        a4 = [zeros(1, i+1); zeros(i,1) transition_matrix]

        transition_matrix = p*a1 + (1-p)*a2 + (1-q)*a3 + q*a4
        transition_matrix[2:i, :] = transition_matrix[2:i, :]/2
    end

    for i = 1:num_states
       transition_matrix[i,:] = transition_matrix[i,:]/sum(transition_matrix[i,:])
    end

    # get stationary distribution to normalize effective labour to L=1
    pi = eigvecs(transition_matrix')[:,num_states]

    # normalize pi
    pi = pi./sum(pi)

    # exponentiate
    ygrid = exp.(ygrid)

    # normalize effective labour
    ygrid = ygrid/sum(pi.*ygrid)

    return transition_matrix, ygrid
end

function tauchen(mean, sd, rho, num_states; q=3)

    """
    Tauchen's method to approximate AR(1) process with Markov Chain

    ##### Fields

    - 'num_states': Number of points in markov process
    - 'sd' : Standard deviation of innovation
    - 'rho' : Autocorrelation coefficient
    - 'mean': Unconditional mean
    -  'q' : The number of standard deviations to each side of the process

    ##### Returns

    - 'transition_matrix': num_states x num_states array where
    transition_matrx[i,j] is prob. of going from i to j
    - 'ygrid': vector of income state space of length num_states

    """

      uncond_sd = sd/sqrt(1-rho^2)
      y = range(-q*uncond_sd, stop = q*uncond_sd, length = num_states)
      d = y[2]-y[1]

      Pi = zeros(num_states,num_states)

      for row = 1:num_states
        # end points
            Pi[row,1] = cdf(Normal(),(y[1] - rho*y[row] + d/2)/sd)
            Pi[row,num_states] = 1 - cdf(Normal(), (y[num_states] - rho*y[row] - d/2)/sd)

        # middle columns
            for col = 2:num_states-1
                Pi[row, col] = (cdf(Normal(),(y[col] - rho*y[row] + d/2) / sd) -
                               cdf(Normal(),(y[col] - rho*y[row] - d/2) / sd))
            end
      end

    yy = y .+ mean # center process around its mean

    Pi = Pi./sum(Pi, dims = 2) # renormalize

    return Pi, yy
end


"""
`cmean(x)`

Compute circular mean of an array.

**Arguments**
- `x`: Array of angles in radians.

**Returns**
- `m`: Circular mean of `x`.
"""
function cmean(x)
    S = mean(sin.(x))
    C = mean(cos.(x))
    m = atan(S, C)
    return m
end

"""
`cstd(x)`

Compute circular standard deviation of an array.

**Arguments**
- `x`: Array of angles in radians.

**Returns**
- `sd`: Circular standard deviation of `x`.
"""
function cstd(x)
    S = mean(sin.(x))
    C = mean(cos.(x))
    R = sqrt(S^2 + C^2)
    x = -2*log(R)
    if x < 0
        @warn "Negative value $x set to 0!"
        x = 0
    end
    sd = sqrt(x)
    return sd
end


using Distributions
using Statistics
using CSV
using StringEncodings
using Printf

#MathKurtosis - kurtosis (from Distributions)
#MathSkewness  - skewness (from Distributions)
#Normal Distribution =
# MathCumulativeDistributionNormal  - cdf (nd,x)

"""
  returns a series of returns given a serie of prices
"""
function calcReturnsFromPrice(prices::Array{Float64})
   ret=1;
   s=size(prices)[1]
   rreturns=Array{Float64,1}()
   for  i=1:(s-1)
      push!(rreturns,prices[i+1]/prices[i]-1)
   end
   return rreturns;
end

"""
  returns a series of prices  given a serie of returns
"""
function calcPricesFromReturns(rreturns::Array{Float64})
   ret=1;
   s=size(rreturns)[1]
   equities=Array{Float64,1}()
   push!(equities,1000000)
   for  i=1:s
      push!(equities,(rreturns[i]+1)*equities[i])
   end
   return equities;
end

"""
returns the Total return of a series of returns given of the n first returns

"""
function calcTotalReturn(returns::Array{Float64})
   ret=1;
   s=size(returns)[1]
   for  i=1:s
      ret*=(1+returns[i])
   end
   return ret-1;
end

"""
returns the arithmetic average return of the series of returns given of the n first returns

"""
function calcAvgReturn(returns::Array{Float64})
   sum=Float64(0)
   s=size(returns)[1]
   for  i=1:s
      sum+=returns[i]
   end
   return sum/s;
end

function calcAnnualReturn(returns::Array{Float64}, numberOfDays)
    gReturn=calcTotalReturn(returns);
   return (1+gReturn)^(252.0/numberOfDays)-1;
end

function calcAnnualSR(returns::Array{Float64}, riskfree, numberOfDays)
   return sqrt(252)*calcSR(returns,riskfree)
end

"""
 returns the sample standard deviation
"""
function calcDesvPad(x::Array{Float64})
   avg=calcAvgReturn(x);
   n=size(x)[1];
   s=0;
    for i=1:n
        s+= (x[i]-avg)^2;
    end
  s=s/n;
  return sqrt(s);
end

function calcSR(returns::Array{Float64}, riskfree)
  avg=calcAvgReturn(returns);
   sigma=calcDesvPad(returns);

    if(sigma!=0)
       return (avg-riskfree)/sigma;
    end
    return -1;
end

"""
  returns Prob that SR is greater than Threshold
"""
function ProbSRgreaterThreshold(returns::Array{Float64},threshold)
   n=size(returns)[1];
   sre=calcSR(returns,0);
   return Z( (sre-threshold)*sqrt(n-1)/
              sqrt(1 - skewness(returns)*sre + (kurtosis(returns)-1)*sre*sre/4)
            );
end

"""
 returns the  mininum Track Record(minTRL) lenght (in years) to have returns above threshold wiht level of conficence (alpha)
"""
function minTRL(returns::Array{Float64},  threshold,  alpha)
   sre=calcSR(returns,0);
   minT= 1.0+ ( 1.0 -skewness(returns)*sre
         + (kurtosis(returns)-1)*sre*sre/4.0)*
         (Z(alpha)/(sre-threshold))^2.0 ;
   return minT/252
end

function Z( x)
   nd=Normal(0,1)
  return cdf(nd,x)
end

"""
  Print Strategy performance STSE report. It is calculated from a equity time series (equities parameter) observed in a given number of days (numberOfDays parameters).
     Equity series should be an array of float64, from oldest to newest equity value marked to market.
     You should provide at least 30 data points and the number of days can not be smaller than the number of points.
     You may define a desired threshold. The results for zero threshold is always presentes. The default value for threshold is 0.1.
     You may also define a risk free return rate for Sharpe ratio calculation.
"""

function analyzeEquitySeries(equities::Array{Float64},numberOfDays,threshold=0.1,risk_free_rate=0.0)
   n=size(equities)[1]
   if n<30 || numberOfDays>n
      println("ERROR!!")
      println("        You should provide at least 30 data points and the number of days can not be bigger than the number of points")
      println("         ",n," points where provided!!")
      return
   end

   rreturns=calcReturnsFromPrice(equities);
   rreturns=convert(Array{Float64},rreturns);
   rreturns=convert(Array{Float64},rreturns);

   @printf("calcTotalReturn() (%%)=%.2f\n", calcTotalReturn(rreturns)*100);
   @printf("calcAnnualReturn (%%)=%.2f\n",calcAnnualReturn(rreturns,numberOfDays)*100)
   @printf("calcDevPad=%.2f\n",calcDesvPad(rreturns))
   @printf("calcSR=%.2f\n",calcSR(rreturns,risk_free_rate))
   @printf("calcAnnualSR=%.2f\n",calcAnnualSR(rreturns,risk_free_rate,numberOfDays ))
   @printf("ProbSRgreaterThreshold(0)=%.3f\n",ProbSRgreaterThreshold(rreturns,0))
   @printf("minTRL(0)=%.1f\n",minTRL(rreturns,0,0.95))
   @printf("ProbSRgreaterThreshold(%.2f)=%.3f\n",threshold,ProbSRgreaterThreshold(rreturns,0.05))
   @printf("minTRL(%.2f)=%.1f\n",threshold,minTRL(rreturns,0.05,0.95))

end

"""
Print Strategy performance STSE report.

Parameters:
fileName
numberOfDays
delim='\t'

Given a equity file with the following format (UTF16 codification):

<DATE>	<BALANCE>	<EQUITY>	<DEPOSIT LOAD>
2019.10.01 00:00	100000.00	100000.00	0.0000
2019.10.03 17:59	99962.79	100009.43	0.0022

This kind of files are generated by Metatrader5

Note: The deposit load value shows the percent of account funds used to open positions. Load calculation formula:

Load = Margin / Equity * 100%
https://www.mql5.com/en/articles/2704#load

"""
function processEquityFileUtf16(fileName,numberOfDays,delim='\t')
   s=read(fileName,String,enc"UTF-16");
    print("delim=",delim)
    path = tempname();
    f = open(path, enc"UTF-8", "w");
    write(f, s);
    close(f); # Essential to complete encoding

    cv=CSV.read(path;delim=delim,types=Dict(1=>String,2=>String),normalizenames=true,dateformat="yyyy.mm.dd hh:ss")

   equities=convert(Array{Float64},cv[!,3])

   analyzeEquitySeries(equities,numberOfDays)
    return cv;
 end

"""
Print Strategy performance STSE report.
Parameters:fileName,numberOfDays,delim[opcional]\n

fileName -
numberOfDays -
delim=','

O arquivo apontado em fileName deve estar no formato abaixo:

<DATE>,	<BALANCE>	<EQUITY>	<DEPOSIT LOAD>
2019.10.01 00:00,	100000.00,	100000.00,	0.0000
2019.10.03 17:59,	99962.79,	100009.43,	0.0022

e CODIFICAÇÃO UTF-8 (arquivos gravados pelo Metatrader5 geralmente são UTF16 enquanto gravado pelo python em geral são UTF-8)

Observacao: The deposit load value shows the percent of account funds used to open positions. Load calculation formula:

Load = Margin / Equity * 100%
https://www.mql5.com/en/articles/2704#load

"""
function processEquityFileUtf8(fileName,numberOfDays,delim=',')
  """ s=read(fileName,String,enc"UTF-16");
   print("delim=",delim)
   path = tempname();
   f = open(path, enc"UTF-8", "w");
   write(f, s);
   close(f); # Essential to complete encoding
   """
   cv=CSV.read(fileName;delim=delim,types=Dict(1=>String,2=>String),normalizenames=true,dateformat="yyyy.mm.dd hh:ss")

  # print(head(cv))
  equities=convert(Array{Float64},cv[!,3])

  analyzeEquitySeries(equities,numberOfDays)
  return cv;
end

"""
Print Strategy performance STSE report.
Parameters:fileName,numberOfDays,delim[opcional]\n

fileName -
numberOfDays -
delim=','

The file must be in csv format with the following columns ( UTF-8 codification):
	row_number,date,balance,equity,load,orders

 These file may created by analyze_TestReport.py, using as raw data the XLS file Report created by StrategyTester in MetatTrader5 platform (StrategyTester | tab Backtest | Report (Open XML (MS office Excel)))

Note: The deposit load value shows the percent of account funds used to open positions. Load calculation formula:

Load = Margin / Equity * 100%
https://www.mql5.com/en/articles/2704#load

"""
function processEquityFilePy(fileName,numberOfDays,delim=',')
  """ s=read(fileName,String,enc"UTF-16");
   print("delim=",delim)
   path = tempname();
   f = open(path, enc"UTF-8", "w");
   write(f, s);
   close(f); # Essential to complete encoding
   """
   cv=CSV.read(fileName;delim=delim,types=Dict(1=>String,2=>String),normalizenames=true,dateformat="yyyy.mm.dd hh:ss");

  # print(head(cv))
   equities=convert(Array{Float64},cv[!,4])

     analyzeEquitySeries(equities,numberOfDays);
    return cv;
end

"""
Analisa desempenho de trader usando arquivo com formato abaixo.
Parameters:fileName,numberOfDays,delim[opcional]\n

fileName -
numberOfDays -
delim=','

The file must be in csv format with the following columns ( UTF-8 codification):
	row_number,date,balance,equity,load,orders

O arquivo apontado em fileName deve estar no formato abaixo:

   "counter","date-hour","tickReturn","asset Return","currentBalance"
"""
function processFileRF(fileName, numberOfDays,delim=',')

 cv=CSV.read(fileName;delim=delim,types=Dict(1=>String,2=>String),normalizenames=true,dateformat="yyyy-mm-dd hh:mm:ss")

   rreturns=convert(Array{Float64},cv[!,3])
   rreturns=rreturns.-1 # em formato RF o arquivo usa 1.0 para retorno zero!!!
   equities=calcPricesFromReturns(rreturns)
   println(" ---equities--- ")
   print(equities)
   println(" ---equities end--- ")
   #equities=convert(Array{Float64},cv[!,3])
   analyzeEquitySeries(equities,numberOfDays)
end

"""
Analisa desempenho de trader
Parâmetros:fileName,numberOfDays,delim[opcional]\n

fileName - string com nome do arquivo
numberOfDays - numero de dias de operação equivalente ao periodo observado
delim=';'  - caracter delimitador, por default ;

O arquivo apontado em fileName deve estar no formato abaixo:

   "counter","date-hour","tickReturn","asset Return","currentBalance"
"""
function processFile(fileName,numberOfDays,delim=';')
processFile(fileName,"",numberOfDays,delim);
end

function processFile(fileName,fileName2, numberOfDays,delim)

 #  FileWrite(fileHandle,cont,datahora,StringFormat("%.8f", tickReturn),StringFormat("%.8f", assetReturn),StringFormat("%.8f",currentBalance));

 s=read(fileName,String,enc"UTF-16");
 #e=encode(s,"UTF-8");
 #d=decode(e,"UTF-8");

 path = tempname();
 f = open(path, enc"UTF-8", "w");
 write(f, s);

 close(f); # Essential to complete encoding

if fileName2!=""
   s2=read(fileName,String,enc"UTF-16");

   f = open(path, enc"UTF-8", "w");
   write(f, s2);

   close(f); # Essential to complete encoding
end

 cv=CSV.read(path;delim=delim,types=Dict(1=>String,2=>String),normalizenames=true,dateformat="yyyy.mm.dd hh:ss")

   rreturns=convert(Array{Float64},cv[!,3])
   equities=calcPricesFromReturns(rreturns)
   #equities=convert(Array{Float64},cv[!,3])
   analyzeEquitySeries(equities,numberOfDays)
end


"""
    zero_one_pack!(capacity::N, weights::V, values::V, dp::V) where {N <: Number,V <: AbstractVector}
This does 0-1 (each item can be chosen only once) knapsack :
pack capacity = capacity
weight of each item = weights
value of each item = values
dp array is what the function works on
It returns the ans (dp[capacity])
```
julia> dp=zeros(Int,30)
julia> zero_one_pack!(20,[1,3,11],[2,5,30],dp)
37
```
"""
function dp_zero_one_pack!(capacity::N, weights::V, values::V, dp::V
) where {N <: Number,V <: AbstractVector}
    for i in 1:length(weights)
        j = capacity
        while j > weights[i] # reversed loop
            dp[j] = max(dp[j], dp[j - weights[i]] + values[i])
            j -= 1
        end
        dp[weights[i]] = max(dp[weights[i]], values[i]) # dp[j]=max(dp[j],dp[0]+values[i])
    end
    return dp[capacity]
end

"""
This does complete/infinite (each item can be chosen infinite times) knapsack :
pack capacity = capacity
weight of each item = weights
value of each item = values
dp array is what the function works on
It returns the ans (dp[capacity])
```
julia> dp=zeros(Int,30)
julia> complete_pack!(20,[1,2,9],[1,3,20],dp)
43
```
"""
function dp_complete_pack!(capacity::N, weights::V, values::V, dp::V
) where {N <: Number,V <: AbstractVector}
    for i in 1:length(weights)
        dp[weights[i]] = max(dp[weights[i]], values[i]) # dp[j]=max(dp[j],dp[0]+values[i])
        for j in weights[i] + 1:capacity
            dp[j] = max(dp[j], dp[j - weights[i]] + values[i])
        end
    end
    return dp[capacity]
end


using Distributions, HypothesisTests

p = [0.08, 0.12, 0.2, 0.2, 0.15, 0.25]
O = [3, 2, 9, 11, 8, 27]
M = length(O)
n = sum(O)
E = n*p

testStatistic = sum((O-E).^2 ./E)
pVal = ccdf(Chisq(M-1), testStatistic)

println("Manually calculated test statistic: ", testStatistic)
println("Manually calculated p-value: ", pVal,"\n")

println(ChisqTest(O,p))


"""
    pythagorean_triplets(n)

Find all positive integer triplets `(a, b, c)` s.t. `a + b + c = n` and `a < b < c` and `a^2 + b^2 == c^2`.
"""
# cmcaine's answer, with thanks to akshu3398.
function pythagorean_triplets(n)
    triplets = NTuple{3, Int}[]
    # Lower bound because the smallest triple is 3, 4, 5.
    # Upper bound implied by a < b < c && a + b + c == n.
    for a in 3:cld(n, 3) - 1
        # Derived by eliminating c from these simultaneous
        # equations and solving for b:
        #     a^2 + b^2 = c^2
        #     a + b + c = n
        b = (n^2 - 2n * a) / (2 * (n - a))
        if a < b && isinteger(b)
            c = n - a - b
            # Proof that b < c:
            # Let b = c. Then a^2 + b^2 = c^2 ≡ a^2 = 0 but we know that a ≠ 0.
            # Let b > c. a^2 + b^2 = c^2 ≡ a^2 = c^2 - b^2.
            #    If b > c, a^2 < 0, but that is impossible for real numbers.
            push!(triplets, (a, b, c))
        end
    end
    return sort!(triplets)
end


function dare(A,B,Q,R,S=0)
	#
	# solves discrete-time algebraic Riccati equation
	# X = A'XA - (A'XB+S)(R+B'XB)^{-1}(B'XA+S') + Q
	#
	# R^{-1} is not computed
	#
	# javier.cara@upm.es, 2016-02
	#

	# Dimensions
	n,m = size(B)

	if S== 0
		S = zeros(n,m)
	end

	L = [A zeros(n,n) B;Q -eye(n) S;S' zeros(m,n) R]
	M = [eye(n) zeros(n,n) zeros(n,m); zeros(n,n) -A' zeros(n,m);zeros(m,n) -B' zeros(m,m)]

	# Compute the eigenvalue decomposition
	(d,v) = eig(L,M)

	# Sort the eigenvalues
	ew = abs(d)
	pos = sortperm(ew)

	# Compute X
	v1 = v[1:n,pos[1:n]]
	v2 = v[n+1:2*n,pos[1:n]]
	X = real(v2/v1)

	return X

end

######
function dare_test()
    #=
    BibText
    @TECHREPORT{Benner95acollection,
    author = {Peter Benner and Alan J. Laub and Volker Mehrmann},
    title = {A Collection of Benchmark Examples for the Numerical Solution of
    Algebraic Riccati Equations II: Discrete-Time Case},
    institution = {FAK. F. MATHEMATIK, TU CHEMNITZ--ZWICKAU},
    year = {1995}
    }

    nov-2016
    =#

	# ------------------------------------------------------------------
	# ejm1 collection of benchmark examples
	A1 = [4 3;-4.5 -3.5]
	B1 = [1 -1]'
	R1 = 1
	Q1 = [9 6;6 4]
	X1 = (1+sqrt(5))/2*[9 6;6 4]

	X11 = dare(A1,B1,Q1,R1)

	# --------------------------------------------------------------------
	# ejm computed with mathematica
	Am = [1 -1 1;0 1 1;0 0 1]
	Bm = [1 0;1 0;0 1]
	Rm = [10 0;0 0.1]
	Qm = [10 0 0;0 1 0;0 0 0.1]
	Xm = [42.2835 -68.5247 -3.94783;-68.5247 154.043 16.0017;-3.94783 16.0017 8.33197]

	Xmm = dare(Am,Bm,Qm,Rm)

	#####

	return X1,X11,Xm,Xmm

end


using LinearAlgebra
using SparseArrays
using Plots
using ForwardDiff

"This routine solves Poisson's equation using piecewise linear finite element methods.
The matrix is assembled locally, and boundary nodes are not removed."

m = 100 # number of elements

α = 1.0
β = 0.0
f(x) = x + 0*2*sin(2*pi*x)
κ(x) = 1.0 + x

# define spatial grid
x = LinRange(-1,1,m+2) # x_0, x_1, ..., x_m, x_{m+1} = x_0
xint = x[1:end-1]
h = x[2]-x[1]

# x = x + randn(size(x))/(2*m)
# x = @. x + (1+x)*(1-x)/3

# construct local FEM matrix
map_point(x,a,b) = a + (b-a) * (1+x)/2 # maps x ∈ [-1,1] to interval [a,b]
A = spzeros(m+2,m+2)
b = zeros(m+2)
A_local = [1 -1;-1 1]

function ϕ(r)
    ϕ1 = @. (1-r)/2
    ϕ2 = @. (1+r)/2
    return [ϕ1 ϕ2]
end
function dϕ(r)
    dϕ1 = @. -.5 + 0*r
    dϕ2 = @. .5 + 0*r
    return [dϕ1 dϕ2]
end

r = 0.0
w = 2.0

# r = [-1; 1] / sqrt(3)
# w = ones(2)
for e = 1:m+1
    h_e = x[e+1] - x[e]
    ids = e:e+1

    # use 1-point quadrature to approximate integrals
    x_e = map_point(0, x[e], x[e+1])

    # accumulate local contributions
    A[ids,ids] += A_local * κ(x_e) / h_e
    b[ids] += h_e * w * f(x_e) * [.5;.5]

    # # use 2-point Gauss quadrature to approximate integrals
    # x_e = map_point.(r, x[e], x[e+1])
    #
    # # build and accumulate local contributions
    # A_local = dϕ(r)'*diagm(w_e.*κ.(x_e))*dϕ(r)
    # A[ids,ids] += A_local / h_e
    # b[ids] += h_e * (ϕ(r)' * (w.*f.(x_e)))
end

# modify for boundary conditions
function impose_Dirichlet_BC!(A,b,i,val)
    b .-= A[:,i]*val
    b[i] = val
    A[i,:] .= 0
    A[:,i] .= 0
    A[i,i] = 1.0
end
function impose_Neumann_BC!(A,b,i,val)
    b[i] += val
end

impose_Dirichlet_BC!(A,b,1,α)
impose_Dirichlet_BC!(A,b,m+2,β)
# impose_Neumann_BC!(A,b,m+2,β)

u = A\b
plot(x,u,legend=false,mark=:dot,ms=2)


# # Trade-off curves
using Random
Random.seed!(1)
m = 25;
n = 10;
A = randn(m, n);
b = randn(m, 1);

#-

using Convex, SCS, LinearAlgebra

gammas = exp10.(range(-4, stop=2, length=100));

x_values = zeros(n, length(gammas));
x = Variable(n);
for i=1:length(gammas)
    cost = sumsquares(A*x - b) + gammas[i]*norm(x,1);
    problem = minimize(cost, [norm(x, Inf) <= 1]);
    solve!(problem, SCS.Optimizer(verbose=0));
    x_values[:,i] = evaluate(x);
end

#-

# Plot the regularization path.

using Plots
plot(title = "Entries of x vs lambda", xaxis=:log, xlabel="lambda", ylabel="x" )
for i = 1:n
    plot!(gammas, x_values[i,:], label="x$i")
end
plot!()


"""
	euler_characteristic(V::Lar.Points, copEV::Lar.ChainOp,	copFE::Lar.ChainOp)

Return the Euler Characteristic number of a 2D- or 3D-model.
`V` is given by rows.
"""
function euler_characteristic(
		V::Lar.Points, # by rows
		copEV::Lar.ChainOp,
		copFE::Lar.ChainOp)

	# χ = size(V,1) - size(copEV,1) + size(copFE,1)
	# return χ
	return euler_characteristic_general(V, copEV, copFE)
end

"""
	euler_characteristic_general(V::Lar.Points, cc::Lar.ChainOp...)

Return the Euler Characteristic number of a n-dimensional model.
`V` is given by rows.
"""
function euler_characteristic_general(V::Lar.Points, cc::Lar.ChainOp...)
	χ = size(V,1)
	for k in 1:length(cc)
		χ = χ + (-1)^k * size(cc[k],1)
	end
	return χ
end


# In the 5 by 5 matrix below, the minimal path sum from the top left to the
# bottom right, by only moving to the right and down, is indicated in bold red
# and is equal to 2427.
#
# 131 673 234 103 18
# 201 96  342 965 150
# 630 803 746 422 111
# 537 699 497 121 956
# 805 732 524 37  331
#
# Find the minimal path sum from the top left to the bottom right by only moving
# right and down in matrix.txt (right click and "Save Link/Target As..."), a 31K
# text file containing an 80 by 80 matrix.

using ProjectEulerSolutions

# Simple dynamic programming solution to fill in a matrix starting with the
# corner and then picking the min path to each consecutive cell.
function p081solution(path::String="")::Integer

    if !isfile(path)
        if path != ""
            println("Cannot find path: '", path, "'")
        end
        return -1
    end

    # Read in matrix from file
    matrix = reduce(hcat, [parse.(Int32, split(line, ',')) for line in eachline(path)])
    n, m = size(matrix)

    best_path = zeros(Integer, size(matrix))
    best_path[1,1] = matrix[1,1]

    # Initialize the edges since they only have a single path.  Make use of the
    # built-in cumulative sum function
    best_path[:,1] = cumsum(matrix[:,1])
    best_path[1,:] = cumsum(matrix[1,:])

    # Very simple dynamic programming solution for the rest
    for i = 2:n
        for j = 2:m
            best_path[i,j] = min(best_path[i-1,j], best_path[i,j-1]) + matrix[i,j]
        end
    end

    return best_path[end,end]
end

p081 = Problems.Problem(p081solution)

input_path = "data/p081_matrix.txt"

Problems.benchmark(p081, input_path)


"""Julia program to check if a number is automorphic or not.
An Automorphic Number is such a number whose square ends in the same digits as  the number itself."""

function check_automophic(num)
    temp = num * num

    while(num > 0)
        # Extract the last digits of the given number and its square.
        lastNum = num % 10
        lastSquare = temp % 10

        # Check if they are equal
        if(lastNum != lastSquare)
            return "The given number $n is not an Automorphic Number."
        end

        num = num ÷ 10
        temp = temp ÷ 10
    end

    return "The given number $n is an Automorphic Number."
end

print("Enter the number: ")
n = readline()
n = parse(Int, n)
res = check_automophic(abs(n))
print(res)

"""
Time Complexity: O(log(n)), where 'n' is the given number
Space Complexity: O(1)

SAMPLE INPUT AND OUTPUT

SAMPLE 1

Enter the number: 24
The given number 24 is not an Automorphic Number.

SAMPLE 2

Enter the number: 25
The given number 25 is an Automorphic Number.
"""


"""
# module TestDoolittle

# Examples

```jldoctest
julia>
```
"""
module TestDoolittle

using Test

using APMAE4001.LUFactorization.Doolittle

@testset "Set 1" begin
     A = [1  1  0  0  0  0
          1  2  1  0  0  0
          0  1  2  1  0  0
          0  0  1  2  1  0
          0  0  0  1  2  1
          0  0  0  0  1  2]
     L, U = doolittle_lu(A)
     @test L == [1.0  0.0  0.0  0.0  0.0  0.0
                 1.0  1.0  0.0  0.0  0.0  0.0
                 0.0  1.0  1.0  0.0  0.0  0.0
                 0.0  0.0  1.0  1.0  0.0  0.0
                 0.0  0.0  0.0  1.0  1.0  0.0
                 0.0  0.0  0.0  0.0  1.0  1.0]
     @test U == [1.0  1.0  0.0  0.0  0.0  0.0
                 0.0  1.0  1.0  0.0  0.0  0.0
                 0.0  0.0  1.0  1.0  0.0  0.0
                 0.0  0.0  0.0  1.0  1.0  0.0
                 0.0  0.0  0.0  0.0  1.0  1.0
                 0.0  0.0  0.0  0.0  0.0  1.0]
end

@testset "Example from notes" begin
     A = [1  1   0
          2  5   1
          1  4  -1]
     b = [1, 1, 2]
     L, U = doolittle_lu(A)
     @test L == [1.0  0.0  0.0
                 2.0  1.0  0.0
                 1.0  1.0  1.0]
     @test U == [1.0  1.0   0.0
                 0.0  3.0   1.0
                 0.0  0.0  -2.0]
     solutions = polysolve(A, b)
     @test solutions == A \ b
end

end


module IterativeMethod

using LinearAlgebra

export
    is_diagonally_dominant,
    jacobi_method,
    gauss_seidel_method, gauss_seidel_method!

#===== Jacobi method =====#

#=

A x = b
A = D + R
D x = -R x + b
  x = -D^{-1} R x + D^{-1} b
⟹
x = B x + z, where B = -D^{-1} R, z = D^{-1}

x_{i + 1} = B x_i + z
          = B (x^* + e_i) + z
          = x^* + B e_i
⟹
e_{i + 1} = B e_i

=#

function is_diagonally_dominant(A::AbstractMatrix{T})::Bool where T<:AbstractFloat
    A = abs.(A)
    d2 = diag(A) * 2
    all(sum(A, dims=1)' .<= d2) && all(sum(A, dims=2) .<= d2)
end

const MAX_ITERATIONS = 1000

function jacobi_method(
    A::Matrix{T},
    b::Vector{T},
    x0::Vector{T} = zero(b),
)::Vector{T} where T<:AbstractFloat
    (nr, nc) = size(A)
    @assert nr == nc "Jacobi: A is not square."
    if !is_diagonally_dominant(A)
        println("Jacobi: A is not diagonally dominant")
    end

    D = Diagonal(A)
    invD = inv(D)
    negR = D - A
    x = x0
    for _ in 1:MAX_ITERATIONS
        x = invD * (negR * x0 + b)
        if x ≈ x0
            return x
        end
        x0 = x
    end
    x
end

#===== Gauss–Seidel method =====#

#=

In Jacobi method,

    x_i^{(k + 1)} = -1/a_ii * ((L + U)_{i, :} * x^{(k)}) + b_i / a_ii

where $U$ ($L$) is the strictly upper (lower) triangular and $R = L + U$. If in the iteration we
use the elements of $x^{(k+1)}$ that have already been computed to compute the unknown ones of
$x^{(k+1)}$,

    x_i^{(k + 1)} = -1/a_ii * ((L_*)_{i, :} * x^{(k + 1)} + U_{i, :} * x^{(k)}) + b_i / a_ii

where $L_* = D + L$ is a lower triangular matrix. Or more compcatly,

    A x = b
    (L_* + U) x = b
    L_* x = b - U x

    ⟹

    L_* x^{(k + 1)} = b - U x^{(k)}
    x^{(k + 1)} = L_*^{-1} (b - U x^{(k)})

=#

function gauss_seidel_method(A::Matrix{T}, b::Vector{T})::Vector{T} where T<:AbstractFloat
    x0 = zero(b)
    gauss_seidel_method!(A, b, x0)
end

# The initial guess `x0` will be updated in-place.
function gauss_seidel_method!(
    A::Matrix{T},
    b::Vector{T},
    x0::Vector{T},
)::Vector{T} where T<:AbstractFloat
    (nr, nc) = size(A)
    @assert nr == nc "Gauss–Seidel: A is not square."
    n = nr
    if !is_diagonally_dominant(A)
        println("Gauss–Seidel: A is not diagonally dominant")
    end

    x = x0
    for _ in 1:MAX_ITERATIONS
        updated = false
        for r in 1:n
            old = x[r]
            x[r] = (b[r] - ((@view A[r, :])' * x - A[r, r] * x[r])) / A[r, r]
            if !(x[r] ≈ old)
                updated = true
            end
        end
        if !updated
            return x
        end
    end
    x
end

end # module


# Algorithm 6.2
# Gaussian Elimination with partial pivoting

function swap_rows(mat, row1, row2)
	for i in 1:size(mat)[2]
		p1 = mat[row1, i]
		p2 = mat[row2, i]
		tmp = p1
		mat[row1, i] = p2
		mat[row2, i] = tmp
	end
	return mat
end

function solve_gauss(mat)
	n, m = size(mat)
	for k in 1:n
		max_row = 1
		max_val = 0
		for i in 1:n
			if mat[i, k] > max_val
				max_row = i
				max_val = mat[i, k]
			end
		end
		if max_val == 0
			println("No unique solution")
			return
		end

		if max_row != 1
			mat = swap_rows(mat, k, max_row)
		end

		for i in (k + 1):n
			m = mat[i, k]/mat[k, k]
			for j in k:(n + 1)
				mat[i, j] -= m*mat[k, j]
			end
		end
	end
	println(mat)
	x = zeros(1, n)
	x[n] = mat[n, n + 1]/mat[n ,n]
	for i in (n - 1):-1:1
		sum = 0
		for j in (i + 1):n
			sum = mat[i, j]*x[j]
		end
		x[i] = (mat[i, n + 1] - sum)*mat[i, i]
	end
	return x
end

A = [[1.0 1.0 2]; [2.0 -2.0 3.0];]
println(A)
A = solve_gauss(A)
println(A)