#===============================================================================

The palindromic number 595 is interesting because it can be written as the sum
of consecutive squares: 6^2 + 7^2 + 8^2 + 9^2 + 10^2 + 11^2 + 12^2.

There are exactly eleven palindromes below one-thousand that can be written as
consecutive square sums, and the sum of these palindromes is 4164. Note that 1 =
0^2 + 1^2 has not been included as this problem is concerned with the squares of
positive integers.

Find the sum of all the numbers less than 108 that are both palindromic and can
be written as the sum of consecutive squares.

--------------------------------------------------------------------------------

The solution is easy: we generate all i^2 + ... + j^2 below the limit and we
check that the number obtained is smaller than the limit.

Trap: a same number can be the sum of different sums of squares...

===============================================================================#

include("Euler.jl")

function euler(limit)
    i = 1
    ns = Set{Int}()

    while true
        n = i^2
        if limit <= n
            break
        end

        j = i+1
        n += j^2

        while n < limit
            if is_palindromic(n)
                push!(ns, n)
            end
            j += 1
            n += j^2
        end

        i += 1
    end

    sum(ns)
end

println("10^3: $(euler(10^3))")
println("10^8: $(euler(10^8))")


### A Pluto.jl notebook ###
# v0.15.1

using Markdown
using InteractiveUtils

# ╔═╡ 7f621598-8ce7-11eb-1d78-89ab395c5aac
using MultivariateStats

# ╔═╡ 1e960125-0783-4932-9b98-7b09c72bc951
md"[Reference Chapter](https://www.deeplearningbook.org/contents/linear_algebra.html)"

# ╔═╡ 811b4842-775b-11eb-3690-830857abe1b5
md"### Eigenvectors"

# ╔═╡ ec3044cc-7768-11eb-17ff-c19d9ac19ae4
md"
Cosider a matrix of order ``n × n``

$A = \begin{bmatrix}a_{11} & a_{12} & a_{13} & a_{14} & ... & a_{1n}\\a_{21} & a_{22} & a_{23} & a_{24} & ... & a_{2n}\\a_{31} & a_{32} & a_{33} & a_{34} & ... & a_{3n}\\a_{41} & a_{42} & a_{43} & a_{44} & ... & a_{4n}\\\vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\a_{n1} & a_{n2} & a_{n3} & a_{n4} & ... & a_{nn}\end{bmatrix}$

When we multiply it by a vector of length `` n ``...

$B = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \\ b_4 \\\vdots \\ b_n \end{bmatrix}$

`` C = AB ``

Therefore, ``C = \begin{bmatrix}a_{11} & a_{12} & a_{13} & a_{14} & ... & a_{1n}\\a_{21} & a_{22} & a_{23} & a_{24} & ... & a_{2n}\\a_{31} & a_{32} & a_{33} & a_{34} & ... & a_{3n}\\a_{41} & a_{42} & a_{43} & a_{44} & ... & a_{4n}\\\vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\a_{n1} & a_{n2} & a_{n3} & a_{n4} & ... & a_{nn}\end{bmatrix}$ $\begin{bmatrix} b_1 \\ b_2 \\ b_3 \\ b_4 \\\vdots \\ b_n \end{bmatrix}``

which can be rewritten as,

$C = \begin{bmatrix} a_{11} \times b_1 + a_{12} \times b_2 + a_{13} \times b_3 + a_{14} \times b_4 ... a_{1n} \times b_n \\ a_{21} \times b_1 + a_{22} \times b_2 + a_{23} \times b_3 + a_{24} \times b_4 ... a_{2n} \times b_n \\ a_{31} \times b_1 + a_{32} \times b_2 + a_{33} \times b_3 + a_{34} \times b_4 ... a_{3n} \times b_n \\ a_{41} \times b_1 + a_{42} \times b_2 + a_{43} \times b_3 + a_{44} \times b_4 ... a_{4n} \times b_n \\\vdots \\ a_{n1} \times b_1 + a_{n2} \times b_2 + a_{n3} \times b_3 + a_{n4} \times b_4 ... a_{nn} \times b_n \end{bmatrix}$

If there exists a scalar `` \lambda ``, such that `` B \times \lambda = C ``, then `` B `` is the eigenvector of `` A ``, and `` \lambda `` is its eigenvalue.
"

# ╔═╡ dd42dfa8-7aa9-11eb-3312-fbed21eb12d5
md"## Eigendecomposition"

# ╔═╡ ec9fae4a-7aa9-11eb-10a9-bfed672dfaca
md"If a matrix 𝐴 has 𝑛 linearly independent eigenvectors and 𝑛 corresponding eigenvalues, then if we make a matrix out of the eigenvectors, and we call it 𝑉, where every column in 𝑉 is an eigenvector of 𝐴, and make a (column)vector 𝜆 which has eigenvalues, corresponding to eigenvectors of 𝐴, then we can say that :

Eigendecomposition of 𝐴 = 𝑉_diag_(𝜆)𝑉$^{-1}$
"

# ╔═╡ 6dca1270-7aab-11eb-23a6-c5db84f32eb5
let
	import LinearAlgebra
	A = rand(1:10,3,3)
	F = LinearAlgebra.eigen(A)
	A, F
end

# ╔═╡ 5b37c0c6-7ab5-11eb-2fe2-4fefc5a9e193
md"""
In the above code, we find the eigendecomposition of 𝐴, and store it in 𝐹, where `F.vectors` give us all the eigenvalues(or the matrix 𝑉 in the above context), and `F.values` stores the corresponding eigenvalues(the vector 𝜆 in the above context).
"""

# ╔═╡ 8f59c323-4326-4c5a-b013-6e0cfa442e02
md"""
!!! note "Some terms"

	If a matrix's eigenvalues are all positive, its called _**positive definite**_.

	If a matrix's eigenvalues are all positive or zero, its called _**positive semidefinite**_.

	If a matrix's eigenvalues are all negative, its called _**negative definite**_.

	If a matrix's eigenvalues are all negative or zero, its called _**negative semidefinite**_.

	In a **_positive semidefinite_** matrix, _for all x, **xᵀ𝐴x ≥ 𝟢**_.

	In a **_positive semidefinite_** matrix, _**xᵀ𝐴x = 𝟢**, implies that,_ **_x = 𝟢_**.
"""

# ╔═╡ 11a7a0f0-7ad8-11eb-12c7-77f971d32a1a
md"## Singular Value Decomposition"

# ╔═╡ 2c663d90-7ad8-11eb-0e6c-039ecc187c0d
md"""
There's another method of decomposing (or factorising) a matrix into sigular vectors and singular values called Singular Value Decomposition.
Every real matrix has a singular value decomposition, but not every matrix has can be decomposed into eigenvalues. For example, matrices which are not square.

#### What is it?
We can write a matrix 𝐴 as a product of 𝟥 matrices --

`` 𝐴 = 𝑈𝐷𝑉^T ``

1. 𝐴 and 𝐷, both have dimensions `` m × n ``, 𝑈 has dimensions `` m × m ``, and 𝑉 has dimensions `` n × n ``.
2. 𝑈 and 𝑉 are both orthogonal matrices.
3. 𝐷 is a diagonal matrix and is not necessarily square.
4. The diagonal elements of 𝐷, also known as singular values of matrix 𝐴, are the are the square roots of eigenvalues of 𝐴ᵀ𝐴 or 𝐴𝐴ᵀ(eigenvalues of both are equal).
5. The columns of 𝑈, also known as left-singular vectors of 𝐴, are the eigenvectors of 𝐴𝐴ᵀ.
6. The columns of 𝑉, also known as right-singular vectors of 𝐴, are the eigenvectors of 𝐴ᵀ𝐴.
"""

# ╔═╡ bdef0196-7b91-11eb-2d26-7b02fa67233c
let
	A = convert(Matrix,rand(0:9, 3,3))
	LinearAlgebra.svd(A)
end

# ╔═╡ d0fd5526-7d00-11eb-113b-47a224413bac
md"## Trace operator
A simple demonstration would be easier to understand."

# ╔═╡ 22247216-7ddf-11eb-24da-65455796685f
let
	A = convert(Matrix,rand(0:9, 3,3))
	A, LinearAlgebra.tr(A)
end

# ╔═╡ 052770a6-787d-11eb-1816-11fc224a68bb
md"It gives the sum of diagonal elements."

# ╔═╡ 0085a0f2-7ded-11eb-18c3-8b40f3a71ab9
md"## Moore Penrose Pseudo-Inverse
Assume there exists a matrix 𝐴, and its left-inverse 𝐵. Such that they satisfy,

`` 𝐴𝑥 = 𝑦 ``

`` 𝑥 = 𝐵𝑦  ``

The pseudo-inverse of 𝐴 is $A^+$, where $A^+$ given by the matrix,

`` 𝐴^+ = VD^+U^T ``

𝑈, 𝑉 and 𝐷 are from the Singular Value Decomposition of 𝐴, as defined above

`` D^+ `` is given by calculating the reciprocal of every non-zero element in 𝐷, and then the resulting matrix's transpose.

Lets just say that `` A^+ `` is pretty close to 𝐵.
"

# ╔═╡ 98be78fe-7dfd-11eb-0812-59df1018f0af
let
	A = convert(Matrix,rand(0:9, 3,3))
	A, LinearAlgebra.pinv(A), inv(transpose(A) * A) * transpose(A)#original matrix, pseudo-inverse and left-inverse
end

# ╔═╡ 0def7a46-787d-11eb-189c-8151f676c8f1
md"## Try It Yourself"

# ╔═╡ dce0bdfb-23c3-415a-9b5c-833782e223a8
md"
**Try understanding the _Principal Component Analysis_ from the book and implementing it in Julia.**
"

# ╔═╡ 22557a0e-7de0-11eb-2fa0-07c807d98310
md"""
Lets understand some terms!

##### [Features](https://www.datarobot.com/wiki/feature/)
Assume you've to predict the price of houses. The data you have is, size of houses, size of lawn, number of rooms, and of course, price.
Here, size of house, size of lawn, number of rooms, and price act as features. Also note that here, price is the _dependent variable_, while rest are independent variables.

##### Feature Elimination
It can be hard to work with multiple features. From the above features, you can try eliminating one or more features, which will better help in prediction. The accuracy of prediction will fall though. For an example, I can assume that the size of lawn variable doesn't exist here. Working with the two remaining independent variables to predict the dependent variable (_price_ in the above case), would be easier, but accuracy of prediction will fall.

##### Feature Extraction
Dropping a whole feature can't be very reliable, you say? Yeah it isn't (in many cases)! Hence, introducing _feature extraction_! We can try making new independent features from the ones we already have, but has overall lesser dimensions in a way that we can make the data in the new features, as accurate as possible compared to the old ones. Since every new feature is made from the combining old ones, all new features have much of the information of old ones!

So, is there some specified way to make those new features I talk about, in feature extraction, you ask? Yes! There are many, and one such way is called **_Principle Component Analysis_**.

Principle Component Analyis is a method of doing feature extraction.

To start, the reference book and [this article](https://towardsdatascience.com/a-one-stop-shop-for-principal-component-analysis-5582fb7e0a9c) are good to read.
"""

# ╔═╡ 73ef6fad-08fc-4294-8306-68ed5722a8bb
md"
_Hint_: [MultivariateStats.jl](https://multivariatestatsjl.readthedocs.io/en/latest/pca.html)
"

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
MultivariateStats = "6f286f6a-111f-5878-ab1e-185364afe411"

[compat]
MultivariateStats = "~0.8.0"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

julia_version = "1.7.0-beta3.0"
manifest_format = "2.0"

[[deps.ArgTools]]
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"

[[deps.Arpack]]
deps = ["Arpack_jll", "Libdl", "LinearAlgebra"]
git-tree-sha1 = "2ff92b71ba1747c5fdd541f8fc87736d82f40ec9"
uuid = "7d9fca2a-8960-54d3-9f78-7d1dccf2cb97"
version = "0.4.0"

[[deps.Arpack_jll]]
deps = ["Libdl", "OpenBLAS_jll", "Pkg"]
git-tree-sha1 = "e214a9b9bd1b4e1b4f15b22c0994862b66af7ff7"
uuid = "68821587-b530-5797-8361-c406ea357684"
version = "3.5.0+3"

[[deps.Artifacts]]
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"

[[deps.Base64]]
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"

[[deps.Compat]]
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
git-tree-sha1 = "727e463cfebd0c7b999bbf3e9e7e16f254b94193"
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
version = "3.34.0"

[[deps.CompilerSupportLibraries_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "e66e0078-7015-5450-92f7-15fbd957f2ae"

[[deps.DataAPI]]
git-tree-sha1 = "bec2532f8adb82005476c141ec23e921fc20971b"
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
version = "1.8.0"

[[deps.DataStructures]]
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
git-tree-sha1 = "7d9d316f04214f7efdbb6398d545446e246eff02"
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
version = "0.18.10"

[[deps.Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[deps.DelimitedFiles]]
deps = ["Mmap"]
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"

[[deps.Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[deps.Downloads]]
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"

[[deps.InteractiveUtils]]
deps = ["Markdown"]
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"

[[deps.LibCURL]]
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"

[[deps.LibCURL_jll]]
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"

[[deps.LibGit2]]
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"

[[deps.LibSSH2_jll]]
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"

[[deps.Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[deps.LinearAlgebra]]
deps = ["Libdl", "libblastrampoline_jll"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[deps.Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[deps.Markdown]]
deps = ["Base64"]
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"

[[deps.MbedTLS_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"

[[deps.Missings]]
deps = ["DataAPI"]
git-tree-sha1 = "2ca267b08821e86c5ef4376cffed98a46c2cb205"
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
version = "1.0.1"

[[deps.Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[deps.MozillaCACerts_jll]]
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"

[[deps.MultivariateStats]]
deps = ["Arpack", "LinearAlgebra", "SparseArrays", "Statistics", "StatsBase"]
git-tree-sha1 = "8d958ff1854b166003238fe191ec34b9d592860a"
uuid = "6f286f6a-111f-5878-ab1e-185364afe411"
version = "0.8.0"

[[deps.NetworkOptions]]
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"

[[deps.OpenBLAS_jll]]
deps = ["Artifacts", "CompilerSupportLibraries_jll", "Libdl"]
uuid = "4536629a-c528-5b80-bd46-f80d51c5b363"

[[deps.OrderedCollections]]
git-tree-sha1 = "85f8e6578bf1f9ee0d11e7bb1b1456435479d47c"
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
version = "1.4.1"

[[deps.Pkg]]
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"

[[deps.Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[deps.REPL]]
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"

[[deps.Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[deps.SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[deps.Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[deps.SharedArrays]]
deps = ["Distributed", "Mmap", "Random", "Serialization"]
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"

[[deps.Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[deps.SortingAlgorithms]]
deps = ["DataStructures"]
git-tree-sha1 = "b3363d7460f7d098ca0912c69b082f75625d7508"
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
version = "1.0.1"

[[deps.SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[deps.Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[deps.StatsAPI]]
git-tree-sha1 = "1958272568dc176a1d881acb797beb909c785510"
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
version = "1.0.0"

[[deps.StatsBase]]
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
git-tree-sha1 = "8cbbc098554648c84f79a463c9ff0fd277144b6c"
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
version = "0.33.10"

[[deps.TOML]]
deps = ["Dates"]
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"

[[deps.Tar]]
deps = ["ArgTools", "SHA"]
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"

[[deps.Test]]
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[[deps.UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[deps.Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"

[[deps.Zlib_jll]]
deps = ["Libdl"]
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"

[[deps.libblastrampoline_jll]]
deps = ["Artifacts", "Libdl", "OpenBLAS_jll"]
uuid = "8e850b90-86db-534c-a0d3-1478176c7d93"

[[deps.nghttp2_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"

[[deps.p7zip_jll]]
deps = ["Artifacts", "Libdl"]
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"
"""

# ╔═╡ Cell order:
# ╟─1e960125-0783-4932-9b98-7b09c72bc951
# ╟─811b4842-775b-11eb-3690-830857abe1b5
# ╟─ec3044cc-7768-11eb-17ff-c19d9ac19ae4
# ╟─dd42dfa8-7aa9-11eb-3312-fbed21eb12d5
# ╟─ec9fae4a-7aa9-11eb-10a9-bfed672dfaca
# ╠═6dca1270-7aab-11eb-23a6-c5db84f32eb5
# ╟─5b37c0c6-7ab5-11eb-2fe2-4fefc5a9e193
# ╟─8f59c323-4326-4c5a-b013-6e0cfa442e02
# ╟─11a7a0f0-7ad8-11eb-12c7-77f971d32a1a
# ╟─2c663d90-7ad8-11eb-0e6c-039ecc187c0d
# ╠═bdef0196-7b91-11eb-2d26-7b02fa67233c
# ╟─d0fd5526-7d00-11eb-113b-47a224413bac
# ╠═22247216-7ddf-11eb-24da-65455796685f
# ╟─052770a6-787d-11eb-1816-11fc224a68bb
# ╟─0085a0f2-7ded-11eb-18c3-8b40f3a71ab9
# ╠═98be78fe-7dfd-11eb-0812-59df1018f0af
# ╟─0def7a46-787d-11eb-189c-8151f676c8f1
# ╟─dce0bdfb-23c3-415a-9b5c-833782e223a8
# ╟─22557a0e-7de0-11eb-2fa0-07c807d98310
# ╟─73ef6fad-08fc-4294-8306-68ed5722a8bb
# ╠═7f621598-8ce7-11eb-1d78-89ab395c5aac
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002


using NewtonsMethod
using Test

@testset "NewtonsMethod.jl" begin
    # Testing a well known functions
        # f(x) = (x-2)^2, f'(x) = 2(x-2), root = 2
    f(x)  =  (x - 2)^2
    df(x) = 2(x - 2)
    @test NewtonsMethod.newtonroot(f, df)[1] ≈ 2.0
    @test NewtonsMethod.newtonroot(f)[1] ≈ 2.0

        # f(x) = (x-1)^7, f'(x) = 7(x-1)^6, root = 1
    f(x)  =  (x - 1)^7
    df(x) = 7(x - 1)^6
    @test NewtonsMethod.newtonroot(f, df)[1] ≈ 1.0
    @test NewtonsMethod.newtonroot(f)[1] ≈ 1.0

        # f(x) = log(x), f'(x) = 1/x, root = 1
    f(x)  =  log(x)
    df(x) = 1 / x
    @test NewtonsMethod.newtonroot(f, df)[1] ≈ 1.0
    @test NewtonsMethod.newtonroot(f)[1] ≈ 1.0

    # Testing a non-converging function
        # f(x) = x^2 +2, f'(x) = 2x
    f(x)  =  x^2 + 2
    df(x) = 2x
    @test NewtonsMethod.newtonroot(f, df)[1] === nothing
    @test NewtonsMethod.newtonroot(f)[1] === nothing

    # Testing maxiter. I will choose an arbitrarily small maxiter, so the function should not converge in the first test and converge in the second.
    f(x)  =  (x - 2)^2
    df(x) = 2(x - 2)
    @test NewtonsMethod.newtonroot(f, df; maxiter=5)[1] === nothing
    @test NewtonsMethod.newtonroot(f; maxiter=5)[1]  === nothing

    f(x)  =  2x
    df(x) = 2
    @test NewtonsMethod.newtonroot(f, df; maxiter=5)[1] ≈ 0.0
    @test NewtonsMethod.newtonroot(f; maxiter=5)[1] ≈ 0.0

    # Testing the tolerance. I will choose a very slack tolerance (1e-4) in order to have a ``bad'' solution.
    f(x)  =  (x - 2)^2
    df(x) = 2(x - 2)
    @test !(NewtonsMethod.newtonroot(f, df; tolerance=1e-4)[1] ≈ 2.0)
    @test !(NewtonsMethod.newtonroot(f; tolerance=1e-4)[1]  ≈ 2.0)

    f(x)  =  2x
    df(x) = 2
    @test NewtonsMethod.newtonroot(f, df; maxiter=5)[1] ≈ 0.0
    @test NewtonsMethod.newtonroot(f; maxiter=5)[1] ≈ 0.0

    # Testing the initial value.
    f(x)  =  (x - 2)^2
    df(x) = 2(x - 2)
    @test NewtonsMethod.newtonroot(f, df; x0=10)[1] ≈ 2.0 # x0 = 10
    @test NewtonsMethod.newtonroot(f, df; x0=-10)[1] ≈ 2.0 # x0 = -10
    @test NewtonsMethod.newtonroot(f, df; x0=0)[1] ≈ 2.0 # x0 = 0

    @test NewtonsMethod.newtonroot(f; x0=10)[1]  ≈ 2 # x0 = 10
    @test NewtonsMethod.newtonroot(f; x0=-10)[1]  ≈ 2 # x0 = -10
    @test NewtonsMethod.newtonroot(f; x0=0)[1]  ≈ 2 # x0 = 0

    # Testing BigFloat.
    f(x)  =  (x - 2)^2
    df(x) = 2(x - 2)
    @test BigFloat(NewtonsMethod.newtonroot(f, df)[1]) ≈ 2.0
    @test BigFloat(NewtonsMethod.newtonroot(f; tolerance=1e-13)[1]) ≈ 2.0
    @test NewtonsMethod.newtonroot(f; x0=BigFloat(0.8))[1] ≈ 2.0
    @test typeof(newtonroot(f; x0=BigFloat(0.8))[1]) === BigFloat

    # Breaking something on purpose: I present the result in the README.md
    # f(x)  =  (x - 2)^2
    # df(x) = 2(x - 2)
    # @test NewtonsMethod.newtonroot(f, df)[1] ≈ 3
    # @test NewtonsMethod.newtonroot(f)[1]  ≈ 3

end


using Shamir, Polynomials, Test

@info "Testing creation of shares"
prod_coeffs = [1234, 166, 94]
n = 6
prod_poly = Poly(prod_coeffs)
shares = Shamir.construct_shares(n, prod_poly)
@test shares == [[1, 1494], [2, 1942], [3, 2578], [4, 3402], [5, 4414], [6, 5614]]

@info "Testing Recover secret"
prod_coeffs = [1234, 166, 94]
n = 6 #total number of parties
k = 3 #min num of shares
p = 1613 #field
prod_poly = Poly(prod_coeffs)
shares = Shamir.construct_shares(n, prod_poly)
secret = Shamir.recover_secret(shares, n, k, p)
@test secret == 1234.0

@info "All tests completed"


"""
    chinese_remainder_theorem(array::Array{Int64}, remainder::Array{Int64}))

The Chinese remainder theorem is based on simultaneous congruence. In more detail, if `x`
knows the `remainder` of the Euclidean division of an `array[i]` by several integers, then
`x` can determine the remainder of the division of `array[i]` by the product of these
integers uniquely, following the condition that the divisors are pairwise coprime. For more
information see: [https://en.wikipedia.org/wiki/Chinese_remainder_theorem](https://en.wikipedia.org/wiki/Chinese_remainder_theorem)

# Arguments
- `array::Array{Int64}`: Array with elements.
- `remainder::Array{Int64}`: Number of elements

# Examples
```julia-repl
julia> import ClassicAlgorithmsCollections
julia> num = [3, 4, 5]
julia> rem = [2, 3, 1]
julia> ClassicAlgorithmsCollections.chinese_remainder_theorem(num, rem)
11
```
"""
function chinese_remainder_theorem(array::Array{Int64}, remainder::Array{Int64})
    x = 1
    n = length(array)
    while true
        i = 1
        while i < n
            if x % array[i] != remainder[i]
                break
            end
            i += 1
        end
        if i == n
            return x
        end
        x += 1
    end
end


using LinearAlgebra
using Printf

function _polynomial(a, x)
    accumulator = a[end] * one(x)
    for i in length(a)-1:-1:1
        accumulator = accumulator * x + a[i]
    end
    accumulator
end

# definition of polynom
struct Polynom{C}
    coefficients::C
    Polynom(coefficients::CC) where CC = coefficients[end] == 0 ? throw(ArgumentError("Coefficient of the highest exponent cannot be zero.")) : new{CC}(coefficients)
end

# based on https://github.com/JuliaMath/Polynomials.jl
function from_roots(roots::AbstractVector{T}; aₙ = one(T)) where {T}
    n = length(roots)
    c = zeros(T, n+1)
    c[1] = one(T)
    for j = 1:n
        for i = j:-1:1
            c[i+1] = c[i+1]-roots[j]*c[i]
        end
    end
    return Polynom(aₙ.*reverse(c))
end

(p::Polynom)(x) = _polynomial(p.coefficients, x)
degree(p::Polynom) = length(p.coefficients) - 1

function _derivativeof(p::Polynom)
    n = degree(p)
    n > 1 ? Polynom([(i - 1)*p.coefficients[i] for i in 2:n+1]) : error("Low degree of a polynomial.")
end
LinearAlgebra.adjoint(p::Polynom) = _derivativeof(p)

function Base.show(io::IO, p::Polynom)
    n = degree(p)
    a = reverse(p.coefficients)
    for (i, c) in enumerate(a[1:end-1])
        if (c != 0)
            c < 0 && print(io, " - ")
            c > 0 && i > 1 && print(io, " + ")
            print(io, "$(abs(c))x^$(n - i + 1)")
        end
    end
    c = a[end]
    c > 0 && print(io, " + $(c)")
    c < 0 && print(io, " - $(abs(c))")
end

# default optimization parameters
atol = 1e-12
maxiter = 100
stepsize = 0.95

# definition of optimization methods
abstract type RootFindingMethod end
struct Newton <: RootFindingMethod end
struct Secant <: RootFindingMethod end
struct Bisection <: RootFindingMethod end

init!(::Bisection, p, a, b) = sign(p(a)) != sign(p(b)) ? (a, b) : throw(ArgumentError("Signs at both ends are the same."))
init!(::RootFindingMethod, p, a, b) = (a, b)

function step!(::Newton, poly::Polynom, xᵢ, step_size)
    _, x̃ = xᵢ
    dp = p'
    x̃, x̃ - step_size*p(x̃)/dp(x̃)
end

function step!(::Secant, poly::Polynom, xᵢ, step_size)
    x, x̃ = xᵢ
    dpx = (p(x) - p(x̃))/(x - x̃)
    x̃, x̃ - stepsize*p(x̃)/dpx
end

function step!(::Bisection, poly::Polynom, xᵢ, step_size)
    x, x̃ = xᵢ
    midpoint = (x + x̃)/2
    if sign(p(midpoint)) == sign(p(x̃))
        x̃ = midpoint
    else
        x = midpoint
    end
    x, x̃
end

function find_root(p::Polynom, rfm=Newton, a=-5.0, b=5.0, max_iter=100, step_size=0.95, tol=1e-12)
    x, x̃ = init!(rfm, p, a, b)
    for i in 1:maxiter
        x, x̃ = step!(rfm, p, (x, x̃), step_size)
        val = p(x̃)
        @printf "x = %.5f | x̃ = %.5f | p(x̃) = %g\n" x x̃ val
        abs(val) < atol && return x̃
    end
    println("Method did not converge in $(max_iter) iterations to a root within $(tol) tolerance.")
    return x̃
end

# test code
poly = Polynom(rand(4))
p = from_roots([-3, -2, -1, 0, 1, 2, 3])
dp = p'
p(3.0), dp(3.0)

x₀ = find_root(p, Bisection(), -5.0, 5.0)
x₀ = find_root(p, Newton(), -5.0, 5.0)
x₀ = find_root(p, Secant(), -5.0, 5.0)


using SpecialFunctions

"""
Base type of FractionalDiffEq algorithms
"""
abstract type FractionalDiffEqAlgorithm end

"""
@article{
title={A predictor-corrector approach for the numerical solution of fractional differential equations},
author={Diethelm, Kai and Ford, Neville J. and Freed, Alan D.}
doi={https://doi.org/10.1023/A:1016592219341}
}
"""

"""
Predict-Evaluate-Correct-Evaluate algorithm.

For more details, please refer to [Predictor-Corrector algorithms](https://en.wikipedia.org/wiki/Predictor%E2%80%93corrector_method)

This PECE algorithm is taken from Diethelm's paper.

```tex
@article{
title={A predictor-corrector approach for the numerical solution of fractional differential equations},
author={Diethelm, Kai and Ford, Neville J. and Freed, Alan D.}
doi={https://doi.org/10.1023/A:1016592219341}
}
```
"""
struct PECE <: FractionalDiffEqAlgorithm end
#TODO: Use Richardson extrapolation to refine the PECE algorithms

"""
    FDEProblem(f, α, u0, T, h)

Define a Fractional Differential in time interval [0, T] with initial value y(0)=u₀, α-order derivative and step size h.
"""
struct FDEProblem{T<:Float64}
    f
    α::T
    u0
    T
    h::T
end

"""
    solve(FDEProblem, PECE())

After define the FDEProblem, use **PECE(Predict-Evaluate-Correct-Evaluate) algorithm** to computing the Fractional Differential Equation
"""
function solve(FDE::FDEProblem, ::PECE)
    f, α, u0, T, h = FDE.f, FDE.α, FDE.u0, FDE.T, FDE.h
    N=Int64(T/h)
    y=zeros(N+1)
    leftsum = 0

    if floor(α)==0
        leftsum=u0
    elseif floor(α)==1
        leftsum=u0+T*u0
    end

    @fastmath @inbounds @simd for n in range(0, N, step=1)
        y[Int64(n+1)]=leftsum + h^α/gamma(α+2)*f((n+1)*h, predictor(f, y, α, n, h, u0, T))+h^α/gamma(α+2)*right(f, y, α, n, h)
    end

    return y
end

function right(f, y, α, n, h)
    temp = 0

    for j in range(0, n, step=1)
        temp+=A(j, n, α)*f(j*h, y[Int64(j+1)])
    end

    return temp
end

function predictor(f, y, α, n, h, u0, T)
     predict = 0
     leftsum = 0

     if floor(α)==0
        leftsum=u0
     elseif floor(α)==1
        leftsum=u0+T*u0
     end

     @fastmath @inbounds @simd for j in range(0, n, step=1)
        predict+=B(j, n, α, h)*f(j*h, y[Int64(j+1)])
     end

     #return u0+predict
     return leftsum+predict
end

function A(j, n, α)
    if j==0
        return n^(α+1)-(n-α)*(n+1)^α
    elseif 1 ≤ j ≤ n
        return (n-j+2)^(α+1)+(n-j)^(α+1)-2*(n-j+1)^(α+1)
    elseif j==n+1
        return 1
    end
end

function B(j, n, α, h)
    return h^α/α*((n+1-j)^α-(n-j)^α)
end


export log_pdf, gradlog_pdf, hesslog_pdf

"""
    log_pdf(x)

Compute the log of the density of the univariate standard Gaussian distribution (zero mean and unitary standard deviation).
"""
log_pdf(x) = -0.5*(log(2*π) + x^2)

"""
    gradlog_pdf(x)

Compute the gradient of the log of the density of the univariate standard Gaussian distribution (zero mean and unitary standard deviation).
"""
gradlog_pdf(x) = -x

"""
    hesslog_pdf(x)

Compute the hessian of the log of the density of the univariate standard Gaussian distribution (zero mean and unitary standard deviation).
"""
hesslog_pdf(x) = -1.0


module MonteCarloFun

"
arxiv doc :     1008.4686
                1205.4446

plan:
        1) Fitting a model to data
                Fitting a straight line
                magic of Gaussian

"

#2018_10_05
#fitting a straight line

using Plots
using LinearAlgebra

x = [1., 2., 3., 4.]
y = [14, 26, 37, 43]
sigma = [1, 1, 2, 2]

plot(x, y, seriestype=:scatter, yerr=sigma, legend=false)

Y=y
N = length(Y)
A = zeros(N,2)

A[:,1] .= 1.;
A[:,2] .= x;

W = Diagonal(1. ./ sigma.^2)

MB1 = A \ Y
MB2 = (sqrt(W)*A) \ (sqrt(W)*Y)

plot(x, y, seriestype=:scatter, yerr=sigma, legend=:topleft)
plot!(x, A * MB1, label="Unweighted")
plot!(x, A * MB2, label="Weighted")

C = inv(A' * W * A)

dMB = randn((100,2)) * sqrt(C)

Angle = collect(range(0,stop=2π,length=100))
xa = [sin.(Angle) cos.(Angle)]
cMB = xa * sqrt(C)

plot([MB1[1]], [MB1[2]], seriestype=:scatter, label="Unweighted",xlabel="b", ylabel="m")
plot!([MB2[1]], [MB2[2]], seriestype=:scatter, label="Weighted",xlabel="b", ylabel="m")
plot!(MB2[1] .+ dMB[:,1], MB2[2] .+ dMB[:,2], seriestype=:scatter, label="Weighted")
plot!(MB2[1] .+ cMB[:,1], MB2[2] .+ cMB[:,2], seriestype=:scatter, label="Weighted")

plot(x, y, seriestype=:scatter, yerr=sigma, legend=:topleft)
plot!(x, A * MB2, line=(:blue, 1, 3), label="Weighted")
for i in 1:50
    dmb = MB2 + dMB[i,:]
    plot!(x, A * dmb, label= "" , line=(:black, 0.2, 2))
end
plot!()

function line_log_likehood(x,y,sigma,m,b)
    return -0.5 * sum(@. (y -(m*x+b))^2 / sigma^2)
end

bbest,mbest = MB2
line_log_likehood(x,y,sigma,mbest,bbest)

function line_lnl(b,m)
    return line_log_likehood(x, y, sigma, m, b)
end

function make_line_lnl_function(x,y,sigma)
    function f(m,b)
        return line_log_likehood(x,y,sigma,m,b)
    end
    return f
end

LineLnl = make_line_lnl_function(x,y,sigma)
LineLnl(mbest,bbest)

end # module


struct Basis
    Ψ::Matrix
    r::Int
end

function VandermondeBasis(x,r::Int;verbose::Bool=false)
    """Makes a vandermonde basis given a range and a number of basis modes"""
    Ψ = zeros((length(x),r))
    for (j,col) in enumerate(eachcol(Ψ))
        ψ = make_vander_column(j,x)
        if verbose
            println("ψ","_"*string(j)," size: ",size(ψ))
        end
        Ψ[:,j] = make_vander_column(j,x) #Each col is x^(k-1)
    end
    if verbose
        print("Final Ψ size: ",size(Ψ))
    end
    return Basis(Ψ,r) #Truncated basis matrix
end

function make_vander_column(k,col_k)
    ψ = [x^(k-1) for x in col_k]
    return ψ #Column of basis matrix
end


@testset "Clebsch-Gordan" begin
using PyCall, Test, SphericalHarmonics

sympy = pyimport("sympy")
spin = pyimport("sympy.physics.quantum.spin")

pycg(j1, m1, j2, m2, j3, m3, T=Float64) =
      spin.CG(j1, m1, j2, m2, j3, m3).doit().evalf().__float__()

@info("Testing cg1 implementation against sympy ... ")
for j1 = 0:2, j2=0:2, j3=0:4
   for m1 = -j1:j1, m2=-j2:j2, m3=-j3:j3
      @test cg1(j1,m1,j2,m2,j3,m3) ≈ pycg(j1,m1, j2,m2, j3,m3)
   end
end

@info("Check CG Coefficients for some higher quantum numbers...")
j1 = 8
j2 = 11
j3 = j1+j2
for m1 = -j1:j1, m2=-j2:j2, m3=-j3:j3
   @test cg1(j1,m1,j2,m2,j3,m3) ≈ pycg(j1,m1, j2,m2, j3,m3)
end

@info("Checking the SphH expansion in terms of CG coeffs")
# expansion coefficients of a product of two spherical harmonics in terms a
# single spherical harmonic
# see e.g. https://en.wikipedia.org/wiki/Clebsch–Gordan_coefficients
# this is the magic formula that we need
for ntest = 1:10
   # two random Ylm  ...
   l1, l2 = rand(1:10), rand(1:10)
   m1, m2 = rand(-l1:l1), rand(-l2:l2)
   # ... evaluated at random spherical coordinates
   θ = rand() * π
   φ = (rand()-0.5) * 2*π
   # evaluate all relevant Ylms (up to l1 + l2)
   Ylm = compute_y(l1+l2, cos(θ), φ)
   # evaluate the product p = Y_l1_m1 * Y_l2_m2
   p = Ylm[index_y(l1,  m1)] * Ylm[index_y(l2,m2)]
   # and its expansion in terms of CG coeffs
   p2 = 0.0
   M = m1 + m2  # all other coeffs are zero

   for L = abs(M):(l1+l2)
      p2 += sqrt( (2*l1+1)*(2*l2+1) / (4 * π * (2*L+1)) ) *
            cg1(l1,  0, l2,  0, L, 0) *
            cg1(l1, m1, l2, m2, L, M) *
            Ylm[index_y(L, M)]
   end
   print(@test p ≈ p2); print(" ")
end
println()

end # @testset


"""
    count_pythagorean_elements(array::Array{Int64,1})

Counting the number of `Pythagorean`-elements in the array, which sum is equal to
A^2 + B^2 = C^2.

# Arguments
- `array::Array{Int64,1}`: Unsorted array

# Examples
```julia-repl
julia> import ClassicAlgorithmsCollections
julia> arr = [5, 1, 3, 4, 17, 8, 15, 2, 2, 13 ,12]
julia> ClassicAlgorithmsCollections.count_pythagorean_elements(arr, sum)
3
```
"""
function count_pythagorean_elements(array::Array{Int64,1})

    # Initialize result
    ans = 0
    n = length(array)

    for i in 1:n-2
        for j in i+1:n-1
            for k in j+1:n
                AA = array[i]^2
                BB = array[j]^2
                CC = array[k]^2
                if (AA + BB == CC)
                    ans += 1
                elseif (AA + CC == BB)
                    ans += 1
                elseif (BB + CC == AA)
                    ans += 1
                end
            end
        end
    end
    return ans
end


using TuringModels

Turing.setadbackend(:reverse_diff);
#nb Turing.turnprogress(false);

# Can't really set a Uniform[-Inf,Inf] on σ

# Turing model
@model m8_4(y) = begin
    α₁ ~ Uniform(-3000, 1000)
    α₂ ~ Uniform(-1000, 3000)
    σ ~ Truncated(Cauchy(0,1), 0, Inf)

    for i ∈ 1:length(y)
        y[i] ~ Normal(α₁ + α₂, σ)
    end
end

# Observations

y = rand(Normal(0,1), 100);

# Sample

chns = sample(m8_4(y), Turing.NUTS(0.95), 2000);

# Results rethinking

m84rethinking = "
         mean      sd     5.5%   94.5% n_eff Rhat
 a1    -861.15 558.17 -1841.89  -31.04     7 1.43
 a2     861.26 558.17    31.31 1842.00     7 1.43
 sigma    0.97   0.07     0.89    1.09     9 1.17
";

# Describe the posterior samples

describe(chns)

# End of `08/m8.4t.jl`


# Tests for Dirichlet distribution

using Distributions
using Base.Test

d = Dirichlet(3, 2.0)

@test dim(d) == 3
@test d.alpha == [2.0, 2.0, 2.0]
@test d.alpha0 == 6.0

@test_approx_eq mean(d) fill(1.0/3, 3)
@test_approx_eq cov(d) [8 -4 -4; -4 8 -4; -4 -4 8] / (36 * 7)
@test_approx_eq var(d) diag(cov(d))

@test_approx_eq pdf(d, [0.2, 0.3, 0.5]) 3.6
@test_approx_eq pdf(d, [0.4, 0.5, 0.1]) 2.4
@test_approx_eq logpdf(d, [0.2, 0.3, 0.5]) log(3.6)
@test_approx_eq logpdf(d, [0.4, 0.5, 0.1]) log(2.4)

x = rand(d, 100)
p = pdf(d, x)
lp = logpdf(d, x)
for i in 1 : size(x, 2)
	@test_approx_eq lp[i] logpdf(d, x[:,i])
	@test_approx_eq p[i] pdf(d, x[:,i])
end

v = [2.0, 1.0, 3.0]
d = Dirichlet(v)

@test dim(d) == length(v)
@test d.alpha == v
@test d.alpha0 == sum(v)

@test_approx_eq mean(d) v / sum(v)
@test_approx_eq cov(d) [8 -2 -6; -2 5 -3; -6 -3 9] / (36 * 7)
@test_approx_eq var(d) diag(cov(d))

@test_approx_eq pdf(d, [0.2, 0.3, 0.5]) 3.0
@test_approx_eq pdf(d, [0.4, 0.5, 0.1]) 0.24
@test_approx_eq logpdf(d, [0.2, 0.3, 0.5]) log(3.0)
@test_approx_eq logpdf(d, [0.4, 0.5, 0.1]) log(0.24)

x = rand(d, 100)
p = pdf(d, x)
lp = logpdf(d, x)
for i in 1 : size(x, 2)
	@test_approx_eq p[i] pdf(d, x[:,i])
	@test_approx_eq lp[i] logpdf(d, x[:,i])
end

# Sampling

x = rand(d)
@test isa(x, Vector{Float64})
@test length(x) == 3

x = rand(d, 10)
@test isa(x, Matrix{Float64})
@test size(x) == (3, 10)

# Test MLE

n = 10000
x = rand(d, n)
x = x ./ sum(x, 1)

r = fit_mle(Dirichlet, x)
@test_approx_eq_eps r.alpha d.alpha 0.25

r = fit_mle(Dirichlet, x, fill(2.0, n))
@test_approx_eq_eps r.alpha d.alpha 0.25


"""
    rekursif(f,a,b,n)

berisi program untuk mencari integral numerik dari fungsi
`f(x)` pada interval `[a,b]` menggunakan Aturan Rekursif Trapesium.
Program ini secara default berisi 4 masukan, yaitu fungsi `f(x)`,
titik ujung interval `[a,b]` dan banyaknya iterasi `n`

# Example
```jl
julia> f(x) = 1/x;

julia> T = rekursif(f,1,5,5)
6×1 Array{Float64,2}:
 2.4
 1.8666666666666667
 1.6833333333333333
 1.628968253968254
 1.6144063238103485
 1.6106858960792332
```
return solusi integral numerik pada setiap iterasi `T`
"""
function rekursif(f,a,b,n)
  M = 1;
  h = b-a;
  T = Array{Float64}(undef,n+1,1)
  T[1] = h/2*(f(a)+f(b))
  for j = 1:n
    h = h/2;
    s = 0;
    for k = 1:M
      x = a+h*(2*k-1);
      s = s+f(x);
    end
    M = 2*M;
    T[j+1] = T[j]/2+h*s;
  end
  return T
end


using Distributions, Plots, LaTeXStrings; pyplot()

dist = TriangularDist(0,2,1)
xGrid = 0:0.01:2
uGrid = 0:0.01:1

p1 = plot( xGrid, pdf.(dist,xGrid), c=:blue,
		xlims=(0,2), ylims=(0,1.1),
		xlabel="x", ylabel="f(x)")
p2 = plot( xGrid, cdf.(dist,xGrid), c=:blue,
		xlims=(0,2), ylims=(0,1),
		xlabel="x", ylabel="F(x)")
p3 = plot( uGrid,quantile.(dist,uGrid), c=:blue,
		xlims=(0,1), ylims=(0,2),
		xlabel="u", ylabel=(L"F^{-1}(u)"))

plot(p1, p2, p3, legend=false, layout=(1,3), size=(1200, 400))


module ZernikePolynomials

export mn2OSA, mn2Noll, OSA2mn, Noll2mn, Noll2OSA, OSA2Noll, Zernike, Zernikecoefficients, evaluateZernike, normalization

# Zernike polynomial-based functions
# Some formulas from Wikipedia, some from: "Standards for Reporting the Optical Aberrations of Eyes", Journal of Refractive Surgery Volume 18 September/October 2002
# Reinier Doelman, 23-12-2018

"""
    mn2OSA(m::Int,n::Int)

Convert the integer pair (n,m) that defines the Zernike polynomial Z_n^m(ρ,θ) to the sequential OSA/ANSI stardard index number.

Returns NaN for invalid integer pairs.

See also: [`mn2Noll`], [`OSA2mn`]

Source: "Standards for Reporting the Optical Aberrations of Eyes", Journal of Refractive Surgery Volume 18 September/October 2002

# Example:
```julia-repl
julia> [mn2OSA(n,m) for n in 0:4, m in -5:5]
```
"""
function mn2OSA(m::Int,n::Int)
  if n < abs(m) || isodd(n-m)
    return NaN
  else
    return Int((1//2)*(n*(n+2)+m))
  end
end

"""
    mn2Noll(m::Int,n::Int)

Convert the integer pair (n,m) that defines the Zernike polynomial Z_n^m(ρ,θ) to the sequential Noll index number.

Returns NaN for invalid integer pairs.

See also: [`mn2OSA`], [`Noll2mn`]

Source: (https://en.wikipedia.org/wiki/Zernike_polynomials)

# Example:
```julia-repl
julia> [mn2Noll(n,m) for n in 0:4, m in -5:5]
```
"""
function mn2Noll(m::Int,n::Int)
  if n < abs(m) || isodd(n-m)
    return NaN
  else
    if m > 0 && (mod(n,4) ∈ (0,1))
      p = 0
    elseif m < 0 && (mod(n,4) ∈ (2,3))
      p = 0
    elseif m ≥ 0 && (mod(n,4) ∈ (2,3))
      p = 1
    elseif m ≤ 0 && (mod(n,4) ∈ (0,1))
      p = 1
    else
      error("Invalid combination of (m,n) in Noll indexing")
    end
    return Int((1//2)*n*(n+1) + abs(m) + p)
  end
end

"""
    OSA2mn(j::Int)

Convert the sequential OSA/ANSI stardard index number j to the integer pair (n,m) that defines the Zernike polynomial Z_n^m(ρ,θ).

See also: [`Noll2mn`], [`mn2OSA`], [`OSA2Noll`]

Source: "Standards for Reporting the Optical Aberrations of Eyes", Journal of Refractive Surgery Volume 18 September/October 2002

# Example:
```julia-repl
julia> [OSA2mn(j) for j in 0:10]
```
"""
function OSA2mn(j::Int)
  n = Int(ceil((-3 + sqrt(9+8j))/2))
  m = 2j-n*(n+2)
  return (Int(m),Int(n))
end

"""
    Noll2mn(j::Int)

Convert the Noll index number j to the integer pair (n,m) that defines the Zernike polynomial Z_n^m(ρ,θ).

See also: [`OSA2mn`], [`mn2Noll`], [`Noll2OSA`]

Source: (https://en.wikipedia.org/wiki/Zernike_polynomials)

# Example:
```julia-repl
julia> [OSA2mn(j) for j in 0:10]
```
"""
function Noll2mn(j::Int)
  n = Int(ceil((-3 + sqrt(1+8j))/2))
  jr = j - Int(n*(n+1)/2)
  if mod(n,4) ∈ (0,1)
    m1 = jr
    m2 = -(jr-1)
    if iseven(n-m1)
      m = m1
    else
      m = m2
    end
  else # mod(n,4) ∈ (2,3)
    m1 = jr-1
    m2 = -(jr)
    if iseven(n-m1)
      m = m1
    else
      m = m2
    end
  end
  return (m,n)
end

"""
    Noll2OSA(j::Int)

Convert the Noll index number j to the OSA/ANSI stardard index number for Zernike polynomials.

See also: [`OSA2Noll`], [`Noll2mn`]

# Example:
```julia-repl
julia> [Noll2OSA(OSA2Noll(j)) for j = 0:10]
```
"""
function Noll2OSA(j::Int)
  return mn2OSA(Noll2mn(j)...)
end

"""
    OSA2Noll(j::Int)

Convert OSA/ANSI stardard index number to the Noll index number for Zernike polynomials.

See also: [`OSA2Noll`], [`Noll2mn`]

# Example:
```julia-repl
julia> [Noll2OSA(OSA2Noll(j)...) for j = 0:10]
```
"""
function OSA2Noll(j::Int)
  return mn2Noll(OSA2mn(j)...)
end

"""
    R(m::Int,n::Int)

Obtain the function ρ -> R_n^|m|(ρ), where R is the radial polynomial in Zernike polynomials

# Example:
```julia-repl
julia> R(1,1)
```
"""
function R(m::Int,n::Int) # radial polynomial
  p = s -> ((-1)^s * factorial(n-s))/ (factorial(s)*factorial(Int(0.5*(n+abs(m))-s))*factorial(Int(0.5*(n-abs(m))-s)))
  f = x -> sum([p(s)*x.^(n-2s) for s in 0:Int((n-abs(m))/2)])
  return f
end

function normalization(m::Int,n::Int) # normalization constant of the zernike polynomial
  δ = (x,y) -> Int(x == y)
  c = sqrt(2(n+1)/(1+δ(m,0)))
  return c
end

"""
    Zernike(m::Int,n::Int;coord=:polar)

Obtain the function (ρ,θ) -> Z_n^m(ρ,θ), where Z is the Zernike polynomial with coefficients n and m. ρ is the radius and θ the angle.

If coord=:cartesian the function is (x,y) -> Z_n^m(x,y) in Cartesian coordinates.

# Example:
```julia-repl
julia> Zernike(1,1)
julia> Zernike(1,1,coord=:polar)
julia> Z = Zernike(1,1,:cartesian)
julia> Z(0.5,0.2)
```
"""
function Zernike(m::Int,n::Int;coord=:polar)
  δ = ρ -> (abs(ρ) <= 1.)
  if m ≥ 0
    Z = (ρ,θ) -> normalization(m,n)*R(m,n)(ρ) * cos(m*θ) * δ(ρ)
  else
    Z = (ρ,θ) -> -normalization(m,n)*R(m,n)(ρ) * sin(m*θ) * δ(ρ)
  end

  if coord == :cartesian
    g = (x,y) -> (sqrt(x.^2 + y.^2), atan(y,x))
    return (x,y) -> Z(g(x,y)...)
  elseif coord == :polar
    return Z
  else
    error("Unrecognized coordinate system")
  end
end

"""
    Zernike(j::Int;index=:OSA,coord=:polar)

Obtain the function (ρ,θ) -> Z_j(ρ,θ), where Z is the Zernike polynomial with sequential index j, according to the indexing by :OSA or :Noll.

If coord=:cartesian the function is (x,y) -> Z_j(x,y) in Cartesian coordinates.

# Example:
```julia-repl
julia> Zernike(1)
julia> Zernike(1,coord=:polar)
julia> Z = Zernike(5,index=:Noll,coord=:cartesian)
julia> Z(0.5,0.2)
```
"""
function Zernike(j::Int;index=:OSA,coord=:polar)
  if index == :OSA
    return Zernike(OSA2mn(j)...,coord=coord)
  elseif index == :Noll
    return Zernike(Noll2mn(j)...,coord=coord)
  else
    error("Unknown Zernike sequential index")
  end
end

"""
    Zernikecoefficients(phase::AbstractArray{Float64,2}, J::Vector{Int}; index=:OSA)

Compute the Zernike coefficients (OSA normalization) that in a least-squares sense optimally describe the phase.
The Zernike polynomials used are specified with an index vector J, according to OSA indexing.

"""
function Zernikecoefficients(phase::AbstractArray{Float64,2}, J::Vector{Int}; index=:OSA)
  s = size(phase)
  X = range(-1.,stop=1,length=s[1])
  Y = range(-1.,stop=1,length=s[2])

  D = [[Zernike(j,coord=:cartesian,index=index)(x,y) for x in X, y in Y] for j in J ]
  G = cat([D[i][:] for i in 1:length(J)]...; dims=2)
  return G\phase[:]
end

"""
    Zernikecoefficients(x::AbstractArray{<: AbstractFloat,1}, phase::AbstractArray{Float64,2}, J::Vector{Int}; index=:OSA)

Compute the Zernike coefficients (OSA normalization) that in a least-squares sense optimally describe the phase.
The Zernike polynomials used are specified with an index vector J, according to OSA indexing.

"""
function Zernikecoefficients(X::AbstractArray{<: AbstractFloat,1}, phase::AbstractArray{Float64,2}, J::Vector{Int}; index=:OSA)
  s = size(phase)
  if !(s[1] == s[2] && s[1] == length(X))
    error("Non-matching size")
  end

  D = [[Zernike(j,coord=:cartesian,index=index)(x,y) for x in X, y in X] for j in J ]
  G = cat([D[i][:] for i in 1:length(J)]...; dims=2)
  return G\phase[:]
end

"""
    evaluateZernike(N::Int, J::Vector{Int}, coefficients::Vector{Float64}; index=:OSA)

Evaluate the Zernike polynomials on an N-by-N grid as specified by the Zernike coefficients of the polynomials J

# Example:
```julia-repl
julia> W = evaluateZernike(64,[5, 6],[0.3, 4.1])
```
"""
function evaluateZernike(N::Int, J::Vector{Int}, coefficients::AbstractArray{Float64,1}; index=:OSA)
  X = range(-1.,stop=1,length=N)
  Y = range(-1.,stop=1,length=N)

  D = [[Zernike(j,coord=:cartesian,index=index)(x,y) for x in X, y in Y] for j in J ]
  return reduce(+,map(*,D,coefficients))
end

"""
    evaluateZernike(x::AbstractArray{<: AbstractFloat,1}, J::Vector{Int}, coefficients::Vector{Float64}; index=:OSA)

Evaluate the Zernike polynomials on a grid as specified by the Zernike coefficients of the polynomials J, on a grid with points in x

# Example:
```julia-repl
julia> W = evaluateZernike(64,[5, 6],[0.3, 4.1])
```
"""
function evaluateZernike(X::AbstractArray{<: AbstractFloat,1}, J::Vector{Int}, coefficients::Vector{Float64}; index=:OSA)
  D = [[Zernike(j,coord=:cartesian,index=index)(x,y) for x in X, y in X] for j in J ]
  return reduce(+,map(*,D,coefficients))
end

function evaluateZernike(n::Int, J::Int, coefficients::Float64; index=:OSA)
  return evaluateZernike(n, [J], [coefficients]; index=index)
end

function evaluateZernike(X::AbstractArray{<: AbstractFloat,1}, J::Int, coefficients::Float64; index=:OSA)
  return evaluateZernike(X, [J], [coefficients]; index=index)
end

end # module


# Model Lion & Zebra (predetor vs. prey) over the course of 100 years.

# Using Lotka-Volterra equation, we have:
# dx/dt=x(α−βy)  dy/dt=−y(γ−δx)

start_time = 0;
end_time = 100;

# How much time passes between each successive calculation
time_step = 1/4 # In years
end_step = int(((end_time - start_time)/ time_step));

# The number of zebras when the simulation is started
initial_zebras = 30000;

# The number of lions when the simulation is started
initial_lions = 15;

# The number of zebras that must be killed for a lion to be born. (Zebras / Lion)

zebras_per_lion = 1000;

# The chance a zebra will die when a zebra lion cross paths
fighting_chance = 0.50;

meeting_chance = 0.02;

zebra_growth_rate = 0.20;

lion_death_rate = 0.10;

# Initialization

zebras_over_time = fill(0.0, end_step+1);
lions_over_time = fill(0.0, end_step+1);
model_time = fill(0.0, end_step+1);

zebras = initial_zebras;
lions = initial_lions;

zebras_over_time[1]	= zebras;
lions_over_time[1] = lions;

# Run the model

for sim_step = 1:end_step

	sim_time = start_time + sim_step * time_step;
	model_time[sim_step] = sim_time;

	# First we must calculate our flows (our rates)
	zebra_births = zebras * zebra_growth_rate;
	zebra_deaths  = min(zebras, meeting_chance*fighting_chance*zebras*lions);

	lion_births = 1/zebras_per_lion * zebra_deaths;
	lion_deaths = lions * lion_death_rate;

	# Update the stock levels
	zebras = zebras + zebra_births - zebra_deaths;
	lions = lions + lion_births - lion_deaths

	# Stock values always update in the next time step
	zebras_over_time[sim_step+1] = zebras;
	lions_over_time[sim_step+1] = lions;
end

println("Time, Zebras (Thousands), Lions");

for i = 1:end_step
	print(model_time[i]);
	print(",");
	print(zebras_over_time[i]/1000);
	print(",");
	println(lions_over_time[i]);
end


using NumericalMethodsforEngineers, DataFrames, Plots

@sym begin
  LagrangePolynomial(xi_, yi_) := Module([N, sum, num, den],
    begin
      N = Length(xi)
      sum = 0
      For( i=1, i <= N, Increment(i),
        begin
          num = 1
          den = 1
          For( j=1, j <= N, Increment(j),
            begin
              If(j != i, num *= (x-xi[j]))
              If(j != i, den *= (xi[i]-xi[j]))
            end
          )
          sum += yi[i] * num/den
        end
      ),
      Return(Simplify(sum))
    end
  )
end

println()
@sym begin
  xi = [1, 3, 6]
  yi = [1, 5, 10]
  # LagrangePolynomial defined in ../../src/ch07/sym_fcns.jl
  Q(x_) = LagrangePolynomial(xi, yi)
  SetJ(qs, ToString(Q(x)))
  Print("Q(x) = $(Q(x))\n")
end

@sym Print("Q(4.5) = $(N(Q(4.5)))\n");
println("\n")

ProjDir = dirname(@__FILE__)
cd(ProjDir) #do

  x = [1.0, 3.0, 6.0]
  y = [1.0, 5.0, 10.0]
  xi = [2.0, 4.5]
  (dfin, dfxi) = lagrangianpolynomial(length(x), x, y, xi)

  dfin |> display
  println()

  dfxi |> display
  println()

  @eval f(x) = $(Meta.parse(qs))

  xint = 1:1:10
  p = plot(xint, f.(xint), line=(:path, 1), label="interpolated curve")
  scatter!(p, dfin[:x], dfin[:y], marker=(:circle, 4), label="input points", color=:blue)
  scatter!(p, dfxi[:xi], dfxi[:yi], marker=(:star, 8), color=:red, label="interpolated points")

  plot(p)
  savefig("ex.5.1_symata.png")
  #gui()

  #end


using NumericalMethodsforEngineers, Test

a = [1. 2. -2.; 2. 5. -4.; -2. -4. 5.]

println("\n|a[1, 1]| = $(det(a[1, 1]))\n")
println("|a[1:2, 1:2]| = $(det(a[1:2, 1:2])))\n")
println("|a| = $(det(a))")
println("\ncholesky(a):")
cholesky(a) |> display

a = [16. 4. 8.; 4. 5. -4.; 8. -4. 22.]

println("\n|a[1, 1]| = $(det(a[1, 1]))\n")
println("|a[1:2, 1:2]| = $(det(a[1:2, 1:2])))\n")
println("|a| = $(det(a))")

println("\ncholesky(a) - upper:")
upper = cholesky(a).U
upper |> display

println("\ncholesky(a) - lower:")
lower = cholesky(a).L
lower |> display

println("\nlower * upper:")
lower * upper |> display

@test a == lower * upper


module Factorions
using Memoize, ProgressMeter, Distributed

export is_factorion, fact_upper_bound, find_factorions, string_base
export find_in_range, find_in_parallel

@memoize function fast_fact(n::Int)
    return factorial(n)
end

"""
`sum_fact_digs(n)` returns the sum of the factorials of the
base-10 digits of `n`.
"""
function sum_fact_digs(n::Int, b::Int = 10)
    digs = digits(n, base = b)
    sum(map(fast_fact, digs))
end

"""
`is_factorion(n,b=10)` determines if `n` is a base-`b` *factorion*;
that is, if `n` is equal to the sum of the factorials of its base-`b` digits.
"""
function is_factorion(n::Int, b::Int = 10)::Bool
    if n <= 0
        return false
    end
    return n == sum_fact_digs(n, b)
end

"""
`fact_upper_bound(b=10)` returns an upper bound on the number of digits in
a base-`b` factorion.
"""
function fact_upper_bound(b::Int = 10)
    n::Int = 1
    fmax = fast_fact(b - 1)
    while b^(n - 1) < fmax * n
        n += 1
    end
    return n
end

function find_factorions(b::Int = 10)
    stop = b^fact_upper_bound(b)
    println("Trying $stop possibilities (up to $(fact_upper_bound(b)) digits)")
    list = Int[]
    PM = Progress(stop)
    for n = 1:stop
        if is_factorion(n, b)
            push!(list, n)
        end
        next!(PM)
    end
    for x in list
        println("$(string_base(x,b)) = $x")
    end

    return Set(list)

end

function find_in_range(rng, b::Int = 10)
    list = Int[]
    for x in rng
        if is_factorion(x, b)
            push!(list, x)
        end
    end
    return Set(list)
end

function find_in_parallel(b::Int = 10)
    stop = b^fact_upper_bound(b)
    println("Trying $stop possibilities (up to $(fact_upper_bound(b)) digits)")
    np = Threads.nthreads()
    #println(addprocs(np))
    results = Vector{Any}(undef, np)

    for j = 1:np
        rng = j:np:stop
        @info "Spawning process #$j"
        results[j] = @spawnat j + 1 find_in_range(rng, b)
    end
    sets = fetch.(results)
    X = union(sets...)
    for n in sort(collect(X))
        println("$n \t $(string_base(n,b))")
    end
    return union(sets...)
end

"""
`string_base(n,b)` renders the nonnegative integer `n` in
base `b` (we require `1 < b < 37`).
"""
function string_base(n::Int, b::Int = 10)
    symbs = "0123456789abcdefghijklmnopqrstuvwxyz"
    digs = digits(n, base = b)
    chars = [symbs[k+1] for k in digs]
    return join(reverse(chars)) * "_$b"
end

end # module


n = 2
m = 1

A = [1.0 1.0; 0.0 1.0]
B = [0.0; 1.0]

Q = [1.0 0.0; 0.0 1.0]
R = [1.0]

# x_init = [1.0; 0.0]

nz = n + m + n

function unpack(z)
	x2 = z[1:n]
	u1 = z[n .+ (1:m)]
	y2 = z[n + m .+ (1:n)]

	return x2, u1, y2
end

z0 = rand(nz)

function lagrangian(z, x1)
	x2, u1, y2 = unpack(z)

	L = 0.0

	# objective
	L += transpose(x2) * Q * x2
	L += (transpose(u1) * R * u1)[1]

	# constraints
	L += transpose(y2) * (A * x1 + B * u1[1] - x2)

	return L
end

@variables z_sym[1:nz]
@variables x1_sym[1:n]

L = lagrangian(z_sym, x1_sym)
L = simplify.(L)

dL = Symbolics.gradient(L, z_sym)
ddL = Symbolics.hessian(L, z_sym)
ddLx1 = Symbolics.jacobian(dL, x1_sym)

L_grad = eval(Symbolics.build_function(dL, z_sym, x1_sym)[1])
L_hess = eval(Symbolics.build_function(ddL, z_sym, x1_sym)[1])
L_hess_x1 = eval(Symbolics.build_function(ddLx1, z_sym, x1_sym)[1])

function solve(z0, x1)
	z = copy(z0)

	∇L = L_grad(z, x1)

	println()
	for i = 1:10
		r_norm = norm(∇L)
		println("iter ($i)")
		println("	grad norm: $(r_norm)")
		r_norm < 1.0e-8 && break

		∇²L = L_hess(z, x1)

		Δ = ∇²L \ ∇L

		α = 1.0
		ẑ = z - α * Δ

		∇L̂ = L_grad(ẑ, x1)

		iter = 1
		while norm(∇L̂) > norm(∇L)
			α *= 0.5
			ẑ = z - α * Δ
			∇L̂ = L_grad(ẑ, x1)
			iter += 1

			if iter > 100
				break
			end
		end

		z = ẑ
		∇L = ∇L̂
	end

	return z, ∇L, L_hess(z, x1)
end

z_sol, ∇L_sol, ∇²L_sol = solve(z0, x1)
K_direct = (∇²L_sol \ L_hess_x1(z_sol, x1))[n .+ (1:m), :]

# LQR solution
K, P = tvlqr([A], [B], [Q, Q], [R[1,1]])

norm(K_direct - K[1])


# Helpful packages for working with images and factorizations
# using Pkg; Pkg.add("Images")
# using Pkg; Pkg.add("ImageMagick") # And this allows us to load JPEG-encoded images
using Images, LinearAlgebra

# ------------------------------------------------------------------------------------------
# ### Using a SVD to compress an image
#
# In this exercise, we'll use a singular value decomposition (SVD) to compress an image --
# so that we can store an image without keeping around "unnecessary" information.
#
# To start, let's define a singular value decomposition. In a SVD, we take a matrix $A$ and
# factorize it so that
#
# $$A = USV^T$$
#
# where matrices $U$ and $V$ are unitary and hold our singular vectors. Matrix $S$ is
# diagonal and stores our singular values in decreasing order from top/left to bottom/right.
#
# In Julia, our images are stored as arrays, so we can think of `yellowbanana` as a matrix
# ------------------------------------------------------------------------------------------

yellowbanana = load("images/104_100.jpg")

size(yellowbanana)

yellowbanana[60,50] # Each element in the array is a color

# ------------------------------------------------------------------------------------------
# That means we can take the SVD of this image. So, we can store this picture of a banana as
# sets of singular vectors and singular values.
#
# **The reason this is important** is that we'll find that we do **not** need to keep track
# of *all* the singular vectors and *all* the singular values to store an image that still
# looks like a banana! This means we can choose to keep only the important information,
# throw away the rest, and thereby "compress" the image.
#
# Working with grayscale images is a bit easier, so let's work with the gray version of this
# banana.
# ------------------------------------------------------------------------------------------

banana = Gray.(yellowbanana)

# ------------------------------------------------------------------------------------------
# Note that `banana` is a (100 x 100) matrix that stores a pixel -- information about the
# color -- at each entry.
# ------------------------------------------------------------------------------------------

size(banana)

# ------------------------------------------------------------------------------------------
# If we take the `channelview` of `banana`, we'll see the numbers that describe the color at
# each point in the array:
# ------------------------------------------------------------------------------------------

channelview(banana)

# ------------------------------------------------------------------------------------------
# Note that you can call `Gray` on the `channelview` of an image to display the image!
# ------------------------------------------------------------------------------------------

Gray.(channelview(banana))

# ------------------------------------------------------------------------------------------
# **So how can we use a SVD to determine what information in an image is really important?**
#
# The singular values tell us!
#
# If we have matrices $U$, $S$, and $V$ from our image, we can rebuild that image with the
# matrix product $USV^T$.
#
# Taking this matrix product is the same as adding together the outer products of each
# corresponding pair of vectors from $U$ and $V$, scaled by a singular value ($\sigma$) from
# $S$. In other words, for a (100 x 100) pixel image,
#
# $$A_{image} = USV^T = \sum_{i = 1}^{100} \sigma_i \mathbf{u_i}\mathbf{v_i'} $$
#
# Every outer product $u_i * v_i'$ creates a (100 x 100) matrix. Here we're summing together
# one hundred (100 x 100) matrices in order to create the original matrix $A_{image}$. The
# matrices at the beginning of the series -- those that are scaled by **large** singular
# values -- will be **much** more important in recreating the original matrix $A_{image}$.
#
# This means we can approximate $A_{image}$ as
#
# $$A_{image} \approx \sum_{i = 1}^{n} \sigma_i \mathbf{u_i}\mathbf{v_i'}$$
#
# where $n < 100$.
#
# For example, if we rebuild our image of a banana using only 30 (instead of all 100)
# singular values, we get
#
#
#
# Using 10, 5, and then 3 singular values, we get
#
#
#
#
#
# #### Exercise
#
# Write a function called `compress_image`. Its input arguments should be an image and the
# factor by which you want to compress the image. A compressed grayscale image should
# display when `compress_image` is called.
#
# For example,
#
# ```julia
# compress_image("images/104_100.jpg", 33)
# ```
#
# will return a compressed image of a grayscale banana built using 3 singular values. (This
# image has 100 singular values, so use `fld(100, 33)` to determine how many singular values
# to keep. `fld` performs "floor" division.)
#
# *Hints*:
#
# * Perform the SVD on the `channelview` of a grayscale image.
# * In an empty input cell, execute `?svd` to find a function that wil perform an SVD for
# you.
#
# ### Solution 1
# ------------------------------------------------------------------------------------------

function compress_image(img, factor)
    img = channelview(Gray.(img))
    img_svd = svd(img)
    U, S, V = img_svd.U, img_svd.S, img_svd.V
    num_sv = size(S)[1]
    use_sv = fld(num_sv, factor) # `fld` returns an integer
    # Reconstruct image via matrix multiplies
    compressed_image = U[:, 1:use_sv] * Diagonal(S[1:use_sv]) * V[:, 1:use_sv]'
    Gray.(compressed_image)
end

compress_image(banana, 10)

# ------------------------------------------------------------------------------------------
# ### Solution 2
# ------------------------------------------------------------------------------------------

function compress_image(img, factor)
    img = channelview(Gray.(img))
    img_svd = svd(img)
    U, S, V = img_svd.U, img_svd.S, img_svd.V
    num_sv = size(S)[1]
    use_sv = fld(num_sv, factor) # `fld` returns an integer
    # Reconstruct image with a `for` loop that sequentially performs outer products
    compressed_image = fill(zero(N0f8), (num_sv, num_sv))
    for i in 1:use_sv
       compressed_image += S[i] * U[:, i] * V[:, i]'
    end
    Gray.(compressed_image)
end

compress_image(banana, 20)


#
# Test script for least square solver
#
# author: Atsushi Sakai
#

using ArgMin

# ====Least square ====

function test_least_square()
	println("test_least_square")
	A = [ 0.97  1.86  0.41;
		 1.23  2.18  0.53;
		 0.80  1.24  0.62;
		 1.29  0.98  0.51;
		 1.10  1.23  0.69;
		 0.67  0.34  0.54;
		 0.87  0.26  0.62;
		 1.10  0.16  0.48;
		 1.92  0.22  0.71;
		 1.29  0.12  0.62];
	m, n = size(A)
	b = 1e3 * ones(m);
	xhat = solve_least_square(A, b)
	# println(xhat)

	@test xhat[1] ≈ 62.0766 atol=0.01
	@test xhat[2] ≈ 99.985 atol=0.01
	@test xhat[3] ≈ 1442.84 atol=0.01
end

function test_multi_objective_least_square()
	println("test_multi_objective_least_square")
	As = [[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0],
   		  [9.0, 2.0, 1.0, 4.0, 3.0, 1.6, 1.0, 2.0, -9.5, -19.0]];
	bs = [[1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0, 10.0],
   		  [5.0, 2.0, 8.0, -4.0, 1.0, 0.6, 17.0, 21.0, -9.0, -1.0]];
	lambdas = [0.1, 1.0]

	xhat = solve_multi_objective_least_square(As, bs, lambdas)

	@test xhat[1] ≈ 0.405977 atol=0.01
end

function test_solve_constrained_least_square()
	println("test_solve_constrained_least_square")
	A = randn(10,5)
	b = randn(10)
	C = randn(2,5)
	d = randn(2)
	xhat = solve_least_square(A,b,C,d);
end

function test_solve_nonlinear_least_square_with_newton_raphson()
	println("test_solve_nonlinear_least_square_with_newton_raphson")

	f(x) = (exp(x)-exp(-x)) / (exp(x)+exp(-x))
	Df(x) = 4.0 / (exp(x) + exp(-x))^2;

	xhat = solve_nonlinear_least_square_with_newton_raphson(f,Df,0.95)
end

function test_solve_nonlinear_least_square_with_gauss_newton()
	println("test_solve_nonlinear_least_square_with_gauss_newton")
	f(x) = (exp.(x) - exp.(-x)) ./ (exp.(x) + exp.(-x))
	Df(x) = 4 ./ (exp.(x) + exp.(-x)).^2
	xhat = solve_nonlinear_least_square_with_gauss_newton(f,Df,[1.15])
end

function test_solve_nonlinear_least_square_with_levenberg_marquardt()
	println("test_solve_nonlinear_least_square_with_levenberg_marquardt")
	f(x) = (exp.(x) - exp.(-x)) ./ (exp.(x) + exp.(-x))
	Df(x) = 4.0 ./ (exp.(x) + exp.(-x)).^2
	xhat = solve_nonlinear_least_square_with_levenberg_marquardt(f,Df,[0.95], 1.0)
end

function test_solve_constrained_nonlinear_least_square_with_augmented_lagragian()
	println("test_solve_constrained_nonlinear_least_square_with_augmented_lagragian")
	f(x) = [ x[1] + exp(-x[2]), x[1]^2 + 2*x[2] + 1 ]
	Df(x) = [ 1.0 -exp(-x[2]);
  			2*x[1]  2 ]
	g(x) = [ x[1] + x[1]^3 + x[2] + x[2]^2 ]
	Dg(x) = [ 1 + 3*x[1]^2  1 + 2*x[2] ]
	x, z, hist = solve_constrained_nonlinear_least_square_with_augmented_lagragian(f, Df, g, Dg, [0.5, -0.5], 1.0)
end

test_least_square()
test_multi_objective_least_square()
test_solve_constrained_least_square()
test_solve_nonlinear_least_square_with_newton_raphson()
test_solve_nonlinear_least_square_with_gauss_newton()
test_solve_nonlinear_least_square_with_levenberg_marquardt()
test_solve_constrained_nonlinear_least_square_with_augmented_lagragian()


using LinearAlgebra

# k_local(Elastic Modulus,thickness,height,length,number of nodes,row/column one, row/column two)

#creates local stiffness matrix and maps to global
function k_local(k,l,n,rc_1,rc_2)
    k_global = zeros(n,n)
    k_global[rc_1,rc_1] = k
    k_global[rc_1,rc_2] = -k
    k_global[rc_2,rc_1] = -k
    k_global[rc_2,rc_2] = k
    return k_global
end

# constants from problem statement
k1 = 40
k2 = 30
k3 = 60
k4 = 50
# combining all globally mapped matrices
kG = k_local(k1,l,5,1,2)+
     k_local(k2,l,5,2,4)+
     k_local(k3,l,5,3,4)+
     k_local(k4,l,5,4,5)

# Adjusts Global Matrix for fixed-end reactions
function adj_fix(kG,x,n)
    default = zeros(Float64,1,n)
    default[x] = 1
    kG[x,:] = default
    kG[:,x] = transpose(default)
end
adj_fix(kG,1,5)
adj_fix(kG,3,5)

# Force Vector
f = Float64[0 0 0 100 60]
F = transpose(f)

# Displacement vector
u = kG\F
println("Nodal displacement vector (mm):")
display(u)


@testset "Fornberg FD weights tests:" begin

    # all stencils provide derivatives at point labelled with 0

    # centered 1st order derivatives i.e. ...,-1,0,1,...
    @test Jecco.calculate_weights(1,2,1) == [-1//2, 0//1, 1//2]
    @test Jecco.calculate_weights(1,4,2) == [1//12, -2//3, 0//1, 2//3, -1//12]
    @test Jecco.calculate_weights(1,6,3) == [-1//60, 3//20, -3//4, 0//1, 3//4, -3//20, 1//60]
    @test Jecco.calculate_weights(1,8,4) == [1//280, -4//105, 1//5, -4//5, 0//1,
                                       4//5, -1//5, 4//105, -1//280 ]

    # forward 1st order derivatives i.e. 0,1,...
    @test Jecco.calculate_weights(1,1,0) == [-1//1, 1//1]
    @test Jecco.calculate_weights(1,2,0) == [-3//2, 2//1, -1//2]
    @test Jecco.calculate_weights(1,3,0) == [-11//6, 3//1, -3//2, 1//3]
    @test Jecco.calculate_weights(1,4,0) == [-25//12, 4//1, -3//1, 4//3, -1//4]
    @test Jecco.calculate_weights(1,5,0) == [-137//60, 5//1, -5//1, 10//3, -5//4, 1//5]
    @test Jecco.calculate_weights(1,6,0) == [-49//20, 6//1, -15//2, 20//3, -15//4, 6//5, -1//6]

    # forward 1st order derivatives i.e. -1,0,1,2,...
    @test Jecco.calculate_weights(1,3,1) == [-1//3, -1//2, 1//1, -1//6]
    @test Jecco.calculate_weights(1,4,1) == [-1//4, -5//6, 3//2, -1//2, 1//12]
    @test Jecco.calculate_weights(1,5,1) == [-1//5, -13//12, 2//1, -1//1, 1//3, -1//20]
    @test Jecco.calculate_weights(1,6,1) == [-1//6, -77//60, 5//2, -5//3, 5//6, -1//4, 1//30]
    @test Jecco.calculate_weights(1,7,1) == [-1//7, -29//20, 3//1, -5//2, 5//3, -3//4, 1//5, -1//42]

    # forward 1st order derivatives i.e. -2,-1,0,1,2,3,...
    @test Jecco.calculate_weights(1,5,2) == [1//20, -1//2, -1//3, 1//1, -1//4, 1//30]
    @test Jecco.calculate_weights(1,6,2) == [1//30, -2//5, -7//12, 4//3, -1//2, 2//15, -1//60]

    # backward 1st order derivatives i.e. ...,-1,0
    @test Jecco.calculate_weights(1,1,1) == [-1//1, 1//1]
    @test Jecco.calculate_weights(1,2,2) == [1//2, -2//1, 3//2]
    @test Jecco.calculate_weights(1,3,3) == [-1//3, 3//2, -3//1, 11//6]
    @test Jecco.calculate_weights(1,4,4) == [1//4, -4//3, 3//1, -4//1, 25//12]

    # backward 1st order derivatives i.e. ...,-2,-1,0,1
    @test Jecco.calculate_weights(1,3,2) == [1//6, -1//1, 1//2, 1//3]
    @test Jecco.calculate_weights(1,4,3) == [-1//12, 1//2, -3//2, 5//6, 1//4]
    @test Jecco.calculate_weights(1,5,4) == [1//20, -1//3, 1//1, -2//1, 13//12, 1//5]

    # backward 1st order derivatives i.e. ...,-3,-2,-1,0,1,2
    @test Jecco.calculate_weights(1,5,3) == [-1//30, 1//4, -1//1, 1//3, 1//2, -1//20]
    @test Jecco.calculate_weights(1,6,4) == [1//60, -2//15, 1//2, -4//3, 7//12, 2//5, -1//30]

    # centered 2nd order derivatives i.e. ...,-1,0,1,...
    @test Jecco.calculate_weights(2,2,1) == [1//1, -2//1, 1//1]
    @test Jecco.calculate_weights(2,4,2) == [-1//12, 4//3, -5//2, 4//3, -1//12]
    @test Jecco.calculate_weights(2,6,3) == [1//90, -3//20, 3//2, -49//18,
                                       3//2, -3//20, 1//90]
    @test Jecco.calculate_weights(2,8,4) == [-1//560, 8//315, -1//5, 8//5, -205//72,
                                       8//5, -1//5, 8//315, -1//560]

    # forward 2nd order derivatives i.e. 0,1,...
    @test Jecco.calculate_weights(2,2,0) == [1//1, -2//1, 1//1]
    @test Jecco.calculate_weights(2,3,0) == [2//1, -5//1, 4//1, -1//1]
    @test Jecco.calculate_weights(2,4,0) == [35//12, -26//3, 19//2, -14//3, 11//12]
    @test Jecco.calculate_weights(2,5,0) == [15//4, -77//6, 107//6, -13//1, 61//12, -5//6]
    @test Jecco.calculate_weights(2,6,0) == [203//45, -87//5, 117//4, -254//9, 33//2, -27//5, 137//180]
    @test Jecco.calculate_weights(2,7,0) == [469//90, -223//10, 879//20, -949//18, 41//1, -201//10,
                                       1019//180, -7//10]

    # forward 2nd order derivatives i.e. -1,0,1,2,...
    @test Jecco.calculate_weights(2,3,1) == [1//1, -2//1, 1//1, 0//1]
    @test Jecco.calculate_weights(2,4,1) == [11//12, -5//3, 1//2, 1//3, -1//12]
    @test Jecco.calculate_weights(2,5,1) == [5//6, -5//4, -1//3, 7//6, -1//2, 1//12]
    @test Jecco.calculate_weights(2,6,1) == [137//180, -49//60, -17//12, 47//18, -19//12,
                                       31//60, -13//180]

    # forward 2nd order derivatives i.e. -2,-1,0,1,2,3,...
    @test Jecco.calculate_weights(2,5,2) == [-1//12, 4//3, -5//2, 4//3, -1//12, 0//1]
    @test Jecco.calculate_weights(2,6,2) == [-13//180, 19//15, -7//3, 10//9,
                                       1//12, -1//15, 1//90]

    # backward 2nd order derivatives i.e. ...,-1,0
    @test Jecco.calculate_weights(2,2,2) == [1//1, -2//1, 1//1]
    @test Jecco.calculate_weights(2,3,3) == [-1//1, 4//1, -5//1, 2//1]
    @test Jecco.calculate_weights(2,4,4) == [11//12, -14//3, 19//2, -26//3, 35//12]

    # backward 2nd order derivatives i.e. ...,-2,-1,0,1
    @test Jecco.calculate_weights(2,3,2) == [0//1, 1//1, -2//1, 1//1]
    @test Jecco.calculate_weights(2,4,3) == [-1//12, 1//3, 1//2, -5//3, 11//12]
    @test Jecco.calculate_weights(2,5,4) == [1//12, -1//2, 7//6, -1//3, -5//4, 5//6]

    # backward 2nd order derivatives i.e. ...,-3,-2,-1,0,1,2
    @test Jecco.calculate_weights(2,5,3) == [0//1, -1//12, 4//3, -5//2, 4//3, -1//12]
    @test Jecco.calculate_weights(2,6,4) == [1//90, -1//15, 1//12, 10//9, -7//3, 19//15, -13//180]

    # centered 3rd order derivatives i.e ...,-3,-2,-1,0,1,2,3
    @test Jecco.calculate_weights(3,6,3) == [1//8, -1//1, 13//8, 0//1, -13//8, 1//1, -1//8]

    # forward 3rd order derivatives i.e 0,1,2,3,...
    @test Jecco.calculate_weights(3,3,0) == [-1//1, 3//1, -3//1, 1//1]
    @test Jecco.calculate_weights(3,4,0) == [-5//2, 9//1, -12//1, 7//1, -3//2]

    # forward 3rd order derivatives i.e -1,0,1,2,...
    @test Jecco.calculate_weights(3,3,1) == [-1//1, 3//1, -3//1, 1//1]
    @test Jecco.calculate_weights(3,4,1) == [-3//2, 5//1, -6//1, 3//1, -1//2]

    # backward 3rd order derivatives i.e -3,-2,-1,0,...
    @test Jecco.calculate_weights(3,3,3) == [-1//1, 3//1, -3//1, 1//1]
    @test Jecco.calculate_weights(3,4,4) == [3//2, -7//1, 12//1, -9//1, 5//2]

    # backward 3rd order derivatives i.e -2,-1,0,1...
    @test Jecco.calculate_weights(3,3,2) == [-1//1, 3//1, -3//1, 1//1]
    @test Jecco.calculate_weights(3,4,3) == [1//2, -3//1, 6//1, -5//1, 3//2]

end

@testset "EqualSizeStencilFD tests:" begin

    Dx = EqualSizeStencilFD{1}(1, 4, 1.0, 10)
    @test 12*Dx[1,:]  ≈ [-25.0, 48.0, -36.0, 16.0, -3, 0.0, 0.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[2,:]  ≈ [-3.0, -10.0, 18.0, -6.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[3,:]  ≈ [1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[4,:]  ≈ [0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[5,:]  ≈ [0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[6,:]  ≈ [0.0, 0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[7,:]  ≈ [0.0, 0.0, 0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0] atol=1e-15
    @test 12*Dx[8,:]  ≈ [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0] atol=1e-15
    @test 12*Dx[9,:]  ≈ [0.0, 0.0, 0.0, 0.0, 0.0, -1.0, 6.0, -18.0, 10.0, 3.0] atol=1e-15
    @test 12*Dx[10,:] ≈ [0.0, 0.0, 0.0, 0.0, 0.0, 3.0, -16.0, 36.0, -48.0, 25.0] atol=1e-15

end

@testset "Periodic Derivative tests:" begin

    Dx = CenteredDiff{1}(1, 4, 1.0, 10)
    @test 12*Dx[1,:]  ≈ [0.0, 8.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, -8.0] atol=1e-15
    @test 12*Dx[2,:]  ≈ [-8.0, 0.0, 8.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0] atol=1e-15
    @test 12*Dx[3,:]  ≈ [1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[4,:]  ≈ [0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[5,:]  ≈ [0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[6,:]  ≈ [0.0, 0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0, 0.0] atol=1e-15
    @test 12*Dx[7,:]  ≈ [0.0, 0.0, 0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0, 0.0] atol=1e-15
    @test 12*Dx[8,:]  ≈ [0.0, 0.0, 0.0, 0.0, 0.0, 1.0, -8.0, 0.0, 8.0, -1.0] atol=1e-15
    @test 12*Dx[9,:]  ≈ [-1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, -8.0, 0.0, 8.0] atol=1e-15
    @test 12*Dx[10,:] ≈ [8.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, -8.0, 0.0] atol=1e-15

    # 1D case

    xmin   = -2.0*pi
    xmax   =  2.0*pi
    xnodes =  600
    ord    =  4

    hx     = (xmax - xmin) / xnodes

    x  = collect(xmin:hx:xmax-hx)
    f  = sin.(x)

    D1 = CenteredDiff(1, ord, hx, length(x))
    D2 = CenteredDiff(2, ord, hx, length(x))

    df  = D1 * f
    d2f = D2 * f

    @test df  ≈ cos.(x) rtol=1e-8
    @test d2f ≈ -f rtol=1e-8

    # now for the callable, point-wise, methods
    df0 = similar(df)
    for i in eachindex(f)
        df0[i] = D1(f,i)
    end
    @test df0 == df

    # 3D case

    ymin   = -1.0*pi
    ymax   =  1.0*pi
    ynodes =  20
    zmin   = -1.0*pi
    zmax   =  1.0*pi
    znodes =  300

    hy     = (ymax - ymin) / ynodes
    hz     = (zmax - zmin) / znodes

    y      = collect(ymin:hy:ymax-hy)
    z      = collect(zmin:hz:zmax-hz)

    f      = [sin.(x1) .* sin.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dxf0   = [cos.(x1) .* sin.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dyf0   = [sin.(x1) .* cos.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dzf0   = [sin.(x1) .* sin.(x2) .* cos.(x3) for x1 in x, x2 in y, x3 in z]
    dxzf0  = [cos.(x1) .* sin.(x2) .* cos.(x3) for x1 in x, x2 in y, x3 in z]

    Dx     = CenteredDiff{1}(1, ord, hx, length(x))
    Dz     = CenteredDiff{3}(1, ord, hz, length(z))

    Dxx    = CenteredDiff{1}(2, ord, hx, length(x))
    Dzz    = CenteredDiff{3}(2, ord, hz, length(z))

    dxf    = Dx * f
    dzf    = Dz * f
    dxzf   = Dx * (Dz * f)

    @test dxf  ≈ dxf0
    @test dzf  ≈ dzf0
    @test dxzf ≈ dxzf0

    d2xf   = Dxx * f
    d2zf   = Dzz * f

    @test d2xf ≈ -f
    @test d2zf ≈ -f

    # now for callable, point-wise, methods
    @test Dx(f,2,10,120)  == dxf[2,10,120]
    @test Dx(f,42,20,300) == dxf[42,20,300]
    @test Dz(f,2,10,120)  == dzf[2,10,120]
    @test Dz(f,42,20,300) == dzf[42,20,300]
end

@testset "EqualSizeStencilFD Derivative tests:" begin

    # 1D case

    xmin   = -2.0*pi
    xmax   =  2.0*pi
    xnodes =  600
    ord    =  6

    hx     = (xmax - xmin) / xnodes

    x  = collect(xmin:hx:xmax-hx)
    f  = sin.(x)

    D1 = EqualSizeStencilFD(1, ord, hx, length(x))
    D2 = EqualSizeStencilFD(2, ord, hx, length(x))

    df  = D1 * f
    d2f = D2 * f

    @test df  ≈ cos.(x)
    @test d2f ≈ -f

    # now for the callable, point-wise, methods
    df0 = similar(df)
    for i in eachindex(f)
        df0[i] = D1(f,i)
    end
    @test df0 == df

    # 3D case

    ymin   = -1.0*pi
    ymax   =  1.0*pi
    ynodes =  20
    zmin   = -1.0*pi
    zmax   =  1.0*pi
    znodes =  300

    hy     = (ymax - ymin) / ynodes
    hz     = (zmax - zmin) / znodes

    y      = collect(ymin:hy:ymax-hy)
    z      = collect(zmin:hz:zmax-hz)

    f      = [sin.(x1) .* sin.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dxf0   = [cos.(x1) .* sin.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dyf0   = [sin.(x1) .* cos.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dzf0   = [sin.(x1) .* sin.(x2) .* cos.(x3) for x1 in x, x2 in y, x3 in z]
    dxzf0  = [cos.(x1) .* sin.(x2) .* cos.(x3) for x1 in x, x2 in y, x3 in z]

    Dx     = EqualSizeStencilFD{1}(1, ord, hx, length(x))
    Dz     = EqualSizeStencilFD{3}(1, ord, hz, length(z))

    Dxx    = EqualSizeStencilFD{1}(2, ord, hx, length(x))
    Dzz    = EqualSizeStencilFD{3}(2, ord, hz, length(z))

    dxf    = Dx * f
    dzf    = Dz * f
    dxzf   = Dx * (Dz * f)

    @test dxf  ≈ dxf0
    @test dzf  ≈ dzf0
    @test dxzf ≈ dxzf0

    d2xf   = Dxx * f
    d2zf   = Dzz * f

    @test d2xf ≈ -f
    @test d2zf ≈ -f

    # now for callable, point-wise, methods
    @test Dx(f,2,10,120)  == dxf[2,10,120]
    @test Dx(f,42,20,300) == dxf[42,20,300]
    @test Dz(f,2,10,120)  == dzf[2,10,120]
    @test Dz(f,42,20,300) == dzf[42,20,300]
end

@testset "Spectral Derivative tests:" begin

    # 1D case
    xmin   = -2.0
    xmax   =  2.0
    xnodes =  32

    x, = Jecco.cheb(xmin, xmax, xnodes)
    f = 0.5 * x.^2

    Dx  = ChebDeriv(1, xmin, xmax, xnodes)
    Dxx = ChebDeriv(2, xmin, xmax, xnodes)

    dxf  = Dx * f
    dxxf = Dxx * f
    @test dxf  ≈ x
    @test dxxf ≈ fill(1.0, size(dxxf))

    # now for the callable, point-wise, methods
    dxf0 = similar(dxf)
    for i in eachindex(f)
        dxf0[i] = Dx(f,i)
    end
    @test dxf0 ≈ dxf

    # 3D case

    ymin   = -1.0
    ymax   =  1.0
    ynodes =  8

    zmin   = -1.0
    zmax   =  1.0
    znodes =  16

    y, = Jecco.cheb(ymin, ymax, ynodes)
    z, = Jecco.cheb(zmin, zmax, znodes)

    Dy  = ChebDeriv{2}(1, ymin, ymax, ynodes)
    Dyy = ChebDeriv{2}(2, ymin, ymax, ynodes)
    Dz  = ChebDeriv{3}(1, zmin, zmax, znodes)
    Dzz = ChebDeriv{3}(2, zmin, zmax, znodes)

    f     = [0.5 * x1.^2 .* cos.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dxf0  = [x1 .* cos.(x2) .* sin.(x3)          for x1 in x, x2 in y, x3 in z]
    dzf0  = [0.5 * x1.^2 .* cos.(x2) .* cos.(x3) for x1 in x, x2 in y, x3 in z]
    dxxf0 = [cos.(x2) .* sin.(x3)                for x1 in x, x2 in y, x3 in z]
    dzzf0 = -copy(f)

    dxf = Dx * f
    dzf = Dz * f
    d2xf = Dxx * f
    d2zf = Dzz * f

    @test dxf ≈ dxf0
    @test dzf ≈ dzf0

    @test d2xf ≈ dxxf0
    @test d2zf ≈ dzzf0

    # now for callable, point-wise, methods
    @test Dx(f,2,4,16)  ≈ dxf[2,4,16]
    @test Dx(f,1,6,12)  ≈ dxf[1,6,12]
    @test Dz(f,2,3,8)   ≈ dzf[2,3,8]
    @test Dz(f,16,8,1)  ≈ dzf[16,8,1]
end

@testset "Cross FD derivative tests:" begin
    # 2D FD case
    xmin   = -2.0*pi
    xmax   =  2.0*pi
    xnodes =  600
    ymin   = -1.0*pi
    ymax   =  1.0*pi
    ynodes =  300
    ord    =  4

    hx     = (xmax - xmin) / xnodes
    hy     = (ymax - ymin) / ynodes

    x      = collect(xmin:hx:xmax-hx)
    y      = collect(ymin:hy:ymax-hy)

    f      = [sin.(x1) .* sin.(x2) for x1 in x, x2 in y]

    Dx     = CenteredDiff{1}(1, ord, hx, length(x))
    Dy     = CenteredDiff{2}(1, ord, hy, length(y))

    dxyf   = Dx * (Dy * f)

    dxyf0  = similar(dxyf)
    for j in 1:ynodes
        for i in 1:xnodes
            dxyf0[i,j] = Dx(Dy, f,i, j)
        end
    end
    @test dxyf0 ≈ dxyf rtol=1e-12 atol=1e-12
end

@testset "PeriodicFD cross derivative for general arrays tests:" begin
    xmin   = -2.0*pi
    xmax   =  2.0*pi
    xnodes =  600
    ord    =  4

    ymin   = -1.0
    ymax   =  1.0
    ynodes =  16

    zmin   = -1.0*pi
    zmax   =  1.0*pi
    znodes =  300

    hx     = (xmax - xmin) / xnodes
    hz     = (zmax - zmin) / znodes

    x      = collect(xmin:hx:xmax-hx)
    y,     = Jecco.cheb(ymin, ymax, ynodes)
    z      = collect(zmin:hz:zmax-hz)

    f   = [0.5 * sin.(x1) .* x2.^2 .* sin.(x3)  for x1 in x, x2 in y, x3 in z]

    Dx  = CenteredDiff{1}(1, ord, hx, length(x))
    Dz  = CenteredDiff{3}(1, ord, hz, length(z))

    dxzf  = Dx * (Dz * f)

    dxzf0  = similar(dxzf)
    for k in 1:znodes
        for j in 1:ynodes
            for i in 1:xnodes
                dxzf0[i,j,k] = Dx(Dz, f, i, j, k)
            end
        end
    end
    @test dxzf0 ≈ dxzf rtol=1e-12 atol=1e-12

    g   = [sin.(x3) .* 0.5 * sin.(x2) .* x1.^2 for x3 in z, x1 in y, x2 in x]

    Dz  = CenteredDiff{1}(1, ord, hz, length(z))
    Dx  = CenteredDiff{3}(1, ord, hx, length(x))

    dxzg  = Dx * (Dz * g)

    @test Dz(Dx, g, 100,2,1)     ≈ dxzg[100,2,1]
    @test Dz(Dx, g, 1,1,1)       ≈ dxzg[1,1,1]
    @test Dz(Dx, g, 300,16,600)  ≈ dxzg[300,16,600]
    @test Dz(Dx, g, 10,8,150)    ≈ dxzg[10,8,150]
end

@testset "Fourier Derivative tests:" begin
    # 1D case
    xmin   = -2.0*pi
    xmax   =  2.0*pi
    xnodes =  600
    ord    =  4

    hx     = (xmax - xmin) / xnodes

    x  = collect(xmin:hx:xmax-hx)
    f  = sin.(x)

    D1 = FourierDeriv(1, xmax-xmin, length(x))
    D2 = FourierDeriv(2, xmax-xmin, length(x))

    df  = D1 * f
    d2f = D2 * f

    @test df  ≈ cos.(x)
    @test d2f ≈ -f

    # now for the callable, point-wise, methods
    df0  = similar(df)
    d2f0 = similar(d2f)
    for i in eachindex(f)
        df0[i]  = D1(f,i)
        d2f0[i] = D2(f,i)
    end
    @test df0  ≈ df  atol=1e-12
    @test d2f0 ≈ d2f atol=1e-10

    # 3D case
    ymin   = -1.0*pi
    ymax   =  1.0*pi
    ynodes =  20
    zmin   = -1.0*pi
    zmax   =  1.0*pi
    znodes =  300

    hy     = (ymax - ymin) / ynodes
    hz     = (zmax - zmin) / znodes

    y      = collect(ymin:hy:ymax-hy)
    z      = collect(zmin:hz:zmax-hz)

    f      = [sin.(x1) .* sin.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dxf0   = [cos.(x1) .* sin.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dyf0   = [sin.(x1) .* cos.(x2) .* sin.(x3) for x1 in x, x2 in y, x3 in z]
    dzf0   = [sin.(x1) .* sin.(x2) .* cos.(x3) for x1 in x, x2 in y, x3 in z]
    dxzf0  = [cos.(x1) .* sin.(x2) .* cos.(x3) for x1 in x, x2 in y, x3 in z]

    Dx     = FourierDeriv{1}(1, xmax-xmin, length(x))
    Dz     = FourierDeriv{3}(1, zmax-zmin, length(z))

    Dxx    = FourierDeriv{1}(2, xmax-xmin, length(x))
    Dzz    = FourierDeriv{3}(2, zmax-zmin, length(z))

    dxf    = Dx * f
    dzf    = Dz * f
    dxzf   = Dx * (Dz * f)

    @test dxf  ≈ dxf0
    @test dzf  ≈ dzf0
    @test dxzf ≈ dxzf0

    d2xf   = Dxx * f
    d2zf   = Dzz * f

    @test d2xf ≈ -f
    @test d2zf ≈ -f

    # now for callable, point-wise, methods
    @test Dx(f,2,10,120)  ≈ dxf[2,10,120]
    @test Dx(f,42,20,300) ≈ dxf[42,20,300]
    @test Dz(f,2,10,120)  ≈ dzf[2,10,120]
    @test Dz(f,42,20,300) ≈ dzf[42,20,300]
end

@testset "EqualSizeStencilFD cross derivative for general arrays tests:" begin
    xmin   = -2.0*pi
    xmax   =  2.0*pi
    xnodes =  600
    ord    =  6

    ymin   = -1.0
    ymax   =  1.0
    ynodes =  16

    zmin   = -1.0*pi
    zmax   =  1.0*pi
    znodes =  300

    hx     = (xmax - xmin) / xnodes
    hz     = (zmax - zmin) / znodes

    x      = collect(xmin:hx:xmax-hx)
    y,     = Jecco.cheb(ymin, ymax, ynodes)
    z      = collect(zmin:hz:zmax-hz)

    f   = [0.5 * sin.(x1) .* x2.^2 .* sin.(x3)  for x1 in x, x2 in y, x3 in z]

    Dx  = EqualSizeStencilFD{1}(1, ord, hx, length(x))
    Dz  = EqualSizeStencilFD{3}(1, ord, hz, length(z))

    dxzf  = Dx * (Dz * f)

    dxzf0  = similar(dxzf)
    for k in 1:znodes
        for j in 1:ynodes
            for i in 1:xnodes
                dxzf0[i,j,k] = Dx(Dz, f, i, j, k)
            end
        end
    end
    @test dxzf0 ≈ dxzf rtol=1e-12 atol=1e-12

    g   = [sin.(x3) .* 0.5 * sin.(x2) .* x1.^2 for x3 in z, x1 in y, x2 in x]

    Dz  = EqualSizeStencilFD{1}(1, ord, hz, length(z))
    Dx  = EqualSizeStencilFD{3}(1, ord, hx, length(x))

    dxzg  = Dx * (Dz * g)

    @test Dz(Dx, g, 100,2,1)     ≈ dxzg[100,2,1]
    @test Dz(Dx, g, 1,1,1)       ≈ dxzg[1,1,1]
    @test Dz(Dx, g, 300,16,600)  ≈ dxzg[300,16,600]
    @test Dz(Dx, g, 10,8,150)    ≈ dxzg[10,8,150]
end

@testset "PeriodicFD and EqualSizeStencilFD cross derivative for general arrays tests:" begin
    xmin   = -2.0*pi
    xmax   =  2.0*pi
    xnodes =  600
    ord    =  6

    ymin   = -1.0
    ymax   =  1.0
    ynodes =  16

    zmin   = -1.0*pi
    zmax   =  1.0*pi
    znodes =  300

    hx     = (xmax - xmin) / xnodes
    hz     = (zmax - zmin) / znodes

    x      = collect(xmin:hx:xmax-hx)
    y,     = Jecco.cheb(ymin, ymax, ynodes)
    z      = collect(zmin:hz:zmax-hz)

    f   = [0.5 * sin.(x1) .* x2.^2 .* sin.(x3)  for x1 in x, x2 in y, x3 in z]

    Dx  = EqualSizeStencilFD{1}(1, ord, hx, length(x))
    Dz  = CenteredDiff{3}(1, ord, hz, length(z))

    dxzf  = Dx * (Dz * f)

    dxzf0  = similar(dxzf)
    for k in 1:znodes
        for j in 1:ynodes
            for i in 1:xnodes
                dxzf0[i,j,k] = Dx(Dz, f, i, j, k)
            end
        end
    end
    @test dxzf0 ≈ dxzf rtol=1e-12 atol=1e-12

    g   = [sin.(x3) .* 0.5 * sin.(x2) .* x1.^2 for x3 in z, x1 in y, x2 in x]

    Dz  = CenteredDiff{1}(1, ord, hz, length(z))
    Dx  = EqualSizeStencilFD{3}(1, ord, hx, length(x))

    dxzg  = Dx * (Dz * g)

    @test Dz(Dx, g, 100,2,1)     ≈ dxzg[100,2,1]
    @test Dz(Dx, g, 1,1,1)       ≈ dxzg[1,1,1]
    @test Dz(Dx, g, 300,16,600)  ≈ dxzg[300,16,600]
    @test Dz(Dx, g, 10,8,150)    ≈ dxzg[10,8,150]
end


#========================================================================================#
#	Laboratory 3
#
# Ranges, Arrays and Vectors
#
# Author: Niall Palfreyman, 10/01/2022
#========================================================================================#
[
	Activity(
		"""
		In this laboratory we look at a very important structure in Julia: arrays. Once you
		understand how arrays work, and how amazingly useful they are, you will (almost)
		never need to write a for-loop again!

		... But first we start with Ranges. A Range is an interval of values between start
		and stop boundaries. Here is a range of values between 1 and 5:

			r = 1:5

		What is the type of r?
		""",
		"typeof(1:5)",
		(x -> x <: UnitRange)
	),
	Activity(
		"""
		We can collect() the individual values of a Range into a collection:

			collect(1:5)

		What is the type of this data structure?
		""",
		"typeof(collect(1:5))",
		(x -> x <: Vector)
	),
	Activity(
		"""
		We can also construct ranges for other types, and we can also specify our own
		steplength between the values:

			0.0:0.2:1.0

		Use collect() to list the values in this range and find out how many different
		values the range contains:
		""",
		"length(collect(0.0:0.2:1.0))",
		x -> x==6
	),
	Activity(
		"""
		collect() turns a Range into an Array. Arrays are the daily workhorse of data science:
		they collect elements into tables of 1 or more dimensions. A 1-dimensional Array is
		called a Vector. Here is a COLUMN vector:

			v = [5,4,7,6,3]

		What is its size()?
		""",
		"size(v)",
		x -> x==(5,)
	),
	Activity(
		"""
		What is the size of this ROW vector: w = [5 4 7 6 3 2 1] ?
		""",
		"size(w)",
		x -> x==(1,7)
	),
	Activity(
		"""
		As you can see, the FIRST dimension of an array runs down its columns, and
		the SECOND dimension runs across its rows.

		If you want to write a really super-duper-speedy program, you can create arrays
		using the values that are accidentally hanging around in memory:

			A = Array{Int}(undef,2,3)

		However, more usually we want to INITIALISE the elements of a new array. What
		is the value of the elements in this array:

			A = ones(2,3)
		""",
		"A[1]",
		x -> x == 1.0
	),
	Activity(
		"""
		What about this array: zeros(3,4). What is its size?
		""",
		"size(zeros(3,4))",
		x -> x==(3,4)
	),
	Activity(
		"""
		A MATRIX is an Array with just two dimensions. We often want to fill a matrix
		with a particular value. For example:

			B = Matrix{Int}(undef,3,2)
			fill!( B, 42)

		What is the size of this array?
		""",
		"size(B)",
		x -> x==(3,2)
	),
	Activity(
		"""
		We can create an array literal by explicitly naming the array's elements
		in square brackets:

			C = [2 4 7;3 9 1.2]

		Which value determines the type of the array elements here?
		""",
		"The element from the least restrictive number set",
		x -> x==1.2
	),
	Activity(
		"""
		We can also use square brackets to concatenate arrays, for example:

			[ones(2,3) zeros(2,4)]

		Find the symbol for using square brackets to concatenate two arrays vertically:
		""",
		"",
		x -> occursin( ";", x)
	),
	Activity(
		"""
		A very powerful way of constructing arrays is by COMPREHENSION. That is, we
		write a formula in square brackets that specifies (comprehends) the entire
		contents of the array. For example, here is a vector of square values:

			[x^2 for x in 1:12]

		What do you get if you add the condition "if isodd(x)" to the end of this
		comprehension?
		""",
		"[x^2 for x in 1:12 if isodd(x)]",
		x -> x == [x^2 for x in 1:12 if isodd(x)]
	),
	Activity(
		"""
		We can use several comprehension variables:

			[x*y for x in 1:5, y in 1:5]

		Use array comprehension to generate a matrix of values for the function
		exp(-(x^2 + y^2)) with x and y in the range from -1 to +1 in steps of 0.2. This
		function is a 'hill' with a peak value of 1.0 at the centre where x = y = 0.
		""",
		"[exp(-(x^2 + y^2)) for x in -1:0.2:1, y in -1:0.2:1]",
		x -> abs(sum(x) - 61.02) < 0.01
	),
	Activity(
		"""
		Once we have created an array, we usually want to inspect it. First create
		this array:

			my_matrix = ((1:3)*(1:4)')//3

		Now use eltype() to find the type of the elements in the array my_matrix:
		""",
		"eltype(my_matrix)",
		x -> x <: Rational
	),
	Activity(
		"""
		The length() of a matrix is the number of elements in it. What is the length
		of my_matrix?
		""",
		"length(my_matrix)",
		x -> x==12
	),
	Activity(
		"""
		What is the ndims() of my_matrix?
		""",
		"ndims(my_matrix)",
		x -> x==2
	),
	Activity(
		"""
		You have already used size() to return the size of a matrix, but you can also
		add a second argument to specify which dimension of the matrix you wish to know
		the size of. What is the size of the second dimension of my_matrix?
		""",
		"size(my_matrix,2)",
		x -> x==4
	),
	Activity(
		"""
		Matrix-based languages like Julia offer very many different ways of inspecting
		individual parts of an array. To experiment with this various ways, first define
		the following two arrays:

			my_vector = collect(1:5)
			my_matrix = [1 2 3;4 5 6;7 8 9]

		We can access individual elements of these arrays easily using square brackets
		and subscripts. How do we use subscripts to view the element 8 in my_matrix?
		""",
		"my_matrix[r,c]",
		x -> occursin("my_matrix[3,2]",replace(x," "=>""))
	),
	Activity(
		"""
		The keyword 'end' denotes the last subscript in an array dimension. For example,
		what value is denoted by my_matrix[1,end-1]?
		""",
		"my_matrix[2,end-1]",
		x -> x==2
	),
	Activity(
		"""
		Use subscripting with '=' to change the first entry in my_vector from 1 to 0, then
		give me the new vector that results from this change:
		""",
		"reply(my_vector)",
		x -> x==[0,2,3,4,5]
	),
	Activity(
		"""
		Often, we wish to work not just with individual array elements, but with a whole
		slice of the array. What is the result of the expression my_vector[2:4]?
		""",
		"my_vector[2:4]",
		x -> x==Main.my_vector[2:4]
	),
	Activity(
		"""
		The colon also stands for a complete row or column of an array. What is the
		result of the expression my_matrix[:,3]?
		""",
		"my_matrix[:,3]",
		x -> x==Main.my_matrix[:,3]
	),
	Activity(
		"""
		What is the result of the expression my_matrix[begin+1:end,end]?
		""",
		"my_matrix[begin+1:end,end]",
		x -> x==Main.my_matrix[begin+1:end,end]
	),
	Activity(
		"""
		We can also use slices to change the values in an array. What is the new
		value of the array my_matrix if we perform the following operation:

			my_matrix[3,:] = [17,18,19]
		""",
		"reply(my_matrix)",
		x -> x==[1 2 3;4 5 6;17 18 19]
	),
	Activity(
		"""
		What is the value of the array new_matrix if we construct it like this:

			new_matrix = reshape(my_matrix,1,9)
		""",
		"reshape(my_matrix,1,9)",
		x -> x==reshape(Main.my_matrix,1,9)
	),
	Activity(
		"""
		Notice the strange order of elements in new_matrix. This order arises
		because Julia uses "column-major" ordering of matrix elements: that is,
		these elements have a linear ordering that runs first down the columns
		and then across the rows. You can see this if you list all elements of
		my_matrix using ':'. Show me the result of doing this:
		""",
		"my_matrix[:]",
		x -> x==Main.my_matrix[:]
	),
	Activity(
		"""
		We can perform any function on all elements of an array by using the
		broadcast operator "." like this:

			log.(my_matrix)

		What is the sin() of all elements of my_matrix?
		""",
		"sin.(my_matrix)",
		x -> x==sin.(Main.my_matrix)
	),
	Activity(
		"""
		We can also use broadcasting with infix operators, for example:

			my_matrix .+ 5

		What is the result of taking the reciprocal (1/x) of each element
		in my_matrix?
		""",
		"1 ./ my_matrix",
		x -> x == 1 ./ Main.my_matrix
	),
	Activity(
		"""
		Another way of operating on all elements in an array is to map() the
		operation over the array:

			map(log,my_matrix)

		What is the sin() of my_matrix?
		""",
		"map(sin,my_matrix)",
		x -> x==map(sin,Main.my_matrix)
	),
	Activity(
		"""
		The map() function is particularly useful when we are using anonymous
		functions:

			map(x->5x,my_matrix)

		What do we obtain if we apply the function (x->5sin(x+3)) to my_matrix?
		""",
		"map(x->5sin(x+3),my_matrix)",
		x -> x==map(x->5sin(x+3),Main.my_matrix)
	),
	Activity(
		"""
		We can combine mapping with slicing:

			map(x->5x,my_matrix[:,3])

		What is the result of applying the function (x->5sin(x+3)) to the central
		column of my_matrix?
		""",
		"map(x->5sin(x+3),my_matrix[:,2])",
		x -> x==map(x->5sin(x+3),Main.my_matrix[:,2])
	),
	Activity(
		"""
		We can also iterate (i.e.: loop) over the elements in any array:

			for m in my_array
				print( m, ", ")
			end

		Write a loop to calculate the sum of all elements in my_matrix:
		""",
		"reply(ans)",
		x -> x==sum(Main.my_matrix)
	),
]


"""
	normalizemean!(A)

In place removal of the variable mean from the P×N matrix `A` where P is the number of variables and N is the number of samples.
"""
normalizemean!(A::AbstractMatrix) = A .-= mean(A,dims=2)

"""
	normalizemean(A)

Remove the variable mean from the P×N matrix `A` where P is the number of variables and N is the number of samples.
"""
normalizemean(A::AbstractMatrix) = A .- mean(A,dims=2)

"""
	normalizemeanstd!(A)

In place normalization of variables to be mean zero and standard deviation one of the P×N matrix `A` where P is the number of variables and N is the number of samples.
"""
function normalizemeanstd!(A::AbstractMatrix)
	normalizemean!(A::AbstractMatrix)
	A ./= max.(1e-12, std(A,dims=2)) # max to avoid div by zero
end
"""
	normalizemeanstd!(A)

Normalize variables to be mean zero and standard deviation one in the P×N matrix `A` where P is the number of variables and N is the number of samples.
"""
function normalizemeanstd(A::AbstractMatrix)
	A = normalizemean(A)
	A ./= max.(1e-12, std(A,dims=2)) # max to avoid div by zero
end


function finitehess{T <: Number}(
    f::Function,
    x::T,
    direction::Symbol = :central,
)
    ε = @hessianrule(x)
    return (f(x + ε) - 2 * f(x) + f(x - ε)) / (ε * ε)
end

function finitehess!{T <: Number}(
    H::Array{T},
    f::Function,
    x::Vector{T},
    direction::Symbol = :central,
)
    n = length(x)
    ε = nan(T)
    # TODO: Remove all these copies
    xpp, xpm, xmp, xmm = copy(x), copy(x), copy(x), copy(x)
    f_x = f(x)
    for i in 1:n
        x_i = x[i]
        ε = @hessianrule(x[i])
        xpp[i], xmm[i] = x_i + ε, x_i - ε
        H[i, i] = (f(xpp) - 2 * f_x + f(xmm)) / (ε * ε)
        ε_i = @centralrule(x[i])
        xp = x_i + ε_i
        xm = x_i - ε_i
        xpp[i], xpm[i], xmp[i], xmm[i] = xp, xp, xm, xm
        for j in (i + 1):n
            x_j = x[j]
            ε_j = @centralrule(x[j])
            xp = x_j + ε_j
            xm = x_j - ε_j
            xpp[j], xpm[j], xmp[j], xmm[j] = xp, xm, xp, xm
            H[i, j] = (f(xpp) - f(xpm) - f(xmp) + f(xmm)) / (4 * ε_i * ε_j)
            xpp[j], xpm[j], xmp[j], xmm[j] = x_j, x_j, x_j, x_j
        end
        xpp[i], xpm[i], xmp[i], xmm[i] = x_i, x_i, x_i, x_i
    end
    Base.LinAlg.copytri!(H, 'U')
    return H
end

function finitehess{T <: Number}(
    f::Function,
    x::Vector{T},
    direction::Symbol = :central
)
    n = length(x)
    H = Array(T, n, n)
    finitehess!(f, x, H, direction)
    return H
end

function taylor_finite_difference_hessian(
    f::Function,
    x::Real,
    h::Real,
)
    f_x = f(x)
    d = 4^6 * (2^4 * (f(x + h) + f(x - h) - 2 * f_x) - (f(x + 2 * h) + f(x - 2 * h) - 2 * f_x))
    d += - (2^4 * (f(x + 4 * h) + f(x - 4 * h) - 2 * f_x) - (f(x + 8 * h) + f(x - 8 * h) - 2 * f_x))
    return d / (3 * 2^6 * (2^8 - 1) * h^2)
end


module FresnelIntegrals
using SpecialFunctions
export fresnelc
export fresnels
export fresnel

"""
        fresnelc(z)
Calculates the Fresnel cosine integral for the number z for
    ``C(z) = \\int_{0}^{z} \\cos{\\left(\\frac{\\pi t^2}{2}\\right)}dt``
"""
fresnelc(z::Number) = 0.25*(1-1im)*(1im*erf(0.5*(1-1im)*z*√(π)) + erf(0.5*(1+1im)*z*√(π)))

"""
        fresnels(z)
Calculates the Fresnel sine integral for the number z for
    ``S(z) = \\int_{0}^{z} \\sin{\\left(\\frac{\\pi t^2}{2}\\right)}dt``
"""
fresnels(z::Number) = 0.25*(1+1im)*(-1im*erf(0.5*(1-1im)*z*√(π)) + erf(0.5*(1+1im)*z*√(π)))

"""
        fresnel(z)
Calculates the cosine and sine fresnel integrals
"""
fresnel(z::Number) = (fresnelc(z),fresnels(z))

end # module


import RowEchelon
import LinearAlgebra

#=
Homework 7
Section 3.2
Problems 26
=#

# Create a matrix with the vectors as columns
A = [3 2 -2 0 ;
     5 -6 -1 0 ;
     -6 0 3 0 ;
     4 7 0 -3]

# Find the detA
detA = LinearAlgebra.det(A)

println("Original Matrix:")
println(A)
println("Determinate:")
println(detA)

# See if the Determine was 0
# Dont forget floating point error
if (detA > 0.000001 || detA < -0.000001)
    println("Hence the original matrix is invertable, and its columns form a linearly independent set.")
else
    println("Hence the original matrix is NOT invertable, and its columns form a linearly dependent set.")
end


mutable struct MeanSquaredError <: DiffableFunction
    grad_field :: GradField
end

function forward(::MeanSquaredError, y_true, y_pred)
    diff  = y_true .- y_pred
    loss = Base.sum((x -> x^2), diff) / length(diff)
    return loss
end

function backward(f::MeanSquaredError, gy)
    y_true, y_pred = f.grad_field.inputs
    diff = y_true .- y_pred
    gy = broadcast_to(gy, size(diff.values))
    gx1 = gy .* diff .* (2. / length(diff.values))
    gx2 = -gx1
    return (gx1, gx2)
end

"""
    mean_squared_error(y_true, y_pred)

# Examples
```julia-repl
julia> y_true = Variable([10, 20, 30])
name: nothing
values: [10, 20, 30]
creator: User-Defined(nothing)

julia> y_pred = Variable([12, 20, 25])
name: nothing
values: [12, 20, 25]
creator: User-Defined(nothing)

julia> mean_squared_error(y_true, y_pred)
name: nothing
values: 9.666666666666666
creator: JITrench.MeanSquaredError
```
"""
mean_squared_error(y_true, y_pred) = MeanSquaredError(GradField())(y_true, y_pred)


using FdeSolver
using Plots

## inputs
tSpan = [0, 5]     # [intial time, final time]
y0 = 1             # intial value
β = 0.6            # order of the derivative

λ = -10
par = λ
F(t, y, par)= par * y
JF(t, y, par) = par

## Numerical solution
t,y = FDEsolver(F, tSpan, y0, β, par)
t_J, y_J = FDEsolver(F, tSpan, y0, β, par, JF = JF)

using MittagLeffler # for the exact solution

plot(t, y, linewidth = 5, title = "Solution to a linear fractional equation",
    xaxis = "Time (t)", yaxis = "y(t)", ls = :dot, label = "Approximation")
plot!(t, t -> mittleff(β, λ * t .^ β), linewidth = 5, label = "Exact solution")
plot!(t_J, y_J, linewidth = 5, ls = :dash, label = "Approximation with jacobian")


# ## Adding LaTeX to plots (2)

#md # ![FILE_NAME.png](images/FILE_NAME.png)

# For pyplot backend

## ref https://jp.mathworks.com/matlabcentral/fileexchange/35237-matlab-plot-gallery-adding-latex-to-plots?focused=6792946&tab=example

using LaTeXStrings, Measures
pyplot()
Plots.PyPlot.rc("text", usetex ="true")

fib = zeros(12);
for i = 1:12
    fib[i] = (((1+sqrt(5))/2)^i - ((1-sqrt(5))/2)^i)/sqrt(5);
end

plot(fib,
    marker=:circle,
    xlabel=L"n",
    ylabel=L"F_n",
    annotation=(5, 100, L"F_n = \frac{1}{\sqrt{5}} \left[\left( \frac{1+\sqrt{5}}{2} \right)^n - \left( \frac{1-\sqrt{5}}{2} \right)^n \right]")
)