text:
"""
    DataFrame <: AbstractDataFrame

An AbstractDataFrame that stores a set of named columns

The columns are normally AbstractVectors stored in memory,
particularly a Vector or CategoricalVector.

# Constructors
```julia
DataFrame(pairs::Pair...; makeunique::Bool=false, copycols::Bool=true)
DataFrame(pairs::AbstractVector{<:Pair}; makeunique::Bool=false, copycols::Bool=true)
DataFrame(ds::AbstractDict; copycols::Bool=true)
DataFrame(kwargs..., copycols::Bool=true)

DataFrame(columns::AbstractVecOrMat,
          names::AbstractVector;
          makeunique::Bool=false, copycols::Bool=true)

DataFrame(table; copycols::Union{Bool, Nothing}=nothing)
DataFrame(::DataFrameRow)
DataFrame(::GroupedDataFrame; keepkeys::Bool=true)
```

# Keyword arguments

- `copycols` : whether vectors passed as columns should be copied; by default set
  to `true` and the vectors are copied; if set to `false` then the constructor
  will still copy the passed columns if it is not possible to construct a
  `DataFrame` without materializing new columns. Note the `copycols=nothing`
  default in the Tables.jl compatible constructor; it is provided as certain
  input table types may have already made a copy of columns or the columns may
  otherwise be immutable, in which case columns are not copied by default.
  To force a copy in such cases, or to get mutable columns from an immutable
  input table (like `Arrow.Table`), pass `copycols=true` explicitly.
- `makeunique` : if `false` (the default), an error will be raised

(note that not all constructors support these keyword arguments)

# Details on behavior of different constructors

It is allowed to pass a vector of `Pair`s, a list of `Pair`s as positional
arguments, or a list of keyword arguments. In this case each pair is considered
to represent a column name to column value mapping and column name must be a
`Symbol` or string. Alternatively a dictionary can be passed to the constructor
in which case its entries are considered to define the column name and column
value pairs. If the dictionary is a `Dict` then column names will be sorted in
the returned `DataFrame`.

In all the constructors described above column value can be a vector which is
consumed as is or an object of any other type (except `AbstractArray`). In the
latter case the passed value is automatically repeated to fill a new vector of
the appropriate length. As a particular rule values stored in a `Ref` or a
`0`-dimensional `AbstractArray` are unwrapped and treated in the same way.

It is also allowed to pass a vector of vectors or a matrix as as the first
argument. In this case the second argument must be
a vector of `Symbol`s or strings specifying column names, or the symbol `:auto`
to generate column names `x1`, `x2`, ... automatically. Note that in this case
if the first argument is a matrix and `copycols=false` the columns of the created
`DataFrame` will be views of columns the source matrix.

If a single positional argument is passed to a `DataFrame` constructor then it
is assumed to be of type that implements the
[Tables.jl](https://github.com/JuliaData/Tables.jl) interface using which the
returned `DataFrame` is materialized.

Finally it is allowed to construct a `DataFrame` from a `DataFrameRow` or a
`GroupedDataFrame`. In the latter case the `keepkeys` keyword argument specifies
whether the resulting `DataFrame` should contain the grouping columns of the
passed `GroupedDataFrame` and the order of rows in the result follows the order
of groups in the `GroupedDataFrame` passed.

# Notes

The `DataFrame` constructor by default copies all columns vectors passed to it.
Pass the `copycols=false` keyword argument (where supported) to reuse vectors without
copying them.

By default an error will be raised if duplicates in column names are found. Pass
`makeunique=true` keyword argument (where supported) to accept duplicate names,
in which case they will be suffixed with `_i` (`i` starting at 1 for the first
duplicate).

If an `AbstractRange` is passed to a `DataFrame` constructor as a column it is
always collected to a `Vector` (even if `copycols=false`). As a general rule
`AbstractRange` values are always materialized to a `Vector` by all functions in
DataFrames.jl before being stored in a `DataFrame`.

`DataFrame` can store only columns that use 1-based indexing. Attempting
to store a vector using non-standard indexing raises an error.

The `DataFrame` type is designed to allow column types to vary and to be
dynamically changed also after it is constructed. Therefore `DataFrame`s are not
type stable. For performance-critical code that requires type-stability either
use the functionality provided by `select`/`transform`/`combine` functions, use
`Tables.columntable` and `Tables.namedtupleiterator` functions, use barrier
functions, or provide type assertions to the variables that hold columns
extracted from a `DataFrame`.

# Examples
```jldoctest
julia> DataFrame((a=[1, 2], b=[3, 4])) # Tables.jl table constructor
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      3
   2 │     2      4

julia> DataFrame([(a=1, b=0), (a=2, b=0)]) # Tables.jl table constructor
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      0
   2 │     2      0

julia> DataFrame("a" => 1:2, "b" => 0) # Pair constructor
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      0
   2 │     2      0

julia> DataFrame([:a => 1:2, :b => 0]) # vector of Pairs constructor
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      0
   2 │     2      0

julia> DataFrame(Dict(:a => 1:2, :b => 0)) # dictionary constructor
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      0
   2 │     2      0

julia> DataFrame(a=1:2, b=0) # keyword argument constructor
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      0
   2 │     2      0

julia> DataFrame([[1, 2], [0, 0]], [:a, :b]) # vector of vectors constructor
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      0
   2 │     2      0

julia> DataFrame([1 0; 2 0], :auto) # matrix constructor
2×2 DataFrame
 Row │ x1     x2
     │ Int64  Int64
─────┼──────────────
   1 │     1      0
   2 │     2      0
```
"""
struct DataFrame <: AbstractDataFrame
    columns::Vector{AbstractVector}
    colindex::Index

    # the inner constructor should not be used directly
    function DataFrame(columns::Union{Vector{Any}, Vector{AbstractVector}},
                       colindex::Index; copycols::Bool=true)
        if length(columns) == length(colindex) == 0
            return new(AbstractVector[], Index())
        elseif length(columns) != length(colindex)
            throw(DimensionMismatch("Number of columns ($(length(columns))) and number of " *
                                    "column names ($(length(colindex))) are not equal"))
        end

        len = -1
        firstvec = -1
        for (i, col) in enumerate(columns)
            if col isa AbstractVector
                if len == -1
                    len = length(col)
                    firstvec = i
                elseif len != length(col)
                    n1 = _names(colindex)[firstvec]
                    n2 = _names(colindex)[i]
                    throw(DimensionMismatch("column :$n1 has length $len and column " *
                                            ":$n2 has length $(length(col))"))
                end
            end
        end
        len == -1 && (len = 1) # we got no vectors so make one row of scalars

        # we write into columns as we know that it is guaranteed
        # that it was freshly allocated in the outer constructor
        @static if VERSION >= v"1.4"
            if copycols && len >= 1_000_000 && length(columns) > 1 && Threads.nthreads() > 1
                @sync for i in eachindex(columns)
                    Threads.@spawn columns[i] = _preprocess_column(columns[i], len, copycols)
                end
            else
                for i in eachindex(columns)
                    columns[i] = _preprocess_column(columns[i], len, copycols)
                end
            end
        else
            for i in eachindex(columns)
                columns[i] = _preprocess_column(columns[i], len, copycols)
            end
        end

        for (i, col) in enumerate(columns)
            firstindex(col) != 1 && _onebased_check_error(i, col)
        end

        new(convert(Vector{AbstractVector}, columns), colindex)
    end
end

function _preprocess_column(col::Any, len::Integer, copycols::Bool)
    if col isa AbstractRange
        return collect(col)
    elseif col isa AbstractVector
        return copycols ? copy(col) : col
    elseif col isa Union{AbstractArray{<:Any, 0}, Ref}
        x = col[]
        return fill!(Tables.allocatecolumn(typeof(x), len), x)
    elseif col isa AbstractArray
        throw(ArgumentError("adding AbstractArray other than AbstractVector " *
                            "as a column of a data frame is not allowed"))
    else
        return fill!(Tables.allocatecolumn(typeof(col), len), col)
    end
end

DataFrame(df::DataFrame; copycols::Bool=true) = copy(df, copycols=copycols)

function DataFrame(pairs::Pair{Symbol, <:Any}...; makeunique::Bool=false,
                   copycols::Bool=true)::DataFrame
    colnames = [Symbol(k) for (k, v) in pairs]
    columns = Any[v for (k, v) in pairs]
    return DataFrame(columns, Index(colnames, makeunique=makeunique),
                     copycols=copycols)
end

function DataFrame(pairs::Pair{<:AbstractString, <:Any}...; makeunique::Bool=false,
                   copycols::Bool=true)::DataFrame
    colnames = [Symbol(k) for (k, v) in pairs]
    columns = Any[v for (k, v) in pairs]
    return DataFrame(columns, Index(colnames, makeunique=makeunique),
                     copycols=copycols)
end

# this is needed as a workaround for Tables.jl dispatch
function DataFrame(pairs::AbstractVector{<:Pair}; makeunique::Bool=false,
                   copycols::Bool=true)
    if isempty(pairs)
        return DataFrame()
    else
        if !(all(((k, v),) -> k isa Symbol, pairs) || all(((k, v),) -> k isa AbstractString, pairs))
            throw(ArgumentError("All column names must be either Symbols or strings (mixing is not allowed)"))
        end
        colnames = [Symbol(k) for (k, v) in pairs]
        columns = Any[v for (k, v) in pairs]
        return DataFrame(columns, Index(colnames, makeunique=makeunique),
                         copycols=copycols)
    end
end

function DataFrame(d::AbstractDict; copycols::Bool=true)
    if all(k -> k isa Symbol, keys(d))
        colnames = collect(Symbol, keys(d))
    elseif all(k -> k isa AbstractString, keys(d))
        colnames = [Symbol(k) for k in keys(d)]
    else
        throw(ArgumentError("All column names must be either Symbols or strings (mixing is not allowed)"))
    end

    colindex = Index(colnames)
    columns = Any[v for v in values(d)]
    df = DataFrame(columns, colindex, copycols=copycols)
    d isa Dict && select!(df, sort!(propertynames(df)))
    return df
end

function DataFrame(; kwargs...)
    if isempty(kwargs)
        DataFrame([], Index())
    else
        cnames = Symbol[]
        columns = Any[]
        copycols = true
        for (kw, val) in kwargs
            if kw === :copycols
                if val isa Bool
                    copycols = val
                else
                    throw(ArgumentError("the `copycols` keyword argument must be Boolean"))
                end
            elseif kw === :makeunique
                    throw(ArgumentError("the `makeunique` keyword argument is not allowed " *
                                        "in DataFrame(; kwargs...) constructor"))
            else
                push!(cnames, kw)
                push!(columns, val)
            end
        end
        DataFrame(columns, Index(cnames), copycols=copycols)
    end
end

function DataFrame(columns::AbstractVector, cnames::AbstractVector{Symbol};
                   makeunique::Bool=false, copycols::Bool=true)::DataFrame
    if !(eltype(columns) <: AbstractVector) && !all(col -> isa(col, AbstractVector), columns)
        throw(ArgumentError("columns argument must be a vector of AbstractVector objects"))
    end
    return DataFrame(collect(AbstractVector, columns),
                     Index(convert(Vector{Symbol}, cnames), makeunique=makeunique),
                     copycols=copycols)
end

function _name2symbol(str::AbstractVector)
    if !(all(x -> x isa AbstractString, str) || all(x -> x isa Symbol, str))
        throw(ArgumentError("All passed column names must be strings or Symbols"))
    end
    return Symbol[Symbol(s) for s in str]
end

DataFrame(columns::AbstractVector, cnames::AbstractVector;
          makeunique::Bool=false, copycols::Bool=true) =
    DataFrame(columns, _name2symbol(cnames), makeunique=makeunique, copycols=copycols)

DataFrame(columns::AbstractVector{<:AbstractVector}, cnames::AbstractVector{Symbol};
          makeunique::Bool=false, copycols::Bool=true)::DataFrame =
    DataFrame(collect(AbstractVector, columns),
              Index(convert(Vector{Symbol}, cnames), makeunique=makeunique),
              copycols=copycols)

DataFrame(columns::AbstractVector{<:AbstractVector}, cnames::AbstractVector;
          makeunique::Bool=false, copycols::Bool=true) =
    DataFrame(columns, _name2symbol(cnames); makeunique=makeunique, copycols=copycols)

function DataFrame(columns::AbstractVector, cnames::Symbol; copycols::Bool=true)
    if cnames !== :auto
        throw(ArgumentError("if the first positional argument to DataFrame " *
                            "constructor is a vector of vectors and the second " *
                            "positional argument is passed then the second " *
                            "argument must be a vector of column names or :auto"))
    end
    return DataFrame(columns, gennames(length(columns)), copycols=copycols)
end

function DataFrame(columns::AbstractMatrix, cnames::AbstractVector{Symbol};
                   makeunique::Bool=false, copycols::Bool=true)
    getter = copycols ? getindex : view
    return DataFrame(AbstractVector[getter(columns, :, i) for i in 1:size(columns, 2)],
                     cnames, makeunique=makeunique, copycols=false)
end

DataFrame(columns::AbstractMatrix, cnames::AbstractVector;
          makeunique::Bool=false, copycols::Bool=true) =
    DataFrame(columns, _name2symbol(cnames); makeunique=makeunique, copycols=copycols)

function DataFrame(columns::AbstractMatrix, cnames::Symbol; copycols::Bool=true)
    if cnames !== :auto
        throw(ArgumentError("if the first positional argument to DataFrame " *
                            "constructor is a matrix and a second " *
                            "positional argument is passed then the second " *
                            "argument must be a vector of column names or :auto"))
    end
    return DataFrame(columns, gennames(size(columns, 2)), makeunique=false, copycols=copycols)
end

# Discontinued constructors

DataFrame(matrix::Matrix) =
    throw(ArgumentError("`DataFrame` constructor from a `Matrix` requires " *
                        "passing :auto as a second argument to automatically " *
                        "generate column names: `DataFrame(matrix, :auto)`"))

DataFrame(vecs::Vector{<:AbstractVector}) =
    throw(ArgumentError("`DataFrame` constructor from a `Vector` of vectors requires " *
                        "passing :auto as a second argument to automatically " *
                        "generate column names: `DataFrame(vecs, :auto)`"))

DataFrame(column_eltypes::AbstractVector{T}, cnames::AbstractVector{Symbol},
          nrows::Integer=0; makeunique::Bool=false) where T<:Type =
    throw(ArgumentError("`DataFrame` constructor with passed eltypes is " *
                        "not supported. Pass explicitly created columns to a " *
                        "`DataFrame` constructor instead."))

DataFrame(column_eltypes::AbstractVector{<:Type}, cnames::AbstractVector{<:AbstractString},
          nrows::Integer=0; makeunique::Bool=false) where T<:Type =
    throw(ArgumentError("`DataFrame` constructor with passed eltypes is " *
                        "not supported. Pass explicitly created columns to a " *
                        "`DataFrame` constructor instead."))

##############################################################################
##
## AbstractDataFrame interface
##
##############################################################################

index(df::DataFrame) = getfield(df, :colindex)

# this function grants the access to the internal storage of columns of the
# `DataFrame` and its use is unsafe. If the returned vector is mutated then
# make sure that:
# 1. `AbstractRange` columns are not added to a `DataFrame`
# 2. all inserted columns use 1-based indexing
# 3. after several mutating operations on the vector are performed
#    each element (column) has the same length
# 4. if length of the vector is changed that the index of the `DataFrame`
#    is adjusted appropriately
_columns(df::DataFrame) = getfield(df, :columns)

_onebased_check_error() =
    throw(ArgumentError("Currently DataFrames.jl supports only columns " *
                        "that use 1-based indexing"))
_onebased_check_error(i, col) =
    throw(ArgumentError("Currently DataFrames.jl supports only " *
                        "columns that use 1-based indexing, but " *
                        "column $i has starting index equal to $(firstindex(col))"))

# note: these type assertions are required to pass tests
nrow(df::DataFrame) = ncol(df) > 0 ? length(_columns(df)[1])::Int : 0
ncol(df::DataFrame) = length(index(df))

##############################################################################
##
## DataFrame consistency check
##
##############################################################################

corrupt_msg(df::DataFrame, i::Integer) =
    "Data frame is corrupt: length of column " *
    ":$(_names(df)[i]) ($(length(df[!, i]))) " *
    "does not match length of column 1 ($(length(df[!, 1]))). " *
    "The column vector has likely been resized unintentionally " *
    "(either directly or because it is shared with another data frame)."

function _check_consistency(df::DataFrame)
    cols, idx = _columns(df), index(df)

    for (i, col) in enumerate(cols)
        firstindex(col) != 1 && _onebased_check_error(i, col)
    end

    ncols = length(cols)
    @assert length(idx.names) == length(idx.lookup) == ncols
    ncols == 0 && return nothing
    nrows = length(cols[1])
    for i in 2:length(cols)
        @assert length(cols[i]) == nrows corrupt_msg(df, i)
    end
    nothing
end

_check_consistency(df::AbstractDataFrame) = _check_consistency(parent(df))

##############################################################################
##
## getindex() definitions
##
##############################################################################

# df[SingleRowIndex, SingleColumnIndex] => Scalar
@inline function Base.getindex(df::DataFrame, row_ind::Integer,
                               col_ind::Union{Signed, Unsigned})
    cols = _columns(df)
    @boundscheck begin
        if !checkindex(Bool, axes(cols, 1), col_ind)
            throw(BoundsError(df, (row_ind, col_ind)))
        end
        if !checkindex(Bool, axes(df, 1), row_ind)
            throw(BoundsError(df, (row_ind, col_ind)))
        end
    end

    @inbounds cols[col_ind][row_ind]
end

@inline function Base.getindex(df::DataFrame, row_ind::Integer,
                               col_ind::SymbolOrString)
    selected_column = index(df)[col_ind]
    @boundscheck if !checkindex(Bool, axes(df, 1), row_ind)
        throw(BoundsError(df, (row_ind, col_ind)))
    end
    @inbounds _columns(df)[selected_column][row_ind]
end

# df[MultiRowIndex, SingleColumnIndex] => AbstractVector, copy
@inline function Base.getindex(df::DataFrame, row_inds::AbstractVector, col_ind::ColumnIndex)
    selected_column = index(df)[col_ind]
    @boundscheck if !checkindex(Bool, axes(df, 1), row_inds)
        throw(BoundsError(df, (row_inds, col_ind)))
    end
    @inbounds return _columns(df)[selected_column][row_inds]
end

@inline Base.getindex(df::DataFrame, row_inds::Not, col_ind::ColumnIndex) =
    df[axes(df, 1)[row_inds], col_ind]

# df[:, SingleColumnIndex] => AbstractVector
function Base.getindex(df::DataFrame, row_inds::Colon, col_ind::ColumnIndex)
    selected_column = index(df)[col_ind]
    copy(_columns(df)[selected_column])
end

# df[!, SingleColumnIndex] => AbstractVector, the same vector
@inline function Base.getindex(df::DataFrame, ::typeof(!), col_ind::Union{Signed, Unsigned})
    cols = _columns(df)
    @boundscheck if !checkindex(Bool, axes(cols, 1), col_ind)
        throw(BoundsError(df, (!, col_ind)))
    end
    @inbounds cols[col_ind]
end

function Base.getindex(df::DataFrame, ::typeof(!), col_ind::SymbolOrString)
    selected_column = index(df)[col_ind]
    return _columns(df)[selected_column]
end

# df[MultiRowIndex, MultiColumnIndex] => DataFrame

function _threaded_getindex(selected_rows::AbstractVector,
                            selected_columns::AbstractVector,
                            df_columns::AbstractVector,
                            idx::AbstractIndex)
    @static if VERSION >= v"1.4"
        if length(selected_rows) >= 1_000_000 && Threads.nthreads() > 1
            new_columns = Vector{AbstractVector}(undef, length(selected_columns))
            @sync for i in eachindex(new_columns)
                Threads.@spawn new_columns[i] = df_columns[selected_columns[i]][selected_rows]
            end
            return DataFrame(new_columns, idx, copycols=false)
        else
            return DataFrame(AbstractVector[df_columns[i][selected_rows] for i in selected_columns],
                             idx, copycols=false)
        end
    else
        return DataFrame(AbstractVector[df_columns[i][selected_rows] for i in selected_columns],
                         idx, copycols=false)
    end
end

@inline function Base.getindex(df::DataFrame, row_inds::AbstractVector{T},
                               col_inds::MultiColumnIndex) where T
    @boundscheck if !checkindex(Bool, axes(df, 1), row_inds)
        throw(BoundsError(df, (row_inds, col_inds)))
    end
    selected_columns = index(df)[col_inds]

    u = _names(df)[selected_columns]
    lookup = Dict{Symbol, Int}(zip(u, 1:length(u)))
    # use this constructor to avoid checking twice if column names are not
    # duplicate as index(df)[col_inds] already checks this
    idx = Index(lookup, u)

    if length(selected_columns) == 1
        return DataFrame(AbstractVector[_columns(df)[selected_columns[1]][row_inds]],
                         idx, copycols=false)
    else
        # Computing integer indices once for all columns is faster
        selected_rows = T === Bool ? _findall(row_inds) : row_inds
        _threaded_getindex(selected_rows, selected_columns, _columns(df), idx)
    end
end

@inline function Base.getindex(df::DataFrame, row_inds::AbstractVector{T}, ::Colon) where T
    @boundscheck if !checkindex(Bool, axes(df, 1), row_inds)
        throw(BoundsError(df, (row_inds, :)))
    end
    idx = copy(index(df))

    if ncol(df) == 1
        return DataFrame(AbstractVector[_columns(df)[1][row_inds]], idx, copycols=false)
    else
        # Computing integer indices once for all columns is faster
        selected_rows = T === Bool ? _findall(row_inds) : row_inds
        _threaded_getindex(selected_rows, 1:ncol(df), _columns(df), idx)
    end
end

@inline Base.getindex(df::DataFrame, row_inds::Not, col_inds::MultiColumnIndex) =
    df[axes(df, 1)[row_inds], col_inds]

# df[:, MultiColumnIndex] => DataFrame
Base.getindex(df::DataFrame, row_ind::Colon, col_inds::MultiColumnIndex) =
    select(df, index(df)[col_inds], copycols=true)

# df[!, MultiColumnIndex] => DataFrame
Base.getindex(df::DataFrame, row_ind::typeof(!), col_inds::MultiColumnIndex) =
    select(df, index(df)[col_inds], copycols=false)

##############################################################################
##
## setindex!()
##
##############################################################################

# Will automatically add a new column if needed
function insert_single_column!(df::DataFrame, v::AbstractVector, col_ind::ColumnIndex)
    if ncol(df) != 0 && nrow(df) != length(v)
        throw(ArgumentError("New columns must have the same length as old columns"))
    end
    dv = isa(v, AbstractRange) ? collect(v) : v
    firstindex(dv) != 1 && _onebased_check_error()

    if haskey(index(df), col_ind)
        j = index(df)[col_ind]
        _columns(df)[j] = dv
    else
        if col_ind isa SymbolOrString
            push!(index(df), Symbol(col_ind))
            push!(_columns(df), dv)
        else
            throw(ArgumentError("Cannot assign to non-existent column: $col_ind"))
        end
    end
    return dv
end

function insert_single_entry!(df::DataFrame, v::Any, row_ind::Integer, col_ind::ColumnIndex)
    if haskey(index(df), col_ind)
        _columns(df)[index(df)[col_ind]][row_ind] = v
        return v
    else
        throw(ArgumentError("Cannot assign to non-existent column: $col_ind"))
    end
end

# df[!, SingleColumnIndex] = AbstractVector
function Base.setindex!(df::DataFrame, v::AbstractVector, ::typeof(!), col_ind::ColumnIndex)
    insert_single_column!(df, v, col_ind)
    return df
end

# df.col = AbstractVector
# separate methods are needed due to dispatch ambiguity
Base.setproperty!(df::DataFrame, col_ind::Symbol, v::AbstractVector) =
    (df[!, col_ind] = v)
Base.setproperty!(df::DataFrame, col_ind::AbstractString, v::AbstractVector) =
    (df[!, col_ind] = v)
Base.setproperty!(::DataFrame, col_ind::Symbol, v::Any) =
    throw(ArgumentError("It is only allowed to pass a vector as a column of a DataFrame. " *
                        "Instead use `df[!, col_ind] .= v` if you want to use broadcasting."))
Base.setproperty!(::DataFrame, col_ind::AbstractString, v::Any) =
    throw(ArgumentError("It is only allowed to pass a vector as a column of a DataFrame. " *
                        "Instead use `df[!, col_ind] .= v` if you want to use broadcasting."))

# df[SingleRowIndex, SingleColumnIndex] = Single Item
function Base.setindex!(df::DataFrame, v::Any, row_ind::Integer, col_ind::ColumnIndex)
    insert_single_entry!(df, v, row_ind, col_ind)
    return df
end

# df[SingleRowIndex, MultiColumnIndex] = value
# the method for value of type DataFrameRow, AbstractDict and NamedTuple
# is defined in dataframerow.jl

for T in MULTICOLUMNINDEX_TUPLE
    @eval function Base.setindex!(df::DataFrame,
                                  v::Union{Tuple, AbstractArray},
                                  row_ind::Integer,
                                  col_inds::$T)
        idxs = index(df)[col_inds]
        if length(v) != length(idxs)
            throw(DimensionMismatch("$(length(idxs)) columns were selected but the assigned " *
                                    "collection contains $(length(v)) elements"))
        end
        for (i, x) in zip(idxs, v)
            df[row_ind, i] = x
        end
        return df
    end
end

# df[MultiRowIndex, SingleColumnIndex] = AbstractVector
for T in (:AbstractVector, :Not, :Colon)
    @eval function Base.setindex!(df::DataFrame,
                                  v::AbstractVector,
                                  row_inds::$T,
                                  col_ind::ColumnIndex)
        if row_inds isa Colon && !haskey(index(df), col_ind)
            df[!, col_ind] = copy(v)
            return df
        end
        x = df[!, col_ind]
        x[row_inds] = v
        return df
    end
end

# df[MultiRowIndex, MultiColumnIndex] = AbstractDataFrame
for T1 in (:AbstractVector, :Not, :Colon),
    T2 in MULTICOLUMNINDEX_TUPLE
    @eval function Base.setindex!(df::DataFrame,
                                  new_df::AbstractDataFrame,
                                  row_inds::$T1,
                                  col_inds::$T2)
        idxs = index(df)[col_inds]
        if view(_names(df), idxs) != _names(new_df)
            throw(ArgumentError("column names in source and target do not match"))
        end
        for (j, col) in enumerate(idxs)
            df[row_inds, col] = new_df[!, j]
        end
        return df
    end
end

for T in MULTICOLUMNINDEX_TUPLE
    @eval function Base.setindex!(df::DataFrame,
                                  new_df::AbstractDataFrame,
                                  row_inds::typeof(!),
                                  col_inds::$T)
        idxs = index(df)[col_inds]
        if view(_names(df), idxs) != _names(new_df)
            throw(ArgumentError("Column names in source and target data frames do not match"))
        end
        for (j, col) in enumerate(idxs)
            # make sure we make a copy on assignment
            df[!, col] = new_df[:, j]
        end
        return df
    end
end

# df[MultiRowIndex, MultiColumnIndex] = AbstractMatrix
for T1 in (:AbstractVector, :Not, :Colon, :(typeof(!))),
    T2 in MULTICOLUMNINDEX_TUPLE
    @eval function Base.setindex!(df::DataFrame,
                                  mx::AbstractMatrix,
                                  row_inds::$T1,
                                  col_inds::$T2)
        idxs = index(df)[col_inds]
        if size(mx, 2) != length(idxs)
            throw(DimensionMismatch("number of selected columns ($(length(idxs))) " *
                                    "and number of columns in " *
                                    "matrix ($(size(mx, 2))) do not match"))
        end
        for (j, col) in enumerate(idxs)
            df[row_inds, col] = (row_inds === !) ? mx[:, j] : view(mx, :, j)
        end
        return df
    end
end

# insertcols!
# TODO: move to abstractdataframe/abstractdataframe.jl

"""
    insertcols!(df::AbstractDataFrame[, col], (name=>val)::Pair...;
                after::Bool=false, makeunique::Bool=false, copycols::Bool=true)

Insert a column into a data frame in place. Return the updated data frame.
If `col` is omitted it is set to `ncol(df)+1`
(the column is inserted as the last column).

# Arguments
- `df` : the data frame to which we want to add columns
- `col` : a position at which we want to insert a column, passed as an integer
  or a column name (a string or a `Symbol`); the column selected with `col`
  and columns following it are shifted to the right in `df` after the operation
- `name` : the name of the new column
- `val` : an `AbstractVector` giving the contents of the new column or a value of any
  type other than `AbstractArray` which will be repeated to fill a new vector;
  As a particular rule a values stored in a `Ref` or a `0`-dimensional `AbstractArray`
  are unwrapped and treated in the same way
- `after` : if `true` columns are inserted after `col`
- `makeunique` : defines what to do if `name` already exists in `df`;
  if it is `false` an error will be thrown; if it is `true` a new unique name will
  be generated by adding a suffix
- `copycols` : whether vectors passed as columns should be copied

If `val` is an `AbstractRange` then the result of `collect(val)` is inserted.

If `df` is a `SubDataFrame` then it must have been created with `:` as column selector
(otherwise an error is thrown). In this case the `copycols` keyword argument
is ignored (i.e. the added column is always copied) and the parent data frame's
column is filled with `missing` in rows that are filtered out by `df`.

If `df` isa `DataFrame` that has no columns and only values
other than `AbstractVector` are passed then it is used to create a one-element
column.
If `df` isa `DataFrame` that has no columns and at least one `AbstractVector` is
passed then its length is used to determine the number of elements in all
created columns.
In all other cases the number of rows in all created columns must match
`nrow(df)`.
.

# Examples
```jldoctest
julia> df = DataFrame(a=1:3)
3×1 DataFrame
 Row │ a
     │ Int64
─────┼───────
   1 │     1
   2 │     2
   3 │     3

julia> insertcols!(df, 1, :b => 'a':'c')
3×2 DataFrame
 Row │ b     a
     │ Char  Int64
─────┼─────────────
   1 │ a         1
   2 │ b         2
   3 │ c         3

julia> insertcols!(df, 2, :c => 2:4, :c => 3:5, makeunique=true)
3×4 DataFrame
 Row │ b     c      c_1    a
     │ Char  Int64  Int64  Int64
─────┼───────────────────────────
   1 │ a         2      3      1
   2 │ b         3      4      2
   3 │ c         4      5      3

julia> insertcols!(df, :b, :d => 7:9, after=true)
3×5 DataFrame
 Row │ b     d      c      c_1    a
     │ Char  Int64  Int64  Int64  Int64
─────┼──────────────────────────────────
   1 │ a         7      2      3      1
   2 │ b         8      3      4      2
   3 │ c         9      4      5      3
```
"""
function insertcols!(df::AbstractDataFrame, col::ColumnIndex, name_cols::Pair{Symbol, <:Any}...;
                     after::Bool=false, makeunique::Bool=false, copycols::Bool=true)
    if !is_column_insertion_allowed(df)
        throw(ArgumentError("insertcols! is only supported for DataFrame, or for " *
                            "SubDataFrame created with `:` as column selector"))
    end
    if !(copycols || df isa DataFrame)
        throw(ArgumentError("copycols=false is only allowed if df isa DataFrame "))
    end
    if col isa SymbolOrString
        col_ind = Int(columnindex(df, col))
        if col_ind == 0
            throw(ArgumentError("column $col does not exist in data frame"))
        end
    else
        col_ind = Int(col)
    end

    if after
        col_ind += 1
    end

    if !(0 < col_ind <= ncol(df) + 1)
        throw(ArgumentError("attempt to insert a column to a data frame with " *
                            "$(ncol(df)) columns at index $col_ind"))
    end

    if !makeunique
        if !allunique(first.(name_cols))
            throw(ArgumentError("Names of columns to be inserted into a data frame " *
                                "must be unique when `makeunique=true`"))
        end
        for (n, _) in name_cols
            if hasproperty(df, n)
                throw(ArgumentError("Column $n is already present in the data frame " *
                                    "which is not allowed when `makeunique=true`"))
            end
        end
    end

    if ncol(df) == 0 && df isa DataFrame
        target_row_count = -1
    else
        target_row_count = nrow(df)
    end

    for (n, v) in name_cols
        if v isa AbstractVector
            if target_row_count == -1
                target_row_count = length(v)
            elseif length(v) != target_row_count
                if target_row_count == nrow(df)
                    throw(DimensionMismatch("length of new column $n which is " *
                                            "$(length(v)) must match the number " *
                                            "of rows in data frame ($(nrow(df)))"))
                else
                    throw(DimensionMismatch("all vectors passed to be inserted into " *
                                            "a data frame must have the same length"))
                end
            end
        elseif v isa AbstractArray && ndims(v) > 1
            throw(ArgumentError("adding AbstractArray other than AbstractVector as " *
                                "a column of a data frame is not allowed"))
        end
    end
    if target_row_count == -1
        target_row_count = 1
    end

    for (name, item) in name_cols
        if !(item isa AbstractVector)
            if item isa Union{AbstractArray{<:Any, 0}, Ref}
                x = item[]
                item_new = fill!(Tables.allocatecolumn(typeof(x), target_row_count), x)
            else
                @assert !(item isa AbstractArray)
                item_new = fill!(Tables.allocatecolumn(typeof(item), target_row_count), item)
            end
        elseif item isa AbstractRange
            item_new = collect(item)
        elseif copycols && df isa DataFrame
            item_new = copy(item)
        else
            item_new = item
        end

        if df isa DataFrame
            dfp = df
        else
            @assert df isa SubDataFrame
            dfp = parent(df)
            item_new_orig = item_new
            T = eltype(item_new_orig)
            item_new = similar(item_new_orig, Union{T, Missing}, nrow(dfp))
            fill!(item_new, missing)
            item_new[rows(df)] = item_new_orig
        end

        firstindex(item_new) != 1 && _onebased_check_error()

        if ncol(dfp) == 0
            dfp[!, name] = item_new
        else
            if hasproperty(dfp, name)
                @assert makeunique
                k = 1
                while true
                    nn = Symbol("$(name)_$k")
                    if !hasproperty(dfp, nn)
                        name = nn
                        break
                    end
                    k += 1
                end
            end
            insert!(index(dfp), col_ind, name)
            insert!(_columns(dfp), col_ind, item_new)
        end
        col_ind += 1
    end
    return df
end

insertcols!(df::AbstractDataFrame, col::ColumnIndex, name_cols::Pair{<:AbstractString, <:Any}...;
            after::Bool=false, makeunique::Bool=false, copycols::Bool=true) =
    insertcols!(df, col, (Symbol(n) => v for (n, v) in name_cols)...,
                after=after, makeunique=makeunique, copycols=copycols)

insertcols!(df::AbstractDataFrame, name_cols::Pair{Symbol, <:Any}...;
            after::Bool=false, makeunique::Bool=false, copycols::Bool=true) =
    insertcols!(df, ncol(df)+1, name_cols..., after=after,
                makeunique=makeunique, copycols=copycols)

insertcols!(df::AbstractDataFrame, name_cols::Pair{<:AbstractString, <:Any}...;
            after::Bool=false, makeunique::Bool=false, copycols::Bool=true) =
    insertcols!(df, (Symbol(n) => v for (n, v) in name_cols)...,
                after=after, makeunique=makeunique, copycols=copycols)

function insertcols!(df::AbstractDataFrame, col::Int=ncol(df)+1; makeunique::Bool=false, name_cols...)
    if !(0 < col <= ncol(df) + 1)
        throw(ArgumentError("attempt to insert a column to a data frame with " *
                            "$(ncol(df)) columns at index $col"))
    end
    if !isempty(name_cols)
        # an explicit error is thrown as keyword argument was supported in the past
        throw(ArgumentError("inserting colums using a keyword argument is not supported, " *
                            "pass a Pair as a positional argument instead"))
    end
    return df
end

"""
    copy(df::DataFrame; copycols::Bool=true)

Copy data frame `df`.
If `copycols=true` (the default), return a new  `DataFrame` holding
copies of column vectors in `df`.
If `copycols=false`, return a new `DataFrame` sharing column vectors with `df`.
"""
function Base.copy(df::DataFrame; copycols::Bool=true)
    return DataFrame(copy(_columns(df)), copy(index(df)), copycols=copycols)
end

"""
    deleteat!(df::DataFrame, inds)

Delete rows specified by `inds` from a `DataFrame` `df` in place and return it.

Internally `deleteat!` is called for all columns so `inds` must be:
a vector of sorted and unique integers, a boolean vector, an integer,
or `Not` wrapping any valid selector.

# Examples
```jldoctest
julia> df = DataFrame(a=1:3, b=4:6)
3×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      4
   2 │     2      5
   3 │     3      6

julia> deleteat!(df, 2)
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      4
   2 │     3      6
```
"""
function Base.deleteat!(df::DataFrame, inds)
    if !isempty(inds) && size(df, 2) == 0
        throw(BoundsError(df, (inds, :)))
    end

    # workaround https://github.com/JuliaLang/julia/pull/41646
    if VERSION <= v"1.6.2" && inds isa UnitRange{<:Integer}
        inds = collect(inds)
    end

    # we require ind to be stored and unique like in Base
    # otherwise an error will be thrown and the data frame will get corrupted
    return _deleteat!_helper(df, inds)
end

function Base.deleteat!(df::DataFrame, inds::AbstractVector{Bool})
    if length(inds) != size(df, 1)
        throw(BoundsError(df, (inds, :)))
    end
    drop = _findall(inds)
    # workaround https://github.com/JuliaLang/julia/pull/41646
    if VERSION <= v"1.6.2" && drop isa UnitRange{<:Integer}
        drop = collect(drop)
    end
    return _deleteat!_helper(df, drop)
end

Base.deleteat!(df::DataFrame, inds::Not) = deleteat!(df, axes(df, 1)[inds])

function _deleteat!_helper(df::DataFrame, drop)
    cols = _columns(df)
    isempty(cols) && return df

    n = nrow(df)
    col1 = cols[1]
    deleteat!(col1, drop)
    newn = length(col1)

    for i in 2:length(cols)
        col = cols[i]
        if length(col) == n
            deleteat!(col, drop)
        end
    end

    for i in 1:length(cols)
        # this should never happen, but we add it for safety
        @assert length(cols[i]) == newn corrupt_msg(df, i)
    end

    return df
end

"""
    empty!(df::DataFrame)

Remove all rows from `df`, making each of its columns empty.
"""
function Base.empty!(df::DataFrame)
    foreach(empty!, eachcol(df))
    return df
end

##############################################################################
##
## hcat!
##
##############################################################################

# hcat! for 2 arguments, only a vector or a data frame is allowed
function hcat!(df1::DataFrame, df2::AbstractDataFrame;
               makeunique::Bool=false, copycols::Bool=true)
    u = add_names(index(df1), index(df2), makeunique=makeunique)
    for i in 1:length(u)
        df1[!, u[i]] = copycols ? df2[:, i] : df2[!, i]
    end
    return df1
end

# TODO: after deprecation remove AbstractVector methods

function hcat!(df::DataFrame, x::AbstractVector; makeunique::Bool=false, copycols::Bool=true)
    Base.depwarn("horizontal concatenation of data frame with a vector is deprecated. " *
                 "Pass DataFrame(x1=x) instead.", :hcat!)
    return hcat!(df, DataFrame(AbstractVector[x], [:x1], copycols=false),
                 makeunique=makeunique, copycols=copycols)
end

function hcat!(x::AbstractVector, df::DataFrame; makeunique::Bool=false, copycols::Bool=true)
    Base.depwarn("horizontal concatenation of data frame with a vector is deprecated. " *
                 "Pass DataFrame(x1=x) instead.", :hcat!)
    return hcat!(DataFrame(AbstractVector[x], [:x1], copycols=copycols), df,
                 makeunique=makeunique, copycols=copycols)
end

# hcat! for 1-n arguments
hcat!(df::DataFrame; makeunique::Bool=false, copycols::Bool=true) = df
hcat!(a::DataFrame, b::Union{AbstractDataFrame, AbstractVector},
      c::Union{AbstractDataFrame, AbstractVector}...;
      makeunique::Bool=false, copycols::Bool=true) =
    hcat!(hcat!(a, b, makeunique=makeunique, copycols=copycols),
          c..., makeunique=makeunique, copycols=copycols)

##############################################################################
##
## Missing values support
##
##############################################################################
"""
    allowmissing!(df::DataFrame, cols=:)

Convert columns `cols` of data frame `df` from element type `T` to
`Union{T, Missing}` to support missing values.

`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).

If `cols` is omitted all columns in the data frame are converted.
"""
function allowmissing! end

function allowmissing!(df::DataFrame, col::ColumnIndex)
    df[!, col] = allowmissing(df[!, col])
    return df
end

function allowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex})
    for col in cols
        allowmissing!(df, col)
    end
    return df
end

function allowmissing!(df::DataFrame, cols::AbstractVector{Bool})
    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))
    for (col, cond) in enumerate(cols)
        cond && allowmissing!(df, col)
    end
    return df
end

allowmissing!(df::DataFrame, cols::MultiColumnIndex) =
    allowmissing!(df, index(df)[cols])

allowmissing!(df::DataFrame, cols::Colon=:) =
    allowmissing!(df, axes(df, 2))

"""
    disallowmissing!(df::DataFrame, cols=:; error::Bool=true)

Convert columns `cols` of data frame `df` from element type `Union{T, Missing}` to
`T` to drop support for missing values.

`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).

If `cols` is omitted all columns in the data frame are converted.

If `error=false` then columns containing a `missing` value will be skipped instead
of throwing an error.
"""
function disallowmissing! end

function disallowmissing!(df::DataFrame, col::ColumnIndex; error::Bool=true)
    x = df[!, col]
    if Missing <: eltype(x)
        try
            df[!, col] = disallowmissing(x)
        catch e
            row = findfirst(ismissing, x)
            if row !== nothing
                if error
                    col_name = only(names(df, col))
                    throw(ArgumentError("Missing value found in column " *
                                        ":$col_name in row $row"))
                end
            else
                rethrow(e)
            end
        end
    end
    return df
end

function disallowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex};
                          error::Bool=true)
    for col in cols
        disallowmissing!(df, col, error=error)
    end
    return df
end

function disallowmissing!(df::DataFrame, cols::AbstractVector{Bool}; error::Bool=true)
    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))
    for (col, cond) in enumerate(cols)
        cond && disallowmissing!(df, col, error=error)
    end
    return df
end

disallowmissing!(df::DataFrame, cols::MultiColumnIndex; error::Bool=true) =
    disallowmissing!(df, index(df)[cols], error=error)

disallowmissing!(df::DataFrame, cols::Colon=:; error::Bool=true) =
    disallowmissing!(df, axes(df, 2), error=error)

"""
    append!(df::DataFrame, df2::AbstractDataFrame; cols::Symbol=:setequal,
            promote::Bool=(cols in [:union, :subset]))
    append!(df::DataFrame, table; cols::Symbol=:setequal,
            promote::Bool=(cols in [:union, :subset]))

Add the rows of `df2` to the end of `df`. If the second argument `table` is not an
`AbstractDataFrame` then it is converted using `DataFrame(table, copycols=false)`
before being appended.

The exact behavior of `append!` depends on the `cols` argument:
* If `cols == :setequal` (this is the default)
  then `df2` must contain exactly the same columns as `df` (but possibly in a
  different order).
* If `cols == :orderequal` then `df2` must contain the same columns in the same
  order (for `AbstractDict` this option requires that `keys(row)` matches
  `propertynames(df)` to allow for support of ordered dicts; however, if `df2`
  is a `Dict` an error is thrown as it is an unordered collection).
* If `cols == :intersect` then `df2` may contain more columns than `df`, but all
  column names that are present in `df` must be present in `df2` and only these
  are used.
* If `cols == :subset` then `append!` behaves like for `:intersect` but if some
  column is missing in `df2` then a `missing` value is pushed to `df`.
* If `cols == :union` then `append!` adds columns missing in `df` that are present
  in `df2`, for columns present in `df` but missing in `df2` a `missing` value
  is pushed.

If `promote=true` and element type of a column present in `df` does not allow
the type of a pushed argument then a new column with a promoted element type
allowing it is freshly allocated and stored in `df`. If `promote=false` an error
is thrown.

The above rule has the following exceptions:
* If `df` has no columns then copies of columns from `df2` are added to it.
* If `df2` has no columns then calling `append!` leaves `df` unchanged.

Please note that `append!` must not be used on a `DataFrame` that contains
columns that are aliases (equal when compared with `===`).

# See also

Use [`push!`](@ref) to add individual rows to a data frame and [`vcat`](@ref)
to vertically concatenate data frames.

# Examples
```jldoctest
julia> df1 = DataFrame(A=1:3, B=1:3)
3×2 DataFrame
 Row │ A      B
     │ Int64  Int64
─────┼──────────────
   1 │     1      1
   2 │     2      2
   3 │     3      3

julia> df2 = DataFrame(A=4.0:6.0, B=4:6)
3×2 DataFrame
 Row │ A        B
     │ Float64  Int64
─────┼────────────────
   1 │     4.0      4
   2 │     5.0      5
   3 │     6.0      6

julia> append!(df1, df2);

julia> df1
6×2 DataFrame
 Row │ A      B
     │ Int64  Int64
─────┼──────────────
   1 │     1      1
   2 │     2      2
   3 │     3      3
   4 │     4      4
   5 │     5      5
   6 │     6      6
```
"""
function Base.append!(df1::DataFrame, df2::AbstractDataFrame; cols::Symbol=:setequal,
                      promote::Bool=(cols in [:union, :subset]))
    if !(cols in (:orderequal, :setequal, :intersect, :subset, :union))
        throw(ArgumentError("`cols` keyword argument must be " *
                            ":orderequal, :setequal, :intersect, :subset or :union)"))
    end

    if ncol(df1) == 0
        for (n, v) in pairs(eachcol(df2))
            df1[!, n] = copy(v) # make sure df1 does not reuse df2
        end
        return df1
    end
    ncol(df2) == 0 && return df1

    if cols == :orderequal && _names(df1) != _names(df2)
        wrongnames = symdiff(_names(df1), _names(df2))
        if isempty(wrongnames)
            mismatches = findall(_names(df1) .!= _names(df2))
            @assert !isempty(mismatches)
            throw(ArgumentError("Columns number " *
                                join(mismatches, ", ", " and ") *
                                " do not have the same names in both passed " *
                                "data frames and `cols == :orderequal`"))
        else
            mismatchmsg = " Column names :" *
            throw(ArgumentError("Column names :" *
                                join(wrongnames, ", :", " and :") *
                                " were found in only one of the passed data frames " *
                                "and `cols == :orderequal`"))
        end
    elseif cols == :setequal
        wrongnames = symdiff(_names(df1), _names(df2))
        if !isempty(wrongnames)
            throw(ArgumentError("Column names :" *
                                join(wrongnames, ", :", " and :") *
                                " were found in only one of the passed data frames " *
                                "and `cols == :setequal`"))
        end
    elseif cols == :intersect
        wrongnames = setdiff(_names(df1), _names(df2))
        if !isempty(wrongnames)
            throw(ArgumentError("Column names :" *
                                join(wrongnames, ", :", " and :") *
                                " were found in only in destination data frame " *
                                "and `cols == :intersect`"))
        end
    end

    nrows, ncols = size(df1)
    targetrows = nrows + nrow(df2)
    current_col = 0
    # in the code below we use a direct access to _columns because
    # we resize the columns so temporarily the `DataFrame` is internally
    # inconsistent and normal data frame indexing would error.
    try
        for (j, n) in enumerate(_names(df1))
            current_col += 1
            if hasproperty(df2, n)
                df2_c = df2[!, n]
                S = eltype(df2_c)
                df1_c = df1[!, j]
                T = eltype(df1_c)
                if S <: T || !promote || promote_type(S, T) <: T
                    # if S <: T || promote_type(S, T) <: T this should never throw an exception
                    append!(df1_c, df2_c)
                else
                    newcol = similar(df1_c, promote_type(S, T), targetrows)
                    copyto!(newcol, 1, df1_c, 1, nrows)
                    copyto!(newcol, nrows+1, df2_c, 1, targetrows - nrows)
                    firstindex(newcol) != 1 && _onebased_check_error()
                    _columns(df1)[j] = newcol
                end
            else
                if Missing <: eltype(df1[!, j])
                    resize!(df1[!, j], targetrows)
                    df1[nrows+1:targetrows, j] .= missing
                elseif promote
                    newcol = similar(df1[!, j], Union{Missing, eltype(df1[!, j])},
                                     targetrows)
                    copyto!(newcol, 1, df1[!, j], 1, nrows)
                    newcol[nrows+1:targetrows] .= missing
                    firstindex(newcol) != 1 && _onebased_check_error()
                    _columns(df1)[j] = newcol
                else
                    throw(ArgumentError("promote=false and source data frame does " *
                                        "not contain column :$n, while destination " *
                                        "column does not allow for missing values"))
                end
            end
        end
        current_col = 0
        for col in _columns(df1)
            current_col += 1
            @assert length(col) == targetrows
        end
        if cols == :union
            for n in setdiff(_names(df2), _names(df1))
                newcol = similar(df2[!, n], Union{Missing, eltype(df2[!, n])},
                                 targetrows)
                @inbounds newcol[1:nrows] .= missing
                copyto!(newcol, nrows+1, df2[!, n], 1, targetrows - nrows)
                df1[!, n] = newcol
            end
        end
    catch err
        # Undo changes in case of error
        for col in _columns(df1)
            resize!(col, nrows)
        end
        @error "Error adding value to column :$(_names(df1)[current_col])."
        rethrow(err)
    end
    return df1
end

function Base.push!(df::DataFrame, row::Union{AbstractDict, NamedTuple};
                    cols::Symbol=:setequal,
                    promote::Bool=(cols in [:union, :subset]))
    possible_cols = (:orderequal, :setequal, :intersect, :subset, :union)
    if !(cols in possible_cols)
        throw(ArgumentError("`cols` keyword argument must be any of :" *
                            join(possible_cols, ", :")))
    end

    nrows, ncols = size(df)
    targetrows = nrows + 1

    if ncols == 0 && row isa NamedTuple
        for (n, v) in pairs(row)
            setproperty!(df, n, fill!(Tables.allocatecolumn(typeof(v), 1), v))
        end
        return df
    end

    old_row_type = typeof(row)
    if row isa AbstractDict && keytype(row) !== Symbol &&
        (keytype(row) <: AbstractString || all(x -> x isa AbstractString, keys(row)))
        row = (;(Symbol.(keys(row)) .=> values(row))...)
    end

    # in the code below we use a direct access to _columns because
    # we resize the columns so temporarily the `DataFrame` is internally
    # inconsistent and normal data frame indexing would error.
    if cols == :union
        if row isa AbstractDict && keytype(row) !== Symbol && !all(x -> x isa Symbol, keys(row))
            throw(ArgumentError("when `cols == :union` all keys of row must be Symbol"))
        end
        for (i, colname) in enumerate(_names(df))
            col = _columns(df)[i]
            if haskey(row, colname)
                val = row[colname]
            else
                val = missing
            end
            S = typeof(val)
            T = eltype(col)
            if S <: T || promote_type(S, T) <: T
                push!(col, val)
            elseif !promote
                try
                    push!(col, val)
                catch err
                    for col in _columns(df)
                        resize!(col, nrows)
                    end
                    @error "Error adding value to column :$colname."
                    rethrow(err)
                end
            else
                newcol = similar(col, promote_type(S, T), targetrows)
                copyto!(newcol, 1, col, 1, nrows)
                newcol[end] = val
                firstindex(newcol) != 1 && _onebased_check_error()
                _columns(df)[i] = newcol
            end
        end
        for (colname, col) in zip(_names(df), _columns(df))
            if length(col) != targetrows
                for col2 in _columns(df)
                    resize!(col2, nrows)
                end
                throw(AssertionError("Error adding value to column :$colname"))
            end
        end
        for colname in setdiff(keys(row), _names(df))
            val = row[colname]
            S = typeof(val)
            if nrows == 0
                newcol = [val]
            else
                newcol = Tables.allocatecolumn(Union{Missing, S}, targetrows)
                fill!(newcol, missing)
                newcol[end] = val
            end
            df[!, colname] = newcol
        end
        return df
    end

    if cols == :orderequal
        if old_row_type <: Dict
            throw(ArgumentError("passing `Dict` as `row` when `cols == :orderequal` " *
                                "is not allowed as it is unordered"))
        elseif length(row) != ncol(df) || any(x -> x[1] != x[2], zip(keys(row), _names(df)))
            throw(ArgumentError("when `cols == :orderequal` pushed row must " *
                                "have the same column names and in the " *
                                "same order as the target data frame"))
        end
    elseif cols === :setequal
        # Only check for equal lengths if :setequal is selected,
        # as an error will be thrown below if some names don't match
        if length(row) != ncols
            # an explicit error is thrown as this was allowed in the past
            throw(ArgumentError("`push!` with `cols` equal to `:setequal` " *
                                "requires `row` to have the same number of elements " *
                                "as the number of columns in `df`."))
        end
    end
    current_col = 0
    try
        for (col, nm) in zip(_columns(df), _names(df))
            current_col += 1
            if cols === :subset
                val = get(row, nm, missing)
            else
                val = row[nm]
            end
            S = typeof(val)
            T = eltype(col)
            if S <: T || !promote || promote_type(S, T) <: T
                push!(col, val)
            else
                newcol = similar(col, promote_type(S, T), targetrows)
                copyto!(newcol, 1, col, 1, nrows)
                newcol[end] = val
                firstindex(newcol) != 1 && _onebased_check_error()
                _columns(df)[columnindex(df, nm)] = newcol
            end
        end
        current_col = 0
        for col in _columns(df)
            current_col += 1
            @assert length(col) == targetrows
        end
    catch err
        for col in _columns(df)
            resize!(col, nrows)
        end
        @error "Error adding value to column :$(_names(df)[current_col])."
        rethrow(err)
    end
    return df
end

"""
    push!(df::DataFrame, row::Union{Tuple, AbstractArray}; promote::Bool=false)
    push!(df::DataFrame, row::Union{DataFrameRow, NamedTuple, AbstractDict};
          cols::Symbol=:setequal, promote::Bool=(cols in [:union, :subset]))

Add in-place one row at the end of `df` taking the values from `row`.

Column types of `df` are preserved, and new values are converted if necessary.
An error is thrown if conversion fails.

If `row` is neither a `DataFrameRow`, `NamedTuple` nor `AbstractDict` then
it must be a `Tuple` or an `AbstractArray`
and columns are matched by order of appearance. In this case `row` must contain
the same number of elements as the number of columns in `df`.

If `row` is a `DataFrameRow`, `NamedTuple` or `AbstractDict` then
values in `row` are matched to columns in `df` based on names. The exact behavior
depends on the `cols` argument value in the following way:
* If `cols == :setequal` (this is the default)
  then `row` must contain exactly the same columns as `df` (but possibly in a
  different order).
* If `cols == :orderequal` then `row` must contain the same columns in the same
  order (for `AbstractDict` this option requires that `keys(row)` matches
  `propertynames(df)` to allow for support of ordered dicts; however, if `row`
  is a `Dict` an error is thrown as it is an unordered collection).
* If `cols == :intersect` then `row` may contain more columns than `df`,
  but all column names that are present in `df` must be present in `row` and only
  they are used to populate a new row in `df`.
* If `cols == :subset` then `push!` behaves like for `:intersect` but if some
  column is missing in `row` then a `missing` value is pushed to `df`.
* If `cols == :union` then columns missing in `df` that are present in `row` are
  added to `df` (using `missing` for existing rows) and a `missing` value is
  pushed to columns missing in `row` that are present in `df`.

If `promote=true` and element type of a column present in `df` does not allow
the type of a pushed argument then a new column with a promoted element type
allowing it is freshly allocated and stored in `df`. If `promote=false` an error
is thrown.

As a special case, if `df` has no columns and `row` is a `NamedTuple` or
`DataFrameRow`, columns are created for all values in `row`, using their names
and order.

Please note that `push!` must not be used on a `DataFrame` that contains columns
that are aliases (equal when compared with `===`).

# Examples
```jldoctest
julia> df = DataFrame(A=1:3, B=1:3);

julia> push!(df, (true, false))
4×2 DataFrame
 Row │ A      B
     │ Int64  Int64
─────┼──────────────
   1 │     1      1
   2 │     2      2
   3 │     3      3
   4 │     1      0

julia> push!(df, df[1, :])
5×2 DataFrame
 Row │ A      B
     │ Int64  Int64
─────┼──────────────
   1 │     1      1
   2 │     2      2
   3 │     3      3
   4 │     1      0
   5 │     1      1

julia> push!(df, (C="something", A=true, B=false), cols=:intersect)
6×2 DataFrame
 Row │ A      B
     │ Int64  Int64
─────┼──────────────
   1 │     1      1
   2 │     2      2
   3 │     3      3
   4 │     1      0
   5 │     1      1
   6 │     1      0

julia> push!(df, Dict(:A=>1.0, :C=>1.0), cols=:union)
7×3 DataFrame
 Row │ A        B        C
     │ Float64  Int64?   Float64?
─────┼─────────────────────────────
   1 │     1.0        1  missing
   2 │     2.0        2  missing
   3 │     3.0        3  missing
   4 │     1.0        0  missing
   5 │     1.0        1  missing
   6 │     1.0        0  missing
   7 │     1.0  missing        1.0

julia> push!(df, NamedTuple(), cols=:subset)
8×3 DataFrame
 Row │ A          B        C
     │ Float64?   Int64?   Float64?
─────┼───────────────────────────────
   1 │       1.0        1  missing
   2 │       2.0        2  missing
   3 │       3.0        3  missing
   4 │       1.0        0  missing
   5 │       1.0        1  missing
   6 │       1.0        0  missing
   7 │       1.0  missing        1.0
   8 │ missing    missing  missing
```
"""
function Base.push!(df::DataFrame, row::Any; promote::Bool=false)
    if !(row isa Union{Tuple, AbstractArray})
        # an explicit error is thrown as this was allowed in the past
        throw(ArgumentError("`push!` does not allow passing collections of type " *
                            "$(typeof(row)) to be pushed into a DataFrame. Only " *
                            "`Tuple`, `AbstractArray`, `AbstractDict`, `DataFrameRow` " *
                            "and `NamedTuple` are allowed."))
    end
    nrows, ncols = size(df)
    targetrows = nrows + 1
    if length(row) != ncols
        msg = "Length of `row` does not match `DataFrame` column count."
        throw(DimensionMismatch(msg))
    end
    current_col = 0
    try
        for (i, (col, val)) in enumerate(zip(_columns(df), row))
            current_col += 1
            S = typeof(val)
            T = eltype(col)
            if S <: T || !promote || promote_type(S, T) <: T
                push!(col, val)
            else
                newcol = Tables.allocatecolumn(promote_type(S, T), targetrows)
                copyto!(newcol, 1, col, 1, nrows)
                newcol[end] = val
                firstindex(newcol) != 1 && _onebased_check_error()
                _columns(df)[i] = newcol
            end
        end
        current_col = 0
        for col in _columns(df)
            current_col += 1
            @assert length(col) == targetrows
        end
    catch err
        #clean up partial row
        for col in _columns(df)
            resize!(col, nrows)
        end
        @error "Error adding value to column :$(_names(df)[current_col])."
        rethrow(err)
    end
    df
end

"""
    repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)

Update a data frame `df` in-place by repeating its rows. `inner` specifies how many
times each row is repeated, and `outer` specifies how many times the full set
of rows is repeated. Columns of `df` are freshly allocated.

# Example
```jldoctest
julia> df = DataFrame(a=1:2, b=3:4)
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      3
   2 │     2      4

julia> repeat!(df, inner=2, outer=3);

julia> df
12×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      3
   2 │     1      3
   3 │     2      4
   4 │     2      4
   5 │     1      3
   6 │     1      3
   7 │     2      4
   8 │     2      4
   9 │     1      3
  10 │     1      3
  11 │     2      4
  12 │     2      4
```
"""
function repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)
    inner < 0 && throw(ArgumentError("inner keyword argument must be non-negative"))
    outer < 0 && throw(ArgumentError("outer keyword argument must be non-negative"))
    return mapcols!(x -> repeat(x, inner = Int(inner), outer = Int(outer)), df)
end

"""
    repeat!(df::DataFrame, count::Integer)

Update a data frame `df` in-place by repeating its rows the number of times
specified by `count`. Columns of `df` are freshly allocated.

# Example
```jldoctest
julia> df = DataFrame(a=1:2, b=3:4)
2×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      3
   2 │     2      4

julia> repeat(df, 2)
4×2 DataFrame
 Row │ a      b
     │ Int64  Int64
─────┼──────────────
   1 │     1      3
   2 │     2      4
   3 │     1      3
   4 │     2      4
```
"""
function repeat!(df::DataFrame, count::Integer)
    count < 0 && throw(ArgumentError("count must be non-negative"))
    return mapcols!(x -> repeat(x, Int(count)), df)
end

# This is not exactly copy! as in general we allow axes to be different
function _replace_columns!(df::DataFrame, newdf::DataFrame)
    @assert ncol(newdf) == 0 || nrow(df) == nrow(newdf)
    copy!(_columns(df), _columns(newdf))
    copy!(_names(index(df)), _names(newdf))
    copy!(index(df).lookup, index(newdf).lookup)
    return df
end


text:
using WordCloud
using Random
words = [randstring(rand(1:8)) for i in 1:300]
weights = randexp(length(words))
wordcloud(words, weights) |> generate!

ext: jl


text:
typealias FeatureLayer Ptr{GDAL.OGRLayerH}
typealias Feature Ptr{GDAL.OGRFeatureH}
typealias FeatureDefn Ptr{GDAL.OGRFeatureDefnH}
typealias FieldDefn Ptr{GDAL.OGRFieldDefnH}
typealias Geometry Ptr{GDAL.OGRGeometryH}
typealias GeomFieldDefn GDAL.OGRGeomFieldDefnH
typealias SpatialRef Ptr{GDAL.OGRSpatialReferenceH}
typealias CoordTransform Ptr{GDAL.OGRCoordinateTransformationH}
typealias Driver Ptr{GDAL.GDALDriverH}
typealias RasterBand Ptr{GDAL.GDALRasterBandH}
typealias Dataset Ptr{GDAL.GDALDatasetH}
typealias ProgressFunc Ptr{GDAL.GDALProgressFunc}

"return the corresponding `DataType` in julia"
const _jltype = Dict{GDAL.GDALDataType, DataType}(
    GDAL.GDT_Unknown    => Any,
    GDAL.GDT_Byte       => UInt8,
    GDAL.GDT_UInt16     => UInt16,
    GDAL.GDT_Int16      => Int16,
    GDAL.GDT_UInt32     => UInt32,
    GDAL.GDT_Int32      => Int32,
    GDAL.GDT_Float32    => Float32,
    GDAL.GDT_Float64    => Float64
)

"return the corresponding `GDALDataType`"
const _gdaltype = Dict{DataType,GDAL.GDALDataType}(
    Any         => GDAL.GDT_Unknown,
    UInt8       => GDAL.GDT_Byte,
    UInt16      => GDAL.GDT_UInt16,
    Int16       => GDAL.GDT_Int16,
    UInt32      => GDAL.GDT_UInt32,
    Int32       => GDAL.GDT_Int32,
    Float32     => GDAL.GDT_Float32,
    Float64     => GDAL.GDT_Float64
)

const _fieldtype = Dict{GDAL.OGRFieldType, DataType}(
    GDAL.OFTInteger         => Int32,
    GDAL.OFTIntegerList     => Void,
    GDAL.OFTReal            => Float64,
    GDAL.OFTRealList        => Void,
    GDAL.OFTString          => Cstring,
    GDAL.OFTStringList      => Void,
    GDAL.OFTWideString      => Void, # deprecated
    GDAL.OFTWideStringList  => Void, # deprecated
    GDAL.OFTBinary          => Void,
    GDAL.OFTDate            => Date,
    GDAL.OFTTime            => Void,
    GDAL.OFTDateTime        => DateTime,
    GDAL.OFTInteger64       => Int64,
    GDAL.OFTInteger64List   => Void,
    GDAL.OFTMaxType         => Void
)

const _geomname = Dict{GDAL.OGRwkbGeometryType, Symbol}(
    GDAL.wkbUnknown                 => :Unknown,
    GDAL.wkbPoint                   => :Point,
    GDAL.wkbLineString              => :LineString,
    GDAL.wkbPolygon                 => :Polygon,
    GDAL.wkbMultiPoint              => :MultiPoint,
    GDAL.wkbMultiLineString         => :MultiLineString,
    GDAL.wkbMultiPolygon            => :MultiPolygon,
    GDAL.wkbGeometryCollection      => :GeometryCollection,
    GDAL.wkbCircularString          => :CircularString,
    GDAL.wkbCompoundCurve           => :CompoundCurve,
    GDAL.wkbCurvePolygon            => :CurvePolygon,
    GDAL.wkbMultiCurve              => :MultiCurve,
    GDAL.wkbMultiSurface            => :MultiSurface,
    GDAL.wkbNone                    => :None,
    GDAL.wkbLinearRing              => :LinearRing,
    GDAL.wkbCircularStringZ         => :CircularStringZ,
    GDAL.wkbCompoundCurveZ          => :CompoundCurveZ,
    GDAL.wkbCurvePolygonZ           => :CurvePolygonZ,
    GDAL.wkbMultiCurveZ             => :MultiCurveZ,
    GDAL.wkbMultiSurfaceZ           => :MultiSurfaceZ,
    GDAL.wkbPoint25D                => :Point25D,
    GDAL.wkbLineString25D           => :LineString25D,
    GDAL.wkbPolygon25D              => :Polygon25D,
    GDAL.wkbMultiPoint25D           => :MultiPoint25D,
    GDAL.wkbMultiLineString25D      => :MultiLineString25D,
    GDAL.wkbMultiPolygon25D         => :MultiPolygon25D,
    GDAL.wkbGeometryCollection25D   => :GeometryCollection25D
)

const _access = Dict{UInt32, Symbol}(0 => :ReadOnly, 1 => :Update)

"""
    GDALGetDataTypeSize(GDALDataType) -> int
Get data type size in bits.
### Parameters
* **eDataType**: type, such as GDT_Byte.
### Returns
the number of bits or zero if it is not recognised.
"""
getdatatypesize(dt::Integer) =
    ccall((:GDALGetDataTypeSize,GDAL.libgdal),Cint,(GDAL.GDALDataType,),dt)

"""
    GDALGetDataTypeName(GDALDataType) -> const char *
Get name of data type.
### Parameters
* **eDataType**: type to get name of.
### Returns
string corresponding to existing data type or NULL pointer if invalid type given.
"""
getdatatypename(dt::Integer) =
    bytestring(ccall((:GDALGetDataTypeName,GDAL.libgdal),Cstring,(GDAL.GDALDataType,),dt))

"""
    GDALGetDataTypeByName(const char *) -> GDALDataType
Get data type by symbolic name.
### Parameters
* **pszName**: string containing the symbolic name of the type.
### Returns
GDAL data type.
"""
function getdatatypebyname(name::AbstractString)
    ccall((:GDALGetDataTypeByName,GDAL.libgdal),GDAL.GDALDataType,(Cstring,),name)
end

"""
    GDALDataTypeUnion(GDALDataType,
                      GDALDataType) -> GDALDataType
Return the smallest data type that can fully express both input data types.
### Parameters
* **eType1**: first data type.
* **eType2**: second data type.
### Returns
a data type able to express eType1 and eType2.
"""
function datatypeunion(dt1::Integer,dt2::Integer)
    ccall((:GDALDataTypeUnion,GDAL.libgdal),GDAL.GDALDataType,(GDAL.GDALDataType,GDAL.GDALDataType),dt1,dt2)
end

# """
#     GDALAdjustValueToDataType(GDALDataType eDT,
#                               double dfValue,
#                               int * pbClamped,
#                               int * pbRounded) -> double
# Adjust a value to the output data type.
# ### Parameters
# * **eDT**: target data type.
# * **dfValue**: value to adjust.
# * **pbClamped**: pointer to a integer(boolean) to indicate if clamping has been made, or NULL
# * **pbRounded**: pointer to a integer(boolean) to indicate if rounding has been made, or NULL
# ### Returns
# adjusted value
# """
# function adjustvaluetodatatype(eDT::GDALDataType,dfValue::Real,pbClamped,pbRounded)
#     ccall((:GDALAdjustValueToDataType,libgdal),Cdouble,(GDALDataType,Cdouble,Ptr{Cint},Ptr{Cint}),eDT,dfValue,pbClamped,pbRounded)
# end

"""
`TRUE` if the passed type is complex (one of `GDT_CInt16`, `GDT_CInt32`,
`GDT_CFloat32` or `GDT_CFloat64`)
"""
iscomplex(dtype::GDAL.GDALDataType) = Bool(GDAL.datatypeiscomplex(dtype))

# """
#     GDALGetAsyncStatusTypeName(GDALAsyncStatusType) -> const char *
# Get name of AsyncStatus data type.
# ### Parameters
# * **eAsyncStatusType**: type to get name of.
# ### Returns
# string corresponding to type.
# """
# function getasyncstatustypename(arg1::GDALAsyncStatusType)
#     bytestring(ccall((:GDALGetAsyncStatusTypeName,libgdal),Cstring,(GDALAsyncStatusType,),arg1))
# end

# """
#     GDALGetAsyncStatusTypeByName(const char *) -> GDALAsyncStatusType
# Get AsyncStatusType by symbolic name.
# ### Parameters
# * **pszName**: string containing the symbolic name of the type.
# ### Returns
# GDAL AsyncStatus type.
# """
# function getasyncstatustypebyname(arg1)
#     ccall((:GDALGetAsyncStatusTypeByName,libgdal),GDALAsyncStatusType,(Cstring,),arg1)
# end

"""
Returns a symbolic name for the color interpretation.

This is derived from the enumerated item name with the `GCI_` prefix removed,
but there are some variations. So `GCI_GrayIndex` returns "Gray" and
`GCI_RedBand` returns "Red". The returned strings are static strings and should
not be modified or freed by the application.

### Returns
string corresponding to color interpretation or NULL pointer if invalid enumerator given.
"""
getcolorinterpname(color::Integer) =
    bytestring(ccall((:GDALGetColorInterpretationName,GDAL.libgdal),Cstring,(GDAL.GDALPaletteInterp,),color))

# GDALGetColorInterpretationByName(const char * pszName) -> GDALColorInterp
"Get color interpretation corresponding to the given symbolic name."
getcolorinterp(name::AbstractString) = GDAL.getcolorinterpretationbyname(name)

"""
    GDALGetPaletteInterpretationName(GDALPaletteInterp) -> const char *
Get name of palette interpretation.
### Parameters
* **eInterp**: palette interpretation to get name of.
### Returns
string corresponding to palette interpretation.
"""
getpaletteinterpname(einterp::Integer) =
    bytestring(ccall((:GDALGetPaletteInterpretationName,GDAL.libgdal),Cstring,(GDAL.GDALPaletteInterp,),einterp))

# """
#     OGR_GetFieldTypeName(OGRFieldType eType) -> const char *
# Fetch human readable name for a field type.
# ### Parameters
# * **eType**: the field type to get name for.
# ### Returns
# the name.
# """
# function getfieldtypename(arg1::OGRFieldType)
#     bytestring(ccall((:OGR_GetFieldTypeName,libgdal),Cstring,(OGRFieldType,),arg1))
# end

# """
#     OGR_GetFieldSubTypeName(OGRFieldSubType eSubType) -> const char *
# Fetch human readable name for a field subtype.
# ### Parameters
# * **eSubType**: the field subtype to get name for.
# ### Returns
# the name.
# """
# function getfieldsubtypename(arg1::OGRFieldSubType)
#     bytestring(ccall((:OGR_GetFieldSubTypeName,libgdal),Cstring,(OGRFieldSubType,),arg1))
# end

# """
#     OGR_AreTypeSubTypeCompatible(OGRFieldType eType,
#                                  OGRFieldSubType eSubType) -> int
# Return if type and subtype are compatible.
# ### Parameters
# * **eType**: the field type.
# * **eSubType**: the field subtype.
# ### Returns
# TRUE if type and subtype are compatible
# """
# function aretypesubtypecompatible(eType::OGRFieldType,eSubType::OGRFieldSubType)
#     ccall((:OGR_AreTypeSubTypeCompatible,libgdal),Cint,(OGRFieldType,OGRFieldSubType),eType,eSubType)
# end


text:
using OrdinaryDiffEq
using Trixi

##########  EINSTELLUNGEN     #########

initial_condition = initial_condition_weak_blast_wave
CFL = 0.8
tspan = (0.0, 10)
N = 4
cells_per_dimension = (32, 32)

# CHandrashekar DGSEM Entropy STability and conservation depending on surface flux
# surface Flux can either be Lax Friedrichs or the chandrashekar flux with or without dissipation (Entropy Stability/Conservation)
surface_flux = flux_chandrashekar # FluxPlusDissipation(flux_chandrashekar, DissipationLocalLaxFriedrichs(max_abs_speed_naive)) #
volume_flux  = flux_chandrashekar
volume_integral = VolumeIntegralFluxDifferencing(volume_flux)
basis = LobattoLegendreBasis(N)
solver = DGSEM(basis, surface_flux, volume_integral)

# Standard DGSEM Entropy STability
# polydeg = N
# surface_flux = flux_lax_friedrichs
# volume_integral = VolumeIntegralWeakForm()
# solver = DGSEM(polydeg = polydeg, surface_flux = surface_flux, volume_integral = volume_integral )

###############################################################################
# semidiscretization of the compressible Euler equations

equations = CompressibleEulerEquations2D(1.4)

coordinates_min = (-2.0, -2.0)
coordinates_max = ( 2.0,  2.0)
# mapping as described in the worksheet
function mapping(xi_, eta_)

    xi = xi_
    eta = eta_

    x = xi + 0.15 * cos(0.5 * pi * xi) * cos((3/2) * pi * eta)
    y = eta + 0.15 * cos(2 * pi * xi) * cos(0.5 * pi * eta)

    return SVector(x, y)
  end

  mesh = CurvedMesh(cells_per_dimension, mapping)

semi = SemidiscretizationHyperbolic(mesh, equations, initial_condition, solver)

###############################################################################
# ODE solvers, callbacks etc.

ode = semidiscretize(semi, tspan)

summary_callback = SummaryCallback()

save_solution = SaveSolutionCallback(interval=100,
                                     save_initial_solution=true,
                                     save_final_solution=true,
                                     solution_variables=cons2prim)

stepsize_callback = StepsizeCallback(cfl=CFL)
analysis_interval = 1000
analysis_callback = AnalysisCallback(semi, interval=analysis_interval, save_analysis=true,
                                                 extra_analysis_errors=(:conservation_error,),
                                                 extra_analysis_integrals=(entropy, energy_total,
                                                                           energy_kinetic, energy_internal)

                                                )

callbacks = CallbackSet(summary_callback, stepsize_callback, analysis_callback)

###############################################################################
# run the simulation

sol = solve(ode, CarpenterKennedy2N54(williamson_condition=false),
            dt=1.0, # solve needs some value here but it will be overwritten by the stepsize_callback
            save_everystep=false, callback=callbacks);
summary_callback() # print the timer summary

max_stars_repo_path: src/WR2/MS3 Entropy.jl

max_issues_repo_path: src/WR2/MS3 Entropy.jl

max_forks_repo_path: src/WR2/MS3 Entropy.jl


text:
# Todo: define mols for z
# Dispatch original function on Numbers
function z_ustrip(z::V) where V <: AbstractVector{U} where U <: AbstractFloat
    return z
end

function z_ustrip(z::V) where V <: AbstractVector{U} where U <: Real
    return float.(z)
end

function z_ustrip(z::V) where V <: AbstractVector{U} where U <: Unitful.Amount
    return float.(ustrip.(u"mol", z))
end

function volume(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"m^3")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, volume(model, p, T, z; phase=phase)*u"m^3")
end

function sat_pure(model::SAFT, T::Unitful.Temperature; output=[u"Pa", u"m^3", u"m^3"])
    T = float(ustrip(u"K", T))
    (P_sat, v_l, v_v) = sat_pure(model, T)
    return (P_sat*u"Pa", v_l*u"m^3", v_v*u"m^3")
end

function enthalpy_vap(model::SAFT, T::Unitful.Temperature)
    T = float(ustrip(u"K", T))
    return uconvert(output, enthalpy_vap(model, T)*u"J/mol")
end

function pressure(model::SAFT, v::Unitful.Volume, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"Pa")
    z = z_ustrip(z)
    v = float(ustrip(u"m^3", v)); T = float(ustrip(u"K", T))
    return uconvert(output, pressure(model, z, v, T; phase=phase)*u"Pa")
end

function entropy(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, entropy(model, p, T, z; phase=phase)*u"J")
end

function chemical_potential(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, chemical_potential(model, p, T, z; phase=phase)*u"J")
end

function internal_energy(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, internal_energy(model, p, T, z; phase=phase)*u"J")
end

function enthalpy(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, enthalpy(model, p, T, z; phase=phase)*u"J")
end

function gibbs_free_energy(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, gibbs_free_energy(model, p, T, z; phase=phase)*u"J")
end

function helmholtz_free_energy(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, helmholtz_free_energy(model, p, T, z; phase=phase)*u"J")
end

function isochoric_heat_capacity(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J/K")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, isochoric_heat_capacity(model, p, T, z; phase=phase)*u"J/K")
end

function isobaric_heat_capacity(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"J/K")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, isobaric_heat_capacity(model, p, T, z; phase=phase)*u"J/K")
end

function isothermal_compressibility(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"1/Pa")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, isothermal_compressibility(model, p, T, z; phase=phase)*u"1/Pa")
end

function isentropic_compressibility(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"1/Pa")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, isentropic_compressibility(model, p, T, z; phase=phase)*u"1/Pa")
end

function speed_of_sound(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"m/s")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, speed_of_sound(model, p, T, z; phase=phase)*u"m/s")
end

function isobaric_expansivity(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"1/K")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, isobaric_expansivity(model, p, T, z; phase=phase)*u"1/K")
end

function joule_thomson_coefficient(model::SAFT, p::Unitful.Pressure, T::Unitful.Temperature, z=SA[1.]; phase="unknown", output=u"K/Pa")
    z = z_ustrip(z)
    p = float(ustrip(u"Pa", p)); T = float(ustrip(u"K", T))
    return uconvert(output, joule_thomson_coefficient(model, p, T, z; phase=phase)*u"K/Pa")
end


text:
using Random
using Test
using LinearAlgebra

using Turing
using Turing.DynamicPPL

using StatsFuns: logistic

Turing.setprogress!(false)

Random.seed!(100)

# Load test utilities.
testdir(args...) = joinpath(pathof(Turing), "..", "..", "test", args...)
include(testdir("test_utils", "AllUtils.jl"))

# Test HMC.
include(testdir("inference", "hmc.jl"))


text:
# WIP

@fastmath function ltc_syn_state!(du,u,p,t, n_in, n_neurons, w_sens, w_syns)
  v, sensa, synsa    = ltc_syn_state_unpack_u(n_in, n_neurons, u)
  dv, dsensa, dsynsa = ltc_syn_state_unpack_u(n_in, n_neurons, du)
  stim, C, G_L, E_L, G_sens, α_sensa_A, α_sensa_vh, α_sensa_k, β_sensa_A, β_sensa_vh, β_sensa_k, E_sens, G_syns, α_synsa_A, α_synsa_vh, α_synsa_k, β_synsa_A, β_synsa_vh, β_synsa_k, E_syns = ltc_syn_state_unpack_p(n_in,n_neurons,p)

  I_senss = zeros(eltype(stim), n_in, n_neurons)
  I_synss = zeros(eltype(stim), n_neurons, n_neurons)
  I_L = zeros(eltype(stim), n_neurons)

  @.. I_L = G_L * (v - E_L)

  α_sensa = zeros(eltype(stim), n_in, n_neurons)
  β_sensa = zeros(eltype(stim), n_in, n_neurons)
  α_synsa = zeros(eltype(stim), n_neurons, n_neurons)
  β_synsa = zeros(eltype(stim), n_neurons, n_neurons)

  @inbounds @simd for d in 1:n_neurons
    for s in 1:n_in
      I_senss[s,d] = w_sens[s,d] * G_sens[s,d] * sensa[s,d] * (v[d] - E_sens[s,d])
      α_sensa[s,d] = α_sensa_A[s,d] * exp((stim[s]-α_sensa_vh[s,d])/α_sensa_k[s,d])
      β_sensa[s,d] = β_sensa_A[s,d] * ((stim[s] -β_sensa_vh[s,d]) / (exp((stim[s] -β_sensa_vh[s,d])/β_sensa_k[s,d]) - 1f0))
    end
    for s in 1:n_neurons
      I_synss[s,d] = w_syns[s,d] * G_syns[s,d] * synsa[s,d] * (v[d] - E_syns[s,d])
      α_synsa[s,d] = α_synsa_A[s,d] * exp((v[s]-α_synsa_vh[s,d])/α_synsa_k[s,d])
      β_synsa[s,d] = β_synsa_A[s,d] * ((v[s] -β_synsa_vh[s,d]) / (exp((v[s] -β_synsa_vh[s,d])/β_synsa_k[s,d]) - 1f0))
    end
  end
  I_sens = reshape(sum(I_senss, dims=1), :)
  I_syns = reshape(sum(I_synss, dims=1), :)

  I_tot = I_L .+ I_sens .+ I_syns

  @.. dv = -I_tot / C
  @.. dsensa = w_sens * (α_sensa * (1f0 - sensa) - β_sensa * sensa)
  @.. dsynsa = w_syns * (α_synsa * (1f0 - synsa) - β_synsa * synsa)

  nothing
end

function ltc_syn_state_init_u0p(n_in, n_neurons; T=Float32)
  stim   = fill(T(0), n_in)
  #w_sens = zeros(T, n_in, n_neurons)
  #w_syns = zeros(T, n_neurons, n_neurons)
  C      = rand_uniform(T,  1.0,     1.002,   n_neurons)
  G_L    = rand_uniform(T,  0.0001,  0.1,     n_neurons)
  E_L    = rand_uniform(T, -0.3,     0.3,     n_neurons) #.± 0.1f0
  G_sens = rand_uniform(T,  0.0001,  0.1,     n_in, n_neurons)
  α_sensa_A  = rand_uniform(T, 0.1, 1,     n_in, n_neurons)
  α_sensa_vh = rand_uniform(T, 0.2, 0.4, n_in, n_neurons)
  α_sensa_k  = rand_uniform(T, 0.1, 0.9,     n_in, n_neurons)
  β_sensa_A  = rand_uniform(T, 0.1, 1,     n_in, n_neurons)
  β_sensa_vh = rand_uniform(T, 0.2, 0.4, n_in, n_neurons)
  β_sensa_k  = rand_uniform(T, 0.1, 0.9,     n_in, n_neurons)
  E_sens = rand_uniform(T, -0.5,     0.5,     n_in, n_neurons)
  G_syns = rand_uniform(T,  0.0001,  0.1,     n_neurons, n_neurons)
  α_synsa_A  = rand_uniform(T, 0.1, 1,     n_neurons, n_neurons)
  α_synsa_vh = rand_uniform(T, 0.2, 0.4, n_neurons, n_neurons)
  α_synsa_k  = rand_uniform(T, 0.1, 0.9,     n_neurons, n_neurons)
  β_synsa_A  = rand_uniform(T, 0.1, 1,     n_neurons, n_neurons)
  β_synsa_vh = rand_uniform(T, 0.2, 0.4, n_neurons, n_neurons)
  β_synsa_k  = rand_uniform(T, 0.1, 0.9,     n_neurons, n_neurons)
  E_syns = rand_uniform(T, -0.5,     0.5,     n_neurons, n_neurons)

  v = rand_uniform(T, 0.01, 0.1, n_neurons) #.± 0.1f0
  sensa = rand_uniform(T, 0.001, 0.01, n_in, n_neurons)
  synsa = rand_uniform(T, 0.001, 0.01, n_neurons, n_neurons)

  u0 = vcat(v, vec(sensa), vec(synsa))
  p = vcat(stim,
    C, G_L, E_L,
    vec(G_sens), vec(α_sensa_A),vec(α_sensa_vh),vec(α_sensa_k),vec(β_sensa_A),vec(β_sensa_vh),vec(β_sensa_k), vec(E_sens),
    vec(G_syns), vec(α_synsa_A),vec(α_synsa_vh),vec(α_synsa_k),vec(β_synsa_A),vec(β_synsa_vh),vec(β_synsa_k), vec(E_syns),
  )
  return u0, p
end

function ltc_syn_state_bounds(n_in, n_neurons; T=Float32)
  lb = vcat(
    [1.0 for _ in 1:n_neurons],               # C
    [0 for _ in 1:n_neurons],                 # G_L
    [-1 for _ in 1:n_neurons],                # E_L
    [0 for _ in 1:n_in*n_neurons],            # G_sens
    [0.01 for _ in 1:n_in*n_neurons],            # α_sensa_A
    [0 for _ in 1:n_in*n_neurons],            # α_sensa_vh
    [0.01 for _ in 1:n_in*n_neurons],            # α_sensa_k
    [0.01 for _ in 1:n_in*n_neurons],            # β_sensa_A
    [0 for _ in 1:n_in*n_neurons],            # β_sensa_vh
    [0.01 for _ in 1:n_in*n_neurons],            # β_sensa_k
    [-1 for _ in 1:n_in*n_neurons],           # E_sens
    [0 for _ in 1:n_neurons*n_neurons],       # G_syns
    [0.01 for _ in 1:n_neurons*n_neurons],            # α_synsa_A
    [0 for _ in 1:n_neurons*n_neurons],            # α_synsa_vh
    [0.01 for _ in 1:n_neurons*n_neurons],            # α_synsa_k
    [0.01 for _ in 1:n_neurons*n_neurons],            # β_synsa_A
    [0 for _ in 1:n_neurons*n_neurons],            # β_synsa_vh
    [0.01 for _ in 1:n_neurons*n_neurons],            # β_synsa_k
    [-1 for _ in 1:n_neurons*n_neurons],      # E_syns

    [-0.4 for _ in 1:n_neurons],             # v
    [0 for _ in 1:n_in*n_neurons],            # sensa
    [0 for _ in 1:n_neurons*n_neurons])             # synsa

  ub = vcat(
    [2.0 for _ in 1:n_neurons],
    [0.4 for _ in 1:n_neurons],
    [1 for _ in 1:n_neurons],
    [1 for _ in 1:n_in*n_neurons],
    [1.1 for _ in 1:n_in*n_neurons],            # α_sensa_A
    [1 for _ in 1:n_in*n_neurons],            # α_sensa_vh
    [1 for _ in 1:n_in*n_neurons],            # α_sensa_k
    [1 for _ in 1:n_in*n_neurons],            # β_sensa_A
    [1 for _ in 1:n_in*n_neurons],            # β_sensa_vh
    [1 for _ in 1:n_in*n_neurons],            # β_sensa_k
    [1 for _ in 1:n_in*n_neurons],
    [1 for _ in 1:n_neurons*n_neurons],
    [1.1 for _ in 1:n_neurons*n_neurons],            # α_synsa_A
    [1 for _ in 1:n_neurons*n_neurons],            # α_synsa_vh
    [1 for _ in 1:n_neurons*n_neurons],            # α_synsa_k
    [1 for _ in 1:n_neurons*n_neurons],            # β_synsa_A
    [1 for _ in 1:n_neurons*n_neurons],            # β_synsa_vh
    [1 for _ in 1:n_neurons*n_neurons],            # β_synsa_k
    [1 for _ in 1:n_neurons*n_neurons],

    [0.2 for _ in 1:n_neurons],
    [0.1 for _ in 1:n_in*n_neurons],            # sensa
    [0.1 for _ in 1:n_neurons*n_neurons],

    ) # v)

  T.(lb), T.(ub)
end

function ltc_syn_state_unpack_u(n_in,n_neurons,u)
  vl = n_neurons
  sensal = n_in*n_neurons
  synsal = n_neurons*n_neurons

  s = 1
  v = @view u[s:s+vl-1]
  s += vl
  sensa = @view u[reshape(s:s+sensal-1, n_in, n_neurons)]
  s += sensal
  synsa = @view u[reshape(s:s+synsal-1, n_neurons, n_neurons)]
  return v, sensa, synsa
end

function ltc_syn_state_unpack_p(n_in,n_neurons,p)
  stiml = n_in
  Cl = n_neurons
  G_Ll = n_neurons
  E_Ll = n_neurons
  sensl = n_in*n_neurons
  synsl = n_neurons*n_neurons
  G_sensl = sensl
  α_sensa_Al = sensl
  α_sensa_vhl = sensl
  α_sensa_kl = sensl
  β_sensa_Al = sensl
  β_sensa_vhl = sensl
  β_sensa_kl = sensl
  E_sensl = sensl
  G_synsl = synsl
  α_synsa_Al = synsl
  α_synsa_vhl = synsl
  α_synsa_kl = synsl
  β_synsa_Al = synsl
  β_synsa_vhl = synsl
  β_synsa_kl = synsl
  E_synsl = synsl

  s = 1
  stim = @view p[s:s+stiml-1]
  s += stiml
  C = @view p[s:s+Cl-1]
  s += Cl
  G_L = @view p[s:s+G_Ll-1]
  s += G_Ll
  E_L = @view p[s:s+E_Ll-1]
  s += E_Ll
  G_sens = @view p[reshape(s:s+G_sensl-1, n_in, n_neurons)]
  s += G_sensl
  α_sensa_A = @view p[reshape(s:s+α_sensa_Al-1, n_in, n_neurons)]
  s += α_sensa_Al
  α_sensa_vh = @view p[reshape(s:s+α_sensa_vhl-1, n_in, n_neurons)]
  s += α_sensa_vhl
  α_sensa_k = @view p[reshape(s:s+α_sensa_kl-1, n_in, n_neurons)]
  s += α_sensa_kl
  β_sensa_A = @view p[reshape(s:s+β_sensa_Al-1, n_in, n_neurons)]
  s += β_sensa_Al
  β_sensa_vh = @view p[reshape(s:s+β_sensa_vhl-1, n_in, n_neurons)]
  s += β_sensa_vhl
  β_sensa_k = @view p[reshape(s:s+β_sensa_kl-1, n_in, n_neurons)]
  s += β_sensa_kl
  E_sens = @view p[reshape(s:s+E_sensl-1, n_in, n_neurons)]
  s += E_sensl

  G_syns = @view p[reshape(s:s+G_synsl-1, n_neurons, n_neurons)]
  s += G_synsl
  α_synsa_A = @view p[reshape(s:s+α_synsa_Al-1, n_neurons, n_neurons)]
  s += α_sensa_Al
  α_synsa_vh = @view p[reshape(s:s+α_synsa_vhl-1, n_neurons, n_neurons)]
  s += α_synsa_vhl
  α_synsa_k = @view p[reshape(s:s+α_synsa_kl-1, n_neurons, n_neurons)]
  s += α_synsa_kl
  β_synsa_A = @view p[reshape(s:s+β_synsa_Al-1, n_neurons, n_neurons)]
  s += β_synsa_Al
  β_synsa_vh = @view p[reshape(s:s+β_synsa_vhl-1, n_neurons, n_neurons)]
  s += β_synsa_vhl
  β_synsa_k = @view p[reshape(s:s+β_synsa_kl-1, n_neurons, n_neurons)]
  s += β_synsa_kl
  E_syns = @view p[reshape(s:s+E_synsl-1, n_neurons, n_neurons)]
  s += E_synsl
  return stim, C, G_L, E_L, G_sens, α_sensa_A, α_sensa_vh, α_sensa_k, β_sensa_A, β_sensa_vh, β_sensa_k, E_sens, G_syns, α_synsa_A, α_synsa_vh, α_synsa_k, β_synsa_A, β_synsa_vh, β_synsa_k, E_syns
end

function LTCSynState(wiring, solver, sensealg;
  T=Float32, tspan=T.((0, 1)),
  #wiring=nothing,
  mtkize=true, gen_jac=false, kwargs...)

  w_sens = wiring.matrices[:w_sens]
  w_syns = wiring.matrices[:w_syns]

  s_in = wiring.s_in
  n_neurons = wiring.n_total

  lb, ub = ltc_syn_state_bounds(s_in, n_neurons; T)
  u0, p = ltc_syn_state_init_u0p(s_in, n_neurons; T)

  dudt!(du,u,p,t) = ltc_syn_state!(du,u,p,t, s_in, n_neurons, w_sens, w_syns)

  rnncell = BCTRNNCell(wiring, solver, sensealg, dudt!, u0, tspan, p, lb, ub; mtkize, gen_jac, kwargs...)
  MyRecur(rnncell)
end


text:
"""
        DSA Models
"""

"""
    Kang&Ryu 2007, http://arxiv.org/abs/0704.1521v1
"""
@inline function kr_fitting_function(M::Float64, p::Array{Float64,1})
    mm = M - 1.0
    m2 = M * M
    return ( p[1] + p[2]*mm + p[3]*mm*mm + p[4]*mm*mm*mm + p[5]*mm*mm*mm*mm ) / (m2*m2)
end

function KR07_acc(M::Float64)
    if M <= 2.0
        return 1.96e-3*(M*M - 1.)             # eq. A3
    else
        return kr_fitting_function(M, [5.46, -9.78, 4.17, -0.337, 0.57])
    end
end

"""
    Kang&Ryu 2013, doi:10.1088/0004-637X/764/1/95
"""
function KR13_acc(M::Float64)

    if M < 2.0
        return 0.0
    elseif 2.0 <= M <= 5.0
        param = [-0.0005950569221922047, 1.880258286365841e-5, 5.334076006529829 ]
        return (param[1] + param[2]*M^param[3])
    elseif 5.0 < M <= 15.0
        param = [-2.8696966498579606, 9.667563166507879,
                 -8.877138312318019, 1.938386688261113,
                 0.1806112438315771]
        return kr_fitting_function(M, param)
    else
        return 0.21152
    end
end

"""
    Ryu et al. 2019, https://arxiv.org/abs/1905.04476
    values for 2.25 < M <= 5.0 extrapolated to entire range
"""
function Ryu19_acc(M::Float64)

    if M < 2.25
        return 0.0
    elseif M <= 34.0
        param = [-1.5255114554627316, 2.4026049650156693,
                 -1.2534251472776456, 0.22152323784680614,
                  0.0335800899612107]
        return kr_fitting_function(M, param)
    else
        return 0.0348
    end
end

"""
    Caprioli&Spitkovsky 2015,
"""
function CS14_acc(M::Float64)
    vazza_factor = 0.5
    return vazza_factor * KR13_acc(M)
end

"""
    Constant efficiency as in Pfrommer+ 2016
"""
function P16_acc(M::Float64)
    return 0.5
end

"""
    fallback:
"""
function null_acc(M::Float64)
    return 0.0
end

max_stars_repo_name: LudwigBoess/GadJet

max_issues_repo_name: LudwigBoess/GadJet

max_forks_repo_name: LudwigBoess/GadJet


text:
using HurdleDMR
using Documenter

makedocs(
  modules = [HurdleDMR],
  clean = true,
  format = Documenter.HTML(
        # Use clean URLs, unless built as a "local" build
        prettyurls = !("local" in ARGS),
        canonical = "https://asafmanela.github.io/HurdleDMR.jl/stable/",
        analytics = "UA-4385132-6",
    ),
  sitename = "HurdleDMR.jl",
  authors = "Asaf Manela",
  linkcheck = !("skiplinks" in ARGS),
  pages = [
    "Home" => "index.md",
    "Tutorials" => "tutorials/index.md"
    ],
    )

deploydocs(
  repo = "github.com/AsafManela/HurdleDMR.jl.git",
  )


text:
# Benchmark the atom count and parsing of complete PDB database
# Gazal Kalyan

using Bio.Structure
using Glob
global elapsed
global pdb_filepath
global count
global fh = open("output.txt", "a")

for pdb_filepath in glob("*.ent","/home/gazal/PDB-all/ftp.wwpdb.org/extracted/")
	try
		read(pdb_filepath, PDB)
		elasped = @elapsed struc = read(pdb_filepath, PDB)
		count = countatoms(struc)
		write(fh, "$pdb_filepath,$count,$elasped\n")
		println("$pdb_filepath,$count,$elasped")
	catch y
		println("$pdb_filepath,error,error")
		write(fh, "$pdb_filepath,error,error\n")
	end
end
close(fh)
println("done for now")


text:
for room in loadedState.side.map.rooms
	for entity in room.entities
        if startswith(entity.name, "NerdHelper/DashThroughSpikes") && haskey(entity.data, "textures")
            entity.data["type"] = entity.data["textures"]
            delete!(entity.data, "textures")
        end
    end
end

max_stars_repo_name: Kalobi/NerdHelper

max_issues_repo_name: Kalobi/NerdHelper

max_forks_repo_name: Kalobi/NerdHelper