Implement Online Dictionary Learning with Dask
Implement Online Dictionary Learning with Dask

Provides a working implementation of Online Dictionary Learning with
Dask. This adds an interface that mimics `MiniBatchDictionaryLearning`
from `scikit-learn`. It relies on Online Dictionary Learning to allow
for partial fitting of pieces of the data so as to process the full
dataset incrementally. Though it differs as it internally constructs
Dask graphs to describe the computations to be performed.

The `fit` implementation uses `dict_learning_online` and effectively
fits chunks of the input data through a series of calls to
`dict_learning_online`. This is similar to calling `partial_fit` on
pieces of the data with `MiniBatchDictionaryLearning`. Statistics
accumulated from fitting the dictionary incrementally are kept around
and are reused in any subsequent calls to `fit`. The `transform`
implementation calls `sparse_encode` much like
`MiniBatchDictionaryLearning`, but leverages `atop` to both enact
parallelism and return the result as on Dask Array.

Any checks that `scikit-learn` implements are bypassed and handled at
the Dask level. Further any optional input data that might be computed
within `scikit-learn` functions is computed in Dask instead and provided
to all `scikit-learn` functions. This includes shuffling the input data,
seeding the initial dictionary, seeding the statistics related to the
dictionary, computing the Gramian and Covariance matrices, etc. By doing
this we can avoid repeating this work. Further we can allow the
scheduler to make the best decision about where to compute this and how
best to position it on the cluster.

All intermediate results are correctly represented as Dask Arrays to
ensure the scheduler is able to reason effectively about where the
computation should occur and where data should move (if at all). This
also makes it easy to represent the final results as Dask Arrays. After
`fit` or `transform` constructs the full Dask graph, they call `persist`
on the results to trigger computation in the background and return
references to the result that can be stored in the model or returned.

To aid in viewing the results, we provide a simple viewer leveraging
ipywidgets and matplotlib. This shows the basis images in order along
with their corresponding sparse code. Also it provides a little slider
to allow inspecting an arbitrary basis image and associated sparse code.

nanshe-org/nanshe_workflow,DudLab/nanshe_workflow