This version seems to be working well on all my test problems and is stable in the presence of many latent factors (I tried 100). The interesting solution was to "inhibit" a fraction of the neurons that were strongly competing at each time step. I also re-factored some of the calcuation about MI (so it isn't repeated on each iteration).
This version seems to be working well on all my test problems and is stable in the presence of many latent factors (I tried 100). The interesting solution was to "inhibit" a fraction of the neurons that were strongly competing at each time step.
I also re-factored some of the calcuation about MI (so it isn't repeated on each iteration).

gregversteeg/LinearCorex