How is it possible that the Lomonosov-endgame-tablebase from 2012 still is the most powerful tablebase? Due to Wikipedia: Endgame tablebase the Lomonosov endgame tablebase is still the biggest tablebase (with a size of 140TB). Now this tablebase was computed in the year 2012 with the Lomonosov 1 (Top500-list November 2012, rank 26). The Lomonosov 1 had 78.660 cores with ~900 TFlop/s (peak: 1.700 TFlop/s). When I have a look on the most recent Top500-List (June 2016) I see that the Lomonosov 2 is ranked 41 in the world with ~2.100 TFlop/s (peak: ~2.900 TFlop/s), the No1-ranked super-computer has 93.000 TFlop/s.
So I wonder why the Lomonosov-tablebase after 4 years still is the biggest tablebase. Shouldn't it be easy with the growing computer-power to calculate an 8-men-tablebase? Or is the step from 7 to 8 just too big? Maybe the size would go up exponentially so that standard-storage-systems just cannot handle this? Would appreciate all information about this theme.
I am a developer currently working on software to compute endgame tablebases. The real bottleneck with calculating positions is not memory per se but the working memory and computing cycles needed to calculate a network for an exponentially-growing set of positions.
The number of positions grows exponentially according to the number of men on the board; this means that the working memory needed (whether it be RAM or hard drive reads, which are less expensive and more scalable but significantly slower) increases proportionately to A^N where A is a scaling constant (somewhat less than 64) and N is the number of men in the tablebase. The Lomonosov Supercomputer used 92 TiB of RAM, which is sufficient for a 7-man tablebase; as dfan said in his comment, the jump is too big.
Furthermore, as more nodes are added to the tablebase the number of calculations increases faster than linearly to the number of nodes. For instance, accessing data in a Binary Search Tree (BST) is proportional to ln(M) where M is the number of nodes in the tree; since M = A^N, to access each node of the tree once, A^N * ln(A^N) operations are needed. There are several steps in the tablebase retrograde evaluation algorithm that scale similarly.
In short, memory increases exponentially and the number of operations increases faster than exponentially as more men are added. Computing has not advanced this far in (at the time of writing) 6 years.
Edit
Since this answer is getting a lot of traffic and I re-read my answer, I feel the need to add some clarification. Endgame tablebases are created with retrograde analysis, so deterministically calculating the outcome of one position requires knowledge of enough possible successor positions. As we know, this graph mushrooms quickly. While retrograde analysis is definitely parallelizable, one needs an enormous amount of working memory. However, it is common practice to allocate only 10s of GB of RAM to a single processor, so positions must be communicated over a network. This would require a lot of bandwidth, and multiple processors contending to make changes to the same table state would create synchronization problems that add overhead. Positions must be stored in RAM because disk read latency is horrendous (on the order of milliseconds instead of nanoseconds for cache or microseconds for RAM). It's possible some amount of pipelining or caching could amortize poor latency, but this is still a hard problem to solve at the required scale.
Tl;dr even with a big supercomputer, modern computer architecture is not amenable to this type of problem.