Hyperspherical nature of K-means and similar clustering methods Jain, Murty, and Flynn
state in their article Data Clustering: A Review
all squared error based clustering methods like K-means 
tend to generate hyperspherical clusters. 

However, they do not give a theoretical justification for this.


Is a theoretical justification for this observation?
Can we change the K-means algorithm to generate rectangular clusters?
K-means tends to create compact clusters, which means that geometrically, the distance between every two points is small.

K-means can create clusters which are ball-like if you use $L_2$ norm, pyramid if you use $L_1$ and hypercubes if $L_\infty$ is used.

As opposed to that there are clustering algorithms which first project the data into lower dimension (such as spectral clustering), which means not all of the points in a cluster has to be close, but rather "connected" via other points in the original space, which will make them close after the projection.

This allow you to find very nice looking clusters, such as this and that.