In this paper, we propose a capsule-based neural network model to solve the semantic segmentation problem. By taking advantage of the extractable part-whole dependencies available in capsule layers, we derive the probabilities of the class labels for individual capsules through a recursive, layer-by-layer procedure. We model this procedure as a traceback pipeline and take it as a central piece to build an end-to-end segmentation network. Under the proposed framework, image-level class labels and object boundaries are jointly sought in an explicit manner, which poses a significant advantage over the state-of-the-art fully convolutional network (FCN) solutions. Experiments conducted on modified MNIST and neuroimages demonstrate that our model considerably enhance the segmentation performance compared to the leading FCN variant.

A capsule-based semantic segmentation, in which the probabilities of the class labels are traced back through capsule pipeline.  The authors present a trace-back mechanism to associate lowest level of Capsules with their respective classes Proposes a traceback layer for capsule networks to do semantic segmentation and makes explicit use of part-whole relationship in the capsule layers Proposes a trace-back method based on the CapsNet concept of Sabour to perform a semantic segmentation in parallel to classification.