Remove assertion for filter_shapes in conv_layers as it's already asserted when creating layer variable.
Remove assertion for filter_shapes in conv_layers as it's already asserted when creating layer variable.

PiperOrigin-RevId: 269572222

tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo,tensorflow/lingvo