CI: remove the compilation directories for grcov/etc. to make the image smaller
CI: remove the compilation directories for grcov/etc. to make the image smaller

Part-of: <https://gitlab.gnome.org/GNOME/librsvg/-/merge_requests/747>

GNOME/librsvg,GNOME/librsvg,GNOME/librsvg,GNOME/librsvg