Revert "Free even more unused memory"
Revert "Free even more unused memory"

That memory is not used in this case, so don't free it

This reverts commit bd4aa39bc447a07b0fd9a64bfacfeb2ccc7a7815.

szellmann/visionaray,szellmann/visionaray