If I like both the Caro-Kann and the Slav, is it a good idea to answer 1. d4 with 1... c6? I was just wondering, if as Black I normally play the Caro-Kann and the Slav (but I slightly prefer the Caro-Kann), is it a good idea to answer 1. d4 with 1... c6!?

I looked in an opening database, but it looks like 1... c6 is extremely rare and is only played 0.3% of the time.
What's wrong with it? Doesn't it just transpose into either a Slav or a Caro-Kann?

Does it have a psychological effect on weak opponents? Maybe they will be a little surprised, and thus they will take slightly more time to make their second move, and so it would be an effective subterfuge especially in blitz.
I don't think it's a good idea. Firstly, no 1.d4 player will go for 2.e4, unless they are also 1.e4 players and really, really good at playing against the Caro-Kann. 

More importantly, you may like to play the Slav against d4+c4, and the Caro against d4+e4, but so far white has only played 1.d4. You lose options in case white doesn't follow up with a quick c4 or e4.

Mostly that's lines like the London (say 2.Nf3 and 3.Bf4) or the Colle (with Nf3, e3, Bd3, possibly b3 and Bb2, etc). Against both of those, black's best chance to get an active game are with a quick ...c5, but if you do that you've already given up a tempo.

So there is a downside, and zero upside.