Why is it a good idea to retreat the bishop into the pawn chain in this variation of the Caro Kann? I am reading a book about the Caro-Kann defence.
In this position the author suggests playing 5...Bd7. (Although they assert that Bg6 is not a bad move either). In fact, the whole idea of playing Qb6 early instead of e6 is so that the bishop can retreat to d7 in case of g4.

I find this very confusing. The point of playing the Caro-Kann instead of the French defence is to not have a bad bishop at the price of having to "waste" a tempo with c6.

But in this position not only did black "waste" a tempo with c6, they also had to move the bishop twice AND back inside the pawn chain.

So my question is: why would black want to do that? Why isn't this just a much worse position in a French-like structure considering that ...c5 will come in two moves instead of one, the bishop is inside the pawn chain, and white has got the move g4 for free which might be useful later?

rn2kbnr/pp2pppp/1qp5/3pPb2/3P2P1/2N5/PPP2P1P/R1BQKBNR b KQkq - 9 5
1. e4 c6 2. d4 d5 3. e5 Bf5 4. Nc3 Qb6 5. g4
The immediate point of Bd7 versus Bg6 is to prevent White from playing 6 e6 which could give Black's King a lot of trouble. 

Generally, Black cannot entirely avoid White's g4 anyway (it's always a move to keep in mind in the advance variation).
However, the question is: Does that move actually help White in the long run? Yes, it gains the short-term advantage you mention (free tempo + space), but at the same time it also comes with the long-term disadvantage of fracturing White's kingside pawn structure. Black can then try to play around this and later free the Bishop via e8 &amp; f7-h5 (after that pawn has moved to f6 to attack White's center).