Willingly giving up castling rights in the opening? [fen ""]

1. d4 Nf6 2. c4 d6 3. Nc3 e5


I've seen the Ukrainian/Old Indian Defence played in several GM games before 1970, and was wondering what black gained from willingly giving up castling rights in the opening. According to this source, black actually scores better in the variations where white deprives black of castling rights by trading queens, such as by 4. dxe5 dxe5 5. Qxd8+ Kxd8. So where exactly is black's compensation for the loss of castling rights? I've heard that one can get away without castling if queens are off the board; but this is still the opening and the position looks fairly open (even centre control of both sides is more or less the same), so it seems to me that white can purposefully develop its pieces towards the now exposed black king, while black might have to spend several moves to get its king to safety.
Weaknesses like doubled/tripled pawn structure, color complex holes, and right to castle only matter in chess to the extent to which the opponent can exploit them. In the Old Indian variation you gave above, yes, Black lost the right to castle, but is White in any real position to exploit it? The answer is no. In fact, Black's king is very safe after he plays c6 because he can then tuck his king into c7 and there isn't any real way for White to even check him. The pawn and c6 prevents Nd5 and Nb5 checks, and Bf4 check isn't gonna work because Black will probably post his bishop to d6 and/or play e5, so his king is clearly not "weak" despite the loss of castling rights.

There is another position in the Sicilian Four knights that goes something like:

[fen ""]
1. e4 c5
2. Nf3 Nc6
3. d4 cxd4
4. Nxd4 Nf6
5. Nc3 e6
6. Nxc6 bxc6
7. e5 Nd5
8. Ne4 Qc7
9. f4 Qb6
10. c4 Bb4+
11. Ke2


While Ke2 aesthetically looks ugly because it loses castling rights and also blocks the light-squared bishop from developing, the point is that after Ke2 - f3, the king is extremely safe. If you look at that position yourself, I challenge you to even come up with a plan for Black to check the white king. If you can't, that's okay, because White's king is quite safe on f3. 

Here is a counter example. This is from a game I played recently actually where I'm black.

[fen ""]
1. e4 e5
2. Nf3 Nc6
3. Bc4 Nf6
4. d3 Be7
5. Nc3 O-O
6. Be3 d6
7. h3 Na5
8. Bb3 Nxb3
9. axb3 d5
10. exd5 Nxd5
11. Nxe5 Nxe3
12. fxe3 Bh4+
13. Kf1


Here, Black has sacrificed a pawn to force White to lose castling rights with the annoying Bh4+. Notice though that there are a number of factors in this position that makes losing castling rights much more serious in this position than in the previous position. First, Black has the bishop pair which alone is good compensation for the pawn. Second, White has all these dark square weaknesses on g3, h2, and f2, and Black has the dark-squared bishop. White also lacks his own dark-square bishop to protect those squares, so it only heightens the problems of losing his dark-squared bishop. Third, White's king position here impedes the development of his h1 rook, so his pieces are more awkwardly placed.

You can argue Black has lost a pawn, but I would argue that the pawn sac has forced White to lose a rook, because it will be very difficult to get his h1 rook into action. All these factors make losing castling rights here really really bad for White because Black has these compensating factors that allow him to take advantage of White's compromised king position and loss of castling rights.

So the takeaway from this is that with any weakness&mdash;be it losing castling rights, or pawn structure weaknesses&mdash;a weakness is only a weakness if it can be exploited by the opponent. In all mainline openings where one side loses the right to castle, or takes on any other sort of weakness, it is because that side willingly loses the right to castle because his king position can't be exploited and because other aspects of the position trump king safety/castling rights.