Can you castle with a "ghost" rook? I recently saw this mate-in-2 problem

[title "White to mate in 2"]
[fen "8/8/8/2B5/2p5/1kp5/8/R3K3 w - - 0 1"]


This was posed with the hint that the player of the White pieces was a strong player and Black was a novice. The take away was that White was offering rook odds, started the game without a rook on a1 and the rook now on a1 had started on h1. 

The key was that White castles queenside with the ghost of the rook given as odds so the king moved from e1 to c1 and the rook on a1 stayed where it was. This forces Black to play c2 allowing White to play Ra3 mate.

This raises two questions:


Is castling with a "ghost" rook like this legal where there has
never been a rook on a1 because it was given as odds at the start of
the game?
Is castling with a "ghost" rook like this legal
where there was a rook on a1 at the start of the game but it was
captured before it could move?


The replayer doesn't like it. It will only allow castling with a real rook.
The second part of the question is more straightforward, so I'll look at that first.


  Is castling with a "ghost" rook like this legal where there was a rook
  on a1 at the start of the game but it was captured before it could
  move?


Article 3.8.2 of the FIDE Laws of Chess describes how castling works:


  3.8.2 by ‘castling’. This is a move of the king and either rook of the same colour along the player’s first rank, counting as a single move
  of the king and executed as follows: the king is transferred from its
  original square two squares towards the rook on its original square,
  then that rook is transferred to the square the king has just crossed.


This explicitly states that castling involves two pieces, the king and the rook, and it describes how the rook moves in this manoeuvre. If you don't have an unmoved rook on the board then you can't castle. Therefore castling with a "ghost" rook is illegal.

With this clear the first part becomes easier to answer.


  Is castling with a "ghost" rook like this legal where there has never
  been a rook on a1 because it was given as odds at the start of the
  game?


It depends on the rules of the competition or individual game, if the game is a one-off. If the rules state that apart from the initial position of the board the FIDE Laws of Chess apply then clearly it is illegal.

In the absence of such a clarification the best guide is probably the chess historian Edward Winter. In his chess notes for March 2009, note 6029 he considers exactly this question:


  6029. Castling with a phantom rook From Mark Thornton (Cambridge, England):
  
  ‘In rook-odds games could the odds-giver castle with the “phantom
  rook”? For example, if White gave the odds of his queen’s rook, could
  he play Ke1-c1? And, if so, were the castling rules the same as if the
  rook were present?’


According to Winter there is no conclusive ruling one way or the other. He quotes probably the most sensible approach from Howard Staunton on page 35 of Chess Praxis (London, 1860):


  When a player gives the odds of his king’s or queen’s rook, he must
  not castle (or, more properly speaking, leap his king) on the side
  from whence he takes off the rook, unless before commencing the game
  or match he stipulates to have the privilege of so doing.


There are strong feelings on both sides of the argument. Here is more from Winter:


  However, in a review of the book on pages 88-89 of the March 1890 BCM
  Edward Freeborough disagreed. After stating, with respect to level
  games, that castling should be described as a move of the king and
  that the king should therefore be moved first, Freeborough observed:
  
  ‘It follows logically that the fact of giving the odds of a rook
  ought not to deprive the king of his privilege of taking two steps to
  the right or left as his first move.’
  
  Page 36 of The British Chess Code (London, 1903) stated:
  
  ‘In the absence of agreement to a different effect, a player may
  castle (by moving his king as in ordinary castling) on a side from
  which, before the commencement of the game, the player’s rook has been
  removed, provided that this rook’s square is unoccupied and has been
  unoccupied throughout the game, and that the same conditions as to
  squares and as to the king are fulfilled which are required for
  ordinary castling on this side.’
  
  The above text was quoted on page 275 of the June 1916 Chess Amateur
  when a revised edition of the Code was envisaged. Comments were
  invited, and on page 305 of the July 1916 issue ‘Simplex’ wrote:
  
  ‘This I think sheer nonsense. If a player gave me a rook and wanted
  to castle on this rook’s side, I should say, “No, you don’t, you can’t
  castle without a castle”. Let’s have no pretence. If a player gives a
  rook, let him give it totally not half. Receivers of odds are not
  strong players, and to see the nominal giver of odds move his king a
  couple of squares would be disconcerting. No; if a player gives odds
  let him give them without pretence.’
  
  A contrary view was expressed by W.S. Branch on pages 333-334 of the
  August 1916 Chess Amateur:
  
  ‘Re Chess Laws, page 305 (July), and as to “castling without rook”, I
  would say, first, that you can’t “castle the king” – the full and
  proper term, of which “castles” is an abridgment – without a castle.
  The phrase should be “moving the king as in castling”. I believe that
  the right of the odds-giver to move his king, once in a game, as in
  castling, has always been upheld since “castling” was invented
  (sixteenth century). It existed, as part of the “king’s leap”, long
  before “castling” was invented, and long before the rook was ever
  called a “castle”. The giving of the rook as odds should not deprive
  the king of any of his rights.’
  
  Branch then gave further historical details regarding the king’s leap.
  By 1916, however, the practice of giving odds was disappearing,
  without any formal resolution of the ‘phantom rook’ question.


Winter revisits this topic in note 6035. "Castling with a phantom rook (C.N. 6029)" with some interesting anecdotes regarding castling with a ghost or phantom rook.