John 14:2: a literal house / temple or heaven In John, Jesus is speaking of Heaven.  However in Matthew he's talking about a physical temple.

John 14:2 NIV


  2 My Father’s house has many rooms; if that were not so, would I have
  told you that I am going there to prepare a place for you?


VS

Matthew 21:12-13 NIV


  12 Jesus entered the temple courts and drove out all who were buying
  and selling there. He overturned the tables of the money changers and
  the benches of those selling doves. 13 “It is written,” he said to
  them, “‘My house will be called a house of prayer,’ but you are
  making it ‘a den of robbers.’”


How do we know for sure that he's talking about Heaven in John and not a physical temple / Church?
Well, for sure is never really possible, but let me give this a shot.

There is no difference between the two words in Greek—just like the English translations you provided. Both use the generic Greek term for house, "oikos." However, it's important to note that the second quotation (Mt. 21) is itself a quotation from Isaiah 56:7. In Hebrew, the word for house, bêt, is also used frequently to refer to a "temple" (cf. with the archaeological sites of bet shemesh, et al.), and in fact is the word used in Isa 56:7, which is unquestionably referring to the Solomonic Temple. The Greek word oikos does not include this connotation in contexts other than translation. In other words, referring to the temple as oikos in Matthew is a semiticism preserved by a literal translation and is not part of the usual semantic domain.

So, in John 14 there is no reason contextually to assume that Jesus has in mind a "temple" when he says "house" because there are perfectly good words in Greek that do mean temple.