What would the Jews of Jesus' time have known about heaven? Looking at the cross references for John 12:34, 


  34 So the crowd answered him, “We have heard from the Law that the Christ remains forever. How can you say that the Son of Man must be lifted up? Who is this Son of Man?”


I found several regarding the eternal kingdom of Christ:

Psalm 89:4


  ‘I will establish your offspring forever, and build your throne for all generations.’” Selah


Psalm 110:4


  The Lord has sworn and will not change his mind, “You are a priest forever after the order of Melchizedek.”


Isaiah 9:7


  Of the increase of his government and of peace there will be no end...


Ezekiel 37:25


  They shall dwell in the land that I gave to my servant Jacob, where your fathers lived. They and their children and their children's children shall dwell there forever, and David my servant shall be their prince forever.


Luke 1:33


  and he will reign over the house of Jacob forever, and of his kingdom there will be no end.”


Given the wealth of such references, and the seemingly common misconception that the Messiah would bring about an earthly kingdom, it made me wonder:
What did the Jews of that time know about heaven, or indeed, about the afterlife? Was it a very concrete thing, was their picture of it accurate (whether somewhat primitive or full and detailed), and was it based on Scripture or on tradition?
I believe the best way to approach this question is to start with the following verse Acts 23:8 (RSVCE)


  8For the Sad′ducees say that there is no resurrection, nor
  angel, nor spirit; but the Pharisees acknowledge them all.


Thus some Jews in Jesus' time believed in spirits. When one died, their soul went down to Sheol.

All however, did not suffer the same fate, the wicked one, cut off from his people, was tormented in Hades while the righteous one was gathered to his people, into the bosom of Abraham. [cf. Luke 16:23]

Heaven for them was the dwelling-place of God and where his throne was, the angels also resided there, and it was also the firmament, the region in which sun, moon, and stars are placed, and from where rain, the hail, and the lightning came. [cf. HEAVEN | Jewish Encyclopedia]


It should be noted that the Torah focuses on "Olam Ha Ze," which means "this world." That was the main focus for the Jews.

As for the kingdom of heaven? With reference to the Messianic age, it applies to the time when God will be the sole King on earth, in opposition to the kings of worldly powers. [cf. HEAVEN | Jewish Encyclopedia]



Further reading


Olam Ha-Ba: The Afterlife | Judaism 101
The Afterlife in Judaism, by Ariela Pelaia | About.com






cf. Ancient Hebrew Cosmology | Michæl Paukner.