What happened to Nebuchadnezzar's kingdom during his punishment Reading Daniel 4:29–33, I understand that Nebuchadnezzar was punished by God due to his pride. 

During his punishment, did his kingdom cease to exist or was there a new king?  Who was in charge of his position during his madness?
Daniel 4:29-33 is the culmination of writings, beginning with a Babylonian history, a series of Dead Sea Scroll fragments known as pseudo-Daniel and then the biblical book of Daniel.

M. H. Henze (The Madness of King Nebuchadnezzar, pages 57ff) identifies King Nabodinus (556-539) as the last king of Babylon, although his name never appears in the Hebrew Bible. His history was put together from a number of Babylonian cylinders discovered in 1854 at Tell al-Muqayyar (Ur). One of the cylinders names the king's son as Bel-sarra-usur, who was quickly identified as Belshazzar, given in Daniel as the son of Nebuchadnezzar. Thus it was possible to identify Nabodinus of the Babylonian texts with Nebuchadnezzar of the Book of Daniel.

The Babylonian texts tell us that Nabodinus left Babylon for ten years, to sojourn in Teima in the Arabian Peninsula, where he spent his time in devotion to the moon god, Sin, while his son Belshazzar ruled as regent in Babylon. On page 63, Henze says that according to a wide consensus, the story of Nebuchadnezzar's madness contains a recollection, however distorted, of Nabodinus' self-imposed exile at Teima.

The discovery of the Dead Sea Scrolls between 1946 and 1956 confirmed a link between the Babylonian writings and Daniel chapter 4. One Scroll fragment, known as 4Q The Prayer of Nabonidus closely resembles the account we see in Daniel 4:29-33.


Craig A. Evans, Peter W. Flint (Eschatology, Messianism, and the Dead Sea Scrolls, page 58) say the Aramaic Prayer of Nabonidus mentions Nabonidus and Teima, while agreeing with Daniel in giving the length of the sojourn as seven years. It describes the king as having a bad disease and attributes his recovery to a Jewish exile, without mentioning anyone called Daniel. Evans and Flint say we may conclude that 4QPrNab occupies an intermediate place between the Babylonian accounts of a historical incident and the formation of the Book of Daniel. 

We know from the Babylonian histories that the kingdom continued during the absence of Nabonidus (Nebuchadnezzar), with his son, Belshazzar, acting as regent.