What is the case for interpreting the Genesis creation account literally? I think we can all agree that the entire Bible is not meant to be taken literally.  Consider the Song of Solomon (what would his lover look like if we take it literally?) or the book of Revelation.

So I would like to understand the case for believing the Genesis creation account, among all Biblical texts, is one that is intended to be interpreted literally. I'm not interested in scientific evidence for a young-earth in these answers. I'm interested in Biblical or historical reasons why this text, as a whole (as opposed to individual words), should be considered literal.

I would also discourage answers that address specific words in the Genesis account (such as the Hebrew 'yom'), but if that is the strongest evidence you have, then I won't consider it an invalid answer.
The short version is that there are many places in the Bible that quote/reference the creation account in the book of Genesis literally (Exodus 16:16, Exodus 20:11, Exodus 31:17, Leviticus 23:3, Deuteronomy 5:13, Luke 13:14, etc).

The most obvious is the "reasoning" given for the 4th Commandment:


  Six days you shall labor and do all your work, but the seventh day is a sabbath of the LORD your God .. For in six days the LORD made the heavens and the earth, the sea and all that is in them, and rested on the seventh day


Furthermore, death was not introduced into creation until Adam sinned (Genesis 3:17-19, Romans 5:12, 8:20-22). Without death, you cannot have fossils.

The first of the genealogies of the Bible, which starts with the first man (Adam) and goes through Noah, is given in concrete years of the age of the father when the son was born. While I suppose you could elect to say that the first 5 days of creation were figurative, certainly from day 6 on is distinctly not figurative - otherwise the reckoning of Adam's age at the birth of Seth would be irrational.