Is there only one God (or אֱלֹהִים, or θεός) by nature? Men may be called "gods" (equivalent to Hebrew אֱלֹהִים (elohim) or Greek θεός (theos)) according to office, because they assume God's authority on earth. For example, in Exo. 21:6, it is written,


  Then his master shall bring him to the judges...


The English word "judges" is translated from the Hebrew word הָאֱלֹהִים (ha-elohim), the word most often translated into English as "God" in the KJV. 

But, are there any others besides YHVH who are "G/god/ אֱלֹהִים/ θεός" according to nature, rather than office?

If so, how does one reconcile that acknowledgment with scripture that state there is "one God" (cp. Isa. 45:5; Mark 12:32; 1 Cor. 8:6)?

If not, how does one reconcile that belief with scriptures such as 2 Cor. 4:4?
I think the general understanding among Christians is that God, properly understood, is unique in both nature and office when considering His full nature and full office. That is, there's no other being that is is His own essence (nature) or is the creator of all things (office). (Summa, Part 1, Question 3)

However, your question, as stated, seems to be more of a linguistic question. The words we use to refer to God are ultimately finite and flexible. Even if we had a word for a moment in time that specifically referred to the single being who is His own essence and is the creator of all things, that word would dissipate in its pure meaning very rapidly. Furthermore, words with strict meanings are often used in other contexts hyperbolically or illustratively.

Let's pretend our word for the single being who is His own essence and is the creator of all things is Pift. Thusly, it's proper for me to say,


  There is only one Pift. And Pift alone is worth our praise and
  worship.


But, I can also, in the very next sentence say,


  Cast your other Pifts aside. They're worthless!


Or even,


  That man is a Pift.


The first statement uses Pift in the proper literal form. In fact, the first sentence is tautological; it just paraphrases part of our literal definition. The other two statements abuse the strict, literal denotation of the word, but they each have a proper connotation. They're not necessarily misuses of the word, nor do they necessarily change the meaning of the first statement or the word itself.

In the second statement, something other than Pift is revered as Pift; and to highlight the reverence inappropriately given to these things, they're called Pift -- or perhaps the reverencers simply know them as Pift(s), and I'm using the term to avoid confusion (possibly counterproductive). In either case, there exist (or possible doesn't exist) some perceived things that is given honor and worship on par with Pift. Hence, they are the "Pifts" of our lives.

In the third statement, Pift is clearly just a hyperbole. There exists a man whom we think is awesome for some reason -- perhaps he can run 100 miles without stopping to pee or something equally amazing. To highlight how abnormal or amazing this is, we might call him a Pift. The hyperbole is obvious to an average person, and no harm is done to our word or our first statement.

However, the case with God-terminology in the Bible is often of a somewhat different nature, I think. Take a peek at What are the different names of God in the Bible and what do they mean?. You'll see a slew of "names" like Abba, Alpha, Omega, Good Shepherd, Creator, King, Judge, etc.

These are all words with other meanings, and fulfillable by numerous other things, used to refer explicitly to God in their context. They don't necessarily come with an explanation like, when I say judge here, what I really mean is God. Rather, they're [usually] used and understood by context. Take judge as an example.

Suppose you get a letter in the mail from your local court system that contains this:


  You will appear in court on Friday, February 8th. A jury of your peers
  will determine your guilt or innocence. If guilty, the judge will
  determine your sentence.


The judge in this context is a person, a particular one of many who have the qualifications and can fill the role.

Now, suppose you get a letter from a concerned (and probably tactless) Christian friend:


  You will die. Your sins will be weighed against your good deeds. And
  the Judge will determine your sentence.


We're talking about a very specific being here. The phrase, The judge will determine your sentence, is the same. And the denotation is similar, if not identical. But, the connotations and understood meanings are chasms apart.

The interpretive lens for the titles of God in the Bible fall in this spectrum -- from words that may be intended to mean the only one possible God to words elevated to illustrate particular offices or attributes of God. In neither case do I think there's any real ambiguity in the educated Christian mind. And in general, these sorts of ambiguities tend to arise only when we focus too closely on somethings (literal meaning of a word, in this case) and ignore the context.