What is the process for starting a new Christian denomination? There are thousands of Christian denominations. How did these groups form? 

What is the typical process for organizing a denomination?
Not to be flip, but it sort of goes like this:


Find some theological point on which you find your existing denomination to be so heretical as to be in grave danger of going to hell.  Assume this point is not aidaphora.
Find a bunch of other people who agree, and start meeting together.
Possibly, ordain yourself (as in the case of Joseph Smith - the founder of the LDS denomination, or Joseph Smythe - traditionally considered the founder of the Baptists) or if you already have been ordained (e.g. Richard Allen, founder of the AME Church after being nearly beaten senseless in his home church or the Episcopalian Church in general after the American Revolution) convince a court that you are not, in fact part of your former denomination.
Start forming more than one church, so that you actually are a group of churches, rather than just a splinter one.


Ultimately, the only two entities that you will probably care about are:


Your home denomination - you don't want them claiming you, right?  Just send them a letter saying you are no longer part of them.  If you publicly repudiate them, so much the better!  The point is simple enough - your adherence to a given church is completely voluntary.  Any way in which you no longer volunteer will suffice.
The IRS.  Here's what most groups in the US care about - the tax exemption.  Truth is, it doesn't actually matter to the IRS either. If you want to call yourself "The Church of the Flying Spaghetti Monster," you just need to convince the IRS you are church.  If you affiliate with anyone else, it helps, but ultimately, they don't really decide.


Like nations, each denomination is simply a sovereign entity unto itself.  If you try to call yourself "the Roman Catholic Church," there is an entity and an institution that will say, "No you are not."  But if you want to start your own, there is no one to stop you.

In the end, denominations - even highly organized ones - are voluntary associations of like-minded individuals.  There is no authority - in fact, there isn't even an agreed on definition of denominations.  Scholars simply use the label to distinguish like-minded groups from one another.