Has anyone attempted to characterize chess mathematically? From years of being student, researcher, and practitioner of chess, I feel like it possesses completeness and elegance that are found in mathematical proofs and theories. Thus, it seems only natural to imagine that it should be possible to describe chess in mathematical terms. By that I mean an elegant description, like a compact set of formulas or a proof that chess belongs to such and such field of mathematics. Intuitively, it feels like it should be able to fit in group theory. Does anyone know what attempts have been made in this regard? I would like to specifically exclude engines, neural networks, and the like from the discussion because, despite the obvious success in increasing the quality of play, those attempts do not constitute theoretical proof.
There are many different aspects of chess which can be formalized mathematically. Since the 19th century at least, chess has been mined as a resource to drive mathematical innovation. So when talking about a mathematical characterization of chess, it's not a single modeling that we are talking about, which grabs every feature, but rather a number of models, in which the power of each is paradoxically in the letting go of some aspect of chess which is not relevant for that analysis.

Perhaps the earliest success (not counting the counting :-) of grains of rice on a chessboard) was Zermelo's Theorem (yes by one of the founders of ZF Set Theory) which states that in chess "either White can force a win, or Black can force a win, or both sides can force at least a draw". 

Combinatorial Game Theory (developed by celebrated mathematicians John H. Conway, Elwyn Berlekamp and Richard Guy) has been successfully applied to chess. A couple of assumptions in this theory go against its general applicability to chess though. One is that a win in CGT is solely if your opponent can’t move which does not address stalemate. Secondly there is the notion of “entailing move” (e.g. check) where if one person plays, the other player must respond rather than the first player playing again. But Noam Elkies has derived some non-trivial results in chess endgames by computing their CGT value - references given in his chess page http://www.math.harvard.edu/~elkies/chess.html: "On Numbers and Endgames" and "Higher Nimbers [sic] in Pawn Endgames on Large Chessboards".

You mentioned group theory - well curiously the set of combinatorial games under composition does form an abelian group! To be fair, the two obstacles mentioned above do prevent chess itself from behaving in this way.

There is a massive amount of combinatorial work which has been done around the chessboard. Vaclav Kotesovec has published online an 800 page book just on the subject of non-attacking chess pieces (generalizing the 8 queens problem massively). See http://www.kotesovec.cz/ for the link. This is related to Magic Squares, Experimental Design, etc.

There is also, starting in Finland with Eero Bonsdorff, a long lineage in path enumeration chess problems, counting the number of ways that a position can be reached. This often involves the analysis of Standard Young Tableaux, which also underpin representation theory of symmetric groups. Fibonacci numbers, Catalan numbers and Euler numbers are all frequently found here, together with other combinatorial identities whose realization can be found lurking in ingenious chess compositions. See https://pdb.dieschwalbe.de/search.jsp, and type g='mathematics' in the search box.

The knight's tour problem is also famous for helping to push the study of Hamiltonian graphs, and particularly the challenge of counting the number of such objects. See https://www.mayhematics.com/t/t.htm.

The theory of computation also asks about whether chess is determinate. This is all about arranging sets of pieces in much bigger boards to try to make machines that are equivalent to Turing machines. This brings us to the subject of the limits of computation, which is where I would make the point that if a mathematical theory characterized chess completely, then we should be no more able to reason with it than we are to play at chess itself. That might allow us to escape some of the linguistic challenges of the current laws (e.g. whether pawns can be oriented).

I think there is also research value in a mathematical description of how chess conventions apply consistently to chess problems including fairy chess. Guus Rol has a very ambitious  program which reduces each turn to “micro-phases” and claims to be able to determine with great accuracy how fairy conditions interact in complex “retro-active” situations, where both retro and forward logic is necessary. I don’t know if he will ever complete his theory.

I personally would like to see a more modest theory, which treats each move as atomic and although it doesn’t cover fairy chess so well, at least can cover retro-active aspects of problem conventions such as castling and e.p. Even that has not been done yet.

The mathematical physicist Roger Penrose published a chess position about 2 years ago which was intended to argue his long-held position that there is a fundamentally different kind of reasoning displayed by humans than an AI grounded in "computable functions" can demonstrate. See https://en.chessbase.com/post/a-chess-problem-holds-the-key-to-human-consciousness

Even though the theory of random graphs, and Monte Carlo analysis has been applied very successfully to engines, particularly the most recent generation, I don't think this disqualifies it from being considered a mathematical theory.

There is also a linear algebra approach to counting moves and positions. “If chess is a graph, what is its maximum eigenvalue?” is a very real and interesting question. See Francois Labelle's site at http://wismuth.com/chess/statistics-games.html

Linear algebra can be applied successfully to other chess related questions e.g. how many ways are there for a rook to move from a1 to h8 in exactly n moves? How about a king?

For many years, Noam Elkies &amp; Richard Stanley have been collaborating on a book on chess and mathematics. I don't know when it will finally emerge, or whether they have given up on this. But "The Mathematical Knight" given on Noam's chess page linked earlier perhaps gives a foretaste of that book. Richard Stanley is one of the leading lights in Combinatorics: see http://www-math.mit.edu/~rstan/chess/queue.pdf, and Noam's birthday article https://arxiv.org/pdf/math/0508645.pdf

There is also already a book "Mathematics and Chess" by Miodrag S. Petkovic, but I am not familiar with its contents.

Some of these ideas are listed in a Wikipedia page: https://en.wikipedia.org/wiki/Category:Mathematical_chess_problems, and there's a more specific page https://en.wikipedia.org/wiki/Mathematical_chess_problem.

If anyone else has other examples of chess and mathematics, please mention them in comments and I will try to incorporate them in this response. If anyone has links that illustrate the topics I am mentioning, please add them, thanks.

Finally: math.stackexchange.com has 62 pages of chess entries!?!