What are theoretically sound programming languages for graph problems? There are numerous graph theoretic tools/packages. Each with its pros and cons. What should be the semantics/syntax of a programming language meant to solve graph theoretic problems?
You might want to look at The Graph Programming Language GP. From the linked page:


  GP (for Graph Programs) is a rule-based, nondeterministic programming language for solving graph problems at a high level of abstraction, freeing programmers from handling low-level data structures. The core of GP consists of four constructs: single-step application of a set of conditional graph-transformation rules, sequential composition, branching and iteration. 


Sandra Steinert devoted her PhD thesis to the topic.

There's also a Hoare logic for reasoning about the correctness of such programs.