Is there a name for "physical things out of which one can build a Turing machine"? One of the amazing things about computer science is that the physical implementation is in some sense "irrelevant".
People have successfully built computers out of several different substrates -- relays, vacuum tubes, discrete transistors, etc.
People may soon succeed in building Turing-complete computers out of non-linear optical materials, various biomolecules, and a few other substrates.
In principle, it seems possible to build a billiard-ball computer.

However, the physical substrate is not completely irrelevant.
People have found that certain sets of components -- in particular,
diode-resistor logic --
are "incomplete": no matter how many of them you connect to a power supply and to each other, there are certain very simple things that it cannot do.
(The diode-resistor logic can implement AND, OR, but fails to implement NOT).
Also, certain ways of connecting components -- in particular, single-layer perceptrons -- are "incomplete": there are certain very simple things that they cannot do.
(A single-layer perceptron can implement AND, OR, NOT, but fails to implement XOR).

Is there a less-awkward phrase for "physical things out of which one can build a Turing machine"?
Or for the opposite, "physical things that, no matter how many of them one has, cannot form a Turing machine"?

For a while I used the phrase "functionally complete set" or "universal set of gates" -- or, when speaking to mathematicians, "physical things that can implement a functionally complete set" -- but I've been told that isn't quite correct.
Some sets of components can implement a functionally complete set;
and yet it is not possible to build a Turing-complete machine entirely out of these components.
For example, light bulbs and manually-operated 4-way light switches can implement a functionally complete set (AND, OR, NOT, XOR, etc.);
and yet it is not possible to build a Turing-complete machine entirely out of light switches and light bulbs, since the (electrical or optical) output of one cannot be fed into the (mechanically rotating) input of the next.

related: Is there an official name for a notion of &quot;reusably universal&quot;? and Is there a name for “chips out of which one can build a CPU”?
Physics models reality with theories that define a concept of time-dependent state associated to a system and a time-evolution operator that describes how this state evolves. As soon as you find a physical system that (after some state-space discretization) implements the state space of your Turing machine, and that features interaction terms that implement (maybe after some time discretization) the time evolution according to the state transition table of your Turing machine on its state space, you have found a Turing-complete physical model of your system. Thus you can arguable say, your system "is" Turing-complete. 

When looking at quantum computing, you will find discussions of implications physical theories have on the Turing model of computation. For example, physical theories have to be reversible. A property, which is not shared by ordinary Turing machines. Yet there is no loss in generality, as any Turing machine can be simulated by a reversible one, with some overhead that can trade-off time vs. space, etc.