Can it be determined if language L lies in NP? Given a language L defined by a Turing Machine that decides it, is it possible to determine algorithmically whether L lies in NP?
No.  First, by Rice's Theorem, this is a property of TMs that depends only on the language they compute, so it cannot be computable.

But, more than that, it is known that the index set of $NP$ (that is, the set of TMs that compute languages in $NP$) is $\Sigma^0_3$-complete ($\Sigma^0_3$ in the arithmetic hierarchy of computability, not the polynomial hierarchy).

Questions like this were first investigated by Hajek.  For more, see e.g. this article by Ken Regan.

A few more great nuggets from Hajek's paper:


The index set of $P$ is $\Sigma^0_3$-complete.
$\{i : P^{L(M_i)} \neq NP^{L(M_i)}\}$ is $\Pi^0_2$-complete
There is a total Turing machine (halts on all inputs) $M_i$ such that $P^{L_i} = NP^{L_i}$ but the statement "$P^{L_i} = NP^{L_i}$" is independent (where $L_i = L(M_i)$).  Similarly for relativizations where $P \neq NP$.