Is there a sense in which we can say error returned from a function is the reverse of context? This came up in a discussion about golang, but I think it applies more generally.

Context in which a function is executed (specially in when we have RPCs) and error returned from a function seem to be kind of special input and return values. Is that really the case? Are there any sense that they are related or say reverse or dual of each other?
A duality between exceptions and states was observed and worked out by Jean-Guillaume Dumas, Dominique Duval, Laurent Fousse, and Jean-Claude Reynaud.

TL;DR: Reader and exception monads are obviously dual.

In your setup, I think it works as follows. Let $S$ be the type of environments in which a program might run (you call them contexts, but I'd rather save that word for typing contexts), and let $E$ be a type of errors that a program may raise. The environments here are read-only.

A program which runs in environment of type $S$, takes input of type $A$, and outputs a result of type $B$ has type
$$S \times A \to B.$$
A program which may raise an error of type $E$, takes input of type $A$, and outputs a result of type $B$ has type
$$A \to E + B.$$
We clearly have a duality here. If we imagine that this is all hapenning in some category $\mathcal{C}$, then $X \times A \to B$ in $\mathcal{C}$ is the same thing as $B \to X + A$ in the dual category $\mathcal{C}^\mathrm{op}$.