<p>(I misunderstood the question at the beginning) 
The model is a binary non parametric regression. 
I don't know of a general model, I know of a semiparametric model by Klein and Spady 
<a href="http://www.ssc.wisc.edu/~bhansen/718/NonParametrics7.pdf" rel="nofollow">http://www.ssc.wisc.edu/~bhansen/718/NonParametrics7.pdf</a>
The model is as follows let $x´=[x_1´ \quad x_2´]$ 
Now $P(y=1|x)=E[y=1|x]=g(x'\beta)=f(x)$. 
The details in the link. 
Another possibility is to estimate the joint distribution (that has both discrete and continuous variables). Let g(y,x) be the joint pdf, then estimate $g(y|x)=\frac{g(y,x)}{g(x)}=f(x)$. The estimation of a joint of this type needs a product kernel that you can check 
Nonparametric estimation of regression functions with both categorical and continuous data
Journal of Econometrics, Volume 119, Issue 1, March 2004, Pages 99–130
Jeff Racine, Qi Li</p>

<p>For $y$ continuous the following answer works.
First of all is your data deterministic or random. 
If deterministic you may wish to do interpolation. Splines are your best option. 
If you think your dataset has noise or you have unobserved variables then you can do a non-parametric regression, I think this is what you want. 
The first step is to actually make assumptions about the structure of the randomness. 
The usual assumption is that 
$y=f(x_1,x_2)+\epsilon$ and you let $x´=[x_1´ \quad x_2´]$ be orthogonal -indepdent- of the unobserved variable $x\perp\epsilon$. Then you can replace $f$ by some high degree polynomial on $x$ and estimate this using Non linear least squares, technically this is called series or polynomial estimators (NLS is used when you know f, but you can argue that a high degree polynomial approximates the true f).<br>
Another way is using a ¨kernel¨ approach. 
In fact  you can directly assume that f is the conditional expectation. 
$E[y|x]=f(x_1,x_2)$. In that case you can estimate it using the joint distribution recall the definition of conditional expectation 
$E[y|x]=\int yf(y|x)dy$, with $f(y|x)=\frac{f(y,x)}{f(x)}$. Replacing this elements by its kernel estimators you get the Nadaraya-Watson estimator, you can google it. 
A third option is to use local linear regression. A more advanced option is to use Sieves estimators that use function basis to estimate the model. 
If f is not the conditional expectation then you have to use non parametric regression with instrumental variables, this is more complicated there is a paper by Renault (and others) on Econometrica about this. 
In summary it depends on the randomness structure that you are assuming. 
The book to go if you want to learn non-parametric regression is Li &amp; Racine <a href="http://press.princeton.edu/titles/8355.html" rel="nofollow">http://press.princeton.edu/titles/8355.html</a>
There is also a nice implementation of these econometric procedures in the open source statistical language "R" under the name (package) "np". </p>

CC BY-SA 3.0
2015-05-10T15:56:33.780Z
2015-05-11T04:34:57.150Z
2015-05-11T04:34:57.150Z
user157623
<p>The probability of observing a binary outcome <code>y</code> is a function of two variables, <code>y = f(x1, x2)</code>.</p>

<p>Both <code>x1</code> and <code>x2</code> are continuous, and I have no prior on the functional form of <code>f</code>. I have shallow knowledge of kernel density estimation, but have only seen that done in the case of a univariate variable. How should I proceed here / is there some basic literature on this? </p>

<econometrics><nonparametrics>
Nonparametric estimation: Interaction of two continuous variables
FooBar
# Nonparametric estimation: Interaction of two continuous variables

(Asked by: username_0 on 2015-05-10)

The probability of observing a binary outcome `y` is a function of two variables, `y = f(x1, x2)`.

Both `x1` and `x2` are continuous, and I have no prior on the functional form of `f`. I have shallow knowledge of kernel density estimation, but have only seen that done in the case of a univariate variable. How should I proceed here / is there some basic literature on this?

**Tags:** econometrics, nonparametrics

---

## 1 answers

### Answer by username_1 on 2015-05-10. Score: 1

(I misunderstood the question at the beginning)
The model is a binary non parametric regression.
I don't know of a general model, I know of a semiparametric model by Klein and Spady
<http://www.ssc.wisc.edu/~bhansen/718/NonParametrics7.pdf>
The model is as follows let $x´=[x\_1´ \quad x\_2´]$
Now $P(y=1|x)=E[y=1|x]=g(x'\beta)=f(x)$.
The details in the link.
Another possibility is to estimate the joint distribution (that has both discrete and continuous variables). Let g(y,x) be the joint pdf, then estimate $g(y|x)=\frac{g(y,x)}{g(x)}=f(x)$. The estimation of a joint of this type needs a product kernel that you can check
Nonparametric estimation of regression functions with both categorical and continuous data
Journal of Econometrics, Volume 119, Issue 1, March 2004, Pages 99–130
Jeff Racine, Qi Li

For $y$ continuous the following answer works.
First of all is your data deterministic or random.
If deterministic you may wish to do interpolation. Splines are your best option.
If you think your dataset has noise or you have unobserved variables then you can do a non-parametric regression, I think this is what you want.
The first step is to actually make assumptions about the structure of the randomness.
The usual assumption is that
$y=f(x\_1,x\_2)+\epsilon$ and you let $x´=[x\_1´ \quad x\_2´]$ be orthogonal -indepdent- of the unobserved variable $x\perp\epsilon$. Then you can replace $f$ by some high degree polynomial on $x$ and estimate this using Non linear least squares, technically this is called series or polynomial estimators (NLS is used when you know f, but you can argue that a high degree polynomial approximates the true f).  
Another way is using a ¨kernel¨ approach.
In fact you can directly assume that f is the conditional expectation.
$E[y|x]=f(x\_1,x\_2)$. In that case you can estimate it using the joint distribution recall the definition of conditional expectation
$E[y|x]=\int yf(y|x)dy$, with $f(y|x)=\frac{f(y,x)}{f(x)}$. Replacing this elements by its kernel estimators you get the Nadaraya-Watson estimator, you can google it.
A third option is to use local linear regression. A more advanced option is to use Sieves estimators that use function basis to estimate the model.
If f is not the conditional expectation then you have to use non parametric regression with instrumental variables, this is more complicated there is a paper by Renault (and others) on Econometrica about this.
In summary it depends on the randomness structure that you are assuming.
The book to go if you want to learn non-parametric regression is Li & Racine <http://press.princeton.edu/titles/8355.html>
There is also a nice implementation of these econometric procedures in the open source statistical language "R" under the name (package) "np".


---

