Parameter Estimation for a Regression Model

Abstract

Exercise 3.1.Let a regression function f(⋅) be represented by a linear combination of basis functions TeXΨ1(⋅),…,Ψp(⋅){\textbackslash}Psi _{1}({\textbackslash}cdot ),{\textbackslash}ldots,{\textbackslash}Psi _{p}({\textbackslash}cdot ) .Suppose that for TeXx {\textbackslash}in \{{\textbackslash}mathbb{R}\}{\textasciicircum}{d} the regression function f(⋅) is quadratic in x. Describe the basis and the corresponding vector of coefficients in these cases.The function f(⋅ ) being quadratic in x, means when d = 1, TeXf(x) ={\textbackslash}theta _{1} +{\textbackslash}theta _{2}x +{\textbackslash}theta _{3}{x\}{\textasciicircum}{2} , which obviously leads to TeX{\textbackslash}displaystyle{\textbackslash}begin{array}{rcl} {\textbackslash}Psi _{1}(x) = 1,{\textbackslash}quad {\textbackslash}Psi _{2}(x) = x,{\textbackslash}quad {\textbackslash}Psi _{3}(x) = {x\}{\textasciicircum}{2}& & {\}{\textbackslash}{\textbackslash} {\textbackslash}end{array} When d {\textgreater} 1, TeX{\textbackslash}quad f(x) ={\textbackslash}theta _{1} + {A\}{\textasciicircum}\{{\textbackslash}top }x + {x\}{\textasciicircum}\{{\textbackslash}top \}{\textbackslash}mathit{Bx} , where TeXA {\textbackslash}in \{{\textbackslash}mathbb{R}\}{\textasciicircum}{d} , B is a d × d matrix.Then we can write 3.1 TeX{\textbackslash}displaystyle{\textbackslash}begin{array}{rcl} f(x) ={\textbackslash}theta _{1} +{\textbackslash}sum _{ j=1\}{\textasciicircum}{d}A_{ j}x_{j} +{\textbackslash}sum _{ j=1\}{\textasciicircum}{d\}{\textbackslash}sum _{ k=1\}{\textasciicircum}{d}B_\{{\textbackslash}mathit{ jk}}x_{j}x_{k}& &{\}{\textbackslash}end{array} where Aj is the jth element in A, and Bjk is the element in jth row and kth column of B.