6 Instrumental Variables and GMM: Theory

Maximum likelihood is one of the main estimation methods in spatial econometrics, but it is not the only one. Instrumental variables (IV) and the generalized method of moments (GMM) provide an alternative route that does not require a Gaussian likelihood and, in many spatial models, avoids repeated evaluation of large matrix determinants.

This chapter develops the theoretical foundations of IV and GMM estimation for spatial regression models. We begin with the general GMM framework and then study spatial two-stage least squares for the Spatial Lag Model, efficient IV and GMM procedures, moment-based estimation of the Spatial Error Model, and multistep estimation of the Spatial Autoregressive Combined model. The implementation of these estimators in R is deferred to a separate chapter so that the present discussion can focus on identification, moment conditions, asymptotic properties, and efficiency.

6.1 A Review of GMM

Before presenting GMM-based estimators for the SLM, SEM, and SAC models, we review the core ideas of the generalized method of moments (GMM) as they are used in spatial econometrics. The emphasis is on what is genuinely specific to spatial settings. First, spatial interaction typically creates endogeneity (e.g., through $\mathbf W_n \mathbf y$) and motivates an instrumental-variables interpretation of GMM. Second, spatial dependence means that laws of large numbers and central limit theorems cannot be taken for granted and must be justified under conditions adapted to spatial data. Third, many influential spatial GMM procedures exploit quadratic moment restrictions that are naturally expressed through spatial quadratic forms. Our presentation follows the IV/MoM perspective in Prucha (2014).

6.1.1 Model Specification and Moment Conditions

Let $\mathbf W_n=(w_{ij,n})$ denote the $n\times n$ spatial weights matrix, with $w_{ii,n}=0$ for all $i$, and let $\mathbf z_{in}$ collect all observable variables for unit $i$ (e.g., $y_{in}$, $\mathbf x_{in}$, and instruments $\mathbf h_{in}$). Suppose the data are generated by the structural relationship \[ f(y_{in},\mathbf x_{in},\boldsymbol\theta_0)=\epsilon_{in},\qquad i=1,\ldots,n, \tag{6.1}\] where $\boldsymbol\theta_0\in\boldsymbol\Theta\subset\mathbb R^k$ is unknown and $f(\cdot)$ is known. In spatial models, the regressor vector $\mathbf x_{in}$ may include spatial lags and thus depend on $n$. A convenient generic representation is \[ \mathbf x_{in} = \left[\mathbf x_i,\ (\mathbf W_n\mathbf x)_i,\ (\mathbf W_n y)_i\right] = \left[\mathbf x_i,\ \sum_{j=1}^n w_{ij,n}\mathbf x_{jn},\ \sum_{j=1}^n w_{ij,n}y_{jn}\right], \] where $\mathbf x_i$ is exogenous and $(\mathbf W_n y)_i$ and $(\mathbf W_n\mathbf x)_i$ are spatial lags.

Assume there exists a $p\times 1$ vector of instruments $\mathbf h_{in}$ such that suitable orthogonality restrictions hold. For simplicity, we motivate moment conditions under i.i.d. disturbances with $\mathbb E(\epsilon_{in})=0$ and $\operatorname{Var}(\epsilon_{in})=\sigma^2$, but the spatial GMM literature routinely relaxes this to heteroskedasticity and spatial dependence; these generalizations matter primarily for the asymptotic theory and for how the optimal weighting matrix is constructed.

We summarize the sample information through a $p\times 1$ vector of moments \[ \mathbf g_n(\boldsymbol\theta) = \mathbf g_n(\mathbf z_{1n},\ldots,\mathbf z_{nn};\boldsymbol\theta) = \begin{pmatrix} g_{1n}(\mathbf z_{1n},\ldots,\mathbf z_{nn};\boldsymbol\theta)\\ \vdots\\ g_{pn}(\mathbf z_{1n},\ldots,\mathbf z_{nn};\boldsymbol\theta) \end{pmatrix}, \qquad p\ge k, \tag{6.2}\] and impose the standard identifying restriction \[ \mathbb E\!\left[\mathbf g_n(\boldsymbol\theta)\right]=\mathbf 0 \quad\Longleftrightarrow\quad \boldsymbol\theta=\boldsymbol\theta_0. \]

Given a symmetric positive semidefinite weighting matrix $\boldsymbol\Upsilon_n$, the one-step GMM estimator is defined by \[ \widehat{\boldsymbol\theta}_n = \operatorname*{argmin}_{\boldsymbol\theta\in\boldsymbol\Theta}\; Q_n(\boldsymbol\theta), \qquad Q_n(\boldsymbol\theta)\equiv \mathbf g_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_n \mathbf g_n(\boldsymbol\theta). \tag{6.3}\]

If $p=k$ (just identification), $\boldsymbol\Upsilon_n$ is irrelevant and $\widehat{\boldsymbol\theta}_n$ solves $\mathbf g_n(\widehat{\boldsymbol\theta}_n)=\mathbf 0$.

The classical GMM literature often builds $\mathbf g_n(\boldsymbol\theta)$ from linear IV-type restrictions. In the spatial setting these remain important, but they are frequently complemented by quadratic restrictions that exploit the geometry encoded in $\mathbf W_n$. We briefly describe both classes because they reappear repeatedly in spatial GMM estimators.

6.1.1.1 Linear (IV-type) moments

A canonical linear moment condition takes the form \[ \mathbf g_n^{L}(\boldsymbol\theta) = \frac{1}{n}\sum_{i=1}^n \mathbf h_{in}\,\epsilon_{in}(\boldsymbol\theta), \qquad \epsilon_{in}(\boldsymbol\theta)\equiv f(y_{in},\mathbf x_{in},\boldsymbol\theta), \] so that $\mathbb E[\mathbf h_{in}\epsilon_{in}(\boldsymbol\theta_0)]=\mathbf 0$ under instrument exogeneity. In spatial models, this is the natural starting point whenever spatial lags create endogeneity, since valid instruments can be constructed from exogenous covariates and their spatial lags (e.g., $\mathbf W_n\mathbf x$, $\mathbf W_n^2\mathbf x$, etc.), subject to the usual rank and relevance requirements.

6.1.1.2 Quadratic (spatially distinctive) moments

A distinctive feature of spatial GMM is the use of quadratic moments built from spatial quadratic forms in the structural disturbance. Let $\mathbf u_n(\boldsymbol\theta)$ denote the $n\times 1$ residual vector. For a given $n$, choose an $n\times n$ nonstochastic matrix $\mathbf A_{q,n}$ and define the quadratic form \[ m_{q,n}(\boldsymbol\theta)\;=\;\mathbf u_n(\boldsymbol\theta)^\top \mathbf A_{q,n}\,\mathbf u_n(\boldsymbol\theta), \qquad g^{Q}_{q,n}(\boldsymbol\theta)\;=\;\frac{1}{n}m_{q,n}(\boldsymbol\theta). \]

Following the matrix classification in Lee (2007), we restrict $\{\mathbf A_{q,n}\}_{n\ge1}$ to belong to a class of matrices that is compatible with the usual regularity conditions imposed on $\mathbf W_n$ (uniformly bounded row and column sums). Define \[ \mathcal P_1 \;\equiv\;\Big\{\{\mathbf A_n\}_{n\ge1}:\ \operatorname{tr}(\mathbf A_n)=0,\ \sup_{n}\|\mathbf A_n\|_{1}<\infty,\ \sup_{n}\|\mathbf A_n\|_{\infty}<\infty\Big\}, \] where $\|\mathbf A\|_{1}=\max_{j}\sum_{i}|a_{ij}|$ and $\|\mathbf A\|_{\infty}=\max_{i}\sum_{j}|a_{ij}|$ are the maximum absolute column- and row-sum norms (see Definition 3.20). The trace restriction is the key identification device for the quadratic moment under independent disturbances: if $\mathbb E(\mathbf u_n(\boldsymbol\theta_0)\mathbf u_n(\boldsymbol\theta_0)^\top)=\sigma^2\mathbf I_n$, then \[ \mathbb E\!\left[g^{Q}_{q,n}(\boldsymbol\theta_0)\right] = \frac{1}{n}\operatorname{tr}\!\big(\mathbf A_{q,n}\mathbb E[\mathbf u_n(\boldsymbol\theta_0)\mathbf u_n(\boldsymbol\theta_0)^\top]\big) = \frac{\sigma^2}{n}\operatorname{tr}(\mathbf A_{q,n}) = 0. \]

A useful subclass is obtained by imposing a zero diagonal: \[ \mathcal P_0 \;\equiv\;\Big\{\{\mathbf A_n\}_{n\ge1}\in\mathcal P_1:\ \operatorname{diag}(\mathbf A_n)=\mathbf 0\Big\}. \]

This corresponds to Lee’s $P_{2n}\subset P_{1n}$. The restriction $\operatorname{diag}(\mathbf A_{q,n})=\mathbf 0$ implies \[ \mathbf u_n(\boldsymbol\theta)^\top \mathbf A_{q,n}\mathbf u_n(\boldsymbol\theta) = \sum_{i\neq j} a_{ij,q,n}\,u_{in}(\boldsymbol\theta)\,u_{jn}(\boldsymbol\theta), \] so the quadratic moment depends only on cross-products with $i\neq j$.

The practical advantage of $\mathcal P_0$ is variance simplification under independent errors. If the components of $\mathbf u_n(\boldsymbol\theta_0)$ are independent with $\mathbb E u_{in}=0$, $\mathbb E u_{in}^2=\sigma^2$, and finite fourth moments, then the variance of a quadratic form typically decomposes into a “Gaussian part” plus an extra term involving diagonal elements and the excess kurtosis. A convenient representation is \[ \operatorname{Var}\!\big(\mathbf u^\top \mathbf A \mathbf u\big) = 2\sigma^4\,\operatorname{tr}(\mathbf A\mathbf A^\top) \;+\; \kappa\,\sigma^4\sum_{i=1}^n a_{ii}^2, \qquad \kappa \equiv \frac{\mathbb E u_i^4}{\sigma^4}-3, \] (up to symmetry conventions on $\mathbf A$). Hence, when $\mathbf A\in\mathcal P_0$ the second term vanishes, and the variance (and therefore the optimal weighting for these quadratic moments) does not depend on the unknown fourth-moment/kurtosis parameter. This is the main reason $\mathcal P_0$ moments are attractive in practice.

In spatial applications, $\mathbf A_{q,n}$ is usually constructed from $\mathbf W_n$ and its transformations (e.g., $\mathbf W_n$, $\mathbf W_n^\top$, $\mathbf W_n^\top\mathbf W_n$, or trace-adjusted versions), so these quadratic moments turn spatial structure into informative restrictions on residual cross-products.

6.1.2 One-Step and Two-Step Constructions

Many spatial procedures are implemented as multi-step methods. The reason is mainly practical: a full one-step nonlinear minimization over all parameters can be computationally demanding when spatial dependence enters through $(\mathbf W_n \mathbf y)$ and/or through spatially correlated errors. Two-step constructions reduce the problem to simpler subproblems, often by using preliminary residuals to identify spatial dependence parameters and then estimating regression coefficients conditional on those estimates.

To describe this generically, partition $\boldsymbol\theta_0=[\lambda_0,\boldsymbol\delta_0^\top]^\top$, where $\lambda_0$ denotes a spatial dependence parameter (e.g., an autoregressive coefficient) and $\boldsymbol\delta_0$ collects the remaining parameters. Suppose the moment vector admits the decomposition \[ \mathbf g_n(\boldsymbol\theta)= \begin{pmatrix} \mathbf g_n^{\boldsymbol\lambda}(\lambda,\boldsymbol\delta)\\ \mathbf g_n^{\boldsymbol\delta}(\lambda,\boldsymbol\delta) \end{pmatrix}, \] and that the population restrictions satisfy \[ \mathbb E[\mathbf g_n^{\boldsymbol\lambda}(\lambda,\boldsymbol\delta_0)]=\mathbf 0\iff \lambda=\lambda_0, \qquad \mathbb E[\mathbf g_n^{\boldsymbol\delta}(\lambda_0,\boldsymbol\delta)]=\mathbf 0\iff \boldsymbol\delta=\boldsymbol\delta_0. \]

Given a preliminary estimator $\widetilde{\boldsymbol\delta}_n\xrightarrow{p}\boldsymbol\delta_0$, define the first-step estimator \[ \widehat{\lambda}_n = \operatorname*{argmin}_{\lambda}\; \big[\mathbf g_n^{\boldsymbol\lambda}(\lambda,\widetilde{\boldsymbol\delta}_n)\big]^\top \boldsymbol\Upsilon_n^{\lambda\lambda} \big[\mathbf g_n^{\boldsymbol\lambda}(\lambda,\widetilde{\boldsymbol\delta}_n)\big], \tag{6.4}\] and then estimate $\boldsymbol\delta_0$ by \[ \widehat{\boldsymbol\delta}_n = \operatorname*{argmin}_{\boldsymbol\delta}\; \big[\mathbf g_n^{\boldsymbol\delta}(\widehat{\lambda}_n,\boldsymbol\delta)\big]^\top \boldsymbol\Upsilon_n^{\delta\delta} \big[\mathbf g_n^{\boldsymbol\delta}(\widehat{\lambda}_n,\boldsymbol\delta)\big]. \tag{6.5}\]

When the model is correctly specified, an efficient one-step GMM estimator is asymptotically no worse than multi-step alternatives, but in spatial applications the computational savings of two-step methods can be substantial and the efficiency loss is often modest.

6.1.3 Asymptotic Distribution of the One-Step GMM Estimator

Assume $\widehat{\boldsymbol\theta}_n$ is an interior minimizer of $Q_n(\boldsymbol\theta)$. The first-order condition is \[ \mathbf 0 = \frac{\partial Q_n(\widehat{\boldsymbol\theta}_n)}{\partial\boldsymbol\theta} = 2\,\mathbf G_n(\widehat{\boldsymbol\theta}_n)^\top \boldsymbol\Upsilon_n \mathbf g_n(\widehat{\boldsymbol\theta}_n), \qquad \mathbf G_n(\boldsymbol\theta)\equiv \frac{\partial \mathbf g_n(\boldsymbol\theta)}{\partial\boldsymbol\theta^\top}. \tag{6.6}\]

A mean-value expansion of $\mathbf g_n(\widehat{\boldsymbol\theta}_n)$ around $\boldsymbol\theta_0$ yields \[ \mathbf g_n(\widehat{\boldsymbol\theta}_n) = \mathbf g_n(\boldsymbol\theta_0) + \mathbf G_n(\overline{\boldsymbol\theta}_n)\,(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0), \] for some $\overline{\boldsymbol\theta}_n$ between $\widehat{\boldsymbol\theta}_n$ and $\boldsymbol\theta_0$. Substituting into Equation 6.6 and rearranging gives \[ \sqrt{n}(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0) = -\Big[\mathbf G_0^\top\boldsymbol\Upsilon_0\mathbf G_0\Big]^{-1}\mathbf G_0^\top\boldsymbol\Upsilon_0\;\sqrt{n}\,\mathbf g_n(\boldsymbol\theta_0) + o_p(1), \] provided \[ \mathbf G_n(\boldsymbol\theta_0)\xrightarrow{p} \mathbf G_0,\qquad \boldsymbol\Upsilon_n\xrightarrow{p} \boldsymbol\Upsilon_0, \qquad \sqrt{n}\,\mathbf g_n(\boldsymbol\theta_0)\xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Psi_0), \tag{6.7}\] with $\boldsymbol\Psi_0$ positive definite. Consequently, \[ \sqrt{n}(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0)\xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Phi_0), \qquad \boldsymbol\Phi_0 = \Big[\mathbf G_0^\top\boldsymbol\Upsilon_0\mathbf G_0\Big]^{-1} \mathbf G_0^\top\boldsymbol\Upsilon_0\,\boldsymbol\Psi_0\,\boldsymbol\Upsilon_0\mathbf G_0 \Big[\mathbf G_0^\top\boldsymbol\Upsilon_0\mathbf G_0\Big]^{-1}. \]

The efficient choice is $\boldsymbol\Upsilon_0=\boldsymbol\Psi_0^{-1}$, which yields \[ \boldsymbol\Phi_0=\big[\mathbf G_0^\top\boldsymbol\Psi_0^{-1}\mathbf G_0\big]^{-1}. \] In spatial applications, both the CLT in Equation 6.7 and the consistent estimation of $\boldsymbol\Psi_0$ require care because $\mathbf g_n(\boldsymbol\theta_0)$ typically involves linear and/or quadratic forms under spatial dependence. This is why subsequent sections introduce dependence conditions and matrix classes for $\mathbf A_{q,n}$ that guarantee well-behaved limits.

6.1.4 Asymptotic Distribution of the Two-Step GMM Estimator

The limiting distribution of two-step GMM estimators differs from the one-step case because the first-step plug-in affects both the centering and the variability of the moments used in the second step. To see this, expand the relevant moments in Equation 6.4 around $(\lambda_0,\boldsymbol\delta_0)$: \[ \sqrt{n}\big(\widehat{\lambda}_n-\lambda_0\big) = -\Big[(\mathbf G_0^{\lambda\lambda})^\top\boldsymbol\Upsilon_0^{\lambda\lambda}\mathbf G_0^{\lambda\lambda}\Big]^{-1} (\mathbf G_0^{\lambda\lambda})^\top\boldsymbol\Upsilon_0^{\lambda\lambda} \Big[ \sqrt{n}\,\mathbf g_n^{\boldsymbol\lambda}(\lambda_0,\boldsymbol\delta_0) + \mathbf G_0^{\lambda\delta}\sqrt{n}\,(\widetilde{\boldsymbol\delta}_n-\boldsymbol\delta_0) \Big] + o_p(1), \tag{6.8}\] where $\mathbf G_0^{\lambda\lambda}$ and $\mathbf G_0^{\lambda\delta}$ are the probability limits of the corresponding Jacobians and $\boldsymbol\Upsilon_n^{\lambda\lambda}\xrightarrow{p}\boldsymbol\Upsilon_0^{\lambda\lambda}$.

In many implementations the preliminary estimator is asymptotically linear: \[ \sqrt{n}\,(\widetilde{\boldsymbol\delta}_n-\boldsymbol\delta_0) = \frac{1}{\sqrt{n}}\mathbf T_n^\top\boldsymbol\varepsilon_n + o_p(1), \] for some nonstochastic matrix $\mathbf T_n$. Defining the augmented moment vector \[ \mathbf g_{*n}^{\boldsymbol\lambda}(\lambda_0,\boldsymbol\delta_0) \equiv \mathbf g_n^{\boldsymbol\lambda}(\lambda_0,\boldsymbol\delta_0) + \frac{1}{n}\mathbf G_0^{\lambda\delta}\mathbf T_n^\top\boldsymbol\varepsilon_n, \] Equation 6.8 becomes \[ \sqrt{n}\big(\widehat{\lambda}_n-\lambda_0\big) = -\Big[(\mathbf G_0^{\lambda\lambda})^\top\boldsymbol\Upsilon_0^{\lambda\lambda}\mathbf G_0^{\lambda\lambda}\Big]^{-1} (\mathbf G_0^{\lambda\lambda})^\top\boldsymbol\Upsilon_0^{\lambda\lambda}\; \sqrt{n}\,\mathbf g_{*n}^{\boldsymbol\lambda}(\lambda_0,\boldsymbol\delta_0) + o_p(1). \tag{6.9}\]

If \[ \sqrt{n}\,\mathbf g_{*n}^{\boldsymbol\lambda}(\lambda_0,\boldsymbol\delta_0)\xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Psi_*^{\lambda\lambda}), \] then \[ \sqrt{n}\big(\widehat{\lambda}_n-\lambda_0\big)\xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Phi_*^{\lambda\lambda}), \] with the corresponding sandwich form and efficient weighting $\boldsymbol\Upsilon_0^{\lambda\lambda}=(\boldsymbol\Psi_*^{\lambda\lambda})^{-1}$. The key point, emphasized in Prucha (2014), is that generally $\boldsymbol\Psi_*^{\lambda\lambda}\neq \boldsymbol\Psi^{\lambda\lambda}$ because the plug-in step contributes extra variability unless $\mathbf G_0^{\lambda\delta}=\mathbf 0$.

To illustrate this difference transparently, consider the special case in Prucha (2014) where $\epsilon_i$ are i.i.d. $\mathcal N(0,\sigma^2)$ and the moments are quadratic. For simplicity let \[ \mathbf g_n^{\boldsymbol\lambda}(\lambda_0,\boldsymbol\delta_0) = \begin{pmatrix} \frac{1}{n}\boldsymbol\varepsilon^\top \mathbf A_{1,n}\boldsymbol\varepsilon\\ \frac{1}{n}\boldsymbol\varepsilon^\top \mathbf A_{2,n}\boldsymbol\varepsilon \end{pmatrix}, \qquad \operatorname{tr}(\mathbf A_{q,n})=0. \]

If the augmentation contributes a linear term $\frac{1}{n}\sum_i a_{iq}\epsilon_i$ to component $q$, then the asymptotic variance becomes \[ \Psi^{\lambda\lambda}_{rs} = 2\sigma^4\sum_{i=1}^n\sum_{j=1}^n a_{ijr}a_{ijs}, \qquad \Psi^{\lambda\lambda}_{*rs} = 2\sigma^4\sum_{i=1}^n\sum_{j=1}^n a_{ijr}a_{ijs} + \sigma^2\sum_{i=1}^n a_{ir}a_{is}. \]

Because the extra term depends on the first-step construction (through $\mathbf T_n$), correct implementation of two-step spatial GMM typically prescribes a specific sequence of residual construction and variance estimation.

Finally, for orientation, one-step GMM-type estimators include, among others, the SLM under homoskedasticity (Lee 2007), heteroskedastic SLM (Lin and Lee 2010), extensions with additional endogenous regressors (Liu and Saraiva 2015), and efficient GMM for SAC-type structures (Lee and Liu 2010; Liu et al. 2010). Two-step / IV-MoM estimators include GS2SLS and related procedures for SAC/SEM models, starting with Kelejian and Prucha (1998); Kelejian and Prucha (1999) and later heteroskedastic refinements such as Kelejian and Prucha (2010).

6.2 Spatial Two-Stage Least Squares Estimation of the SLM

In this section we present the Spatial Two-Stage Least Squares (S2SLS) estimator for the SLM. Our discussion follows Kelejian and Prucha (1998), who establish large-sample properties for this estimator, and we also note the heteroskedastic extension in Kelejian and Prucha (2010).¹

This approach has two practical advantages:

It avoids likelihood-based computations such as the log-determinant term and nonlinear optimization over $(\boldsymbol\beta,\rho)$.
It relies on moment restrictions rather than a full parametric distribution for $\epsilon_{i,n}$, so normality is not required for consistency.

To understand the essence of this procedure, let us revisit the formulation of the SLM, expressed as: \[ \mathbf y_n = \mathbf X_n\boldsymbol\beta_0 + \rho_0\mathbf W_n\mathbf y_n + \boldsymbol\varepsilon_n. \]

Alternatively, we can write the model more concisely as: \[ \mathbf y_n = \mathbf Z_n \boldsymbol\delta_0 + \boldsymbol\varepsilon_n, \] where $\mathbf Z_n = \left[\mathbf X_n, \mathbf W_n\mathbf y_n\right]$ is an $n \times (k + 1)$ matrix that combines the exogenous regressors and the spatially lagged dependent variable, and the $(k + 1)\times 1$ coefficient column vector is rearranged as $\boldsymbol\delta_0 = (\boldsymbol\beta^\top_0, \rho_0)^\top$. As discussed in Section 4.1, the inclusion of the spatially lagged dependent variable, $\mathbf W_n\mathbf y_n$, on the right-hand side of the equation introduces endogeneity or simultaneous equation bias. Therefore, the OLS estimates are inconsistent.

To address this issue, we may use an instrumental variables (IV) approach rather than resorting to QML or ML methods. The IV principle relies on the existence of a matrix of instruments, $\mathbf H_n$, that is strongly correlated with $\mathbf Z_n$ but asymptotically uncorrelated with the error term $\boldsymbol\varepsilon_n$.

In what follows we maintain that all columns of $\mathbf X_n$ are exogenous, so the only endogenous regressor is $\mathbf W_n\mathbf y_n$. Consequently, $\mathbf H_n$ should contain $\mathbf X_n$ and additional instruments for $\mathbf W_n\mathbf y_n$.

6.2.1 Instruments in the Spatial Context

What constitutes the best instruments for $\mathbf W_n\mathbf y_n$? To construct the instrument matrix $\mathbf H_n$, it is essential to refer to the literature on optimal instrumental variables. Broadly, this literature suggest that the ‘best instruments’ for the r.h.s variables are their conditional means. Consequently, the ideal instruments are expressed as: \[ \begin{aligned} \mathbb E\left(\mathbf Z_n|\mathbf X_n\right) & = \left[\mathbb E\left(\mathbf X_n|\mathbf X_n\right), \mathbb E\left(\mathbf W_n\mathbf y_n|\mathbf X_n\right)\right], \\ & = \left[\mathbf X_n, \mathbf W_n \mathbb E\left(\mathbf y_n|\mathbf X_n\right)\right]\quad \mbox{since $\mathbf W_n$ is non-stochastic}. \end{aligned} \]

Given that $\mathbf X_n$ is exogenous, it serves as its own best instrument, whereas the best instruments for $\mathbf W_n\mathbf y_n$ are given by $\mathbf W_n\mathbb E\left(\left.\mathbf y_n\right|\mathbf X_n\right)$. Noting that the reduced-form equation, for a given $\boldsymbol\theta = (\boldsymbol\beta^\top, \rho)^\top$ is $\mathbf y_n = (\mathbf I_n - \rho\mathbf W_n)^{-1}\left(\mathbf X_n\boldsymbol\beta + \boldsymbol\varepsilon_n \right)$, and applying Neumann Expansion, the expected value of the reduced form is: \[ \mathbb E(\mathbf y_n|\mathbf X_n) = (\mathbf I_n - \rho\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta = \left[\mathbf I_n + \rho\mathbf W_n + \rho^2\mathbf W_n^2 + \cdots \right]\mathbf X_n\boldsymbol\beta = \left[\sum_{l = 0}^{\infty}\rho^{l}\mathbf W^l_n\right]\mathbf X_n\boldsymbol\beta. \tag{6.10}\]

The challenge lies in approximating $\mathbb E(\mathbf y_n|\mathbf X_n)$ without directly inverting the $n\times n$ matrix $(\mathbf I_n - \rho\mathbf W_n)$. Equation 6.10 reveals that $\mathbb E(\mathbf y_n|\mathbf X_n)$ can be expressed as a linear function of $\mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W^2_n\mathbf X_n, \ldots$ As a result, and given that the roots of $\rho\mathbf W_n$ are less than one in absolute value (or $\rho$ is in its parameter space), the expectation can also be written as: \[ \begin{aligned} \mathbb E(\mathbf W_n\mathbf y_n|\mathbf X_n) & = \mathbf W_n\left(\mathbf I_n - \rho \mathbf W_n\right)^{-1}\mathbf X_n\boldsymbol\beta, \\ & = \mathbf W_n\left[\mathbf I_n + \rho\mathbf W_n + \rho^2\mathbf W^2_n + \rho^3\mathbf W^3_n + \cdots\right]\mathbf X_n\boldsymbol\beta, \\ & = \mathbf W_n\left[\sum_{l = 0}^{\infty}\rho_0^{l}\mathbf W^l_n\right]\mathbf X_n\boldsymbol\beta, \\ & = \mathbf W_n\mathbf X_n\boldsymbol\beta + \mathbf W^2_n\mathbf X_n(\rho\boldsymbol\beta) + \mathbf W^3_n\mathbf X_n(\rho^2\boldsymbol\beta) + \mathbf W^4_n\mathbf X_n(\rho^3\boldsymbol\beta) + \cdots \end{aligned} \]

To avoid issues associated with the computation of the inverse of $(\mathbf I_n - \rho\mathbf W_n)$, Kelejian and Prucha (1998) propose an approximation of the optimal instruments. Recognizing that $\mathbb E(\mathbf y_n|\mathbf X_n)$ is linear in $\mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W^2_n\mathbf X_n, \ldots$, they suggest using a set of instruments $\mathbf H_n$ which consists of the linearly independent (LI) columns of $\mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W^2_n\mathbf X_n, \ldots, \mathbf W^l_n\mathbf X_n$, where $l$ is a pre-selected finite constant and is generally set to 2 in applied studies. Thus, for $l=2$, \[ \mathbf H_n = (\mathbf X_n,\ \mathbf W_n\mathbf X_n,\ \mathbf W_n^{\,2}\mathbf X_n)_{LI}. \]

Remark

The exogeneity argument is straightforward. Because $\mathbf W_n$ is nonstochastic, \[ \mathbb E\!\left(\mathbf X_n^\top\boldsymbol\varepsilon_n\right)=\mathbf 0 \qquad\Longrightarrow\qquad \mathbb E\!\left((\mathbf W_n^l\mathbf X_n)^\top\boldsymbol\varepsilon_n\right) = \mathbf X_n^\top(\mathbf W_n^l)^\top \mathbb E(\boldsymbol\varepsilon_n) = \mathbf 0 \] for any fixed integer $l\ge 1$. Hence spatial lags of the exogenous regressors remain valid instruments. The relevance argument comes from the conditional mean of the spatial lag: \[ \mathbb E(\mathbf W_n\mathbf y_n\mid \mathbf X_n) = \mathbf W_n(\mathbf I_n-\rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0. \] Using the Neumann expansion, \[ \mathbf W_n(\mathbf I_n-\rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0 = \sum_{l=0}^{\infty}\rho_0^l \mathbf W_n^{\,l+1}\mathbf X_n\boldsymbol\beta_0, \] so the conditional mean of $\mathbf W_n\mathbf y_n$ lies in the linear span generated by \[ \mathbf W_n\mathbf X_n,\ \mathbf W_n^2\mathbf X_n,\ \mathbf W_n^3\mathbf X_n,\ \ldots \] This is the sense in which spatial lags of $\mathbf X_n$ are natural instruments for the endogenous regressor $\mathbf W_n\mathbf y_n$.

6.2.2 Defining the S2SLS Estimator

With $\mathbf H_n$ in hand, S2SLS is the usual 2SLS estimator applied to the regression $\mathbf y_n$ on $\mathbf Z_n$, using $\mathbf H_n$ as instruments. The spatial content enters through the construction of $\mathbf H_n$ and through regularity conditions on $\mathbf W_n$ ensuring well-behaved spatial multipliers as $n\to\infty$.

We begin by introducing the assumptions concerning the structure of the error term. Specifically, we assume that the errors form triangular arrays and exhibit heteroskedasticity. Note that Kelejian and Prucha (1998) derived the asymptotic properties under the assumption of homoskedastic errors, while Kelejian and Prucha (2010) extended this framework to accommodate heteroskedasticity.

Assumption 1 (Heteroskedastic independent errors)

The innovations $\{\epsilon_{i,n}: 1\le i\le n,\ n\ge1\}$ satisfy $\mathbb E(\epsilon_{i,n})=0$, $\mathbb E(\epsilon_{i,n}^2)=\sigma^2_{i,n}$ with $0<\underline{a}^\sigma\le \sigma^2_{i,n}\le \overline{a}^\sigma<\infty$. Moreover, there exist $\eta>0$ and $C<\infty$ such that $\sup_{i,n}\mathbb E|\epsilon_{i,n}|^{4+\eta}\le C$. For each $n$, the random variables $\epsilon_{1,n},\ldots,\epsilon_{n,n}$ are independent across $i$.

Assumption 1 characterizes the error terms’ first two moments without imposing assumptions on their full distribution. It explicitly allows for heteroskedasticity, meaning the unobserved variables may have different variances across spatial units. Additionally, this assumption accommodates cases where the innovations depend on the sample size $n$ by allowing the innovations to form a triangular array. See Section 3.7 for the general definition and motivation.

For reference, we also present the assumption of homoskedastic errors as initially proposed by Kelejian and Prucha (1998):

Assumption 2 (Homoskedastic errors)

Following Kelejian and Prucha (1998), the innovations $\left\lbrace \epsilon_{i,n}, 1 \leq i \leq n, n\geq 1\right\rbrace$ are distributed identically. Further, the errors $\left\lbrace \epsilon_{i,n}, 1 \leq i \leq n\right\rbrace$ are for each $n$ distributed jointly independent with $\mathbb E(\epsilon_{i,n})= 0$ and $\mathbb E(\epsilon_{i,n}^2) = \sigma^2_{\epsilon}$, with $0 < \sigma^2_{\epsilon} < b <\infty$. Additionally the errors are assumed to possess fourth moments.

We now outline several key assumptions regarding the behavior of the spatial weight matrix, $\mathbf W_n$.

Assumption 3 (Zero diagonal of $\mathbf W_n$)

Following Kelejian and Prucha (1998), all diagonal elements of the spatial weighting matrix $\mathbf W_n$ are zero.

Assumption 3 (Diagonal elements of $\mathbf W_n$) is a normalization of the model and it also implies that no spatial unit is viewed as its own neighbor.

Assumption 4 (Nonsingularity)

The matrix $(\mathbf I_n-\rho_0\mathbf W_n)$ is nonsingular. Moreover, $\rho_0$ belongs to the admissible parameter space ensuring that the spatial multiplier $(\mathbf I_n-\rho_0\mathbf W_n)^{-1}$ is well-defined.

The nonsingularity condition in Assumption 4 allows us to express the reduced form of the true model as: \[ \mathbf y_n = (\mathbf I_n - \rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0 + (\mathbf I_n - \rho_0\mathbf W_n)^{-1}\boldsymbol\varepsilon_n. \]

This assumption ensures that the model is well-defined, enabling us to solve for $\mathbf y_n$. Additionally, Kelejian and Prucha (1998) note that the elements of $(\mathbf I_n - \rho_0\mathbf W_n)^{-1}$ depend on the sample size $n$, even if the elements of $\mathbf W_n$ do not vary with $n$. Therefore, the elements of $\mathbf y_n$ also depend on $n$, meaning that $\mathbf y_n$ forms a triangular array, even in cases where the errors $\epsilon_{i,n}$ are independent of $n$.

Furthermore, Assumption 1 (Heteroskedastic Errors) implies that the population variance-covariance matrix of $\mathbf y_n$ is \[ \operatorname{Var}(\mathbf y_n)=\boldsymbol\Omega_{\mathbf y_n} =(\mathbf I_n-\rho_0\mathbf W_n)^{-1}\boldsymbol\Sigma_n(\mathbf I_n-\rho_0\mathbf W_n^\top)^{-1}, \tag{6.11}\] where $\boldsymbol\Sigma_n=\operatorname{Diag}(\sigma^2_{1,n},\ldots,\sigma^2_{n,n})$.

Under the homoskedasticity assumption (Assumption 2), this variance-covariance matrix simplifies to: \[ \operatorname{Var}(\mathbf y_n) = \boldsymbol\Omega_{y_n} = \sigma^2_{\epsilon}(\mathbf I_n - \rho_0\mathbf W_n)^{-1}(\mathbf I_n - \rho_0\mathbf W^\top_n)^{-1}. \tag{6.12}\]

Assumption 5 (Uniform control of the spatial multiplier)

Following Kelejian and Prucha (1998), the row and column sums of the matrices $\mathbf W_n$ and $(\mathbf I_n - \rho_0\mathbf W_n)^{-1}$ are bounded uniformly in absolute value.

Assumption 5 is the spatial analogue of a regularity condition preventing dependence from exploding as $n\to\infty$. Under this assumption, products involving $\mathbf W_n$, $(\mathbf I_n-\rho_0\mathbf W_n)^{-1}$, and their transposes remain uniformly bounded in row and column sums. This is the key technical device used below to control the covariance matrices of terms such as $n^{-1}\mathbf H_n^\top\mathbf W_n\mathbf y_n$.

Remark

Recall from our Lemma 3.6 if $\mathbf A_n$ and $\mathbf B_n$ are conformable matrices whose row and column sums are bounded uniformly in absolute value, then the same is true for the product $\mathbf A_n\mathbf B_n$. Indeed, \[ \sum_j |(\mathbf A_n\mathbf B_n)_{ij}| = \sum_j \left|\sum_k a_{ik,n}b_{kj,n}\right| \le \sum_k |a_{ik,n}| \sum_j |b_{kj,n}|, \] so the row sums of $\mathbf A_n\mathbf B_n$ are uniformly bounded; the column sums are handled analogously. Equivalently, the maximum row-sum and maximum column-sum matrix norms are submultiplicative: $\|\mathbf A_n\mathbf B_n\|\leq \|\mathbf A_n\|\|\mathbf B_n\|$.

Remark

For $\mathbf W_n$, uniform bounds on row/column sums restrict the aggregate influence that neighbors can exert on any unit. For the spatial multiplier $(\mathbf I_n-\rho_0\mathbf W_n)^{-1}$, the same type of bound ensures that dependence propagated through the network does not explode with $n$.

Assumption 6 (Regressor rank and boundedness)

Following Kelejian and Prucha (1998), the regressor matrices $\mathbf X_n$ have full column rank (for $n$ large enough). Furthermore, the elements of the matrices $\mathbf X_n$ are uniformly bounded in absolute value.

This assumption ensures that the regressors are not perfectly collinear, which is essential for identifying the parameters of the model. Uniform boundedness further ensures the stability of the regression coefficients in large samples

We now introduce the assumptions concerning the instrument matrix, $\mathbf H_n$.

Assumption 7 (Instrument rank and relevance)

Following Kelejian and Prucha (1998), the instrument matrices $\mathbf H_n$ have full column rank $p \geq k + 1$ for all $n$ large enough. Furthermore, the elements of the matrices $\mathbf H_n$ are uniformly bounded in absolute value. They are composed of a subset of the linearly independent columns of $(\mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W^2_n\mathbf X_n, \ldots)$.

Assumption 8 (Limiting behavior of the instruments)

Following Kelejian and Prucha (1998), let $\mathbf H_n$ be a matrix of instruments, then:

$\lim_{n\to \infty} n^{-1}\mathbf H_n^\top\mathbf H_n = \mathbf Q_{HH}$ where $\mathbf Q_{HH}$ is finite and nonsingular.
$\operatorname{plim}_{n\to \infty} n^{-1}\mathbf H_n^\top\mathbf Z_n = \mathbf Q_{HZ}$ where $\mathbf Q_{HZ}$ is finite and has full column rank.

The Rank Condition of Assumption 7 establishes that there exists a least $p$ columns that are linearly independent such that $p \ge k + 1$. That is, the model can be just- or over-identified. Since $\mathbf H_n$ is constructed from $(\mathbf X_n,\mathbf W_n\mathbf X_n,\mathbf W_n^2\mathbf X_n,\ldots)$, it does not depend on $(\boldsymbol\beta_0,\rho_0)$. For simplicity (as in the standard large-sample analysis), we treat $\mathbf X_n$ and $\mathbf H_n$ as nonstochastic arrays, or equivalently we work conditionally on them. This simplifies the asymptotic properties of the 2SLS estimator. This contrast with the Best 2SLS estimator, which requires consistent estimate of the parameters. See Section 6.2.5.

Assumption 8 establishes conditions for identification of the model. The first condition in Assumption 8 ensures that the instrument matrix $\mathbf H_n$ remains well-conditioned as the sample size increases, preventing issues such as multicollinearity among the instruments. Importantly, since $\mathbf H_n$ contains spatially lagged explanatory variables, this condition also implies that implies that $\mathbf W_n\mathbf X_n$ and $\mathbf X_n$ cannot be linearly dependent. This condition can fail, for example, when $\mathbf W_n\mathbf X_n$ reproduces a column already contained in $\mathbf X_n$, as happens with a spatial lag of a constant under row normalization. In that case the limit matrix may lose rank.

The second condition in Assumption 8 (Limits of Instruments) ensures a nonzero correlation between the instruments and the explanatory variables. Specifically, it guarantees that the instruments are valid in the sense of being correlated with the endogenous components of the model.

Note that \[ \frac{1}{n}\mathbf H_n^\top\mathbf Z_n = \Big[\frac{1}{n}\mathbf H_n^\top\mathbf X_n,\ \frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbf y_n\Big]. \]

Thus, Assumption 8(b) requires in particular that the random block \[ \frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbf y_n \] has a finite probability limit. A sufficient route is to show that \[ \frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbf y_n - \frac{1}{n}\mathbf H_n^\top \mathbb E(\mathbf W_n\mathbf y_n) \xrightarrow{p} \mathbf 0, \] and then separately assume or verify that the deterministic sequence $\frac{1}{n}\mathbf H_n^\top \mathbb E(\mathbf W_n\mathbf y_n)$ converges to a finite limit. Under these two ingredients, \[ \operatorname{plim}_{n\to\infty} \frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbf y_n = \lim_{n\to \infty}\frac{1}{n}\mathbf H_n^\top\mathbb E(\mathbf W_n\mathbf y_n). \tag{6.13}\]

Proof. Let $\boldsymbol\psi_n \equiv \frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbf y_n$. We show first that $\boldsymbol\psi_n - \mathbb E(\boldsymbol\psi_n)\xrightarrow{p} \mathbf 0$. If, in addition, the deterministic sequence $\mathbb E(\boldsymbol\psi_n)$ converges to a finite limit, then $\boldsymbol\psi_n$ converges in probability to that limit, which yields Equation 6.13. Since $\mathbf y_n=(\mathbf I_n-\rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0+(\mathbf I_n-\rho_0\mathbf W_n)^{-1}\boldsymbol\varepsilon_n$, we have $\mathbb E(\mathbf y_n)= (\mathbf I_n-\rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0$. Therefore \[ \mathbb E(\boldsymbol\psi_n) = \frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbb E(\mathbf y_n) = \frac{1}{n}\mathbf H_n^\top\mathbf W_n(\mathbf I_n-\rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0. \]

Thus, provided the above deterministic sequence converges, the candidate probability limit is well defined. Next, \[ \operatorname{Var}(\boldsymbol\psi_n) = \operatorname{Var}\!\left(\frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbf y_n\right) = \frac{1}{n^2}\mathbf H_n^\top\mathbf W_n\operatorname{Var}(\mathbf y_n)\mathbf W_n^\top\mathbf H_n. \]

Using Equation 6.11, $\operatorname{Var}(\mathbf y_n)=\boldsymbol\Omega_{\mathbf y_n} = (\mathbf I_n-\rho_0\mathbf W_n)^{-1}\boldsymbol\Sigma_n(\mathbf I_n-\rho_0\mathbf W_n^\top)^{-1}$, hence \[ \operatorname{Var}(\boldsymbol\psi_n) = \frac{1}{n^2}\mathbf H_n^\top\mathbf W_n\boldsymbol\Omega_{\mathbf y_n}\mathbf W_n^\top\mathbf H_n = \frac{1}{n^2}\mathbf H_n^\top\mathbf D_n\mathbf H_n, \] where $\mathbf D_n\equiv\mathbf W_n\boldsymbol\Omega_{\mathbf y_n}\mathbf W_n^\top$. By Assumption 5, the row and column sums of $\mathbf W_n$, $(\mathbf I_n-\rho_0\mathbf W_n)^{-1}$, and their transposes are uniformly bounded in absolute value. Since $\boldsymbol\Sigma_n$ is diagonal with uniformly bounded diagonal entries by Assumption 1, it follows that the row and column sums of $\boldsymbol\Omega_{\mathbf y_n}$ are uniformly bounded, and therefore the same is true for $\mathbf D_n=\mathbf W_n\boldsymbol\Omega_{\mathbf y_n}\mathbf W_n^\top$ (see our Lemma 3.6). Thus there exists a finite constant $c_d$ such that \[ \max_{1\le j\le n}\sum_{i=1}^n |d_{ij,n}|\le c_d \qquad\text{and}\qquad \max_{1\le i\le n}\sum_{j=1}^n |d_{ij,n}|\le c_d. \]

By Assumption 6 and Assumption 7, the entries of $\mathbf H_n$ are uniformly bounded in absolute value; let $c_h$ be such that $|h_{ij,n}|\le c_h$ for all $i,j,n$. Let $\delta_{ij,n}$ denote the $(i,j)$-th entry of $\operatorname{Var}(\boldsymbol\psi_n)$. Then \[ \begin{aligned} \delta_{ij,n} &= \frac{1}{n^2}\sum_{r=1}^n\sum_{s=1}^n h_{si,n}d_{sr,n}h_{rj,n}, \\ |\delta_{ij,n}| &= \frac{1}{n^2}\left|\sum_{r=1}^n\sum_{s=1}^n h_{si,n}d_{sr,n}h_{rj,n}\right| \\ &\le \frac{1}{n^2}\sum_{r=1}^n\sum_{s=1}^n |h_{si,n}|\,|d_{sr,n}|\,|h_{rj,n}| \qquad \text{(triangle inequality)} \\ &\le \frac{c_h}{n^2}\sum_{r=1}^n\sum_{s=1}^n |d_{sr,n}|\,|h_{rj,n}| \qquad \text{since } |h_{si,n}|\le c_h \\ &= \frac{c_h}{n^2}\sum_{r=1}^n |h_{rj,n}|\sum_{s=1}^n |d_{sr,n}| \qquad \text{since } h_{rj,n}\text{ does not depend on } s \\ &\le \frac{c_h c_d}{n^2}\sum_{r=1}^n |h_{rj,n}| \qquad \text{since } \sum_{s=1}^n |d_{sr,n}|\le c_d \\ &\le \frac{c_h c_d}{n^2}\sum_{r=1}^n c_h \qquad \text{since } |h_{rj,n}|\le c_h \\ &= \frac{c_h^2 c_d}{n}. \end{aligned} \]

Hence each entry of $\operatorname{Var}(\boldsymbol\psi_n)$ is $O(n^{-1})$, and therefore $\operatorname{Var}(\boldsymbol\psi_n)\to \mathbf{0}$. Since $\mathbb E(\boldsymbol\psi_n-\mathbb E(\boldsymbol\psi_n))=\mathbf 0$, Chebyshev’s inequality implies \[ \boldsymbol\psi_n-\mathbb E(\boldsymbol\psi_n)\xrightarrow{p} \mathbf 0. \] Consequently, if the deterministic sequence $\mathbb E(\boldsymbol\psi_n)$ converges to a finite limit, then \[ \frac{1}{n}\mathbf H_n^\top\mathbf W_n\mathbf y_n \xrightarrow{p} \lim_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\mathbb E(\mathbf W_n\mathbf y_n), \] which proves Equation 6.13

Given all these assumptions we can define the S2SLS estimator as follows.

Definition 6.1 (Spatial Two-Stage Least Squares Estimator) Let $\mathbf H_n$ be the matrix $(n\times p)$ of instruments. Then the S2SLS is given by: \[ \widehat{\boldsymbol\delta}_{S2SLS} = \left(\widehat{\mathbf Z}_n^\top\mathbf Z_n\right)^{-1}\widehat{\mathbf Z}^\top_n\mathbf y_n, \tag{6.14}\] where: \[ \widehat{\mathbf Z}_n = \mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n = \mathbf P_{H, n}\mathbf Z_n, \tag{6.15}\] where the projection matrix $\mathbf P_{H, n}$ is defined as \[ \mathbf P_{H, n}= \mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n. \tag{6.16}\]

Alternatively, it can also be expressed as \[ \begin{aligned} \widehat{\boldsymbol\delta}_{S2SLS} & =\left[\mathbf Z_n^\top \mathbf H_n\left(\mathbf H_n^\top\mathbf H_n\right)^{-1}\mathbf H_n^\top\mathbf Z_n\right]^{-1}\mathbf Z_n^\top \mathbf H_n\left(\mathbf H_n^\top\mathbf H_n\right)^{-1}\mathbf H_n^\top\mathbf y_n, \\ & = \left[\mathbf Z_n^\top\mathbf P_{H,n}\mathbf Z_n\right]^{-1}\mathbf Z_n^\top\mathbf P_{H,n}\mathbf y_n. \end{aligned} \tag{6.17}\]

The S2SLS estimator in Equation 6.14 is conceptually similar to the standard 2SLS, with adjustment to account for the spatial structure of the model. The estimation proceeds as follows:

First stage: The first stage involves regressing the endogenous variables $\mathbf Z_n$ on the instruments $\mathbf H_n$ via OLS. This regression can be expressed as $\mathbf Z_n = \mathbf H_n\boldsymbol\theta + \boldsymbol\xi_n$, where $\widehat{\boldsymbol\theta}_n = (\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n$. The predicted values $\widehat{\mathbf Z}_n$ are then obtained using Equation 6.15 where $\mathbf P_{H,n}$ is the projection matrix. The matrix $\mathbf P_{H,n}$ is symmetric and idempotent. It leaves $\mathbf X_n$ unchanged when $\mathbf X_n$ is included in $\mathbf H_n$, and replaces $\mathbf W_n\mathbf y_n$ by its fitted value from the first stage: \[ \mathbf P_{H,n}\mathbf Z_n = [\mathbf X_n,\ \mathbf P_{H,n}\mathbf W_n\mathbf y_n] = [\mathbf X_n,\ \widehat{\mathbf W_n\mathbf y_n} ]. \]
Second stage: In the second stage, the regression of $\mathbf y_n$ on $\widehat{\mathbf Z}_n$ is used to estimate the parameters, yielding the S2SLS estimator as defined in Equation 6.14 or Equation 6.17.

6.2.3 Consistency of S2SLS Estimator

In this section, we provide a sketch of the proof for the consistency of the S2SLS estimator. For a formal proof see Kelejian and Prucha (1998) or Kelejian and Prucha (2010). To begin, recall that the $n\times n$ matrix $\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top$ is symmetric and idempotent, implying that $\widehat{\mathbf Z}_n^\top\mathbf Z_n= \widehat{\mathbf Z}_n^\top\widehat{\mathbf Z}_n$. Using this property, we express the S2SLS estimator in terms of the population error term as follows: \[ \begin{aligned} \widehat{\boldsymbol\delta}_n & = \boldsymbol\delta_0 + \left(\widehat{\mathbf Z}^\top_n\widehat{\mathbf Z}_n\right)^{-1}\widehat{\mathbf Z}^\top_n\boldsymbol\varepsilon_n, \\ & = \boldsymbol\delta_0 + \left[\left(\mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n\right)^\top\left(\mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n\right)\right]^{-1}\left(\mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n\right)^\top\boldsymbol\varepsilon_n, \\ & = \boldsymbol\delta_0 + \left[\mathbf Z^\top_n \mathbf H_n (\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n\right]^{-1}\mathbf Z^\top_n\mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\boldsymbol\varepsilon_n. \end{aligned} \tag{6.18}\]

Solving for $\widehat{\boldsymbol\delta}_{n} - \boldsymbol\delta_0$, we obtain: \[ \begin{aligned} (\widehat{\boldsymbol\delta}_{n} - \boldsymbol\delta_0) & = \left[\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z _n\right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n\right)\right]^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n\right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H^\top_n\boldsymbol\varepsilon_n\right), \\ & = \widetilde{\mathbf P}^\top_n\left(\frac{1}{n}\mathbf H^\top_n\boldsymbol\varepsilon_n\right), \end{aligned} \tag{6.19}\] where: \[ \widetilde{\mathbf P}_n = \left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H_n^\top\mathbf Z_n \right)\left[\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n\right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n\right)\right]^{-1}. \tag{6.20}\]

From Assumption 8 (Limitsof Instruments), we know that: \[ \begin{aligned} \lim_{n\to \infty} n^{-1}\mathbf H_n^\top\mathbf H_n &=& \mathbf Q_{HH},\\ \operatorname{plim}_{n\to \infty} n^{-1}\mathbf H_n^\top\mathbf Z_n &=& \mathbf Q_{HZ}. \end{aligned} \]

Therefore, $\widetilde{\mathbf P}_n\xrightarrow{p} \mathbf P_0$, where $\mathbf P_0=\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\left(\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\right)^{-1}$ is a finite matrix and exists because $\mathbf Q_{HH}$ is invertible by Assumption 8 and $(\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ})^{-1}$ exists because $\mathbf Q_{HZ}$ has full column rank by Assumption 8. Thus, \[ \widetilde{\mathbf P}_n - \mathbf P_0 = o_p(1)\implies \widetilde{\mathbf P}_n= \mathbf P_0 + o_p(1), \tag{6.21}\]

Since its dimension is fixed and all of its entries are finite, $\mathbf P_0=O(1)$.

By Assumption 6 and Assumption 7, the entries of $\mathbf H_n$ are uniformly bounded in absolute value. Assumption 1 implies that $\{\epsilon_{i,n}:1\le i\le n,\ n\ge 1\}$ forms an independent triangular array; under heteroskedasticity the variables need not be identically distributed. Moreover, $\mathbb E(\boldsymbol\varepsilon_n)=\mathbf 0$, $\operatorname{Var}(\boldsymbol\varepsilon_n)=\boldsymbol\Sigma_n=\operatorname{Diag}(\sigma^2_{1,n},\ldots,\sigma^2_{n,n})$. Therefore, \[ \mathbb E\left(\frac{1}{n}\mathbf H_n^\top\boldsymbol\varepsilon_n\right)=\mathbf 0, \qquad \operatorname{Var}\left(\frac{1}{n}\mathbf H_n^\top\boldsymbol\varepsilon_n\right)=\frac{1}{n^2}\mathbf H_n^\top\boldsymbol\Sigma_n\mathbf H_n. \]

Write \[ \frac{1}{n^2}\mathbf H_n^\top\boldsymbol\Sigma_n\mathbf H_n = \frac{1}{n}\left(\frac{1}{n}\mathbf H_n^\top\boldsymbol\Sigma_n\mathbf H_n\right). \]

The matrix $n^{-1}\mathbf H_n^\top\boldsymbol\Sigma_n\mathbf H_n$ has entries uniformly bounded in absolute value by the same row- and column-sum argument used above. Hence \[ \operatorname{Var}\left(\frac{1}{n}\mathbf H_n^\top\boldsymbol\varepsilon_n\right)=O\!\left(\frac{1}{n}\right)\to \mathbf{0}. \]

By Chebyshev’s inequality, \[ \frac{1}{n}\mathbf H_n^\top\boldsymbol\varepsilon_n\xrightarrow{p} \mathbf 0. \] Combining this with Equation 6.19 and Equation 6.21 yields \[ \widehat{\boldsymbol\delta}_n\xrightarrow{p} \boldsymbol\delta_0. \]

6.2.4 Asymptotic Distribution of S2SLS Estimator

Multiplying Equation 6.19 by $\sqrt{n}$ yields \[ \sqrt{n}(\widehat{\boldsymbol\delta}_{n}-\boldsymbol\delta_0) = \widetilde{\mathbf P}_n^\top\left(\frac{1}{\sqrt{n}}\mathbf H_n^\top\boldsymbol\varepsilon_n\right), \tag{6.22}\] where $\widetilde{\mathbf P}_n$ is defined in Equation 6.20.

By Assumption 8, $\widetilde{\mathbf P}_n \xrightarrow{p} \mathbf P_0$, with \[ \mathbf P_0 = \mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\left(\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\right)^{-1}, \qquad \mathbf Q_{HH}=\lim_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\mathbf H_n, \qquad \mathbf Q_{HZ}=\operatorname{plim}_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\mathbf Z_n. \]

To characterize the second factor in Equation 6.22, write \[ \frac{1}{\sqrt{n}}\mathbf H_n^\top\boldsymbol\varepsilon_n = \frac{1}{\sqrt{n}}\sum_{i=1}^n \mathbf h_{i,n}\epsilon_{i,n}, \] where $\mathbf h_{i,n}^\top$ denotes the $i$-th row of $\mathbf H_n$. Define \[ \mathbf x_{i,n}\equiv \frac{1}{\sqrt{n}}\mathbf h_{i,n}\epsilon_{i,n}. \]

Then $\{\mathbf x_{i,n}:1\le i\le n\}$ is an independent triangular array with mean zero. Its covariance matrix is \[ \sum_{i=1}^n \operatorname{Var}(\mathbf x_{i,n}) = \frac{1}{n}\sum_{i=1}^n \mathbf h_{i,n}\mathbf h_{i,n}^\top \sigma_{i,n}^2 = \frac{1}{n}\mathbf H_n^\top\boldsymbol\Sigma_n\mathbf H_n. \]

Assume that \[ \boldsymbol\Delta_0\equiv\lim_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\boldsymbol\Sigma_n\mathbf H_n \] exists and is finite. Since the entries of $\mathbf h_{i,n}$ are uniformly bounded and $\sup_{i,n}\mathbb E|\epsilon_{i,n}|^{4+\eta}<\infty$ by Assumption 1, a Lyapunov condition holds for the triangular array $\{\mathbf x_{i,n}\}$. Therefore a multivariate CLT for independent triangular arrays yields \[ \frac{1}{\sqrt{n}}\mathbf H_n^\top\boldsymbol\varepsilon_n \xrightarrow{d} \mathcal N\!\left(\mathbf 0,\boldsymbol\Delta_0\right). \tag{6.23}\]

Combining $\widetilde{\mathbf P}_n=\mathbf P_0+o_p(1)$ with Equation 6.23 and applying Slutsky’s theorem to Equation 6.22 yields \[ \sqrt{n}(\widehat{\boldsymbol\delta}_{n}-\boldsymbol\delta_0) \xrightarrow{d} \mathcal N\!\left(\mathbf 0,\boldsymbol\Omega_0\right), \qquad \boldsymbol\Omega_0=\mathbf P_0^\top\boldsymbol\Delta_0\mathbf P_0. \]

Equivalently, \[ \boldsymbol\Omega_0 = \left(\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\right)^{-1} \mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\boldsymbol\Delta_0\mathbf Q_{HH}^{-1}\mathbf Q_{HZ} \left(\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\right)^{-1}. \tag{6.24}\]

If Assumption 1 is replaced by Assumption 2 with $\boldsymbol\Sigma_n=\sigma_\epsilon^2\mathbf I_n$, then $\boldsymbol\Delta_0=\sigma_\epsilon^2\mathbf Q_{HH}$ and \[ \sqrt{n}(\widehat{\boldsymbol\delta}_{n}-\boldsymbol\delta_0)\xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Omega_0^o), \qquad \boldsymbol\Omega_0^o=\sigma_\epsilon^2\left(\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\right)^{-1}. \]

Theorem 6.1 (Spatial 2SLS Estimator for SLM) Suppose Assumption 3, Assumption 4, Assumption 5, Assumption 6, Assumption 7, Assumption 8 hold, and either:

Assumption 1 holds and $\boldsymbol\Delta_0=\lim_{n\to\infty} n^{-1}\mathbf H_n^\top\boldsymbol\Sigma_n\mathbf H_n$ exists, or
Assumption 2 holds.

Then the S2SLS estimator \[ \widehat{\boldsymbol\delta}_n = \left(\mathbf Z_n^\top\mathbf P_{H,n}\mathbf Z_n\right)^{-1}\mathbf Z_n^\top\mathbf P_{H,n}\mathbf y_n, \qquad \mathbf P_{H,n}=\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top, \] is consistent and satisfies \[ \sqrt{n}(\widehat{\boldsymbol\delta}_{n}-\boldsymbol\delta_0)\xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Omega_0), \] with $\boldsymbol\Omega_0$ given in Equation 6.24; under homoskedasticity, the limit variance reduces to \[ \boldsymbol\Omega_0^o=\sigma_\epsilon^2(\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ})^{-1}. \tag{6.25}\]

The heteroskedasticity-robust (IV/White) estimator of $\boldsymbol\Omega_0$ is obtained by replacing population quantities in Equation 6.24 by sample analogs and using \[ \widehat{\boldsymbol\Delta}_n=\frac{1}{n}\sum_{i=1}^n \widehat{\epsilon}_{i,n}^{\,2}\,\mathbf h_{i,n}\mathbf h_{i,n}^\top, \qquad \widehat{\boldsymbol\varepsilon}_n=\mathbf y_n-\mathbf Z_n\widehat{\boldsymbol\delta}_n. \tag{6.26}\] Under homoskedasticity, $\sigma_\epsilon^2$ can be estimated by $\widehat{\sigma}_n^2=n^{-1}\widehat{\boldsymbol\varepsilon}_n^\top\widehat{\boldsymbol\varepsilon}_n$.

6.2.5 Best S2SLS Estimator

Lee (2003) suggested the so-called optimal instruments matrix, which gives rise to the Best Spatial Two Stage Least Square (BS2SLS) Estimator. Instead of using the IV matrices that are composed of a subset of the linearly independent columns of $(\mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W^2_n\mathbf X_n, \ldots)$, Lee (2003) suggests the instrument matrix: \[ \widetilde{\mathbf H}^{*}_n =\left[\mathbf X_n, \mathbf W_n(\mathbf I_n - \widetilde{\rho}_n\mathbf W_n)^{-1}\mathbf X_n\widetilde{\boldsymbol\beta}_n\right], \] which requires the use of consistent first-stage estimates for $\rho_0$ and $\boldsymbol\beta_0$.² Lee (2003) shows that these instruments are asymptotically optimal in the sense that they provide the smallest asymptotic variance among all the IV estimators of the SLM.³

The resulting S2SLS estimator is called the Best Spatial Two Stage Least Squares (BS2SLS) estimator. However, since $\widetilde{\mathbf H}^{*}_n$ is an $n \times (k + 1)$ matrix, the model is just-identified and the B2SLS estimator is an IV estimator given by \[ \widehat{\boldsymbol\delta}_n = \left[\widetilde{\mathbf H}^{*\top}_n\mathbf Z_n\right]^{-1}\widetilde{\mathbf H}^{*\top}_n\mathbf y_n. \tag{6.27}\]

We summarize Lee (2003)’s assumptions as follows:

Assumption 9 (Assumptions for the BS2SLSE of the SLM)

Following Lee (2003), assume the following:

The errors $\left\lbrace \epsilon_{i,n}, 1 \leq i \leq n, n\geq 1\right\rbrace$ are distributed identically. Further, the errors $\left\lbrace \epsilon_{i,n}, 1 \leq i \leq n\right\rbrace$ are for each $n$ distributed jointly independent with $\mathbb E(\epsilon_{i,n})= 0$ and $\mathbb E(\epsilon_{i,n}^2) = \sigma^2_{\epsilon}$, with $0 < \sigma^2_{\epsilon} < b <\infty$. Additionally the errors are assumed to possess fourth moments.
The matrices $(\mathbf I_n - \rho_0\mathbf W_n)$ are nonsingular.
The row and column sums of the matrices $\mathbf W_n$ and $(\mathbf I_n - \rho_0\mathbf W_n)^{-1}$ are uniformly bounded in absolute value.
The elements of the matrices $\mathbf X_n$ are uniformly bounded in absolute value.
Let $\mathbf G_{0,n} = \mathbf W_n(\mathbf I_n - \rho_0\mathbf W_n)^{-1}$. The limit $\mathbf J^* = \lim_{n \to \infty}(1/n)\mathbf H_n^{*\top}\mathbf H_n^{*}$ exists and is nonsingular, where \[ \mathbf H_n^* = \left[\mathbf X_n, \mathbf W_n(\mathbf I_n - \rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0\right] = \left[\mathbf X_n, \mathbf G_0\mathbf X_n\boldsymbol\beta_0\right]. \]
$\widetilde{\rho}_n\xrightarrow{p} \rho_0$ and $\widetilde{\boldsymbol\beta}_n \xrightarrow{p} \boldsymbol\beta_0$.⁴

Most of the assumptions in Assumption 9 are standard in spatial settings. However, unlike the S2SLSE from previous sections, the BS2SLSE must account for the stochastic nature of $\widetilde{\mathbf H}^{*}_n$, which depends nonlinearly on the initial consistent estimate $\widetilde{\rho}_n$. In particular, the uniform boundedness properties in (c) are imposed only at the true value of $\rho_0$, i.e., a single point in the parameter space. To ensure validity, we require uniform boundedness to hold for all parameter values in a neighborhood of $\rho_0$. Fortunately, Lee (2003) [pag. 313]’s Lemma shows that condition (c) is sufficient to guarantee that the uniform boundedness property extends to a small neighborhood of the true parameter value.

Lemma 6.1 (Lemma) Suppose that $\{\lVert \mathbf W_n\rVert\}$ and $\{\lVert \left(\mathbf I_n - \rho_0\mathbf W_n\right)^{-1}\rVert\}$, where $\lVert \cdot \rVert$ is a matrix norm, are bounded. Then, $\{\lVert \left(\mathbf I_n - \eta\mathbf W_n\right)^{-1}\rVert\}$ are uniformly bounded in $\eta$ in a neighborhood of $\rho_0$.

We next prove Lemma 6.1, closely following Lee (2003). The proof is worth presenting in full because it illustrates a technique that will appear repeatedly later in the chapter, especially when establishing uniform boundedness of matrices over neighborhoods of the true parameter.

Proof. Recall $\mathbf S_n(\rho)\equiv \mathbf I_n-\rho\mathbf W_n$, and fix $\rho_0$ such that $\mathbf S_n(\rho_0)$ is nonsingular for all $n$. For any scalar $\eta$, let $\Delta\equiv \eta-\rho_0$. Then \[ \mathbf S_n(\eta) = \mathbf I_n-\eta\mathbf W_n = \mathbf I_n-(\rho_0+\Delta)\mathbf W_n = \mathbf S_n(\rho_0)-\Delta \mathbf W_n. \tag{6.28}\]

Since $\mathbf W_n=\mathbf W_n\mathbf S_n(\rho_0)^{-1}\mathbf S_n(\rho_0)$, Equation 6.28 can be rewritten as \[ \begin{aligned} \mathbf S_n(\eta) &= \mathbf S_n(\rho_0)-\Delta\mathbf W_n, \nonumber\\ &= \mathbf S_n(\rho_0)-\Delta\bigl(\mathbf W_n\mathbf S_n(\rho_0)^{-1}\bigr)\mathbf S_n(\rho_0), \nonumber\\ &= \bigl(\mathbf I_n-\Delta \mathbf G_{0n}\bigr)\mathbf S_n(\rho_0), \end{aligned} \tag{6.29}\] where $\mathbf G_{0n}\equiv \mathbf W_n\mathbf S_n(\rho_0)^{-1}$.

Next, bound $\|\mathbf G_{0n}\|$. By submultiplicativity of the matrix norm, \[ \|\mathbf G_{0n}\| = \|\mathbf W_n\mathbf S_n(\rho_0)^{-1}\| \le \|\mathbf W_n\|\,\|\mathbf S_n(\rho_0)^{-1}\|. \]

By assumption, there exist constants $c_a<\infty$ and $c_b<\infty$ such that $\|\mathbf W_n\|\le c_a$, $\|\mathbf S_n(\rho_0)^{-1}\|\le c_b$, $\forall n$. Hence \[ \|\mathbf G_{0n}\|\le c_a c_b \equiv c, \qquad \forall n. \tag{6.30}\]

If $c=0$, then $\mathbf W_n=\mathbf 0$ for all $n$, so $\mathbf S_n(\eta)=\mathbf I_n$ for all $\eta$ and the result is immediate. Thus we may assume $c>0$.

Choose $\delta>0$ such that $\delta c<1$, and define the neighborhood $\mathcal B\equiv \{\eta:\ |\eta-\rho_0|<\delta\}$. Then for any $\eta\in\mathcal B$, $|\Delta|=|\eta-\rho_0|<\delta$, and therefore, using Equation 6.30, \[ \|\Delta \mathbf G_{0n}\| \le |\Delta|\,\|\mathbf G_{0n}\| \le |\Delta|\,c < \delta c < 1, \qquad \forall n. \]

Hence, for every $\eta\in\mathcal B$, the matrix $\mathbf I_n-\Delta\mathbf G_{0n}$ is invertible, and by the Neumann series, \[ (\mathbf I_n-\Delta\mathbf G_{0n})^{-1} = \sum_{k=0}^\infty (\Delta\mathbf G_{0n})^k. \tag{6.31}\]

Taking norms and using the triangle inequality and submultiplicativity, \[ \begin{aligned} \|(\mathbf I_n-\Delta\mathbf G_{0n})^{-1}\| &\le \sum_{k=0}^\infty \|(\Delta\mathbf G_{0n})^k\|, \nonumber\\ &\le \sum_{k=0}^\infty |\Delta|^k \|\mathbf G_{0n}\|^k, \nonumber\\ &\le \sum_{k=0}^\infty (|\Delta|c)^k \nonumber,\\ &\le \sum_{k=0}^\infty (\delta c)^k, = \frac{1}{1-\delta c}. \end{aligned} \tag{6.32}\]

The bound on the right-hand side does not depend on $n$ or $\eta$. Now, since $\mathbf S_n(\rho_0)$ is invertible and $\mathbf I_n-\Delta\mathbf G_{0n}$ is invertible, Equation 6.29 implies that $\mathbf S_n(\eta)$ is invertible, with $\mathbf S_n(\eta)^{-1} = \mathbf S_n(\rho_0)^{-1}(\mathbf I_n-\Delta\mathbf G_{0n})^{-1}$. Therefore, \[ \begin{aligned} \sup_{\eta\in\mathcal B}\|\mathbf S_n(\eta)^{-1}\| &\le \|\mathbf S_n(\rho_0)^{-1}\| \sup_{\eta\in\mathcal B}\|(\mathbf I_n-(\eta-\rho_0)\mathbf G_{0n})^{-1}\| \nonumber\\ &\le c_b\cdot \frac{1}{1-\delta c}, \end{aligned} \tag{6.33}\] uniformly in $n$. Hence, \[ \sup_n\sup_{\eta\in\mathcal B} \|(\mathbf I_n-\eta\mathbf W_n)^{-1}\| < \infty, \] which proves the result.

The following theorem provides the asymptotic distribution of the BS2SLSE for the SLM model.

Theorem 6.2 (Asymptotic Distribution of BS2SLSE for SLM) Under Assumption 9, the BS2SLS estimator defined in Equation 6.27 is consistent and \[ \sqrt{n}\left(\widehat{\boldsymbol\delta}_n - \boldsymbol\delta_0\right) \xrightarrow{d} \mathcal N(\mathbf 0, \boldsymbol\Psi_0^o), \] where \[ \boldsymbol\Psi_0^o = \sigma_{\epsilon}^2\left[\lim_{n\to\infty} \frac{1}{n}\mathbf H^{*\top}_n \mathbf H_n^*\right]^{-1} = \sigma_{\epsilon}^2\mathbf J^{*-1}. \tag{6.34}\]

The limiting variance $\boldsymbol\Psi_0^o$ in Equation 6.34 can be compared with the limiting distribution in Equation 6.25, which can be written as \[ \boldsymbol\Omega_0^o =\sigma_\epsilon^2\left[\operatorname{plim}_{n\to\infty}\frac{1}{n}\mathbf Z_n^\top \mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top \mathbf Z_n\right]^{-1} =\sigma_\epsilon^2\left[\operatorname{plim}_{n\to\infty}\frac{1}{n}\mathbf Z_n^\top \mathbf P_{\mathbf H_n}\mathbf Z_n\right]^{-1}. \]

From the argument leading to Equation 6.13, we have: \[ \operatorname{plim}_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\mathbf Z_n =\lim_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\mathbb E(\mathbf Z_n) =\lim_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\mathbf H_n^*. \]

By the generalized Schwarz inequality⁵, \[ \mathbf H_n^{*\top}\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top\mathbf H_n^{*}\ \le\ \mathbf H_n^{*\top}\mathbf H_n^*. \] After dividing by $n$ and taking limits, this implies \[ \operatorname{plim}_{n\to\infty}\frac{1}{n}\mathbf Z_n^\top \mathbf P_{\mathbf H_n}\mathbf Z_n \ \le\ \lim_{n\to\infty}\frac{1}{n}\mathbf H_n^{*\top}\mathbf H_n^*, \] and therefore (by inversion of positive definite matrices) \[ \boldsymbol\Omega_0^o\ \ge\ \sigma_\epsilon^2\left[\lim_{n\to\infty}\frac{1}{n}\mathbf H_n^{*\top}\mathbf H_n^*\right]^{-1} =\boldsymbol\Psi_0^o. \]

This inequality states that the variance matrix of the S2SLS estimator using $\mathbf H_n$ as instruments is at least as large as the variance matrix of an estimator that directly uses the optimal instruments $\mathbf H_n^*$.

A proof for Theorem 6.2 is given in Appendix Section 6.9.1

6.3 GMM Estimator of SLM

6.3.1 GMM Estimator Under Homoskedasticity

In the previous sections, we analyzed the S2SLS and BS2SLS estimators for the Spatial Lag Model (SLM). Both estimators belong to the broader class of Generalized Method of Moments (GMM) estimators. This section reviews the GMM estimator for the SLM, as proposed by Lee (2007), and derives the GMM and Optimal GMM (OGMM) estimators for the model. This GMM estimator can be considered as an one-step GMM estimator (see Section 6.1.3).

To construct the moment conditions, Lee (2007) uses linear and quadratic moments. Let $\mathbf H_n$ denote an $n \times k_x$ matrix of instrumental variables (IVs) constructed as a function of $\mathbf X_n$ and $\mathbf W_n$. This matrix can be constructed using the instruments proposed by Kelejian and Prucha (1998), such that \[ \mathbf H_n = \left[\mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W_n^2\mathbf X_n, \cdots\right], \] or using the asymptotically optimal instruments proposed by Lee (2003): \[ \mathbf H_n = \left[\mathbf X_n, \mathbf W_n(\mathbf I_n - \widetilde{\rho}_n\mathbf W_n)^{-1}\mathbf X_n\widetilde{\boldsymbol\beta}_n\right], \] where $\widetilde{\rho}_n$ and $\widetilde{\boldsymbol\beta}_n$ are consistent estimates of $\rho_0$ and $\boldsymbol\beta_0$.

Recall the error term for the SLM is defined as \[ \boldsymbol\varepsilon_n(\boldsymbol\theta) = \mathbf y_n - \mathbf Z_n\boldsymbol\delta_n = (\mathbf I_n - \rho \mathbf W_n)\mathbf y_n - \mathbf X_n\boldsymbol\beta. \]

Unlike the S2SLSE, the GMME uses additional quadratic moments. Because of this, the optimally weighted GMM estimator can be asymptotically efficient relative to the S2SLSE. For the quadratic population moments, Lee (2007) proposes: \[ \mathbb E(n^{-1}\boldsymbol\varepsilon_n^\top \mathbf P_{jn}\boldsymbol\varepsilon_n) = 0, \tag{6.35}\] where $\mathbf P_{jn}$ are $n\times n$ matrix of constant having zero trace. Lee (2007) refers to this class of matrices as $\mathcal P_{1n} = \{\mathbf P: \text{$\mathbf P$ is $n\times n$ matrix, $\operatorname{tr}(\mathbf P) = 0$}\}$. A subclass $\mathcal P_{2n}$ arises for $n\times n$ matrices having zero diagonal: $\mathcal P_{2n} = \{\mathbf P: \text{$\mathbf P$ is $n\times n$ matrix, $\operatorname{diag}(\mathbf P) = \mathbf 0$}\}$.

To understand the role played by $\mathbf P_{jn}$, note for any nonstochastic matrix $\mathbf A_{n}$, the quadratic moments are given by: \[ \begin{aligned} \mathbb E\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)& = \operatorname{tr}\left[\mathbb E\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)\right], \\ & = \mathbb E\left[\operatorname{tr}\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)\right], \\ & = \mathbb E\left[\operatorname{tr}\left(\mathbf A_n\boldsymbol\varepsilon_n\boldsymbol\varepsilon_n^\top\right)\right], \\ & = \operatorname{tr}\left[\mathbb E\left(\mathbf A_n\boldsymbol\varepsilon_n\boldsymbol\varepsilon_n^\top\right)\right], \\ & = \sigma_0^2\operatorname{tr}\left(\mathbf A_n\right). \end{aligned} \]

Thus $\mathbb E\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)\neq 0$, unless $\operatorname{tr}(\mathbf A_n) = 0$. If $\operatorname{tr}(\mathbf A_n) \neq 0$, we can always create for example $\mathbf P_n = \mathbf A_n - \operatorname{Diag}(\mathbf P_n)$ such that $\operatorname{tr}(\mathbf P_n) = 0$.

It is important to stress the intuitions of the instruments. The regressor $\mathbf W_n\mathbf y_n$ is endogenous, as \[ \mathbb E\left[\left(\mathbf W_n\mathbf y_n\right)^\top \boldsymbol\varepsilon_n\right] = \sigma_0^2\operatorname{tr}\left(\mathbf G_n(\rho_0)\right) \neq 0, \] where $\mathbf G_n(\rho_0) = \mathbf W_n\mathbf S_0^{-1}$, i.e., the elements of $\mathbf W_n\mathbf y_n$ are correlated with the elements of $\boldsymbol\varepsilon_n$. This can be observed from the expression $\mathbf W_n\mathbf y_n = \mathbf G_n(\rho_0)\mathbf X_n\boldsymbol\beta_0 + \mathbf G_n(\rho_0)\boldsymbol\varepsilon_n$, which follows from the reduced form $\mathbf y_n = \mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta + \mathbf S_0^{-1}\boldsymbol\varepsilon_n$. The linear instruments $\mathbf H_n$ is correlated with $\mathbf G_n(\rho_0)\mathbf X_n\boldsymbol\beta_0$ (since $\mathbf H_n$ are mean of $\mathbf W_n\mathbf y_n$), but uncorrelated with $\boldsymbol\varepsilon_n$, because $\mathbb E(\mathbf H_n^\top\boldsymbol\varepsilon_n) = \mathbf H_n^\top\mathbb E(\boldsymbol\varepsilon_n) = \mathbf 0$. Note that we require $\mathbf P_{jn}$ to be correlated with $\mathbf G_0$ in the sense that $\operatorname{tr}(\mathbf P_{jn}\mathbf G_0)\neq 0$. As long as $\operatorname{tr}(\mathbf P_{jn}) = 0$, $\mathbf P_{jn}\boldsymbol\varepsilon_n$ is uncorrelated with $\boldsymbol\varepsilon_n$, and thus it may be used as an instrument of $\mathbf W_n\mathbf y_n$.

The quadratic moments in Equation 6.35 can be derived under homoskedasticity, where $\mathbb E(\epsilon_{in}^2) = \sigma_0^2$ (Lee 2001). For example, we can use the following moment conditions provided by Kelejian and Prucha (1999): \[ \begin{aligned} \mathbb E\left[\frac{1}{n}\boldsymbol\varepsilon_n^\top \boldsymbol\varepsilon_n\right] & = \sigma^2_0, \\ \mathbb E\left[\frac{1}{n}\boldsymbol\varepsilon_n^\top\mathbf W_n^\top\mathbf W_n\boldsymbol\varepsilon_n\right] & = \sigma^2_0\frac{1}{n}\operatorname{tr}\left(\mathbf W_n^\top\mathbf W_n\right), \\ \mathbb E\left[\frac{1}{n}\boldsymbol\varepsilon_n^\top \mathbf W_n \boldsymbol\varepsilon_n\right] & = 0, \end{aligned} \tag{6.36}\]

These three moment conditions can be reduced to two. Substituting out $\sigma^2_0$ into the second moment equation yields: \[ \begin{aligned} \mathbb E\left[\frac{1}{n}\boldsymbol\varepsilon^\top_n\mathbf W_n\mathbf W_n\boldsymbol\varepsilon_n\right] - \mathbb E\left[\frac{1}{n}\boldsymbol\varepsilon^\top_n \boldsymbol\varepsilon_n\right] \frac{1}{n}\operatorname{tr}\left(\mathbf W^\top_n\mathbf W_n\right) & = 0, \\ \frac{1}{n}\mathbb E\left[\boldsymbol\varepsilon^\top_n\mathbf W_n\mathbf W_n\boldsymbol\varepsilon_n - \boldsymbol\varepsilon^\top_n \boldsymbol\varepsilon_n\frac{1}{n}\operatorname{tr}\left(\mathbf W^\top_n\mathbf W_n\right)\right] & = 0, \\ \frac{1}{n}\mathbb E\left[\boldsymbol\varepsilon^\top_n\mathbf W_n\mathbf W_n\boldsymbol\varepsilon_n - \boldsymbol\varepsilon^\top_n \frac{1}{n}\operatorname{tr}\left(\mathbf W^\top_n\mathbf W_n\right)\boldsymbol\varepsilon_n\right] & = 0,\\ \frac{1}{n}\mathbb E\left[\boldsymbol\varepsilon^\top_n\left(\mathbf W_n\mathbf W_n - \frac{1}{n}\operatorname{tr}\left(\mathbf W^\top_n\mathbf W_n\right)\mathbf I_n\right)\boldsymbol\varepsilon_n\right] & = 0, \\ \frac{1}{n}\mathbb E\left[\boldsymbol\varepsilon^\top_n\mathbf P_{2n}\boldsymbol\varepsilon\right] & = 0, \end{aligned} \] where $\mathbf P_{2n}$ is symmetric with $\operatorname{tr}(\mathbf P_{2n}) = 0$, but its diagonal elements are non zero (In the heteroskedasticity case it is!). Based on these results, Lee (2007) proposes: \[ \begin{aligned} \mathbf P_{1n} & = \mathbf W_n, \\ \mathbf P_{2n} & = \mathbf W^2_n - \left(\operatorname{tr}(\mathbf W^2_n)/n\right)\mathbf I_n. \end{aligned} \]

The vector of moments functions is then defined as: \[ \underset{(2 + k_x)\times 1}{\mathbf g_n(\boldsymbol\theta)} = \begin{pmatrix} \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \\ \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \\ \mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \end{pmatrix}, \tag{6.37}\] which has dimension $(2 + k_k) \times 1$. It is important to note that the moment conditions at the population hold. At $\boldsymbol\theta_0$: \[ \begin{aligned} \mathbb E(\mathbf H_n^\top\boldsymbol\varepsilon_n) & = \mathbf H_n^\top\mathbb E(\boldsymbol\varepsilon_n) = \mathbf 0, \\ \mathbb E(\boldsymbol\varepsilon_n^\top \mathbf P_{jn}\boldsymbol\varepsilon_n) & = \sigma_0^2\operatorname{tr}(\mathbf P_{jn}) = \mathbf 0\;\mbox{for $j = 1,2$} \end{aligned} \]

Let $\boldsymbol\Upsilon_n$ be some $(2 + k_k) \times (2 + k_k)$ symmetric positive semidefinite weighting matrix, then the corresponding GMM estimator is defined as: \[ \widehat{\boldsymbol\theta}_{n} = \underset{\boldsymbol\theta \in \boldsymbol\Theta}{\operatorname*{argmin}}\;\;Q_n(\boldsymbol\theta) = \mathbf g_n(\boldsymbol\theta)^\top\boldsymbol\Upsilon_n\mathbf g_n(\boldsymbol\theta), \tag{6.38}\] where $\boldsymbol\theta = (\rho , \boldsymbol\beta^\top)^\top$ is an $(1 + k)$-dimensional vector, and where $\mathbf g_n(\boldsymbol\theta)$ is the $(2 + k_x)\times 1$ vector of moments.

Since $\mathbb E(\mathbf g_n) = \mathbf 0$, the variance-covariance matrix of the population moment functions are given by: \[ \begin{aligned} \boldsymbol\Omega_n & = \operatorname{Var}(\mathbf g_n) \\ & = \mathbb E(\mathbf g_n\mathbf g_n^\top)\\ & = \mathbb E\begin{pmatrix} \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \left(\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top & \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \left(\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top & \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \boldsymbol\varepsilon_{n}(\boldsymbol\theta)^\top\mathbf H_n \\ \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \left(\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top & \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \left(\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top & \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \boldsymbol\varepsilon_{n}(\boldsymbol\theta)^\top\mathbf H_n \\ \mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \left(\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top & \mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \left(\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top & \mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \boldsymbol\varepsilon_{n}(\boldsymbol\theta)^\top\mathbf H_n \end{pmatrix} \end{aligned} \]

To find these moments, we use Lemma 3.13. Thus, using the fact that $\operatorname{tr}(\mathbf P_{jn}) = 0$, for $j = 1, 2$: \[ \begin{aligned} \mathbb E\left[\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right] & = \mathbf H_n^\top\operatorname{diag}(\mathbf P_{jn})\mu_3, \\ \mathbb E\left[\boldsymbol\varepsilon(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta) \cdot \boldsymbol\varepsilon(\boldsymbol\theta)^\top\mathbf P_{ln}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right]& = (\mu_4 - 3\sigma^4_0)\operatorname{diag}(\mathbf P_{jn})^\top\operatorname{diag}(\mathbf P_{ln}) + \sigma^4_0\left[\operatorname{tr}(\mathbf P_{jn})\operatorname{tr}(\mathbf P_{ln}) + \operatorname{tr}(\mathbf P_{jn}\mathbf P_{ln}^s)\right], \\ & = (\mu_4 - 3\sigma^4_0)\operatorname{diag}(\mathbf P_{jn})^\top\operatorname{diag}(\mathbf P_{ln}) + \sigma^4_0\operatorname{tr}(\mathbf P_{jn}\mathbf P_{ln}^s), \\ \end{aligned} \] where $\mu_3 = \mathbb E(\epsilon_{ni}^3)$, $\mu_4 = \mathbb E(\epsilon_{ni}^4)$, and $\mathbf P_{jn}^s = \mathbf P_{jn} + \mathbf P_{jn}^\top$. Thus, we can write: \[ \underset{(2 + k_x) \times (2 + k_k)}{\boldsymbol\Omega_n} = \begin{pmatrix} \underset{(2 \times 2)}{(\mu_4 - 3\sigma_0^4)\boldsymbol\omega_n^\top\boldsymbol\omega_n} & \underset{(2 \times k_x)}{\mu_3\boldsymbol\omega^\top_n\mathbf H_n} \\ \underset{(k_x \times 2)}{\mu_3 \mathbf H_n^\top\boldsymbol\omega_n} & \underset{(k_x \times k_x)}{\mathbf 0} \end{pmatrix} + \mathbf V_n, \] with $\boldsymbol\omega_n = \left[\operatorname{diag}(\mathbf P_{1n}), \operatorname{diag}(\mathbf P_{2n})\right]$ is $n\times 2$ and \[ \mathbf V_n = \sigma_0^4 \begin{pmatrix} \underset{(1\times 1)}{\operatorname{tr}(\mathbf P_{1n}\mathbf P_{1n}^s)} & \underset{(1\times 1)}{\operatorname{tr}(\mathbf P_{1n}\mathbf P_{2n}^s)} & \underset{(1\times k_x)}{\mathbf 0} \\ \operatorname{tr}(\mathbf P_{2n}\mathbf P_{1n}^s) & \operatorname{tr}(\mathbf P_{2n}\mathbf P_{2n}^s) & \underset{(1\times k_x)}{\mathbf 0} \\ \underset{(k_x\times 1)}{\mathbf 0} & \underset{(k_x\times 1)}{\mathbf 0} & \underset{(k_x\times k_x)}{\frac{1}{\sigma_0^2}\mathbf H_n^\top\mathbf H_n} \end{pmatrix}. \]

As Lee (2007) remarks, when $\boldsymbol\varepsilon_n$ is normally distributed, $\boldsymbol\Omega_n$ is simplified to $\mathbf V_n$ because $\mu_3 = 0$, and $\mu_4 = 3\sigma_0^4$.

For further reference note that the first derivatives of the moment functions in Equation 6.37 are: \[ \begin{aligned} \underset{((2 + k_x) \times (k + 1))}{\frac{\partial \mathbf g_n(\boldsymbol\theta)}{\partial \boldsymbol\theta^\top}} & = \begin{pmatrix} \boldsymbol\varepsilon_n^\top\mathbf P_{1n}^s \\ \boldsymbol\varepsilon_n^\top\mathbf P_{2n}^s \\ \mathbf H_n^\top \end{pmatrix} \frac{\partial \boldsymbol\varepsilon_n}{\partial \boldsymbol\theta^\top}, \\ & = \underset{((2 + k_x)\times n)}{\begin{pmatrix} \boldsymbol\varepsilon_n^\top\mathbf P_{1n}^s \\ \boldsymbol\varepsilon_n^\top\mathbf P_{2n}^s \\ \mathbf H_n^\top \end{pmatrix}} \underset{(n\times(1 + k))}{\begin{pmatrix} -\mathbf W_n\mathbf y_n & -\mathbf X_n \end{pmatrix}}. \end{aligned} \tag{6.39}\]

6.3.1.1 Identification strategy

In Lee (2007)’s GMM framework, identification is formulated in terms of the limiting scaled population moments. More precisely, the true parameter $\boldsymbol\theta_0=(\rho_0,\boldsymbol\beta_0^\top)^\top$ is identified if \[ \lim_{n\to\infty}\frac{1}{n}\mathbb E\!\left[\mathbf g_n(\boldsymbol\theta)\right]=\mathbf 0 \] has a unique solution at $\boldsymbol\theta_0$ (Hansen 1982). To study this condition, we start from the residual evaluated at an arbitrary parameter value $\boldsymbol\theta=(\rho,\boldsymbol\beta^\top)^\top$: \[ \boldsymbol\varepsilon_n(\boldsymbol\theta)=\mathbf S_n(\rho)\mathbf y_n-\mathbf X_n\boldsymbol\beta, \qquad \mathbf S_n(\rho)=\mathbf I_n-\rho\mathbf W_n. \]

Using the reduced form $\mathbf y_n=\mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0+\mathbf S_0^{-1}\boldsymbol\varepsilon_n$, with $\mathbf S_0=\mathbf I_n-\rho_0\mathbf W_n$, and $\mathbf G_0=\mathbf W_n\mathbf S_0^{-1}$, we obtain \[ \boldsymbol\varepsilon_n(\boldsymbol\theta) = \mathbf S_n(\rho)\mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0-\mathbf X_n\boldsymbol\beta + \mathbf S_n(\rho)\mathbf S_0^{-1}\boldsymbol\varepsilon_n. \]

Because $\mathbf S_n(\rho)\mathbf S_0^{-1} = \mathbf I_n+(\rho_0-\rho)\mathbf G_0$ (see Lemma 6.2), it follows that \[ \boldsymbol\varepsilon_n(\boldsymbol\theta) = \mathbf d_n(\boldsymbol\theta)+\bigl(\mathbf I_n+(\rho_0-\rho)\mathbf G_0\bigr)\boldsymbol\varepsilon_n, \] where \[ \mathbf d_n(\boldsymbol\theta) = \mathbf X_n(\boldsymbol\beta_0-\boldsymbol\beta)+(\rho_0-\rho)\mathbf G_0\mathbf X_n\boldsymbol\beta_0. \tag{6.40}\]

This decomposition is the key to the identification argument. It shows that departures from the true parameter value affect the model residual through two channels: first, through the deterministic term $\mathbf d_n(\boldsymbol\theta)$, and second, through the transformed innovation term $\bigl(\mathbf I_n+(\rho_0-\rho)\mathbf G_0\bigr)\boldsymbol\varepsilon_n$.

Identification from the linear moments: Recall that the linear block of the moment vector is $\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)$. Since $\mathbb E(\boldsymbol\varepsilon_n)=\mathbf 0$, we have \[ \mathbb E\!\left[\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)\right] = \mathbf H_n^\top \mathbf d_n(\boldsymbol\theta), \] and therefore \[ \frac{1}{n}\mathbb E\!\left[\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)\right] = \frac{1}{n}\mathbf H_n^\top\mathbf X_n(\boldsymbol\beta_0-\boldsymbol\beta) + (\rho_0-\rho)\frac{1}{n}\mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0. \]

Thus, the linear moments identify $\boldsymbol\theta_0$ if the only solution of \[ \lim_{n\to\infty} \left[ \frac{1}{n}\mathbf H_n^\top\mathbf X_n(\boldsymbol\beta_0-\boldsymbol\beta) + (\rho_0-\rho)\frac{1}{n}\mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 \right] = \mathbf 0 \] is $\boldsymbol\beta=\boldsymbol\beta_0$ and $\rho=\rho_0$.

A convenient sufficient condition is that \[ \lim_{n\to\infty}\frac{1}{n} \Bigl[ \mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0,\, \mathbf H_n^\top\mathbf X_n \Bigr] \tag{6.41}\] has full column rank $(k+1)$. Intuitively, this means that the instruments in $\mathbf H_n$ must be sufficiently informative about both: (i) the exogenous regressors $\mathbf X_n$, and (ii) the endogenous spatial regressor $\mathbf W_n\mathbf y_n$, whose systematic part is captured by $\mathbf G_0\mathbf X_n\boldsymbol\beta_0$.

This is the direct spatial analogue of the usual IV rank condition. If Equation 6.41 holds, then the linear moments alone identify the full parameter vector $(\rho_0,\boldsymbol\beta_0)$.

Why the linear moments may fail to identify $\rho_0$. The previous sufficient condition can only hold if the columns of \[ \Bigl[\mathbf X_n,\;\mathbf G_0\mathbf X_n\boldsymbol\beta_0\Bigr] \] span $(k+1)$ asymptotically distinct directions. If $\mathbf G_0\mathbf X_n\boldsymbol\beta_0$ is asymptotically linearly dependent on $\mathbf X_n$, then the linear moments cannot separately identify $\rho_0$ from $\boldsymbol\beta_0$. In that case, the linear moment equations do not pin down a unique pair $(\rho,\boldsymbol\beta)$, but rather a relation between them. To see this, suppose there exists a constant vector $\mathbf c$ such that \[ \mathbf G_0\mathbf X_n\boldsymbol\beta_0=\mathbf X_n\mathbf c. \] Then Equation 6.40 becomes \[ \mathbf d_n(\boldsymbol\theta)=\mathbf X_n\bigl(\boldsymbol\beta_0-\boldsymbol\beta+(\rho_0-\rho)\mathbf c\bigr), \] and therefore \[ \mathbf H_n^\top\mathbf d_n(\boldsymbol\theta) = \mathbf H_n^\top\mathbf X_n\bigl(\boldsymbol\beta_0-\boldsymbol\beta+(\rho_0-\rho)\mathbf c\bigr). \] If $(1/n)\mathbf H_n^\top\mathbf X_n$ has full column rank $k$, then the linear moments imply \[ \boldsymbol\beta=\boldsymbol\beta_0+(\rho_0-\rho)\mathbf c. \] Hence the linear moments identify $\boldsymbol\beta$ only conditional on $\rho$. Under this scenario, $\rho_0$ must be identified from the quadratic moments.

Identification from the quadratic moments: Now consider the quadratic moment \[ \mathbb E\!\left[\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right], \qquad j=1,2, \] with $\operatorname{tr}(\mathbf P_{jn})=0$. Using the residual decomposition above, \[ \mathbb E\!\left[\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right] = \mathbf d_n(\boldsymbol\theta)^\top\mathbf P_{jn}\mathbf d_n(\boldsymbol\theta) + \sigma_0^2 \operatorname{tr}\!\left( \mathbf S_0^{-1\top}\mathbf S_n(\rho)^\top\mathbf P_{jn}\mathbf S_n(\rho)\mathbf S_0^{-1} \right). \]

Suppose the linear moments have already implied the relation \[ \boldsymbol\beta=\boldsymbol\beta_0+(\rho_0-\rho)\mathbf c, \] so that the remaining identification problem is to determine $\rho_0$. In this case, the quadratic moments are the key source of identification. Write \[ a\equiv \rho_0-\rho. \]

Since \[ \mathbf S_n(\rho)\mathbf S_0^{-1}=\mathbf I_n+a\mathbf G_0, \] we obtain \[ \mathbf S_0^{-1\top}\mathbf S_n(\rho)^\top\mathbf P_{jn}\mathbf S_n(\rho)\mathbf S_0^{-1} = (\mathbf I_n+a\mathbf G_0)^\top \mathbf P_{jn} (\mathbf I_n+a\mathbf G_0). \]

Taking traces and using $\operatorname{tr}(\mathbf P_{jn})=0$ yields \[ \begin{aligned} \operatorname{tr}\!\left( \mathbf S_0^{-1\top}\mathbf S_n(\rho)^\top\mathbf P_{jn}\mathbf S_n(\rho)\mathbf S_0^{-1} \right) &= a\,\operatorname{tr}(\mathbf P_{jn}^s\mathbf G_0) +a^2\,\operatorname{tr}(\mathbf G_0^\top\mathbf P_{jn}\mathbf G_0), \end{aligned} \tag{6.42}\] where \[ \mathbf P_{jn}^s\equiv \mathbf P_{jn}+\mathbf P_{jn}^\top. \]

Equation 6.42 is crucial. It shows that each quadratic moment is, asymptotically, a quadratic function of the scalar deviation $a=\rho_0-\rho$. Therefore, each quadratic moment equation may in principle admit two roots: the true root $a=0$, and possibly a spurious second root.

This is why, in general, one quadratic moment is not enough. The role of using several distinct matrices $\mathbf P_{jn}$ is to rule out a common spurious root. A sufficient condition for uniqueness of the common root is that the two vectors \[ \lim_{n\to\infty}\frac{1}{n} \begin{bmatrix} \operatorname{tr}(\mathbf P_{1n}^s\mathbf G_0)\\ \operatorname{tr}(\mathbf P_{2n}^s\mathbf G_0) \end{bmatrix} \tag{6.43}\] and \[ \lim_{n\to\infty}\frac{1}{n} \begin{bmatrix} \operatorname{tr}(\mathbf G_0^\top\mathbf P_{1n}\mathbf G_0)\\ \operatorname{tr}(\mathbf G_0^\top\mathbf P_{2n}\mathbf G_0) \end{bmatrix} \tag{6.44}\] be linearly independent.

The intuition is simple. The two quadratic moments respond to changes in $\rho$ through two distinct channels: a first-order channel, proportional to $(\rho_0-\rho)$, and a second-order channel, proportional to $(\rho_0-\rho)^2$. If the vectors in Equation 6.43 and Equation 6.44 are linearly independent, then these two channels cannot move in exactly the same direction across the moment equations. As a result, the only common solution is $a=0$, that is, $\rho=\rho_0$.

Summary of the identification strategy: The GMM identification strategy for the SLM can therefore be summarized as follows.

If \[ \lim_{n\to\infty}\frac{1}{n} \Bigl[ \mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0,\, \mathbf H_n^\top\mathbf X_n \Bigr] \] has full column rank $(k+1)$, then the linear moments alone identify $(\rho_0,\boldsymbol\beta_0)$.
If the linear moments do not identify $\rho_0$ because $\mathbf G_0\mathbf X_n\boldsymbol\beta_0$ is asymptotically linearly dependent on $\mathbf X_n$, but \[ \lim_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\mathbf X_n \] still has full column rank $k$, then the linear moments identify $\boldsymbol\beta$ conditional on $\rho$, and the quadratic moments identify $\rho_0$ provided that:
1. at least one first-order trace term is asymptotically nonzero, that is, \[ \lim_{n\to\infty}\frac{1}{n}\operatorname{tr}(\mathbf P_{jn}^s\mathbf G_0)\neq 0 \quad\text{for some } j, \] and
2. the limiting vectors in Equation 6.43 and Equation 6.44 are linearly independent.
Once $\rho_0$ is identified, $\boldsymbol\beta_0$ follows from the linear relation \[ \mathbf H_n^\top\mathbf d_n(\boldsymbol\theta)=\mathbf 0. \]

In short, the linear moments play the same role as in IV/2SLS, while the quadratic moments exploit the covariance structure generated by the spatial propagation mechanism. This is why the GMM approach can identify the SLM even in cases where the usual linear IV strategy is weak or fails altogether.

Assumption 10 (Assumptions for GMM)

Following Lee (2007), assume the following:

The $\epsilon_{ni}$ are i.i.d with zero mean, variance $\sigma_0^2$ and that a moment of order higher than the fourth exists.
The elements of $\mathbf X_n$ are uniformly bounded constants, $\mathbf X_n$ has the full rank $k$, and $\lim_{n\to\infty}(1/n)\mathbf X_n^\top\mathbf X_n$ exists and is nonsingular.
The spatial weights matrices $\{\mathbf W_n\}$ and $\{(\mathbf I_n - \rho\mathbf W_n)^{-1}\}$ at $\rho = \rho_0$ are uniformly bounded in both row and column sums in absolute value.
The matrices $\mathbf P_{1n}$ and $\mathbf P_{2n}$ are uniformly bonded in both row and column sums in absolute value, and elements of $\mathbf H_n$ are uniformly bounded.
Either
1. $\lim_{n\to\infty}(1/n)\mathbf H_n^\top\left[\mathbf G_0\mathbf X_n\boldsymbol\beta_0, \mathbf X_n\right]$ has the full rank $(k + 1)$, or
2. $\lim_{n\to\infty}(1/n)\mathbf H_n^\top\mathbf X_n$ has the full rank $k$, $\lim_{n\to\infty}(1/n)\operatorname{tr}\left(\mathbf P_{jn}^s\mathbf G_0\right)\neq 0$ for some $j$, $\mathbf P_{jn}^s = \mathbf P_{jn} + \mathbf P_{jn}^\top$, and \[ \lim_{n\to\infty}(1/n)\left[\operatorname{tr}\left(\mathbf P_{1n}^s\mathbf G_0\right), \operatorname{tr}\left(\mathbf P_{2n}^s\mathbf G_0\right)\right]^\top, \] is linearly independent of \[ \begin{aligned} \lim_{n\to\infty}(1/n)\left[\operatorname{tr}\left(\mathbf G_0^\top\mathbf P_{1n}\mathbf G_0\right), \operatorname{tr}\left(\mathbf G_0^\top\mathbf P_{2n}\mathbf G_0\right)\right]^\top \end{aligned} \] where $\mathbf G_0 = \mathbf W_n\mathbf S_0^{-1}$.

Theorem 6.3 (GMM estimator for SLM under homoskedasticity) Let $\boldsymbol\Upsilon_n\to \boldsymbol\Upsilon_0$. Under Assumption 10, suppose that $\mathbf P_{jn}$ for $j = 1, 2$, are chosen from $\mathcal P_{1n}$ and $\mathbf H_n$ is a $n\times k_k$ matrix so that \[ \lim_{n \to \infty}\frac{1}{n}\mathbb E\left[\mathbf g_n(\boldsymbol\theta)\right] = \mathbf 0, \] has a unique root at $\boldsymbol\theta_0$ in $\boldsymbol\Theta$. Then, the GMM estimator $\widehat{\boldsymbol\theta}_n$ derived from $\min_{\boldsymbol\theta \in \boldsymbol\Theta}\mathbf g_n(\boldsymbol\theta)^\top\boldsymbol\Upsilon_n\mathbf g_n(\boldsymbol\theta)$ is a consistent estimator of $\boldsymbol\theta_0$, and $\sqrt{n}(\widehat{\boldsymbol\theta}_n - \boldsymbol\theta_0)\xrightarrow{d} \mathcal N(\mathbf 0, \boldsymbol\Sigma_0)$, where \[ \begin{aligned} \boldsymbol\Sigma_0 = & \lim_{n\to\infty}\left[\left(\frac{1}{n}\mathbf D_n^\top\right)\boldsymbol\Upsilon_0\left(\frac{1}{n}\mathbf D_n\right)\right]^{-1}\left(\frac{1}{n}\mathbf D_n^\top\right)\boldsymbol\Upsilon_0\left(\frac{1}{n}\boldsymbol\Omega_n\right)\boldsymbol\Upsilon_0\left(\frac{1}{n}\mathbf D_n\right) \\ & \times \left[\left(\frac{1}{n}\mathbf D_n^\top\right)\boldsymbol\Upsilon_0\left(\frac{1}{n}\mathbf D_n\right)\right]^{-1}, \end{aligned} \] and \[ \underset{(k_x + 2)\times (k + 1)}{\mathbf D_n} = \frac{\partial \mathbb E\left[\mathbf g_n(\boldsymbol\theta_0)\right]}{\partial \boldsymbol\theta^\top} = \begin{pmatrix} \underset{(1 \times 1)}{\sigma_0^2\operatorname{tr}(\mathbf P^s_{1n}\mathbf G_0)} & \underset{(1 \times k)}{\mathbf 0} \\ \underset{(1 \times 1)}{\sigma_0^2\operatorname{tr}(\mathbf P^s_{2n}\mathbf G_0)} & \mathbf 0 \\ \underset{(k_x \times 1)}{\mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0} & \underset{(k_x\times k)}{\mathbf H^\top_n\mathbf X_n} \end{pmatrix} \tag{6.45}\] where $\mathbf G_0 = \mathbf W_n\mathbf S_0^{-1}$, and under the assumption that $\lim_{n\to\infty}\mathbf D_n$ exists and has the full rank $k + 1$.

The GMME uses as linear instruments \[ \mathbf H_n = \left[\mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W_n^2\mathbf X_n, \cdots\right], \] and \[ \begin{aligned} \mathbf P_{1n} & = \mathbf W_n, \\ \mathbf P_{2n} & = \mathbf W^2_n - \left(\operatorname{tr}(\mathbf W^2_n)/n\right)\mathbf I_n. \end{aligned} \]

Assuming that $\boldsymbol\Upsilon_n = \mathbf I_{(2 + k_k)}$, the asymptotic variance can be estimated as \[ \begin{aligned} \widehat{\boldsymbol\Sigma} = & \left[\left(\frac{1}{n}\widehat{\mathbf D}_n^\top\right)\left(\frac{1}{n}\widehat{\mathbf D}_n\right)\right]^{-1}\left(\frac{1}{n}\widehat{\mathbf D}_n^\top\right)\left(\frac{1}{n}\widehat{\boldsymbol\Omega}_n\right)\left(\frac{1}{n}\widehat{\mathbf D}_n\right) \left[\left(\frac{1}{n}\widehat{\mathbf D}_n^\top\right)\left(\frac{1}{n}\widehat{\mathbf D}_n\right)\right]^{-1}, \end{aligned} \] where $\widehat{\boldsymbol\Omega}_n$ is estimated as \[ \widehat{\boldsymbol\Omega}_n = \begin{pmatrix} (\widehat{\mu}_4 - 3\widehat{\sigma}_n^4)\boldsymbol\omega_n^\top\boldsymbol\omega_n & \widehat{\mu}_3\boldsymbol\omega^\top_n\mathbf H_n \\ \widehat{\mu}_3 \mathbf H_n^\top\boldsymbol\omega_n & \mathbf 0 \end{pmatrix} + \widehat{\mathbf V}_n, \tag{6.46}\] with $\widehat{\mu}_4 = (1/n)\sum_{i = 1}^n\widehat{\epsilon}_{in}^4$, $\widehat{\mu}_3 = (1/n)\sum_{i = 1}^n\widehat{\epsilon}_{in}^3$, $\widehat{\sigma}_n^2 = (1/n)\sum_{i = 1}^n\widehat{\epsilon}_{in}^2$, $\widehat{\boldsymbol\varepsilon}_n = (\mathbf I_n - \widehat{\rho}_n\mathbf W_n)\mathbf y_n - \mathbf X_n\widehat{\boldsymbol\beta}_n$, and \[ \widehat{\mathbf V}_n = \widehat{\sigma}_n^4 \begin{pmatrix} \operatorname{tr}(\mathbf P_{1n}\mathbf P_{1n}^s) & \operatorname{tr}(\mathbf P_{1n}\mathbf P_{2n}^s) & \mathbf 0 \\ \operatorname{tr}(\mathbf P_{2n}\mathbf P_{1n}^s) & \operatorname{tr}(\mathbf P_{2n}\mathbf P_{2n}^s) & \mathbf 0 \\ \mathbf 0 & \mathbf 0 & \frac{1}{\widehat{\sigma}_n^2}\mathbf H_n^\top\mathbf H_n \end{pmatrix} \]

The matrix $\widehat{\mathbf D}_n$ can be estimated by replacing $\widehat{\boldsymbol\theta}_n$ in Equation 6.45 or using the Jacobian in Equation 6.39 evaluated at $\widehat{\boldsymbol\theta}_n$.

For a sketch of Proof of Theorem 6.3 see Section 6.9.2

6.3.2 OGMM Estimator

Different weighting matrices $\boldsymbol\Upsilon_n$ lead to different consistent GMM estimators, each with potentially different asymptotic covariance matrices. As established in Theorem 6.3, the GMM estimator remains consistent for any sequence $\boldsymbol\Upsilon_n\to\boldsymbol\Upsilon_0$. Among this class of estimators, an especially important choice is the one that minimizes the asymptotic variance for the given moment vector $\mathbf g_n(\boldsymbol\theta)$.

In Lee’s framework, the optimal weighting matrix is the inverse of the variance matrix of the moments, \[ \boldsymbol\Omega_n=\operatorname{Var}\!\bigl(\mathbf g_n(\boldsymbol\theta_0)\bigr). \]

Since $\boldsymbol\Omega_n$ depends on unknown parameters, the corresponding optimal GMM estimator is implemented in feasible two-step form by replacing $\boldsymbol\Omega_n$ with a consistent estimator $\widehat{\boldsymbol\Omega}_n$. This yields the feasible Optimal GMM Estimator (OGMME).

Theorem 6.4 (OGMM estimator for the SLM under homoskedasticity (Lee 2007)) In addition to Assumption 10, suppose that

the limit of $(1/n)\boldsymbol\Omega_n$ exists and is nonsingular,
\[ \left(\widehat{\boldsymbol\Omega}_n/n\right)^{-1} - \left(\boldsymbol\Omega_n/n\right)^{-1} = o_p(1). \]

Then the feasible OGMM estimator \[ \widehat{\boldsymbol\theta}_{o,n} = \operatorname*{argmin}_{\boldsymbol\theta\in\Theta} \mathbf g_n(\boldsymbol\theta)^\top \widehat{\boldsymbol\Omega}_n^{-1}\mathbf g_n(\boldsymbol\theta), \] with $\mathbf P_{jn}\in\mathcal P_{1n}$, satisfies \[ \sqrt n(\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0) \xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Sigma^o), \] where \[ \boldsymbol\Sigma^o = \left[ \lim_{n\to\infty} \frac1n \mathbf D_n^\top \boldsymbol\Omega_n^{-1}\mathbf D_n \right]^{-1}. \]

Furthermore, \[ \mathbf g_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\boldsymbol\Omega}_n^{-1}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) \xrightarrow{d} \chi^2_{m-p}, \] where \[ m=2+k_x, \qquad p=k+1. \]

6.3.3 Best GMM estimator

Lee (2007) proposes the optimal instrument for the OGMM estimator. The best linear instruments are: \[ \mathbf H_n^* = \mathbb E(\mathbf Z_n|\mathbf X_n) = \left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0, \mathbf X_n\right). \]

There are two cases which can make the selection of best quadratic matrices easier.

6.3.3.1 Selection from $\mathcal P_{2n}$

For the best selection of $\mathbf P_{jn}$, consider $\mathbf P_{jn}\in \mathcal P_{2n}$. In this case $\boldsymbol\omega_n$ is a $n\times 1$ matrix of zeros and $\boldsymbol\Omega_n$ becomes: \[ \boldsymbol\Omega_n = \mathbf V_n = \sigma_0^4 \begin{pmatrix} \operatorname{tr}(\mathbf P_{1n}\mathbf P_{1n}^s) & \operatorname{tr}(\mathbf P_{1n}\mathbf P_{2n}^s) & \mathbf 0 \\ \operatorname{tr}(\mathbf P_{2n}\mathbf P_{1n}^s) & \operatorname{tr}(\mathbf P_{2n}\mathbf P_{2n}^s) & \mathbf 0 \\ \mathbf 0 & \mathbf 0 & \frac{1}{\sigma_0^2}\mathbf H_n^{\top}\mathbf H_n \end{pmatrix} = \sigma_0^4\begin{pmatrix} \boldsymbol\Delta_{2n} & \mathbf 0 \\ \mathbf 0 & \frac{1}{\sigma_0^2}\mathbf H_n^{\top}\mathbf H_n \end{pmatrix}, \] with $\boldsymbol\Delta_{2n} = \left[\operatorname{vec}(\mathbf P_{1n})^\top, \operatorname{vec}(\mathbf P_{2n}^\top)\right]^\top\left[\operatorname{vec}(\mathbf P_{1n}^s), \operatorname{vec}(\mathbf P_{2n}^s)\right]$, because, for any conformable matrices $\mathbf A$ and $\mathbf B$, $\operatorname{tr}(\mathbf A\mathbf B) = \operatorname{vec}(\mathbf A^\top)^\top\operatorname{vec}(\mathbf B)$ and \[ \begin{aligned} \mathbf D_n^\top\boldsymbol\Omega_n^{-1}\mathbf D_n = & \frac{1}{\sigma_0^4} \begin{pmatrix} \sigma_0^2\operatorname{tr}(\mathbf P^s_{1n}\mathbf G_0) & \sigma_0^2\operatorname{tr}(\mathbf P^s_{2n}\mathbf G_0) & (\mathbf G_0\mathbf X_n\boldsymbol\beta_0)^\top\mathbf H_n\\ \mathbf 0 & \mathbf 0 & \mathbf X_n^\top\mathbf H_n \end{pmatrix}\\ & \begin{pmatrix} \operatorname{tr}(\mathbf P_{1n}\mathbf P_{1n}^s)^{-1} & \operatorname{tr}(\mathbf P_{1n}\mathbf P_{2n}^s)^{-1} & \mathbf 0 \\ \operatorname{tr}(\mathbf P_{2n}\mathbf P_{1n}^s)^{-1} & \operatorname{tr}(\mathbf P_{2n}\mathbf P_{2n}^s)^{-1} & \mathbf 0 \\ \mathbf 0 & \mathbf 0 & \sigma_0^2\left(\mathbf H_n^{\top}\mathbf H_n\right)^{-1} \end{pmatrix} \begin{pmatrix} \sigma_0^2\operatorname{tr}(\mathbf P^s_{1n}\mathbf G_0) & \mathbf 0 \\ \sigma_0^2\operatorname{tr}(\mathbf P^s_{2n}\mathbf G_0) & \mathbf 0 \\ \mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \mathbf H^\top_n\mathbf X_n \end{pmatrix}, \\ & = \frac{1}{\sigma_0^4} \begin{pmatrix} \sigma_0^2\mathbf C_{2n} & (\mathbf G_0\mathbf X_n\boldsymbol\beta_0)^\top\mathbf H_n \\ \mathbf 0 & \mathbf X_n^\top\mathbf H_n \end{pmatrix} \begin{pmatrix} \boldsymbol\Delta_{2n}^{-1} & \mathbf 0 \\ \mathbf 0 & \sigma_0^2\left(\mathbf H_n^\top\mathbf H_n\right)^{-1} \end{pmatrix} \begin{pmatrix} \sigma_0^2\mathbf C_{2n}^\top & \mathbf 0 \\ \mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \mathbf H^\top_n\mathbf X_n \end{pmatrix}, \\ & = \frac{1}{\sigma_0^4} \begin{pmatrix} \sigma_0^2\mathbf C_{2n}\boldsymbol\Delta_{2n}^{-1} & \sigma_0^2(\mathbf G_0\mathbf X_n\boldsymbol\beta_0)^\top\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1} \\ \mathbf 0 & \sigma_0^2\mathbf X_n^\top\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1} \end{pmatrix} \begin{pmatrix} \sigma_0^2\mathbf C_{2n}^\top & \mathbf 0 \\ \mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \mathbf H^\top_n\mathbf X_n \end{pmatrix}, \\ & = \begin{pmatrix} \mathbf C_{2n}\boldsymbol\Delta_{2n}^{-1}\mathbf C_{2n}^\top + \frac{1}{\sigma_0^2}(\mathbf G_0\mathbf X_n\boldsymbol\beta_0)^\top\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \frac{1}{\sigma_0^2}(\mathbf G_0\mathbf X_n\boldsymbol\beta_0)^\top\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top\mathbf X_n \\ \frac{1}{\sigma_0^2}\mathbf X_n^\top\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1} \mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \frac{1}{\sigma_0^2}\mathbf X_n^\top\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1} \mathbf H_n^\top\mathbf X_n \end{pmatrix}, \\ & = \begin{pmatrix} \mathbf C_{2n}\boldsymbol\Delta_{2n}^{-1}\mathbf C_{2n}^\top & \mathbf 0 \\ \mathbf 0 & \mathbf 0 \end{pmatrix} + \frac{1}{\sigma_0^2} \begin{pmatrix} (\mathbf G_0\mathbf X_n\boldsymbol\beta_0)^\top \\ \mathbf X_n^\top \end{pmatrix} \mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1} \mathbf H_n^\top \begin{pmatrix} \mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \mathbf X_n \end{pmatrix}, \end{aligned} \] where $\mathbf C_{2n} = \left[\operatorname{tr}\left(\mathbf P_{1n}^s\mathbf G_0\right), \operatorname{tr}\left(\mathbf P_{2n}^s\mathbf G_0\right)\right]$. With the best instruments $\mathbf H^*_n$ for $\mathbf H_n$, the optimal variance-covariance matrix is \[ \mathbf D_n^\top\boldsymbol\Omega_n^{-1}\mathbf D_n = \begin{pmatrix} \mathbf C_{2n}\boldsymbol\Delta_{2n}^{-1}\mathbf C_{2n}^\top & \mathbf 0 \\ \mathbf 0 & \mathbf 0 \end{pmatrix} + \frac{1}{\sigma_0^2} \begin{pmatrix} (\mathbf G_0\mathbf X_n\boldsymbol\beta_0) & \mathbf X_n \end{pmatrix}^\top \begin{pmatrix} \mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \mathbf X_n \end{pmatrix}. \]

Since $\operatorname{tr}(\mathbf P_{jn}\mathbf P_{ln}^s) = \frac{1}{2}\operatorname{tr}(\mathbf P_{jn}^s\mathbf P_{ln}^s)$, we can also write \[ \begin{aligned} \boldsymbol\Delta_{2n} & = \frac{1}{2} \begin{pmatrix} \operatorname{tr}(\mathbf P_{1n}^s\mathbf P_{1n}^s) & \operatorname{tr}(\mathbf P_{1n}^s\mathbf P_{2n}^s) \\ \operatorname{tr}(\mathbf P_{2n}^s\mathbf P_{1n}^s) & \operatorname{tr}(\mathbf P_{2n}^s\mathbf P_{2n}^s) \end{pmatrix},\\ & = \frac{1}{2}\left[\operatorname{vec}(\mathbf P_{1n})^s, \operatorname{vec}(\mathbf P_{2n}^s)\right]^\top\left[\operatorname{vec}(\mathbf P_{1n}^s), \operatorname{vec}(\mathbf P_{2n}^s)\right]. \end{aligned} \]

If $\mathbf P_{jn}$s belong to $\mathcal P_{2n}$ such that $\operatorname{diag}(\mathbf P_{jn}) = \mathbf 0$, then \[ \begin{aligned} \operatorname{tr}(\mathbf P_{jn}^s\mathbf G_0) & = \frac{1}{2}\operatorname{tr}\left[\mathbf P_{jn}^s\left(\mathbf G_0 -\operatorname{Diag}(\mathbf G_0)\right)^s\right], \\ & = \frac{1}{2}\operatorname{vec}\left(\left[\mathbf G_0 - \operatorname{Diag}(\mathbf G_0)\right]^s\right)^\top\operatorname{vec}(\mathbf P_{jn}^s). \end{aligned} \]

Schwartz inequality implies that \[ \begin{aligned} \mathbf C_{2n}\boldsymbol\Delta_{2n}^{-1}\mathbf C_{2n}^\top & \leq \frac{1}{2}\operatorname{vec}\left(\left[\mathbf G_0 - \operatorname{Diag}(\mathbf G_0)\right]^s\right)^\top \frac{1}{2}\operatorname{vec}\left(\left[\mathbf G_0 - \operatorname{Diag}(\mathbf G_0)\right]^s\right) \\ & = \operatorname{tr}\left(\left[\mathbf G_0 - \operatorname{Diag}(\mathbf G_0)\right]^s \mathbf G_0\right). \end{aligned} \]

Therefore, in the subclass $\mathcal P_{2n}$, $\left[\mathbf G_0 - \operatorname{Diag}(\mathbf G_0)\right]$ and $\left[\mathbf G_0\mathbf X_n\boldsymbol\beta_0, \mathbf X_n\right]$ provide the set of best IV functions, and the vector of moment functions is \[ \mathbf g_{b, n} = \begin{pmatrix} \widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta)^\top\left(\widehat{\mathbf G}_n - \operatorname{Diag}(\widehat{\mathbf G}_n)\right)\widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta) \\ \widehat{\boldsymbol\beta}_n^\top\mathbf X_n ^\top\widehat{\mathbf G}_n^\top\widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta)\\ \mathbf X_n^\top\widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta) \end{pmatrix}, \] and the corresponding estimated $\mathbf V_n$ is \[ \widehat{\mathbf V}_n = \widehat{\sigma}^4_n \begin{pmatrix} \operatorname{tr}\left(\left[\widehat{\mathbf G}_n - \operatorname{Diag}(\widehat{\mathbf G}_n)\right]^s\widehat{\mathbf G}_n\right) & \mathbf 0 \\ \mathbf 0 & \frac{1}{\widehat{\sigma}_n^2}\left(\widehat{\mathbf G}_n\mathbf X_n\widehat{\boldsymbol\beta}, \mathbf X_n\right)^\top \left(\widehat{\mathbf G}_n\mathbf X_n\widehat{\boldsymbol\beta}, \mathbf X_n\right) \end{pmatrix}, \] and the Feasible Best GMME (FBGMM) with $\mathcal P_{2n}$ will be from minimizing: \[ Q_n(\boldsymbol\theta) = \mathbf g_{b,n}^\top(\boldsymbol\theta)\widehat{\mathbf V}_n^{-1}\mathbf g_{b,n}(\boldsymbol\theta). \]

6.3.3.2 Assuming Normality

If $\boldsymbol\varepsilon_n \sim \mathcal N(\mathbf 0, \sigma_0^2\mathbf I_n)$, then $\boldsymbol\Omega_n = \mathbf V_n$ even though $\mathbf P_{jn}$’s are selected from the class $\mathcal P_{1n}$.

Because, for any $\mathbf P_{jn}\in\mathcal P_{1n}$, \[ \operatorname{tr}\left(\mathbf P_{jn}^s\mathbf G_0\right)= \frac{1}{2}\operatorname{vec}\left(\left[\mathbf G_0 - \frac{\operatorname{tr}\left(\mathbf G_0\right)}{n}\mathbf I_n\right]^s\mathbf G_0\right)^\top\operatorname{vec}\left(\mathbf P_{jn}^s\right), \] the generalized Schwartz inequality implies that \[ \mathbf C_{2n}\boldsymbol\Delta_{2n}^{-1}\mathbf C_{2n}^\top\leq \operatorname{tr}\left(\left[\mathbf G_0 - \frac{\operatorname{tr}\left(\mathbf G_0\right)}{n}\mathbf I_n\right]^s\mathbf G_0\right). \]

Therefore, in the broader class $\mathcal P_{1n}$, $\left[\mathbf G_0 - \frac{\operatorname{tr}\left(\mathbf G_0\right)}{n}\mathbf I_n\right]$ and $\left[\mathbf G_0\mathbf X_n\boldsymbol\beta_0, \mathbf X_n\right]$ provide the best set of IV functions. Thus, the vector of moment functions is \[ \mathbf g_{b, n} = \begin{pmatrix} \widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta)^\top\left(\widehat{\mathbf G}_n - \frac{\operatorname{tr}(\widehat{\mathbf G}_n)}{n}\mathbf I_n \right)\widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta) \\ \widehat{\boldsymbol\beta}_n^\top\mathbf X_n ^\top\widehat{\mathbf G}_n^\top\widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta)\\ \mathbf X_n^\top\widehat{\boldsymbol\varepsilon}_n(\boldsymbol\theta) \end{pmatrix}, \] and the corresponding estimated $\mathbf V_n$ is \[ \widehat{\mathbf V}_n = \widehat{\sigma}^4_n \begin{pmatrix} \operatorname{tr}\left(\left[\widehat{\mathbf G}_n - \frac{\operatorname{tr}(\widehat{\mathbf G}_n)}{n}\mathbf I_n \right]^s\widehat{\mathbf G}_n\right) & \mathbf 0 \\ \mathbf 0 & \frac{1}{\widehat{\sigma}_n^2}\left(\widehat{\mathbf G}_n\mathbf X_n\widehat{\boldsymbol\beta}, \mathbf X_n\right)^\top \left(\widehat{\mathbf G}_n\mathbf X_n\widehat{\boldsymbol\beta}, \mathbf X_n\right) \end{pmatrix}. \]

Theorem 6.5 (Best GMM estimator for the SLM) Under Assumptions (a)-(c) of Assumption 10, suppose that $\widehat{\rho}_n$ is a $\sqrt{n}$-consistent estimate of $\rho_0$, $\widehat{\boldsymbol\beta}_n$ is a consistent estimate of $\boldsymbol\beta_0$, and $\widehat{\sigma}_n^2$ is a consistent estimate of $\sigma_0^2$. Within the class of GMMEs derived with $\mathcal P_{2n}$, the Best GMM estimator (BGMME) $\widehat{\boldsymbol\theta}_n$ has the limiting distribution that \[ \sqrt{n}\left(\widehat{\boldsymbol\theta}_n - \boldsymbol\theta_0\right)\xrightarrow{d} \mathcal N\left(\mathbf 0, \boldsymbol\Sigma^{b2}\right), \] where \[ \boldsymbol\Sigma^{b2} = \lim_{n\to\infty}\frac{1}{n} \begin{pmatrix} \operatorname{tr}\left(\left[\mathbf G_0 - \operatorname{Diag}(\mathbf G_0)\right]^s \mathbf G_0\right) + \frac{1}{\sigma_0^2}\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right)^\top\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right) & \frac{1}{\sigma_0^2}\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right)^\top\mathbf X_n \\ \frac{1}{\sigma_0^2}\mathbf X_n^\top\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right) & \frac{1}{\sigma_0^2}\mathbf X_n^\top\mathbf X_n \end{pmatrix}, \] with $\mathbf G_0 = \mathbf W_n\left(\mathbf I_n - \rho_0\mathbf W_n\right)^{-1}$, which is assumed to exist.

When $\boldsymbol\varepsilon_n\sim\mathcal N\left(\mathbf 0, \sigma_0^2\mathbf I_n\right)$, within the broader class of GMMEs derived with $\mathcal P_{1n}$, the BGMME $\widehat{\boldsymbol\theta}_{n}$ has the limiting distribution that \[ \sqrt{n}\left(\widehat{\boldsymbol\theta}_n - \boldsymbol\theta_0\right)\xrightarrow{d} \mathcal N\left(\mathbf 0, \boldsymbol\Sigma^{b1}\right), \] where \[ \boldsymbol\Sigma^{b1} = \lim_{n\to\infty}\frac{1}{n} \begin{pmatrix} \operatorname{tr}\left(\left[\mathbf G_0 - \frac{\operatorname{tr}(\mathbf G_0)}{n}\mathbf I_n\right]^s \mathbf G_0\right) + \frac{1}{\sigma_0^2}\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right)^\top\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right) & \frac{1}{\sigma_0^2}\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right)^\top\mathbf X_n \\ \frac{1}{\sigma_0^2}\mathbf X_n^\top\left(\mathbf G_0\mathbf X_n\boldsymbol\beta_0\right) & \frac{1}{\sigma_0^2}\mathbf X_n^\top\mathbf X_n \end{pmatrix}, \] which is assumed to exist.

6.3.4 S2SLS Estimator as GMM Estimator

The moment conditions for the S2SLS estimators are \[ \mathbf g_n = \frac{1}{n}\mathbf H^\top_n\boldsymbol\varepsilon_n= \frac{1}{n}\mathbf H^\top_n\left(\mathbf y_n - \mathbf Z_n\boldsymbol\delta\right), \] whose variance-covariance matrix is: \[ \frac{1}{n}\boldsymbol\Omega_n = \frac{1}{n}\sigma_0^2\mathbf H_n^\top\mathbf H_n. \]

The matrix $\boldsymbol\Upsilon_n^{-1}$ is the optimal weight matrix, which correspond to the inverse of the covariance matrix of the sample moments: \[ \boldsymbol\Upsilon_n^{-1} = \left(\frac{1}{n}\sigma_0^2\mathbf H_n^\top\mathbf H_n\right)^{-1}. \]

Then, the function to minimize is: \[ Q = \frac{1}{n\widehat{\sigma}^2}\left\lbrace \left[\mathbf H^\top_n\mathbf y_n - \mathbf H^\top_n\mathbf Z_n\boldsymbol\delta\right]^\top\left(\mathbf H^\top_n\mathbf H_n\right)^{-1}\left[\mathbf H^\top_n\mathbf y_n - \mathbf H^\top_n\mathbf Z_n\boldsymbol\delta\right]\right\rbrace \]

Obtaining the first order conditions and solving for $\boldsymbol\delta$, we obtain: \[ \widehat{\boldsymbol\delta}_{n} = \left(\mathbf Z^\top_n\mathbf P_{H,n}\mathbf Z_n\right)^{-1}\mathbf Z^\top_n\mathbf P_{H, n}\mathbf y_n. \]

Its asymptotic distribution is \[ \sqrt{n}\left(\widehat{\boldsymbol\delta}_{n} - \boldsymbol\delta_0\right)\xrightarrow{d} \mathcal N\left(\mathbf 0, \sigma_0^2\lim_{n\to\infty}\left\lbrace \frac{1}{n}\left(\mathbf G_n\mathbf X_n\boldsymbol\beta_0\right)^\top \mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top\left(\mathbf G_n\mathbf X_n\boldsymbol\beta_0\right)\right\rbrace^{-1}\right). \]

6.4 Feasible Generalized Least Squares Estimation of the SEM

Kelejian and Prucha (1999) derive a Method of Moments (MOM) estimator for $\lambda$, which is later used in the Feasible Generalized Least Squares (FGLS) estimator. The motivation behind this new estimator, according to Kelejian and Prucha (1999), is that the (quasi) maximum likelihood estimator may not be computationally feasible, especially in cases with moderate or large-sized samples. As they note, the MOM estimator is computationally simple, regardless of sample size, making it highly attractive when dealing with large spatial datasets. Additionally, the IV/GMM estimators avoid the Jacobian term, alleviating many problems associated with matrix inversion, the computation of characteristic roots, and/or Cholesky decomposition. Another motivation for proposing this estimator was that, at the time, there were no formal results regarding the consistency and asymptotic normality of the ML estimator (Prucha 2014, pag. 1608). Recall that Lee formally derived the asymptotic properties of the ML estimator in 2004 for the Spatial Lag Model (SLM).

Recall that the SEM model is given by: \[ \begin{aligned} \mathbf y_n & = \mathbf X_n\boldsymbol\beta_0 + \mathbf u_n, \\ \mathbf u_n & = \lambda_0\mathbf M_n \mathbf u_n + \boldsymbol\varepsilon_n. \end{aligned} \tag{6.47}\]

In brief, Kelejian and Prucha (1999) propose the use of nonlinear least square to obtain a consistent generalized moment estimate for $\lambda_0$, which can be used to obtain consistent estimates for $\boldsymbol\beta_0$ in a FGLS framework. The key distinction between the MOM estimation discussed here and the Generalized Method of Moment (GMM) estimator presented later is that in the MOM approach, there is no inference for the spatial autoregressive coefficient $\lambda$. In other words, $\lambda$ is treated purely as a nuisance parameter, whose only purpose is to aid in obtaining consistent estimates for $\boldsymbol\beta_0$.

Remark

The MOM procedure proposed by Kelejian and Prucha (1999) was originally motivated by the computational difficulties of the ML.

Remark

Kelejian and Prucha (1999) does not provide an asymptotic variance for $\lambda$., which is why some software packages only provide the estimate $\widehat{\lambda}$ without is standard error.

One advantage of the MOM estimator is that it does not rely on the assumption of normality of the disturbances $\boldsymbol\varepsilon_n$. Nonetheless, it does assume that $\epsilon_i$ are independently and identically distributed for all $i$ with zero mean and constant variance $\sigma^2$. We adopt the following assumption about the error terms, as stated by Kelejian and Prucha (1999).

Assumption 11 (Homoskedastic Errors)

The innovations $\left\lbrace \epsilon_{i,n}, 1 \leq i \leq n, n\geq 1\right\rbrace$ are independently and identically distributed for all $n$ with zero mean and variance $\sigma^2$, where $0 < \sigma^2 < b$, with $b < \infty$. Additionally, the innovations are assumed to possess finite fourth moments.

Next, we present the following assumptions regarding the weight matrix:

Assumption 12 (Weight Matrix $\mathbf M_n$)

Assume the following:

All diagonal elements of the spatial weighting matrix $\mathbf M_n$ are zero.
The matrix $(\mathbf I_n - \lambda_0\mathbf M_n)$ is nonsingular with $\left|\lambda_0 \right|<1$.

Given Equation 6.47 and Assumption 12 on the weight matrix, we can express the error term as $\mathbf u_n = (\mathbf I_n - \lambda_0 \mathbf M_n)^{-1}\boldsymbol\varepsilon_n = \mathbf R_0^{-1}\boldsymbol\varepsilon_n$. Therefore, the expectation and variance of $\mathbf u_n$ are $\mathbb E(\mathbf u_n) = 0$ and $\mathbb E(\mathbf u_n\mathbf u^\top_n) = \boldsymbol\Omega_{n0}$, respectively, where: \[ \boldsymbol\Omega_{n0} = \sigma^2_{0}(\mathbf I_n - \lambda_0\mathbf M_n)^{-1}(\mathbf I_n - \lambda_0\mathbf M_n^\top)^{-1} = \sigma_{0}^2\mathbf R_0^{-1}\mathbf R_0^{-1\top}. \]

Note that a row-standardized spatial weight matrix is typically not symmetric, such that $\mathbf M_n\neq \mathbf M_n^\top$ and thus $(\mathbf I_n - \lambda_0\mathbf M_n)^{-1}\neq (\mathbf I_n - \lambda_0\mathbf M_n^\top)^{-1}$.

6.4.1 Generalized Least Squares Estimator

The primary objective in Kelejian and Prucha (1999) is to derive a consistent estimator for $\lambda_0$ to ensure the consistency of the resulting spatially weighted estimator. In this context, Kelejian and Prucha (1999) were not primarily concerned with inference on $\lambda$, but rather viewed it as a tool to obtain consistent estimates for $\boldsymbol\beta$. This implies that $\lambda$ is considered a nuisance parameter.

The Spatially Weighted Least Squares (SWLS) boils down to: \[ \widehat{\boldsymbol\beta}_{SWLS} = \left(\mathbf X_{s, n}^\top\mathbf X_{s, n}\right)^{-1}\mathbf X_{s, n}^\top\mathbf y_{s, n}, \tag{6.48}\] were $\mathbf X_{s, n} = \mathbf X_n - \lambda_0\mathbf M_n\mathbf X_n$ and $\mathbf y_{s, n} = \mathbf y_n - \lambda_0\mathbf M_n\mathbf y_n$, using a consistent estimate $\widehat{\lambda}_n$ for the autoregressive parameter. Note that this model is essentially an OLS applied to spatially filtered variables. Moreover, it is important to note that SWLS is a special case of the Feasible Generalized Least Squares (FGLS) estimator.

Under the assumptions made, the Generalized Least Squares (GLS) estimator—assuming we know $\lambda_0$—for $\boldsymbol\beta$ is: \[ \begin{aligned} \widehat{\boldsymbol\beta}_{GLS} & = \left[\mathbf X^\top_n\boldsymbol\Omega^{-1}_{n0}\mathbf X_n\right]^{-1}\mathbf X^\top_n\boldsymbol\Omega^{-1}_{n0}\mathbf y_n, \\ & = \left[\mathbf X^\top_n\frac{1}{\sigma^2}\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^\top\left(\mathbf I_n - \lambda_0\mathbf M_n\right)\mathbf X_n\right]^{-1}\mathbf X^\top_n\frac{1}{\sigma^2}\left(\mathbf I_n -\lambda_0\mathbf M_n\right)^\top\left(\mathbf I_n - \lambda_0\mathbf M_n\right)\mathbf y_n, \\ & =\left[\mathbf X^\top_n\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^\top\left(\mathbf I_n - \lambda_0\mathbf M_n\right)\mathbf X_n\right]^{-1}\mathbf X^\top_n\left(\mathbf I_n -\lambda_0\mathbf M_n\right)^\top\left(\mathbf I_n - \lambda_0\mathbf M_n\right)\mathbf y_n, \\ & =\left(\mathbf X_n^\top \mathbf R_{n0}^\top\mathbf R_{n0}\mathbf X_n\right)^{-1} \mathbf X_n^\top \mathbf R_{n0}^\top\mathbf R_{n0}\mathbf y_n. \end{aligned} \]

The FGLS estimator substitutes a consistent estimate for $\lambda$ into this expression, resulting in the following: \[ \begin{aligned} \widehat{\boldsymbol\beta}_{FGLS} & =\left[\mathbf X^\top_n\left(\mathbf I_n - \widehat{\lambda}_n\mathbf M_n\right)^\top\left(\mathbf I_n - \widehat{\lambda}_n\mathbf M_n\right)\mathbf X_n\right]^{-1}\mathbf X^\top_n\left(\mathbf I_n - \widehat{\lambda}_n\mathbf M_n\right)^\top\left(\mathbf I_n - \widehat{\lambda}_n\mathbf M_n\right)\mathbf y_n, \\ & = \left(\mathbf X_n^\top \mathbf R_{n}^\top\mathbf R_{n}\mathbf X_n\right)^{-1} \mathbf X_n^\top \mathbf R_{n}^\top\mathbf R_{n}\mathbf y_n, \end{aligned} \] which is precisely the same as Equation 6.48.

6.4.2 Moment Conditions

The core idea behind a MOM estimator is to find a set of population moments equations that relate population moments to the parameters of interest. These population moments are then replaced by sample moments to derive consistent estimates for the parameters, such as $\lambda$, which can be used in Equation 6.48.

Given the DGP in Equation 6.47, we express the relationship as: \[ \boldsymbol\varepsilon_n = \mathbf u_n - \lambda_0\mathbf M_n\mathbf u_n, \] where $\boldsymbol\varepsilon_n$ denotes the idiosyncratic error and $\mathbf u_n$ is the regression error. The Kelejian and Prucha (1999)’s MOM estimator of $\lambda_0$ is based on these three moment conditions (see Section 6.3.1):

Definition 6.2 (Moment Conditions) Under homoskedasticity (Kelejian and Prucha 1999) the moment conditions are: \[ \begin{aligned} \mathbb E\left[n^{-1}\boldsymbol\varepsilon^\top_n \boldsymbol\varepsilon_n\right] &= \sigma^2_0, \\ \mathbb E\left[n^{-1}\boldsymbol\varepsilon^\top_n\mathbf M_n^\top\mathbf M_n\boldsymbol\varepsilon_n\right] &= \frac{\sigma^2_0}{n}\operatorname{tr}\left(\mathbf M^\top_n\mathbf M_n\right),\\ \mathbb E\left[n^{-1}\boldsymbol\varepsilon^\top_n \mathbf M_n \boldsymbol\varepsilon_n \right] & = 0. \end{aligned} \]

To implement the moment conditions, we need to express the conditions on $\boldsymbol\varepsilon_n$ in terms of $\mathbf u_n$, since $\boldsymbol\varepsilon_n$ is not observed. Given that $\mathbf u_n = \lambda \mathbf M_n\mathbf u_n + \boldsymbol\varepsilon_n$, it follows that $\boldsymbol\varepsilon_n = \mathbf u_n - \lambda_0\mathbf M_n\mathbf u_n$, i.e., the spatially filtered regression error terms. This leads to the following relationships: \[ \begin{aligned} \boldsymbol\varepsilon_n^\top\boldsymbol\varepsilon_n &= (\mathbf u_n-\lambda_0\mathbf M_n\mathbf u_n)^\top (\mathbf u_n-\lambda_0\mathbf M_n\mathbf u_n) \\ &= \mathbf u_n^\top\mathbf u_n -2\lambda_0\mathbf u_n^\top\mathbf M_n\mathbf u_n +\lambda_0^2\mathbf u_n^\top\mathbf M_n^\top\mathbf M_n\mathbf u_n. \end{aligned} \tag{6.49}\]

\[ \begin{aligned} \boldsymbol\varepsilon_n^\top\mathbf M_n^\top\mathbf M_n\boldsymbol\varepsilon_n &= (\mathbf u_n-\lambda_0\mathbf M_n\mathbf u_n)^\top \mathbf M_n^\top\mathbf M_n (\mathbf u_n-\lambda_0\mathbf M_n\mathbf u_n) \\ &= \mathbf u_n^\top\mathbf M_n^\top\mathbf M_n\mathbf u_n -2\lambda_0\mathbf u_n^\top\mathbf M_n^\top\mathbf M_n^2\mathbf u_n \\ &\quad+ \lambda_0^2\mathbf u_n^\top (\mathbf M_n^2)^\top\mathbf M_n^2 \mathbf u_n. \end{aligned} \tag{6.50}\]

\[ \begin{aligned} \boldsymbol\varepsilon_n^\top\mathbf M_n\boldsymbol\varepsilon_n &= (\mathbf u_n-\lambda_0\mathbf M_n\mathbf u_n)^\top \mathbf M_n (\mathbf u_n-\lambda_0\mathbf M_n\mathbf u_n) \\ &= \mathbf u_n^\top\mathbf M_n\mathbf u_n -\lambda_0\mathbf u_n^\top (\mathbf M_n^2+\mathbf M_n^\top\mathbf M_n) \mathbf u_n \\ &\quad+ \lambda_0^2\mathbf u_n^\top\mathbf M_n^\top\mathbf M_n^2\mathbf u_n. \end{aligned} \tag{6.51}\]

Defining $\mathbf u_{L} = \mathbf M_n\mathbf u_n$, $\mathbf u_{LL} = \mathbf M_n\mathbf M_n\mathbf u_n$, ⁶ we take expectations in Equation 6.49 under the assumption of homoskedasticity: \[ \begin{aligned} \mathbb E\left[\boldsymbol\varepsilon^\top_n \boldsymbol\varepsilon_n\right] & = \mathbb E\left[\mathbf u^\top_n\mathbf u_n\right] - 2\lambda_0\mathbb E\left[\mathbf u^\top_n\mathbf M_n\mathbf u_n\right] + \lambda^2_0\mathbb E\left[\mathbf u^\top_n\mathbf M^\top_n\mathbf M_n\mathbf u_n\right], \\ \sigma^2_0& = \frac{1}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_n\right] - \lambda_0 \frac{2}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_L\right] + \lambda^2_0\frac{1}{n}\mathbb E\left[\mathbf u_L^\top\mathbf u_L\right] \quad \mbox{since $\mathbb E\left[n^{-1}\boldsymbol\varepsilon^\top_n \boldsymbol\varepsilon_n\right] = \sigma^2_0$},\\ 0 &= \sigma^2_0 - \frac{1}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_n\right] + \lambda_0 \frac{2}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_L\right] - \lambda^2_0\frac{1}{n}\mathbb E\left[\mathbf u_L^\top\mathbf u_L\right],\\ 0 &= \lambda_0 \frac{2}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_L\right] - \lambda^2_0\frac{1}{n}\mathbb E\left[\mathbf u_L^\top\mathbf u_L\right] + \sigma^2_0 - \frac{1}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_n\right], \\ 0 & = \begin{pmatrix} \frac{2}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_L\right] & - \frac{1}{n}\mathbb E\left[\mathbf u_L^\top\mathbf u_L\right] & 1 \end{pmatrix} \begin{pmatrix} \lambda_0 \\ \lambda^2_0 \\ \sigma^2_0 \end{pmatrix} - \frac{1}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_n\right]. \end{aligned} \tag{6.52}\]

Similarly, we derive two additional moment conditions: \[ 0 = \begin{pmatrix} \dfrac{2}{n}\mathbb E(\mathbf u_{LL}^\top\mathbf u_L) & -\dfrac{1}{n}\mathbb E(\mathbf u_{LL}^\top\mathbf u_{LL}) & \dfrac{1}{n}\operatorname{tr}(\mathbf M_n^\top\mathbf M_n) \end{pmatrix} \begin{pmatrix} \lambda_0\\ \lambda_0^2\\ \sigma_0^2 \end{pmatrix} - \frac{1}{n}\mathbb E(\mathbf u_L^\top\mathbf u_L). \tag{6.53}\]

\[ 0 = \begin{pmatrix} \dfrac{1}{n}\mathbb E(\mathbf u_n^\top\mathbf u_{LL}+\mathbf u_L^\top\mathbf u_L) & -\dfrac{1}{n}\mathbb E(\mathbf u_L^\top\mathbf u_{LL}) & 0 \end{pmatrix} \begin{pmatrix} \lambda_0\\ \lambda_0^2\\ \sigma_0^2 \end{pmatrix} - \frac{1}{n}\mathbb E(\mathbf u_n^\top\mathbf u_L). \tag{6.54}\]

At this point, we have three equations and three unknowns: $\lambda_0$, $\lambda_0^2$, and $\sigma_0^2$. Consider the following three-equations system implied by Equation 6.52, Equation 6.53 and Equation 6.54: \[ \boldsymbol\Gamma_n\boldsymbol\alpha_0 = \boldsymbol\gamma_n, \tag{6.55}\] where $\boldsymbol\Gamma_n$ is given in Equation 6.56, and $\boldsymbol\alpha_0 = (\lambda_0, \lambda^2_0, \sigma^2_0)$.⁷ If $\boldsymbol\Gamma_n$ where known, Assumption 15 (Identification) implies that Equation 6.55 determines $\boldsymbol\alpha_0$ as: \[ \boldsymbol\alpha_0 = \boldsymbol\Gamma_n^{-1}\boldsymbol\gamma_n, \] where: \[ \boldsymbol\Gamma_n = \begin{pmatrix} \frac{2}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_L\right] & -\frac{1}{n}\mathbb E\left[\mathbf u_L^\top\mathbf u_L\right] & 1 \\ \frac{2}{n}\mathbb E\left[\mathbf u^\top_{LL}\mathbf u_L\right] & -\frac{1}{n}\mathbb E\left[\mathbf u_{LL}^\top\mathbf u_{LL}\right] & \frac{1}{n} \operatorname{tr}(\mathbf M^\top_n\mathbf M_n)\\ \frac{1}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_{LL} + \mathbf u_L^\top\mathbf u_L\right] & -\frac{1}{n}\mathbb E\left[\mathbf u_{L}^\top\mathbf u_{LL}\right] & 0 \end{pmatrix}, \tag{6.56}\] and \[ \boldsymbol\gamma_n = \begin{pmatrix} \frac{1}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_n\right] \\ \frac{1}{n}\mathbb E\left[\mathbf u^\top_L\mathbf u_L\right] \\ \frac{1}{n}\mathbb E\left[\mathbf u^\top_n\mathbf u_L\right] \end{pmatrix}.. \]

Now, we express the moment conditions $\boldsymbol\gamma_n = \boldsymbol\Gamma_n\boldsymbol\alpha_n$ as sample averages in observables spatial lags of OLS residuals: \[ \mathbf g_n = \mathbf G_n\boldsymbol\alpha_n + \boldsymbol\upsilon_n(\lambda_n, \sigma^2_n), \tag{6.57}\] where $\boldsymbol\upsilon_n(\lambda_n, \sigma^2_n)$ can be viewed as a vector of residuals and $\mathbf G_n$ is an $3\times 3$ matrix given by \[ \mathbf G_n = \begin{pmatrix} \frac{2}{n}\widehat{\mathbf u}^\top_n\widehat{\mathbf u}_L & -\frac{1}{n}\widehat{\mathbf u}_L^\top\widehat{\mathbf u}_L & 1\\ \frac{2}{n}\widehat{\mathbf u}^\top_{LL}\widehat{\mathbf u}_L & -\frac{1}{n}\widehat{\mathbf u}_{LL}^\top\widehat{\mathbf u}_{LL} & \frac{1}{n} \operatorname{tr}(\mathbf M^\top_n\mathbf M_n)\\ \frac{1}{n}\left[\widehat{\mathbf u}^\top_n\widehat{\mathbf u}_{LL} + \widehat{\mathbf u}_L^\top\widehat{\mathbf u}_L\right] & -\frac{1}{n}\widehat{\mathbf u}_{L}^\top\widehat{\mathbf u}_{LL} & 0 \end{pmatrix}, \tag{6.58}\] and \[ \mathbf g_n = \begin{pmatrix} \frac{1}{n}\widehat{\mathbf u}^\top_n\widehat{\mathbf u}_n \\ \frac{1}{n}\widehat{\mathbf u}^\top_L\widehat{\mathbf u}_L \\ \frac{1}{n}\widehat{\mathbf u}^\top_n\widehat{\mathbf u}_L \end{pmatrix}. \tag{6.59}\]

Equation 6.57 can be thought as an OLS regression where (Kelejian and Prucha 1998): \[ \widetilde{\boldsymbol\alpha}_n = \mathbf G_n^{-1}\mathbf g_n. \tag{6.60}\]

However, the estimator in Equation 6.60 is based on an overparameterization in the sense that it does not use the information that the second element of $\boldsymbol\alpha_n$, $\lambda^2_n$, is the squared of the first element. To address this issue, Kelejian and Prucha (1998) and Kelejian and Prucha (1999) define the MOM estimator for $\lambda$ and $\sigma^2$ as the nonlinear least square estimator corresponding to Equation 6.57:⁸ \[ (\widehat{\lambda}_{NLS, n}, \widehat{\sigma}^2_{NLS, n}) = \operatorname*{argmin} \left\lbrace \boldsymbol\upsilon_n(\lambda_n, \sigma^2_n)^\top\boldsymbol\upsilon_n(\lambda_n, \sigma^2_n): \lambda_n \in [-a, a], \sigma^2_n\in [0, b]\right\rbrace \tag{6.61}\]

Note that $(\widehat{\lambda}_{NLS, n}, \widehat{\sigma}^2_{NLS, n})$ are defined as the minimizers of \[ Q_n(\lambda_n, \lambda^2_n, \sigma^2_n) = \left[\mathbf g_n - \mathbf G_n \begin{pmatrix}\lambda_n \\ \lambda ^2_n \\ \sigma^2_n\end{pmatrix}\right]^\top \left[\mathbf g_n - \mathbf G_n \begin{pmatrix}\lambda _n\\ \lambda ^2_n \\ \sigma^2_n\end{pmatrix}\right]. \tag{6.62}\]

We provide the following assumptions.

Assumption 13 (Bounded Matrices)

The row and column sums of the matrices $\mathbf M_n$ and $(\mathbf I_n - \lambda_0\mathbf M_n)$ are bounded uniformly in absolute value.

Assumption 14 (Residuals)

Let $\widetilde{u}_{i, n}$ denote the $i$-th element of $\widetilde{\mathbf u}_n$. We then assume that \[ \widetilde{u}_{i, n} - u_{i,n} = \mathbf d_{i,n}\boldsymbol\Delta_n, \] where $\mathbf d_{i, n}$ and $\boldsymbol\Delta_n$ are $1\times p$ and $p\times 1$ dimensional random vectors. Let $d_{ij,n}$ be the $j$th element of $\mathbf d_{i, n}$. Then, we assume that for some $\delta>0$, $\mathbb E\left|d_{ij,n}\right|^{2 + \delta}\leq c_{d} < \infty$, where $c_d$ does not depend on $n$, and that \[ \sqrt{n}\left\lVert \boldsymbol\Delta_n\right\rVert = O_p(1). \]

The vector $\mathbf d_{i, n}$ denotes the $i$-th row of the regressor matrix, and $\boldsymbol\Delta_n$ denotes de difference between the parameter estimator and the true parameter values. This assumption should be satisfied for most cases in which $\widetilde{\mathbf u}_n$ is based on $\sqrt{n}$-consistent estimators of the regression coefficients (non-linear OLS, linear OLS, S2SLS). Assumption 14 comes from Kelejian and Prucha (2010) and is a bit stronger than the corresponding assumption in Kelejian and Prucha (1999).

Assumption 15 (Identification)

Let $\boldsymbol\Gamma_n$ be the matrix in Equation 6.56. The smallest eigenvalues of $\boldsymbol\Gamma^\top_n\boldsymbol\Gamma_n$ is bounded away from zero, that is, $\omega_{min} (\boldsymbol\Gamma^\top_n\boldsymbol\Gamma_n) \geq \omega_* > 0$, where $\omega_*$ may depend on $\lambda$ and $\sigma^2$

The following theorem establishes the consistency of the NLS estimator.

Theorem 6.6 (Consistency) Let $(\widehat{\lambda}_{NLS, n}, \widehat{\sigma}^2_{NLS, N})$ given by: \[ (\widehat{\lambda}_{NLS, n}, \widehat{\sigma}^2_{NLS, N}) = \operatorname*{argmin} \left\lbrace \boldsymbol\upsilon_n(\lambda, \sigma^2)^\top\boldsymbol\upsilon_n(\lambda, \sigma^2): \rho \in [-a, a], \sigma^2\in [0, b]\right\rbrace \]

Then, given Assumption 1 (Heterokedastic errors), Assumption 12 (Weight Matrix $\mathbf M_n$), 1 (Bounded Matrices), Assumption 14 (Residuals), and Assumption 15 (Identification), \[ (\widehat{\lambda}_{NLS, n}, \widehat{\sigma}^2_{NLS, n}) \xrightarrow{p} (\lambda_0, \sigma^2_0)\quad \mbox{as $n\to \infty$} \]

An important remark is that Theorem 6.6 establishes the consistency of the NLS estimates but does not provide information about the asymptotic distribution of $\widehat{\lambda}_{NLS, n}$.

Proof. Sketch or proof for NLS estimator of $\widehat{\lambda}_n$. The proof is based on Kelejian and Piras (2017) and consist into two steps. First, we prove consistency of $\widehat{\lambda}_n$ for the OLS estimate of $\lambda_0$—which is more simple—and assuming that the vector $\mathbf u_n$ is observed. We then show that $\mathbf u_n$ can be replaced in the GM estimator for $\lambda$ by $\widehat{\mathbf u}_n$. For a more general proof see Kelejian and Prucha (1998); Kelejian and Prucha (1999).

Assuming that $\mathbf u_n$ is observed. Recall that in Equation 6.57 the sample moments are based on the estimated $\widehat{\mathbf u}_n$. But, if $\mathbf u_n$ were observed, then we would use the following sample moments: \[ \mathbf g_n^* = \mathbf G_n^*\boldsymbol\alpha_0 \] where \[ \begin{aligned} \mathbf G_n^* & = \begin{pmatrix} \frac{2}{n}\mathbf u^\top_n\mathbf u_L & -\frac{1}{n}\mathbf u_L^\top\mathbf u_L & 1 \\ \frac{2}{n}\mathbf u^\top_{LL}\mathbf u_L & -\frac{1}{n}\mathbf u_{LL}^\top\mathbf u_{LL} & \frac{1}{n} \operatorname{tr}(\mathbf M^\top_n\mathbf M_n)\\ \frac{1}{n}\left[\mathbf u^\top\mathbf u_{LL} + \mathbf u_L^\top\mathbf u_L\right] & -\frac{1}{n}\mathbf u_{L}^\top\mathbf u_{LL} & 0 \end{pmatrix}, \\ \mathbf g_n^*&= \begin{pmatrix} \frac{1}{n}\mathbf u^\top_n\mathbf u_n \\ \frac{1}{n}\mathbf u^\top_L\mathbf u_L \\ \frac{1}{n}\mathbf u^\top_n\mathbf u_L \end{pmatrix} \end{aligned} \]

Recall that: \[ \begin{aligned} \mathbf u_n & = \left(\mathbf I_n - \lambda_0 \mathbf M_n\right)^{-1}\boldsymbol \varepsilon_n \\ \mathbf u_L & = \mathbf M_n \left(\mathbf I_n - \lambda_0 \mathbf M_n\right)^{-1}\boldsymbol \varepsilon_n \\ \mathbf u_{LL} & = \mathbf M_n \mathbf M_n\left(\mathbf I_n - \lambda_0 \mathbf M_n\right)^{-1}\boldsymbol \varepsilon_n \end{aligned} \] and first and second column of $\mathbf G^*_n$ are quadratic forms of $\boldsymbol\varepsilon_n$. Since $\mathbf M_n$ is uniformly bounded then, using Lemma of consistency of quadratic forms Lemma 3.13, we can state that: \[ \mathbf G^*_n\xrightarrow{p} \boldsymbol\Gamma_n. \]

Also: \[ \begin{aligned} \operatorname{plim} \mathbf g_n^* &= \operatorname{plim} \mathbf G^*_n\boldsymbol\alpha_0, \\ & = \boldsymbol\Gamma_n\boldsymbol\alpha_0. \end{aligned} \]

If $\mathbf u_n$ would be observed, a linear GMM estimator for $\lambda_0$, say $\widetilde{\lambda}$, would be the first element of the least squared estimator $\widetilde{\boldsymbol\alpha}_n$, namely: \[ \widetilde{\boldsymbol\alpha}_n= \mathbf G_n^{-1*}\mathbf g_n^*, \] since $\mathbf G_n^{*}$ is a $3\times 3$ matrix which is nonsingular. Thus, using our previous results: \[ \operatorname{plim} \widetilde{\boldsymbol\alpha}_n=\operatorname{plim} \mathbf G_n^{-1*} \operatorname{plim} \mathbf g_n^* = \boldsymbol\Gamma_n^{-1}\boldsymbol\gamma_n = \boldsymbol\alpha_0. \tag{6.63}\]
Replacing $\mathbf u_n$ by $\widehat{\mathbf u}_n$. Now consider the estimator of $\boldsymbol\alpha_0$ based on $\widehat{\mathbf u}_n$. The OLS estimator is consistent and can be expressed as: \[ \widetilde{\boldsymbol\beta}_n = \boldsymbol\beta_0 + \Delta_n, \quad \Delta_n\xrightarrow{p} \mathbf 0. \]

Then, the OLS estimator $\widehat{\mathbf u}_n$ is:⁹ \[ \begin{aligned} \widehat{\mathbf u}_n & = \mathbf y_n - \mathbf X_n\widehat{\boldsymbol\beta}_n, \\ & = \mathbf y_n - \mathbf X_n\left(\boldsymbol\beta_0 + \Delta_n\right), \\ & = \mathbf y_n - \mathbf X_n\boldsymbol\beta_0 - \mathbf X\Delta_n, \\ & = \mathbf u_n - \mathbf X_n\Delta_n. \end{aligned} \]

Note that, with the exception of the constants in the third column of $\mathbf G^*_n$, every element of $\mathbf G^*_n$ and $\mathbf g_n^*$ can be expressed as a quadratic terms of the form $\boldsymbol\varepsilon^\top_n\mathbf S_n\boldsymbol\varepsilon_n/n$, where $\mathbf S_n$ is an $n\times n$ matrix whose row and columns are uniformly bounded in absolute value given our
1. For example: \[ \frac{1}{n}\mathbf u^\top_n\mathbf u_L = \frac{1}{n} \boldsymbol\varepsilon^{\top}_n\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^{-1\top}\mathbf M_n\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n = \frac{1}{n}\boldsymbol\varepsilon^\top_n\mathbf S_n\boldsymbol\varepsilon_n. \]
Then: \[ \begin{aligned} \frac{\widehat{\mathbf u}^\top_n\mathbf S_n\widehat{\mathbf u}_n}{n} & = \frac{\left(\mathbf u_n - \mathbf X_n\Delta_n \right)^\top\mathbf S_n\left(\mathbf u_n - \mathbf X_n\Delta_n \right)}{n}, \\ & = \frac{\mathbf u^\top_n\mathbf S_n\mathbf u_n}{n}- \frac{2\Delta_n^\top\mathbf X^\top_n\mathbf S_n\mathbf u_n}{n}+\frac{\Delta_n^\top\mathbf X^\top_n\mathbf S_n\mathbf X_n\Delta_n}{n}. \end{aligned} \]

We need to show that: \[ \begin{aligned} \frac{2\Delta_n^\top\mathbf X^\top_n\mathbf S_n\mathbf u_n}{n} & \xrightarrow{p} \mathbf 0, \\ \frac{\Delta_n^\top\mathbf X^\top_n\mathbf S_n\mathbf X_n\Delta_n}{n} & \xrightarrow{p} \mathbf 0, \end{aligned} \] so that we can conclude that: \[ \frac{\widehat{\mathbf u}^\top_n\mathbf S_n\widehat{\mathbf u}_n}{n} \xrightarrow{p} \frac{1}{n}\mathbf u^\top_n\mathbf S_n\mathbf u_n, \] and finally say that: \[ \mathbf g_n\xrightarrow{p} \mathbf g_n^*\xrightarrow{p} \boldsymbol\gamma_n, \quad \mathbf G_n\xrightarrow{p} \mathbf G^*_n\xrightarrow{p} \boldsymbol\Gamma_n. \]

Given @eq-iv-gmm-consisten-error-gm,
consistency is proved.

We can also derive the MOM estimation using a GMM approach without weighting the moments: \[ \min_{\boldsymbol\theta \in \boldsymbol\Theta}\; Q_n(\boldsymbol\theta_n) = \mathbf g_n(\boldsymbol\theta)^\top \mathbf g_n(\boldsymbol\theta). \]

Using the moment conditions under homoskedasticity results in the following empirical moments: \[ \mathbf g_n(\boldsymbol\theta) = \begin{pmatrix} \boldsymbol\varepsilon_n^\top\boldsymbol\varepsilon_n - n\widehat{\sigma}^2 \\ \boldsymbol\varepsilon_n^\top\mathbf M_n^\top\mathbf M_n\boldsymbol\varepsilon_n - \widehat{\sigma}^2\operatorname{tr}(\mathbf M_n^\top\mathbf M_n) \\ \boldsymbol\varepsilon_n^\top\mathbf M_n\boldsymbol\varepsilon_n \end{pmatrix} = \begin{pmatrix} \mathbf u_n^\top\mathbf R_n^\top\mathbf R_n\mathbf u_n - n\widehat{\sigma}^2 \\ \mathbf u_n^\top\mathbf R_n^\top\mathbf M_n^\top\mathbf M_n\mathbf R_n\mathbf u_n - \widehat{\sigma}^2\operatorname{tr}(\mathbf M_n^\top\mathbf M_n) \\ \mathbf u_n^\top\mathbf R_n^\top\mathbf M_n\mathbf R_n\mathbf u_n \end{pmatrix} \] where $\mathbf u_n$ are replaced by least squares residuals.

6.4.3 Feasible Generalized Least Squares Estimator

In Section 6.4.1 we established that the GLS estimator is given by: \[ \boldsymbol\beta_{GLS}(\lambda_0) =\left[\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf y_n, \tag{6.64}\] where $\boldsymbol\Omega(\lambda_0) = (\mathbf I_n - \lambda_0\mathbf M_n)^{-1}(\mathbf I_n - \lambda_0\mathbf M^\top_n)^{-1}$. The MOM procedure provides a consistent estimate for $\lambda_0$, which can be used to obtain the FGLS estimator: \[ \boldsymbol\beta_{FGLS}(\widehat{\lambda}_n) =\left[\mathbf X^\top_n\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}\mathbf X_n\right]^{-1}\mathbf X^\top_n\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}\mathbf y_n. \tag{6.65}\]

The following theorem provides the identification conditions for the Spatial FGLS estimator.

Assumption 16 (Limiting Behavior)

The elements of $\mathbf X_n$ are non-stochastic and bounded in absolute value by $c_X, 0 < c_X < \infty$. Also, $\mathbf X_n$ has full rank, and the matrix $\mathbf Q_X = \lim_{n\to\infty}n^{-1}\mathbf X^\top_n\mathbf X_n$ is finite and nonsingular. Furthermore, the matrices \[ \mathbf Q_X(\lambda_0) = \lim_{n\to \infty} n^{-1}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n \] are finite and nonsingular for all $\left|\lambda_0\right| < 1$

The following theorem proposes the asymptotic distribution for the SFGLS Estimator:

Theorem 6.7 (Asymptotic Properties of FGLS Estimator) If Assumption 11 (Homoskedastic errors), Assumption 12 (Weight Matrix), 1 (Bounded Matrices), and Assumption 16 (Limiting Behavior) hold:

The true GLS estimator $\widehat{\boldsymbol\beta}_{GLS}$ is a consistent estimator for $\boldsymbol\beta_0$, and \[ \sqrt{n}\left(\widehat{\boldsymbol\beta}_{GLS} - \boldsymbol\beta_0\right)\xrightarrow{d} \mathcal N\left(\mathbf 0, \sigma^2 \mathbf Q_X(\lambda_0)^{-1}\right) \]
Let $\widehat{\lambda}_n$ be a consistent estimator for $\lambda_0$. Then the true GLS estimator $\widehat{\boldsymbol\beta}_{GLS}$ and the feasible GLS estimator $\widehat{\boldsymbol\beta}_{FGLS}$ have the same asymptotic distribution.
Suppose further that $\widehat{\sigma}^2_n$ is a consistent estimator for $\sigma^2$. Then $\widehat{\sigma}^2_n\left[n^{-1}\mathbf X^\top_n\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}\mathbf X_n\right]^{-1}$ is a consistent estimator for $\sigma^2 \mathbf Q_X(\lambda_0)^{-1}$.

Note that Theorem 6.7 assumes the existence of a consistent estimator of $\lambda_0$ and $\sigma^2_0$. It can be shown that the OLS estimator: \[ \widehat{\boldsymbol\beta}_n = \left(\mathbf X^\top_n\mathbf X_n\right)^{-1}\mathbf X^\top_n\mathbf y_n, \] is $\sqrt{n}$-consistent. Thus, the OLS residuals $\widetilde{u}_i = y_i - \mathbf x_i^\top\widehat{\boldsymbol\beta}_n$ satisfy Assumption 14 with $d_{i, n} = \left|\mathbf x_i\right|$ and $\Delta_n = \widehat{\boldsymbol\beta}_n -\boldsymbol\beta_0$. Thus, OLS residuals can be used to obtain consistent estimators of $\lambda_0$ and $\sigma^2_0$.

Then, the SFGLS estimator is given by \[ \widehat{\boldsymbol\beta}_{FGLS} = \left[\mathbf X^\top(\widetilde{\lambda})\mathbf X(\widetilde{\lambda})\right]^{-1}\mathbf X^\top(\widetilde{\lambda})\mathbf y(\widetilde{\lambda}), \] where: \[ \mathbf X(\widetilde{\lambda}) = (\mathbf I_n-\widetilde{\lambda}_n\mathbf M_n)\mathbf X_n, \quad \mathbf y(\widetilde{\lambda}) = (\mathbf I_n-\widetilde{\lambda}_n\mathbf M_n)\mathbf y_n. \]

The variance covariance matrix of $\widehat{\boldsymbol\beta}_{FGLS}$ is estimated as: \[ \widehat{\operatorname{Var}}\left(\widehat{\boldsymbol\beta}_{FGLS}\right) = \widehat{\sigma}^2_n\left[\mathbf X^\top(\widetilde{\lambda})\mathbf X(\widetilde{\lambda})\right]^{-1}, \] where: \[ \begin{aligned} \widehat{\sigma}^2_n & = \frac{1}{n}\widehat{\boldsymbol\varepsilon}^\top(\widetilde{\lambda})\widehat{\boldsymbol\varepsilon}(\widetilde{\lambda}), \\ \widehat{\boldsymbol\varepsilon}(\widetilde{\lambda}) & = \mathbf y(\widetilde{\lambda}) - \mathbf X(\widetilde{\lambda})\widehat{\boldsymbol\beta}_{FGLS} = (\mathbf I_n-\widetilde{\lambda}_n\mathbf M_n)\widehat{\mathbf u}_n, \\ \widehat{\mathbf u}_n & = \mathbf y_n-\mathbf X_n\widehat{\boldsymbol\beta}_{FGLS}. \end{aligned} \]

Proof. Sketch of Proof of Theorem 6.7. We first prove part (a). Recall that the GLS and FGLS estimators are given by: \[ \begin{aligned} \widehat{\boldsymbol\beta}_{GLS} & = \left[\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf y_n, \\ \widehat{\boldsymbol\beta}_{FGLS} & = \left[\mathbf X^\top_n\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}\mathbf X_n\right]^{-1}\mathbf X_n^\top\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}\mathbf y_n. \end{aligned} \]

Since $\mathbf y_n = \mathbf X_n\boldsymbol\beta_0 + \mathbf u_n = \mathbf X_n\boldsymbol\beta_0 + \left(\mathbf I_n-\lambda_0\mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n$, the sampling error of $\widehat{\boldsymbol\beta}_{GLS}$ is, \[ \begin{aligned} \widehat{\boldsymbol\beta}_n & = \boldsymbol\beta_0 + \left[\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf u_n, \\ \widehat{\boldsymbol\beta}_n - \boldsymbol\beta_0 & = \left[\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1}\mathbf X_n^\top\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^\top\left(\mathbf I_n - \lambda_0\mathbf M_n\right)\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n, \\ \widehat{\boldsymbol\beta}_n - \boldsymbol\beta_0 & = \left[\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1}\mathbf X^\top_n\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^\top\boldsymbol\varepsilon_n, \\ \sqrt{n}(\widehat{\boldsymbol\beta}_n - \boldsymbol\beta_0) & = \left[\frac{1}{n}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1}\frac{1}{\sqrt{n}}\mathbf A^\top_0\boldsymbol\varepsilon_n, \end{aligned} \] where $\mathbf A_0 = \left(\mathbf I_n - \lambda_0\mathbf M_n\right)\mathbf X_n$. By Assumption 16 (Limiting Behavior): \[ \frac{1}{n}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n \to \mathbf Q_X(\lambda_0). \]

Since $\mathbf Q_X$ is not singular, by continuous mapping theorem: \[ \left[\frac{1}{n}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1} \to \mathbf Q_X^{-1}(\lambda_0). \]

Because the entries of $\mathbf X_n$ are uniformly bounded and the absolute row sums of $\mathbf M_n$ are uniformly bounded, the entries of $\mathbf A_0=(\mathbf I_n-\lambda_0\mathbf M_n)\mathbf X_n$ are uniformly bounded. Therefore, Theorem 3.21 implies that: \[ \frac{1}{\sqrt{n}}\mathbf A^\top_0\boldsymbol\varepsilon_n \xrightarrow{d} \mathcal N\left(\mathbf 0, \lim_{n\to\infty}n^{-1}\sigma^2\mathbf A^\top_0\mathbf A_0\right), \] where $\lim_{n\to\infty}n^{-1}\sigma^2\mathbf A^\top_0\mathbf A_0 = \sigma^2\lim_{n\to\infty}n^{-1}\mathbf X^\top_n\left(\mathbf I_n - \lambda_0\mathbf M_n\right)^\top\left(\mathbf I_n - \lambda_0\mathbf M_n\right)\mathbf X_n=\sigma^2\mathbf Q_X(\lambda_0)$. Consequently: \[ \begin{aligned} \sqrt{n}(\widehat{\boldsymbol\beta}_n - \boldsymbol\beta_0) & = \underbrace{\left[\frac{1}{n}\mathbf X^\top_n\boldsymbol\Omega(\lambda_0)^{-1}\mathbf X_n\right]^{-1}}_{\to \mathbf Q_X^{-1}(\lambda_0)}\underbrace{\frac{1}{\sqrt{n}}\mathbf A_0^\top\boldsymbol\varepsilon_n}_{\xrightarrow{d} \mathcal N(\mathbf 0, \sigma^2\mathbf Q_X(\lambda_0))} \\ & \xrightarrow{d} \mathcal N\left(\mathbf 0, \mathbf Q_X^{-1}(\lambda_0)\sigma^2\mathbf Q_X(\lambda_0)\mathbf Q_X^{-1}(\lambda_0)^\top\right), \\ & \xrightarrow{d} \mathcal N\left(\mathbf 0, \sigma^2\mathbf Q_X^{-1}(\lambda_0)\right). \end{aligned} \]

The asymptotic normality result implies $\widehat{\boldsymbol\beta}_{GLS}-\boldsymbol\beta_0=O_p(n^{-1/2})$ and therefore establishes consistency. To prove part 2, it is enough to show that: \[ \sqrt{n}(\widehat{\boldsymbol\beta}_{GLS} - \widehat{\boldsymbol\beta}_{FGLS}) \xrightarrow{p} \mathbf 0. \]

Recall that $\operatorname{plim}(X_n- Y_n) = 0$ implies that the random variables $X_n$ and $Y_n$ have the same asymptotic distribution. Following Kelejian and Prucha (1999), it suffices to show that \[ \frac{1}{n}\mathbf X^\top_n\left[\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}- \boldsymbol\Omega(\lambda_0)^{-1}\right]\mathbf X_n\xrightarrow{p} \mathbf 0 \tag{6.66}\] and \[ \frac{1}{\sqrt{n}}\mathbf X^\top_n\left[\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}- \boldsymbol\Omega(\lambda_0)^{-1}\right]\mathbf u_n\xrightarrow{p} \mathbf 0. \]

To derive the expression for $\boldsymbol\Omega(\widehat{\lambda}_n)^{-1} - \boldsymbol\Omega(\lambda_0)^{-1}$, we start by expanding both matrices in terms of $\lambda_0$ and $\widehat{\lambda}_n$.

Given that: \[ \boldsymbol\Omega(\lambda_0)^{-1} = (\mathbf I_n - \lambda_0\mathbf M_n^\top)(\mathbf I_n - \lambda_0 \mathbf M_n), \] and \[ \boldsymbol\Omega(\widehat{\lambda}_n)^{-1} = (\mathbf I_n - \widehat{\lambda}_n \mathbf M_n^\top)(\mathbf I_n - \widehat{\lambda}_n \mathbf M_n), \] we subtract these two expressions to obtain: \[ \boldsymbol\Omega(\widehat{\lambda}_n)^{-1} - \boldsymbol\Omega(\lambda_0)^{-1} = \left[ (\mathbf I_n - \widehat{\lambda}_n \mathbf M_n^\top)(\mathbf I_n - \widehat{\lambda}_n \mathbf M_n) - (\mathbf I_n - \lambda_0\mathbf M_n^\top)(\mathbf I_n - \lambda_0\mathbf M_n) \right]. \]

Expanding yields: \[ (\mathbf I_n - \lambda \mathbf M_n^\top)(\mathbf I_n - \lambda \mathbf M_n) = \mathbf I_n - \lambda \mathbf M_n^\top - \lambda \mathbf M_n +\lambda^2\mathbf M_n^\top\mathbf M_n, \]

By subtracting these expansions, we obtain the first-order difference: \[ \begin{aligned} \boldsymbol\Omega(\widehat{\lambda}_n)^{-1} - \boldsymbol\Omega(\lambda_0)^{-1} & = \mathbf I_n - \widehat{\lambda}_n \mathbf M_n^\top - \widehat{\lambda}_n \mathbf M_n+\widehat{\lambda}_n^2\mathbf M_n^\top\mathbf M_n \\ & - \left(\mathbf I_n - \lambda_0 \mathbf M_n^\top - \lambda_0 \mathbf M_n +\lambda^2_0\mathbf M_n^\top\mathbf M_n\right), \\ & = (\lambda_0 - \widehat{\lambda}_n)(\mathbf M_n + \mathbf M_n^\top) + (\widehat{\lambda}_n^2-\lambda_0^2)\mathbf M_n^\top\mathbf M_n. \end{aligned} \]

Using the uniform bounds on the absolute row and column sums of the matrices involved, \[ \frac{1}{n}\mathbf X^\top_n\left[\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}- \boldsymbol\Omega(\lambda_0)^{-1}\right]\mathbf X_n = \underbrace{(\lambda_0 - \widehat{\lambda}_n)}_{\xrightarrow{p} 0}\underbrace{n^{-1}\mathbf X^\top_n(\mathbf M_n + \mathbf M^\top_n)\mathbf X_n}_{O(1)} + \underbrace{(\widehat{\lambda}_n^2-\lambda_0^2)}_{\xrightarrow{p} 0}\underbrace{n^{-1}\mathbf X^\top_n\mathbf M^\top_n\mathbf M_n\mathbf X_n}_{O(1)}, \] where $(\lambda_0 - \widehat{\lambda}_n)=o_p(1)$ because $\widehat{\lambda}_n$ is consistent for $\lambda_0$, and: \[ \begin{aligned} \frac{1}{\sqrt{n}}\mathbf X^\top_n\left[\boldsymbol\Omega(\widehat{\lambda}_n)^{-1}- \boldsymbol\Omega(\lambda_0)^{-1}\right]\mathbf u_n & = \underbrace{(\lambda_0 - \widehat{\lambda}_n)}_{\xrightarrow{p} 0}\underbrace{n^{-1/2}\mathbf X^\top_n(\mathbf M_n + \mathbf M^\top_n)\mathbf u_n}_{O_p(1)} + \underbrace{(\widehat{\lambda}_n^2-\lambda_0^2)}_{\xrightarrow{p} 0}\underbrace{n^{-1/2}\mathbf X_n^\top\mathbf M_n^\top\mathbf M_n\mathbf u_n}_{O_p(1)}, \\ & = o_p(1)O_p(1) + o_p(1)O_p(1), \\ & = o_p(1) + o_p(1), \\ & = o_p(1), \\ & \xrightarrow{p} 0. \end{aligned} \]

To see that $n^{-1/2}\mathbf X^\top_n(\mathbf M_n + \mathbf M^\top_n)\mathbf u_n = O_p(1)$ note that \[ \begin{aligned} \mathbb E\left[n^{-1/2}\mathbf X^\top_n(\mathbf M_n + \mathbf M^\top_n)\mathbf u_n\right] &= 0, \\ \operatorname{Var}[n^{-1/2}\mathbf X^\top_n(\mathbf M_n + \mathbf M^\top_n)\mathbf u_n] &= n^{-1}\underbrace{\mathbf X_n^\top\underbrace{(\mathbf M_n + \mathbf M^\top_n)\boldsymbol\Omega_0(\mathbf M^\top_n + \mathbf M_n)}_{\text{uniformly bounded in absolute row and column sums}}\mathbf X_n}_{O(n)} = O(1). \end{aligned} \]

A similar result holds for $n^{-1/2}\mathbf X^\top_n\mathbf M^\top_n\mathbf M_n\mathbf u_n$.

Part 3 of the theorem follows from Equation 6.66 and the fact that $\widehat{\sigma}^2$ is a consistent estimator for $\sigma^2$.

A Feasible GLS (FGLS) can be obtained along with the following steps:

GLS (FGLS) Algorithm of SEM

The steps are the following:

First of all obtain a consistent estimate of $\boldsymbol\beta$, say $\widetilde{\boldsymbol\beta}$ using either OLS or NLS.
Use this estimate to obtain an estimate of $\mathbf u$, say $\widehat{\mathbf u}$,
Use $\widehat{\mathbf u}$, to estimate $\lambda$, say $\widehat{\lambda}$, using (Equation 6.61),
Estimate $\boldsymbol\beta$ using Equation 6.65

6.5 GMM Estimator for SEM

6.5.1 GMM Estimator Under Homoskedasticity

In this section, we provide the GMM estimator for the SEM model under homoskedasticity following Lee and Liu (2010) and Liu et al. (2010). It is important to note that both articles derive the OGMME for the SAC model, but Lee and Liu (2010) focuses on SAC model with higher orders.

Recall that the SEM model is given by: \[ \begin{aligned} \mathbf y_n & = \mathbf X\boldsymbol\beta_0 + \mathbf u_n \\ \mathbf u_n & =\lambda_0 \mathbf M_n\mathbf u_n + \boldsymbol\varepsilon_n, \end{aligned} \] where $\mathbf P_{jn}, j = 1,2$ are chosen from $\mathcal P_{1n}$, and $\epsilon_{ni}, i = 1, \ldots, n$, of $\boldsymbol\varepsilon_n$ are i.i.d. $(0, \sigma^2_0)$ with zero mean and variance $\sigma_0^2$. For this model, denote $\boldsymbol\varepsilon_n = \left(\mathbf I_n - \lambda_0\mathbf M_n\right)\mathbf u_n = \mathbf R_0\mathbf u_n$, where $\mathbf R_0 = \left(\mathbf I_n - \lambda_0\mathbf M_n\right)$, and $\mathbf u_n = \mathbf y_n - \mathbf X_n\boldsymbol\beta_0$. The reduced form equation of this model is \[ \mathbf y_n = \mathbf X_n\boldsymbol\beta_0 + \mathbf R_0^{-1}\boldsymbol\varepsilon_n. \tag{6.67}\]

Let $\boldsymbol\theta = (\boldsymbol\beta^\top,\lambda)^\top$. The GMM estimator of this model can be based on the following linear and quadratic empirical moments \[ \mathbf g_n(\boldsymbol\theta) = \begin{pmatrix} \mathbf X_n^\top \boldsymbol\varepsilon_n \\ \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \\ \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \end{pmatrix} = \begin{pmatrix} \mathbf X_n^\top\mathbf R_n\mathbf u_n \\ \mathbf u_n^\top\mathbf R_n^\top \mathbf P_{1n}^\top\mathbf R_n\mathbf u_n \\ \mathbf u_n^\top\mathbf R_n^\top \mathbf P_{2n}^\top\mathbf R_n\mathbf u_n \end{pmatrix} = \begin{pmatrix} \mathbf X_n^\top \\ \boldsymbol\varepsilon_n^\top\mathbf P_{1n}^\top \\ \boldsymbol\varepsilon_n^\top\mathbf P_{2n}^\top \end{pmatrix} \mathbf R_n\mathbf u_n, \] where $\mathbf R_n = (\mathbf I_n - \lambda \mathbf M_n)$. At $\boldsymbol\theta = \boldsymbol\theta_0$, the population moments are $\mathbb E\left[\mathbf g_n(\boldsymbol\theta_0)\right] = \mathbb E\left[\left(\mathbf X_n, \mathbf P_{1n}\boldsymbol\varepsilon_n, \mathbf P_{2n}\boldsymbol\varepsilon_n\right)^\top\boldsymbol\varepsilon_n\right] = \mathbf 0$, because under exogeneity $\mathbb E(\mathbf X_n^\top\boldsymbol\varepsilon_n) = \mathbf X_n^\top\mathbb E(\boldsymbol\varepsilon_n) = \mathbf 0$ and $\mathbb E(\boldsymbol\varepsilon_n^\top\mathbf P_{jn}\boldsymbol\varepsilon_n) = \sigma_0^2\operatorname{tr}(\mathbf P_{jn}) = 0$ for $j = 1, 2$.

The derivatives of $\mathbf g_n(\boldsymbol\theta)$ with respect to $\boldsymbol\beta$ and $\lambda$ are \[ \begin{aligned} \frac{\partial \mathbf g_n(\boldsymbol\theta)}{\partial \boldsymbol\theta ^\top} & = \begin{pmatrix} \mathbf X_n^\top\frac{\partial \boldsymbol\varepsilon_n}{\partial \boldsymbol\theta^\top} \\ \boldsymbol\varepsilon_n^\top\mathbf P_{1n}^s\frac{\partial \boldsymbol\varepsilon_n}{\partial \boldsymbol\theta^\top} \\ \boldsymbol\varepsilon_n^\top\mathbf P_{2n}^s\frac{\partial \boldsymbol\varepsilon_n}{\partial \boldsymbol\theta^\top} \end{pmatrix} = \begin{pmatrix} \mathbf X_n^\top \\ \boldsymbol\varepsilon_n^\top \mathbf P_{1n}^s \\ \boldsymbol\varepsilon_n^\top \mathbf P_{2n}^s \end{pmatrix}\frac{\partial \boldsymbol\varepsilon_n}{\partial \boldsymbol\theta^\top}, \end{aligned} \] where $\mathbf P_{jn}^s = \mathbf P_{jn} + \mathbf P_{jn}^\top$, $\boldsymbol\varepsilon_n = \mathbf R_n\mathbf y_n - \mathbf R_n\mathbf X\boldsymbol\beta$, and \[ \frac{\partial \boldsymbol\varepsilon_n}{\partial \boldsymbol\theta^\top} = \begin{pmatrix} \frac{\partial \boldsymbol\varepsilon_n}{\partial \boldsymbol\beta^\top} & \frac{\partial \boldsymbol\varepsilon_n}{\partial \lambda} \end{pmatrix} = \begin{pmatrix} -\mathbf R_n\mathbf X_n & - \mathbf M_n\mathbf u_n \end{pmatrix}. \]

Thus, \[ \frac{\partial \mathbf g_n(\boldsymbol\theta)}{\partial \boldsymbol\theta ^\top} = - \begin{pmatrix} \mathbf X_n^\top\mathbf R_n\mathbf X_n & \mathbf X_n^\top\mathbf M_n\mathbf u_n \\ \boldsymbol\varepsilon_n^\top\mathbf P_{1n}^s\mathbf R_n\mathbf X_n & \boldsymbol\varepsilon_n^\top\mathbf P_{1n}^s\mathbf M_n\mathbf u_n \\ \boldsymbol\varepsilon_n^\top\mathbf P_{2n}^s\mathbf R_n\mathbf X_n & \boldsymbol\varepsilon_n^\top\mathbf P_{2n}^s\mathbf M_n\mathbf u_n \end{pmatrix}. \]

The variance-covariance matrix of the moments is: \[ \underset{(k + 2) \times (k+2)}{\boldsymbol\Omega_n} = \begin{pmatrix} \underset{(k \times k)}{\mathbf 0} & \underset{(k \times 2)}{\mu_3 \mathbf X_n^\top\boldsymbol\omega_n} \\ \underset{(2 \times k)}{\mu_3\boldsymbol\omega^\top_n\mathbf X_n} & \underset{(2 \times 2)}{(\mu_4 - 3\sigma_0^4)\boldsymbol\omega_n^\top\boldsymbol\omega_n} & \\ & \end{pmatrix} + \mathbf V_n, \] with $\boldsymbol\omega_n = \left[\operatorname{diag}(\mathbf P_{1n}), \operatorname{diag}(\mathbf P_{2n})\right]$ is $n\times 2$ and \[ \mathbf V_n = \sigma_0^4 \begin{pmatrix} \underset{(k\times k)}{\frac{1}{\sigma_0^2}\mathbf X_n^\top\mathbf X_n} & \underset{(k\times 1)}{\mathbf 0} & \underset{(k\times 1)}{\mathbf 0} \\ \underset{(1\times k)}{\mathbf 0} & \underset{(1\times 1)}{\operatorname{tr}(\mathbf P_{1n}^s\mathbf P_{1n})} & \underset{(1\times 1)}{\operatorname{tr}(\mathbf P_{1n}^s\mathbf P_{2n})} \\ \underset{(1\times k)}{\mathbf 0} & \operatorname{tr}(\mathbf P_{2n}^s\mathbf P_{1n}) & \operatorname{tr}(\mathbf P_{2n}^s\mathbf P_{2n}) \end{pmatrix}. \]

When $\boldsymbol\varepsilon_n$ is normally distributed, $\boldsymbol\Omega_n$ is simplified to $\mathbf V_n$ because $\mu_3$ and $\mu_4 = 3\sigma_0^3$.

If $\boldsymbol\Upsilon_n = \left(\frac{1}{n}\widehat{\boldsymbol\Omega}_n\right)^{-1}$, then the OGMM estimator has the following asymptotic distribution \[ \sqrt{n}\left(\widehat{\boldsymbol\theta}_n - \boldsymbol\theta_0\right)\xrightarrow{d} \mathcal N(\mathbf 0, \boldsymbol\Sigma^0), \] where \[ \boldsymbol\Sigma^0 = \left[\lim_{n\to\infty}\mathbf D_n^\top\boldsymbol\Omega_n\mathbf D_n\right]^{-1}. \]

At $\boldsymbol\theta_0$, $\frac{\partial \mathbb E\left[g(\boldsymbol\theta_0)\right]}{\partial \boldsymbol\theta^\top} = - \mathbf D_n$, where \[ \underset{(k + 2)\times (k + 1)}{\mathbf D_n} = \begin{pmatrix} \underset{k\times k}{\mathbf X_n^\top\mathbf R_0\mathbf X_n} & \underset{k\times 1}{\mathbf 0} \\ \underset{1 \times k}{\mathbf 0} & \underset{1\times 1}{\sigma_0^2 \operatorname{tr}\left(\mathbf P_{1n}^{s}\mathbf Q_{0n}\right)}\\ \underset{1\times k}{\mathbf 0} & \underset{1\times 1}{\sigma_0^2 \operatorname{tr}\left(\mathbf P_{2n}^{s}\mathbf Q_{0n}\right)} \end{pmatrix} \tag{6.68}\] where $\mathbf Q_0 = \mathbf M_n\mathbf R_0^{-1}$.

6.6 Feasible Generalized Spatial Two-Stage Least Squares Estimation of the SAC Model

6.6.1 Intuition Behind the Procedure

Consider the following SAC model: \[ \begin{aligned} \mathbf y_n & = \mathbf X_n\boldsymbol\beta_0 + \rho_0\mathbf W_n\mathbf y_n + \mathbf u_n \\ & = \mathbf Z_n\boldsymbol\delta_0 + \mathbf u_n,\\ \mathbf u_n & = \lambda_0 \mathbf M_n \mathbf u_n + \boldsymbol\varepsilon_n, \end{aligned} \tag{6.69}\] where $\mathbf Z_n = \left[\mathbf X_n, \mathbf W_n\mathbf y_n\right]$, $\boldsymbol\delta_0 = \left[\boldsymbol\beta_0^\top ,\rho_0\right]^\top$, $\mathbf y_n$ is the $n\times 1$ vector of observations of the dependent variables, $\mathbf X_n$ is the $n \times k$ matrix of observations on nonstochastic (exogenous) regressors, $\mathbf W_n$ and $\mathbf M_n$ are the $n \times n$ non stochastic weights matrices, $\mathbf u_n$ is the $n \times 1$ vector of regression disturbances, $\boldsymbol\varepsilon_n$ is an $n \times 1$ vector of innovations.

It is worth noting that, while the model allows for different spatial weight matrices for each process, in practice, there is rarely a strong justification for assuming different structures.

Remark

This model is generally referred to as the Spatial-ARAR(1, 1) model to emphasize its autoregressive structure both in the dependent variable and the error term.

The SAC model can be estimated using ML procedure (see Anselin 1988). However, ML estimation requires computing the inverses of $\left(\mathbf I_n - \rho\mathbf W_n\right)$ and $\left(\mathbf I_n - \lambda \mathbf M_n\right)$, which is computationally expensive for large samples. Additionally, ML estimation relies on the assumption of normally distributed errors.

To address these challenges, we rely on estimation techniques from the the S2SLS and GMM. To see how this works, consider applying a spatial Cochrane-Orcutt transformation to the SAC model: \[ \begin{aligned} \mathbf y_n & = \mathbf Z_n\boldsymbol\delta_0 + \left(\mathbf I_n - \lambda_0\mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n, \\ \left(\mathbf I_n - \lambda_0 \mathbf M_n\right)\mathbf y_n & = \left(\mathbf I_n - \lambda_0 \mathbf M_n\right) \mathbf Z_n\boldsymbol\delta_0 + \boldsymbol\varepsilon_n, \\ \mathbf y_s(\lambda_0) & = \mathbf Z_s(\lambda_0)\boldsymbol\delta_0 + \boldsymbol\varepsilon_n, \end{aligned} \tag{6.70}\] where the spatially filtered variables are given by: \[ \begin{aligned} \mathbf y_s(\lambda_0) & = & \mathbf y_n - \lambda_0 \mathbf M_n\mathbf y_n \\ & = & \mathbf y_n - \lambda_0 \mathbf y_L, \\ & = & \left(\mathbf I_n - \lambda_0 \mathbf M_n\right)\mathbf y_n, \\ \mathbf Z_s(\lambda_0) & = & \mathbf Z_n - \lambda_0 \mathbf M_n \mathbf Z_n, \\ & = & \mathbf Z_n - \lambda_0 \mathbf Z_L, \\ & = & \left(\mathbf I_n - \lambda_0 \mathbf M_n\right) \mathbf Z_n. \\ \end{aligned} \]

If we knew $\lambda_0$, we would be able to apply an IV approach on the transformed model Equation 6.70. For the discussion below, assume that we know $\lambda_0$. Note that the ideal instruments in this case will be: \[ \mathbb E\left(\mathbf Z_n(\lambda_0)\right) = \mathbb E(\mathbf Z_n) -\lambda_0\mathbb E(\mathbf M_n\mathbf Z_n) \] where \[ \begin{aligned} \mathbb E\left(\mathbf Z_n\right) & = \mathbb E\left[\mathbf X_n, \mathbf W_n\mathbb E\left(\mathbf y_n\right)\right], \\ \mathbb E\left(\mathbf M_n\mathbf Z_n\right) & = \mathbb E\left[\mathbf M_n\mathbf X_n, \mathbf M_n\mathbf W_n\mathbb E\left(\mathbf y_n\right)\right], \end{aligned} \]

Given that all the columns of $\mathbb E(\mathbf Z_n)$ and $\mathbb E(\mathbf M_n\mathbf Z_n)$ are linear in \[ \mathbf X_n, \mathbf W_n\mathbf X_n, \mathbf W^2_n\mathbf X_n, \ldots, \mathbf M_n\mathbf X_n, \mathbf M_n\mathbf W_n\mathbf X_n, \mathbf M_n\mathbf W^2_n\mathbf X_n, \ldots \tag{6.71}\] the matrix of instruments $\mathbf H_n$ is a subset of the linearly independent columns in Equation 6.71, for example \[ \mathbf H_n = \left[\mathbf X_n, \mathbf W_n\mathbf X_n, \ldots, \mathbf W^l_n\mathbf X_n, \mathbf M_n\mathbf X_n, \mathbf M_n\mathbf W_n\mathbf X_n, \ldots,\mathbf M_n\mathbf W^l_n\mathbf X_n\right]_{LI}, \] where typically, $l\leq 2$.

Since we have the instruments $\mathbf H_n$, and we assuming that we obtain a consistent estimate of $\lambda_0$, we might apply a GMM-type procedure using the following moment conditions for the transformed model Equation 6.70: \[ \mathbf m(\lambda_0, \boldsymbol\delta_0) = \mathbb E\left[\frac{1}{\sqrt{n}}\mathbf H_n^\top\boldsymbol\varepsilon_n\right] = \mathbf 0. \]

Now let $\widetilde{\lambda}_n$ some consistent estimator for $\lambda_0$ which can be obtained in a previous step, then the sample moment vector is: \[ \mathbf m^{\delta}(\widetilde{\lambda}_n, \boldsymbol\delta_n) = \frac{1}{\sqrt{n}}\mathbf H^\top_n\underbrace{\left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right]}_{\widetilde{\boldsymbol\varepsilon}_n}, \] where we explicitly state that the moments depends on $\boldsymbol\delta_n$—which will be estimated—and a consistent estimate of $\lambda_0$. Under homoskedasticity the variance-covariance matrix of the moment vector $\mathbf m(\lambda_0, \delta_0)$ is given by: \[ \operatorname{Var}\left[\mathbf m(\lambda_0, \boldsymbol\delta_0)\right] = \mathbb E\left[\mathbf m(\lambda_0, \delta_0)\mathbf m(\lambda_0, \delta_0)^\top\right] = \sigma^2_0n^{-1}\mathbf H^\top_n\mathbf H_n, \] which motivates the following two-step GMM estimator for $\boldsymbol\delta_0$: \[ \widehat{\boldsymbol\delta}_n = \underset{\boldsymbol\delta_n}{\operatorname*{argmin}}\quad \mathbf m^{\delta}_n(\widetilde{\lambda}_n, \boldsymbol\delta_m)^\top \boldsymbol\Upsilon^{\delta\delta}_n\mathbf m^{\delta}_n(\widetilde{\lambda}_m, \boldsymbol\delta_n) \] with \[ \boldsymbol\Upsilon^{\delta\delta}_n = \left[\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right]^{-1}. \]

Note that: \[ \begin{aligned} Q_n & = \left[\frac{1}{\sqrt{n}}\mathbf H^\top_n\left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right]\right]^\top \left[\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right]^{-1}\left[\frac{1}{\sqrt{n}}\mathbf H^\top_n\left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right]\right], \\ & = \frac{1}{n}\left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right]^\top \mathbf H_n\left[\frac{1}{n}\mathbf H_n^\top\mathbf H_n\right]^{-1}\mathbf H_n^\top \left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right], \\ & = \left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right]^\top \mathbf H_n\left[\mathbf H_n^\top\mathbf H_n\right]^{-1}\mathbf H_n^\top \left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right], \\ & = \left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right]^\top \mathbf P_{H,n} \left[\mathbf y_s(\widetilde{\lambda}_n) - \mathbf Z_s(\widetilde{\lambda}_n)\boldsymbol\delta_n\right] \end{aligned} \]

Then, the estimator of $\boldsymbol\delta_0$ will be: \[ \widehat{\boldsymbol\delta}_n = \left[\widehat{\mathbf Z_s}^\top\mathbf Z_s\right]^{-1}\widehat{\mathbf Z_s}^\top\mathbf y_s \] where $\widehat{\mathbf Z_s} = \mathbf H_n\left(\mathbf H_n^\top\mathbf H_n\right)^{-1}\mathbf H_n\mathbf Z_s$. This estimator has been called the feasible generalized spatial two-stage least squares (FGS2SLS) estimator (Kelejian and Prucha 1998). However, this estimator is not fully efficient.

The key question remains: How can we obtain a consistent estimator of $\lambda_0$? As one may anticipate, this consistent estimator is obtained in a preliminary step using the generalized moments (GM) approach.

6.6.2 Moment Conditions Revised

Since we require a consistent estimate of $\lambda_0$, this section focuses on alternative formulations of the moment conditions under both homoskedasticity (Kelejian and Prucha 1999) and heteroskedasticity (Kelejian and Prucha 2010). It is important to recall that the GM approach proposed by Kelejian and Prucha (1999), as presented in Section 6.4.2, does not yield a consistent estimate of $\lambda_0$ in the presence of heteroskedasticity. Specifically, Theorem 6.6 is derived under the assumption of homoskedasticity. Extensions incorporating a generalized method of moments (GMM) framework have been developed by Kelejian and Prucha (2010), Arraiz et al. (2010), and Drukker et al. (2013).

The GMM approach provides three key improvements over the GM estimator. First, it is robust to heteroskedasticity. Second, it yields an asymptotic variance matrix for the parameter $\widehat{\lambda}_n$. Finally, it allows for joint inference on the spatial lag coefficient $\widehat{\rho}_n$ and the spatial error coefficient $\widehat{\lambda}_n$.

For any nonstochastic matrix $\mathbf A_{n}$, the quadratic moments are given by: \[ \begin{aligned} \mathbb E\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)& = \operatorname{tr}\left[\mathbb E\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)\right], \\ & = \mathbb E\left[\operatorname{tr}\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)\right], \\ & = \mathbb E\left[\operatorname{tr}\left(\mathbf A_n\boldsymbol\varepsilon_n\boldsymbol\varepsilon_n^\top\right)\right], \\ & = \operatorname{tr}\left[\mathbb E\left(\mathbf A_n\boldsymbol\varepsilon_n\boldsymbol\varepsilon_n^\top\right)\right], \\ & = \operatorname{tr}\left(\mathbf A_n\boldsymbol\Sigma_0\right), \end{aligned} \] where $\boldsymbol\Sigma_0 = \textrm{Diag}(\sigma_{i, n}^2, \ldots, \sigma_{n, n}^2)$. Note that $\mathbb E\left(\boldsymbol\varepsilon_n^\top \mathbf A_n\boldsymbol\varepsilon_n\right)\neq 0$, unless $\operatorname{tr}(\mathbf A_n) = 0$. But if $\textrm{Diag}(\mathbf A_n) = 0$, meaning that $a_{ii, n} = 0$ for all $i = 1, \ldots, n$, then $\operatorname{tr}\left(\mathbf A_n\boldsymbol\Sigma_0\right) = 0$ and $\mathbf A_n\boldsymbol\varepsilon_n$ is uncorrelated with $\boldsymbol\varepsilon_n$. Since we need $\operatorname{Diag}(\mathbf A_n) = 0$, we can always construct such matrix as $\mathbf A_n = \mathbf P_{n} - \operatorname{Diag}(\mathbf P_n)$.

Thus, we have the following two moment conditions \[ \begin{aligned} \frac{1}{n}\mathbb E\left[\boldsymbol\varepsilon_n^\top\mathbf A_{1n}\boldsymbol\varepsilon_n\right] & = \mathbf 0 \\ \frac{1}{n}\mathbb E\left[\boldsymbol\varepsilon_n^\top\mathbf A_{2n}\boldsymbol\varepsilon_n\right] & = \mathbf 0 \end{aligned} \tag{6.72}\] with \[ \begin{aligned} \mathbf A_{1n} &= \mathbf M^\top_n\mathbf M_n - \operatorname{Diag}(\mathbf M^\top_n\mathbf M_n), \\ \mathbf A_{2n} &= \mathbf M_n. \end{aligned} \]

The sample moments are obtained by replacing $\boldsymbol\varepsilon_n(\boldsymbol\theta) = \left(\mathbf I_n - \lambda\mathbf M_n\right)\left(\mathbf y_n - \mathbf Z_n\boldsymbol\delta\right)$, with $\boldsymbol\theta = (\boldsymbol\delta^\top, \lambda)^\top$, by the their counterpart expressed as a function of the regression residuals. Since $\mathbf u_n(\boldsymbol\delta) = \mathbf y_n - \mathbf Z_n\boldsymbol\delta$, and $\mathbf u_L(\boldsymbol\delta) = \mathbf M_n\mathbf u_n(\boldsymbol\delta)$, it follows that $\boldsymbol\varepsilon_n(\boldsymbol\theta) = \mathbf u_n(\boldsymbol\delta) - \lambda \mathbf u_L(\boldsymbol\delta) = \mathbf y_{s}(\lambda) - \mathbf Z_s(\lambda)\boldsymbol\delta$, the spatially filtered residuals. Then, the set of quadratic moment functions are \[ \begin{aligned} \mathbf m_n(\boldsymbol\theta) & = \frac{1}{n}\begin{pmatrix} \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf A_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \\ \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf A_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta) \end{pmatrix}, \\ & = \frac{1}{n}\begin{pmatrix} \left[\mathbf u_n(\boldsymbol\delta) - \lambda \mathbf u_L(\boldsymbol\delta)\right]^\top \mathbf A_{1n} \left[\mathbf u_n(\boldsymbol\delta) - \lambda \mathbf u_L(\boldsymbol\delta)\right] \\ \left[\mathbf u_n(\boldsymbol\delta) - \lambda \mathbf u_L(\boldsymbol\delta)\right]^\top \mathbf A_{2n} \left[\mathbf u_n(\boldsymbol\delta) - \lambda \mathbf u_L(\boldsymbol\delta)\right] \end{pmatrix}. \end{aligned} \tag{6.73}\]

We need to write the estimator of $\lambda$ as a weighted non-linear LS estimator. Note that: \[ \begin{aligned} \frac{1}{n}\boldsymbol\varepsilon^\top_n(\boldsymbol\theta_0)\mathbf A_{qn}\boldsymbol\varepsilon_n(\boldsymbol\theta_0) & = \left[\mathbf u_n(\boldsymbol\delta) - \lambda \mathbf u_L(\boldsymbol\delta)\right]^\top \mathbf A_{qn} \left[\mathbf u_n(\boldsymbol\delta) - \lambda \mathbf u_L(\boldsymbol\delta)\right], \\ & = \frac{1}{n}\mathbf u^\top_n\mathbf A_{qn}\mathbf u_n - \frac{1}{n}\lambda\left(\mathbf u^\top_n\mathbf A_{qn}\mathbf u_L + \mathbf u_L^\top\mathbf A_{qn}\mathbf u_n\right) +\frac{1}{n}\lambda^2\mathbf u_L^\top\mathbf A_{qn}\mathbf u_L, \\ & = \frac{1}{n}\mathbf u^\top_n\mathbf A_{qn}\mathbf u_n - 2\frac{1}{n}\lambda\mathbf u_L^\top\mathbf A_{qn}\mathbf u_n +\frac{1}{n}\lambda^2\mathbf u_L^\top\mathbf A_{qn}\mathbf u_L, \\ \end{aligned} \tag{6.74}\] for $q = 1, 2$. In the third line of Equation 6.74, we assume that $\mathbf A_{qn}$ is symmetric such that: \[ \begin{aligned} \mathbf u^\top_n\mathbf A_{qn}\mathbf u_L + \mathbf u_L^\top\mathbf A_{qn}\mathbf u_n & = \mathbf u_L^\top \mathbf A_{qn}^\top\mathbf u_n + \mathbf u_L\mathbf A_q\mathbf u_n, \\ & = \mathbf u_L^\top\left(\mathbf A_{qn} + \mathbf A_{qn}^\top\right) \mathbf u_n, \\ & = 2\mathbf u_L^\top\mathbf A_{qn}\mathbf u_n. \end{aligned} \]

Here it is important to note that in some cases $\mathbf A_{2n} = \mathbf M_n$ might not be symmetric. However, we can use Definition 3.22 and set: \[ \mathbf A_{2n} = (1/2)\left(\mathbf M_n + \mathbf M^\top_n\right) \tag{6.75}\]

Taking expectation over Equation 6.74: \[ \begin{aligned} \frac{1}{n}\mathbb E\left(\boldsymbol\varepsilon^\top_n\mathbf A_{qn}\boldsymbol\varepsilon_n\right) & = n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf A_{qn}\mathbf u_n \right) - 2n^{-1}\lambda\mathbb E\left(\mathbf u_L^\top\mathbf A_{qn}\mathbf u_n\right) + \lambda^2n^{-1}\mathbb E\left(\mathbf u_L^\top\mathbf A_{qn}\mathbf u_L\right) \\ 0 & = n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf A_{qn}\mathbf u_n \right) - \begin{pmatrix} 2n^{-1}\mathbb E\left(\mathbf u_L^\top\mathbf A_{qn}\mathbf u_n\right) & -n^{-1}\mathbb E\left(\mathbf u_L^\top\mathbf A_{qn}\mathbf u_L\right) \end{pmatrix} \begin{pmatrix} \lambda \\ \lambda^2 \end{pmatrix} \end{aligned} \]

Then, we have the following system of equations for $q = 1, 2$ (see Kelejian and Prucha 2010, pag 56): \[ \begin{aligned} \begin{pmatrix} n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf A_{1q}\mathbf u_n \right) \\ n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf A_{2q}\mathbf u_n \right) \end{pmatrix} - \begin{pmatrix} 2n^{-1}\mathbb E\left(\mathbf u_L^\top\mathbf A_{1q}\mathbf u_n\right) & -n^{-1}\mathbb E\left(\mathbf u_L^\top\mathbf A_{1q}\mathbf u_L\right) \\ 2n^{-1}\mathbb E\left(\mathbf u_L^\top\mathbf A_{2q}\mathbf u_n\right) & -n^{-1}\mathbb E\left(\mathbf u_L^\top\mathbf A_{2q}\mathbf u_L\right) \end{pmatrix} \begin{pmatrix} \lambda \\ \lambda^2 \end{pmatrix} & = \mathbf 0 \\ \begin{pmatrix} n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf A_{1q}\mathbf u_n \right) \\ n^{-1}\mathbb E\left(\mathbf u^\top_m\mathbf A_{2q}\mathbf u_n \right) \end{pmatrix}- \begin{pmatrix} 2n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf M^\top_n\mathbf A_{1q}\mathbf u_n\right) & -n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf M^\top_n\mathbf A_{1q}\mathbf M_n\mathbf u_n\right) \\ n^{-1}\mathbb E\left(\mathbf u_L^\top\left(\mathbf M_n + \mathbf M^\top_n\right)\mathbf u_n\right) & -n^{-1}\mathbb E\left(\mathbf u^\top_n\mathbf M^\top_n\mathbf A_{2q}\mathbf M_n\mathbf u_n\right) \end{pmatrix} & = \mathbf 0 \\ \boldsymbol\gamma_n - \boldsymbol\Gamma_n\boldsymbol\alpha_n &= \mathbf 0. \end{aligned} \tag{6.76}\] where we use Equation 6.75 for the second moment. Now, we can express the sample moment conditions as in Section 6.4.2: \[ \underset{2 \times 1}{\widetilde{\mathbf m}_n(\widetilde{\boldsymbol\delta}_n, \lambda)} = \underset{2 \times 1}{\widetilde{\mathbf g}_n} - \underset{2 \times 2}{\widetilde{\mathbf G}_n}\begin{pmatrix}\lambda \\ \lambda^2\end{pmatrix} = \mathbf 0 \]

The elements of $\widetilde{\mathbf g}_n$ the following: \[ \begin{aligned} \widetilde{\mathbf g}_{1n} &=& \frac{1}{n}\widetilde{\mathbf u}^\top_n\mathbf A_{1n}\widetilde{\mathbf u}_n \\ \widetilde{\mathbf g}_{2n} &=& \frac{1}{n}\widetilde{\mathbf u}^\top_n\mathbf A_{2n}\widetilde{\mathbf u}_n = \frac{1}{n} \widetilde{\mathbf u}^\top_n\widetilde{\mathbf u}_L \end{aligned} \]

The $\widetilde{\mathbf G}_n$ matrix is given by: \[ \begin{aligned} \widetilde{\mathbf G}_{11,n} &=& 2n^{-1}\widetilde{\mathbf u}^\top_n\mathbf M^\top_n\mathbf A_{1n}\widetilde{\mathbf u}_n \\ \widetilde{\mathbf G}_{12,n} &=& -n^{-1}\widetilde{\mathbf u}^\top_n\mathbf M^\top_n\mathbf A_{1n}\mathbf M_n\widetilde{\mathbf u}_n \\ \widetilde{\mathbf G}_{21,n} &=& -n^{-1}\widetilde{\mathbf u}^\top_n\mathbf M^\top_n\left(\mathbf A_{2n} + \mathbf A_{2n}^\top\right)\widetilde{\mathbf u}_n \\ \widetilde{\mathbf G}_{22,n}&=& -n^{-1}\widetilde{\mathbf u}^\top_n\mathbf M_n\mathbf A_{2n}\mathbf M_n\widetilde{\mathbf u}_n \end{aligned} \]

A more compact notation is: \[ \begin{aligned} \widetilde{\mathbf G}_n & = \frac{1}{n} \begin{pmatrix} \widetilde{\mathbf u}^\top_n\left(\mathbf A_{1n} + \mathbf A^\top_{1n}\right)\widetilde{\mathbf u}_s & - \widetilde{\mathbf u}^\top_s\mathbf A_{1n}\widetilde{\mathbf u}^\top_s \\ \vdots & \vdots \\ \widetilde{\mathbf u}^\top_n\left(\mathbf A_{qn} + \mathbf A^\top_{qn}\right)\widetilde{\mathbf u}_s & - \widetilde{\mathbf u}^\top_s\mathbf A_{qn}\widetilde{\mathbf u}^\top_s \end{pmatrix}, \\ \widetilde{\mathbf g}_n & = \frac{1}{n}\begin{pmatrix} \widetilde{\mathbf u}^\top_n\mathbf A_{1n}\widetilde{\mathbf u}_n \\ \vdots \\ \widetilde{\mathbf u}^\top_n\mathbf A_{qn}\widetilde{\mathbf u}_n \end{pmatrix}, \end{aligned} \tag{6.77}\] for $q = 1, 2$.

6.6.3 Assumptions

In this section, we outline the key assumptions for the SAC model under heteroskedasticity, following Arraiz et al. (2010). These assumptions primarily concern the spatial weight matrices, error structure, regressors, and instruments.

Assumption 17 (Spatial Weights Matrices (Arraiz et al. 2010))

Assume the following:

All diagonal elements $\mathbf W_n$ and $\mathbf M_n$ are zero.
$\lambda_n\in (-1, 1)$, $\rho_n \in (-1, 1)$.
The matrices $\mathbf I_n - \rho_n \mathbf W_n$ and $\mathbf I_n - \lambda_n \mathbf M_n$ are nonsingular for all $\lambda_n\in (-1, 1)$ and $\rho_n \in (-1, 1)$.

Assumption 17(a) is a normalization rule, ensuring that a region cannot be its own neighbor. Assumption 17(b) defines the parameter space, as discussed in Kelejian and Prucha (2010) [section 2.2]. Finally, Assumption 17(c) ensures that the spatial processes for $\mathbf y_n$ and $\mathbf u_n$ are uniquely defined. Under this assumption, the SAC model can be written as: \[ \begin{aligned} \mathbf y_n & = \left(\mathbf I_n - \rho_0 \mathbf W_n\right)^{-1}\left[\mathbf X_n\boldsymbol\beta_0 + \mathbf u_n\right] \\ \mathbf u_n & = \left(\mathbf I_n - \rho_0 \mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n. \end{aligned} \]

The reduced form is: \[ \mathbf y_n = (\mathbf I_n - \rho_0\mathbf W_n)^{-1}\mathbf X_n\boldsymbol\beta_0 +(\mathbf I_n -\rho_0\mathbf W)^{-1}(\mathbf I_n - \lambda_0\mathbf M_n)^{-1}\boldsymbol\varepsilon_n. \]

Assumption 18 (Heteroskedastic Errors (Arraiz et al. 2010))

The error term $\left\lbrace\epsilon_{i,n}: 1 \leq i \leq n, n\geq 1\right\rbrace$ satisfy $\mathbb E(\epsilon_{i,n}) = 0$, $\mathbb E(\epsilon_{i,n}^2) = \sigma^2_{i,n}$, with $0 < \underline{a}^\sigma \leq \sigma^2_{i,n}\leq \overline{a}^\sigma<\infty$. Furthermore, for each $n\geq 1$ the random variables $\epsilon_{1,n}, \ldots, \epsilon_{n,n}$ are totally independent.

This assumption allows for heteroskedasticity in the innovations while ensuring uniformly bounded variances. It also accommodates triangular array structures where variances may depend on the sample size $n$.

Assumption 19 (Bounded Spatial Weight Matrices (Arraiz et al. 2010))

The row and column sums of the matrices $\mathbf W_n$ and $\mathbf M_n$ are bounded uniformly in absolute value, by , respectively, one and some finite constant, and the row and column sums of the matrices $(\mathbf I_n - \rho_0\mathbf W_n)^{-1}$ and $(\mathbf I_n - \lambda_0\mathbf M_n)^{-1}$ are bounded uniformly in absolute value by some finite constant.

This assumption is a technical requirement for deriving large-sample properties of the estimators. It ensures that spatial dependence in $\mathbf y_n$ and $\mathbf u_n$ does not accumulate indefinitely, maintaining a “fading” memory property. Under this assumption: \[ \begin{aligned} \mathbb E\left[\mathbf u_n \right] & = \mathbb E\left[\left(\mathbf I_n - \lambda_0 \mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n\right], \\ & = \left(\mathbf I_n - \lambda_0 \mathbf M_n\right)^{-1} \mathbb E\left[\boldsymbol\varepsilon_n\right], \\ & = \mathbf 0 \;\;\;\mbox{by @ass-iv-gmm-error-hete-gmm (Heteroskedastic Errors)}. \end{aligned} \]

Additionally, the variance of $\mathbf u_n$ is given by: \[ \begin{aligned} \mathbb E\left[\mathbf u_n\mathbf u^\top_n\right] & = \mathbb E\left[\left(\mathbf I_n - \lambda_0 \mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n\boldsymbol\varepsilon^\top_n\left(\mathbf I_n - \lambda_0 \mathbf M^\top_n\right)^{-1}\right], \\ & = \left(\mathbf I_n - \lambda_0\mathbf M_n\right)^{-1}\mathbb E\left[\boldsymbol\varepsilon_n\boldsymbol\varepsilon^\top_n\right]\left(\mathbf I - \lambda_0 \mathbf M^\top_n\right)^{-1}, \\ & = \left(\mathbf I_n - \lambda_0 \mathbf M_n\right)^{-1}\boldsymbol\Sigma_0\left(\mathbf I_n - \lambda_0\mathbf M^\top_n\right)^{-1} \end{aligned} \] where $\boldsymbol\Sigma_0 = \operatorname{Diag}(\sigma^2_{i,n})$.

Assumption 20 (Regressors (Arraiz et al. 2010))

The regressor matrices $\mathbf X_n$ have full column rank (for $n$ large enough). Furthermore, the elements of the matrices $\mathbf X_n$ are uniformly bounded in absolute value.

This assumption prevents multicollinearity and ensures that the regressors remain well-behaved in large samples.

Assumption 21 (Instruments I)

The instruments matrices $\mathbf H_n$ have full column rank $L \geq K + 1$ (for all $n$ large enough). Furthermore, the elements of the matrices $\mathbf H_n$ are uniformly bounded in absolute value. Additionally, $\mathbf H_n$ is assumed to, at least, contain the linearly independent columns of $(\mathbf X_n, \mathbf M_n\mathbf X_n)$

Several papers study the construction of optimal instruments for spatial models (see, for example, Lee 2003, 2007; Das et al. 2003; Kelejian et al. 2004).

Remark

The effect of the selection of instruments on the efficiency of the estimators remains to be further investigated.

Assumption 22 (Instruments II (Identification) (Arraiz et al. 2010))

The instruments $\mathbf H_n$ satisfy furthermore:

$\mathbf Q_{HH} = \lim_{n \to \infty}n^{-1}\mathbf H_n^\top\mathbf H_n$ is finite and nonsingular.
$\mathbf Q_{HZ} = \operatorname{plim}_{n \to \infty}n^{-1}\mathbf H_n^\top\mathbf Z_n$ and $\mathbf Q_{HMZ} = \operatorname{plim}_{n \to \infty}n^{-1}\mathbf H_n^\top\mathbf M_n\mathbf Z_n$ are finite and have full column rank. Furthermore $\mathbf Q_{HZ,s}(\lambda_0) = \mathbf Q_{HZ}-\lambda_0\mathbf Q_{HMZ}$ has full column rank.
$\mathbf Q_{H\Sigma H} = \lim_{n \to \infty}n^{-1}\mathbf H_n^\top\Sigma_n\mathbf H_n$ is finite and nonsingular, where $\boldsymbol\Sigma_n= \operatorname{diag}(\sigma^2_{i,n})$

In treating $\mathbf X_n$ and $\mathbf H_n$ as non-stochastic our analysis should be viewed as conditional on $\mathbf X_n$ and $\mathbf H_n$.

6.6.4 Estimators and Estimation Procedure in a Nutshell

Consider again the transformed model: \[ \mathbf y_{s,n}(\lambda_0) = \mathbf Z_{s,n}(\lambda_0)\boldsymbol\delta_0 + \epsilon_n, \] where $\mathbf y_{s,n}(\lambda_0) = \mathbf y_n - \lambda_0\mathbf M_n\mathbf y_n$ and $\mathbf Z_{s,n}(\lambda_0) = \mathbf Z_n - \lambda_0\mathbf M_n\mathbf Z_n$. If $\lambda_0$ were known, we could directly apply the S2SLS estimator to the transformed model. However, since $\lambda_0$ is unknown, it must be estimated before $\boldsymbol\delta_0$. The estimation procedure consists of the following steps

Obtain an initial IV estimator of $\boldsymbol\delta_0$ to construct consistent residuals.
Use these residuals to derive the moment conditions that provide a consistent estimate of $\lambda_0$ via GMM.
Use the estimate of $\lambda_0$ to define a weighting matrix for the moment conditions to obtain a consistent and efficient estimator.
Estimate $\delta_0$ from the transformed model.
Finally, a consistent and efficient estimate of $\lambda$ is based on GS2SLS residuals.

We now consider each step in detail.

6.6.4.1 Step 1a: S2SLS estimator

In the first step, $\boldsymbol\delta_0$ is estimated by 2SLS applied to untransformed model $\mathbf y_n = \mathbf Z_n\boldsymbol\delta_0 + \mathbf u_n$ using the instruments matrix $\mathbf H_n$. Then: \[ \widetilde{\boldsymbol\delta}_{S2SLS, n} = \left(\widetilde{\mathbf Z}^\top_n\mathbf Z_n\right)^{-1}\widetilde{\mathbf Z}^\top_n\mathbf y_n, \] where $\widetilde{\mathbf Z}_n = \mathbf H_n\left(\mathbf H^\top_n\mathbf H_n\right)^{-1}\mathbf H^\top_n\mathbf Z_n = \mathbf P_{H, n}\mathbf Z_n= (\mathbf X_n, \widetilde{\mathbf W_n\mathbf y_n})$. The estimates $\widetilde{\boldsymbol\delta}_{S2SLS, n}$ yield an initial vector of residuals, $\mathbf u_{n, S2SLS}$ as: \[ \widetilde{\mathbf u}_{S2SLS, n} = \mathbf y_n - \mathbf Z_n\widetilde{\boldsymbol\delta}_{n, S2SLS}. \]

The following theorem states that $\widetilde{\boldsymbol\delta}_{S2SLS, n}$ is consistent:

Theorem 6.8 (Consistency of $\widetilde{\boldsymbol\delta}_{S2SLS, n}$ (Kelejian and Prucha 2010)) Suppose the assumptions hold. Then $\widetilde{\boldsymbol\delta}_{S2SLS, n} = \boldsymbol\delta_0 + O_p(n^{-1/2})$, and hence $\widetilde{\boldsymbol\delta}_{n, S2SLS}$ is consistent for $\boldsymbol\delta_0$: \[ \widetilde{\boldsymbol\delta}_{S2SLS, n}\xrightarrow{p} \boldsymbol\delta_0. \]

Proof. Sketch of proof for Theorem 6.8. The model is: \[ \begin{aligned} \mathbf y_n & = \mathbf Z_n\boldsymbol\delta_0 + \mathbf u_n,\\ \mathbf u_n & = \lambda_0 \mathbf M_n \mathbf u_n + \boldsymbol\varepsilon_n. \end{aligned} \]

The sampling error is given by: \[ \begin{aligned} \widehat{\boldsymbol\delta}_n & = \boldsymbol\delta_0 + \left(\widehat{\mathbf Z}^\top_n\widehat{\mathbf Z}_n\right)^{-1}\widehat{\mathbf Z}^\top_n\mathbf u_n, \\ & = \boldsymbol\delta_0 + \left[\left(\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top\mathbf Z_n\right)^\top\left(\mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n\right)\right]^{-1}\left(\mathbf H_n(\mathbf H_n^\top\mathbf H_n)^{-1}\mathbf H_n^\top\mathbf Z_n\right)^\top\mathbf u_n,\\ & = \boldsymbol\delta_0 + \left[\mathbf Z^\top_n \mathbf H_n (\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\mathbf Z_n\right]^{-1}\mathbf Z^\top_n\mathbf H_n(\mathbf H^\top_n\mathbf H_n)^{-1}\mathbf H^\top_n\left(\mathbf I -\lambda_0\mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n. \end{aligned} \]

Solving for $\widehat{\boldsymbol\delta}_{n} - \boldsymbol\delta_0$ and multiplying by $\sqrt{n}$ we obtain: \[ \begin{aligned} \sqrt{n}(\widehat{\boldsymbol\delta}_{n} - \boldsymbol\delta_0) & = \left[\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n \right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n \right)\right]^{-1} \\ & \times \left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n \right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\frac{1}{\sqrt{n}}\mathbf H^\top_n\left(\mathbf I_n -\lambda_0\mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n, \\ & = \left[\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n \right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n \right)\right]^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n \right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\frac{1}{\sqrt{n}}\mathbf F_n^\top\boldsymbol\varepsilon_n, \end{aligned} \] where: \[ \mathbf F_n^\top = \mathbf H^\top_n\left(\mathbf I_n -\lambda_0\mathbf M_n\right)^{-1} \quad \mbox{whose elements are bounded in absolute value.} \]

Assumption 22 implies that: \[ \begin{aligned} \lim \frac{1}{n}\mathbf H_n^\top\mathbf H_n &= \mathbf Q_{HH}, \\ \operatorname{plim} \frac{1}{n}\mathbf H_n^\top\mathbf Z_n & = \mathbf Q_{HZ}, \end{aligned} \] which are finite and nonsingular.

Furthermore, note that $\mathbb E(n^{-1/2}\mathbf F_n^\top\boldsymbol\varepsilon_n) = \mathbf 0$ and, under homoskedasticity, \[ \begin{aligned} \mathbb E\left[(n^{-1/2}\mathbf F_n^\top\boldsymbol\varepsilon_n)(n^{-1/2}\mathbf F_n^\top\boldsymbol\varepsilon_n)^\top\right] & = \frac{1}{n}\mathbb E\left[\mathbf H_n^\top\left(\mathbf I_n -\lambda\mathbf M_n\right)^{-1}\boldsymbol\varepsilon_n\boldsymbol\varepsilon_n^\top\left(\mathbf I_n -\lambda\mathbf M_n^\top\right)^{-1} \mathbf H_n \right], \\ & = \sigma^2_0\frac{1}{n}\mathbf H_n^\top\left(\mathbf I_n -\lambda_0\mathbf M_n\right)^{-1}\left(\mathbf I_n -\lambda_0\mathbf M_n^\top\right)^{-1} \mathbf H_n. \end{aligned} \]

Assume that \[ \lim_{n\to\infty}\frac{1}{n}\mathbf H_n^\top\left(\mathbf I_n -\lambda_0\mathbf M_n\right)^{-1}\left(\mathbf I_n -\lambda_0\mathbf M_n^\top\right)^{-1} \mathbf H_n = \frac{1}{n}\mathbf F^\top_n\mathbf F_n =\boldsymbol\Phi\quad \mbox{exists} \]

Then using Theorem 3.21: \[ n^{-1/2}\mathbf F_n^\top\boldsymbol\varepsilon_n\xrightarrow{d} \mathcal N(\mathbf 0, \sigma^2_0\boldsymbol\Phi). \]

Therefore: \[ \sqrt{n}(\widehat{\boldsymbol\delta}_{n} - \boldsymbol\delta_0) \xrightarrow{d} \mathcal N(\mathbf 0, \boldsymbol\Delta), \] and \[ \boldsymbol\Delta = \sigma^2_{0}\left[\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\right]^{-1}\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\boldsymbol\Phi\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\left[\mathbf Q_{HZ}^\top\mathbf Q_{HH}^{-1}\mathbf Q_{HZ}\right]^{-1}. \]

Then we can say that $\widetilde{\boldsymbol\delta} = \boldsymbol\delta + O_p(n^{-1/2})$. Consistency follows if $n^{-1}\mathbf F_n^\top\boldsymbol\varepsilon_n\xrightarrow{p} \mathbf 0$. Note that $\mathbb E(n^{-1}\mathbf F_n^\top\boldsymbol\varepsilon_n) = \mathbf 0$ and \[ \operatorname{Var}\left(n^{-1}\mathbf F_n^\top\boldsymbol\varepsilon_n\right)=\sigma^2_0\frac{1}{n^2}\mathbf H_n^\top\left(\mathbf I_n -\lambda_0\mathbf M_n\right)^{-1}\left(\mathbf I_n -\lambda_0\mathbf M_n^\top\right)^{-1} \mathbf H_n, \] which converges to $\mathbf 0$, then using Chebyshev’s Theorem 3.28: \[ n^{-1}\mathbf F_n^\top\boldsymbol\varepsilon_n\xrightarrow{p} \mathbf 0\quad\mbox{and hence}\quad \widetilde{\boldsymbol\delta}_n\xrightarrow{p}\boldsymbol\delta_0. \]

Although $\widetilde{\boldsymbol\delta}_{S2SLS, n}$ is consistent, it does not yet exploit the spatial correlation in the disturbance process. OLS residuals are not appropriate at this stage because the spatial lag $\mathbf W_n\mathbf y_n$ is endogenous, so OLS does not generally provide a consistent estimator of $\boldsymbol\delta_0$ or consistent residuals. The initial S2SLS residuals are therefore used in the second step.

6.6.4.2 Step 1b: Initial GMM estimator of $\lambda$ based on S2SLS residuals

Using the consistent estimate $\mathbf u_n$ in the previous step, now we create the sample moments for $q= 1, 2$ based on the estimated residuals. Define: \[ \begin{aligned} \widetilde{\mathbf u}_{n, \textrm{S2SLS}} & = \mathbf y_n - \mathbf Z_n\widetilde{\boldsymbol\delta}_{n, \textrm{S2SLS}}, \\ \widetilde{\mathbf u}_{s, n} & = \mathbf M_n\widetilde{\mathbf u}_{n, \textrm{S2SLS}}, \end{aligned} \] and the sample moments are: \[ \begin{aligned} \mathbf m_n(\lambda_n,\widetilde{\boldsymbol\delta}_{n, \textrm{S2SLS}}) & = \frac{1}{n} \begin{pmatrix} \widetilde{\mathbf u}_{n, \textrm{S2SLS}}^\top \left(\mathbf I_n - \lambda_n\mathbf M^\top_n\right)\mathbf A_{1,n}\left(\mathbf I_n - \lambda_n\mathbf M_n\right)\widetilde{\mathbf u}_{n, \textrm{S2SLS}} \\ \widetilde{\mathbf u}_{n, \textrm{S2SLS}}^\top \left(\mathbf I_n - \lambda_n\mathbf M^\top_n\right)\mathbf A_{2,n}\left(\mathbf I_n - \lambda_n\mathbf M_n\right)\widetilde{\mathbf u}_{n, \textrm{S2SLS}} \end{pmatrix}, \\ & = \widetilde{\mathbf G}_n\begin{pmatrix} \lambda_n \\ \lambda^2_n \end{pmatrix} - \widetilde{\mathbf g}_n, \end{aligned} \tag{6.78}\] where, \[ \begin{aligned} \widetilde{\mathbf G}_n & = \frac{1}{n} \begin{pmatrix} \widetilde{\mathbf u}^\top_n\left(\mathbf A_{1n} + \mathbf A^\top_{1n}\right)\widetilde{\mathbf u}_{n,s} & - \widetilde{\mathbf u}^\top_{n,s}\mathbf A_{1n}\widetilde{\mathbf u}^\top_{s,n} \\ \vdots & \vdots \\ \widetilde{\mathbf u}^\top_n\left(\mathbf A_{qn} + \mathbf A^\top_{qn}\right)\widetilde{\mathbf u}_{n,s} & - \widetilde{\mathbf u}^\top_{n,s}\mathbf A_{qn}\widetilde{\mathbf u}^\top_{s,n} \end{pmatrix}, \\ \widetilde{\mathbf g}_n & = \frac{1}{n}\begin{pmatrix} \widetilde{\mathbf u}^\top_n\mathbf A_{1n}\widetilde{\mathbf u}_n \\ \vdots \\ \widetilde{\mathbf u}^\top_n\mathbf A_{qn}\widetilde{\mathbf u}_n \end{pmatrix}. \end{aligned} \]

The initial GMM estimator for $\lambda$ is then defined as \[ \breve{\lambda}_{n, \textrm{GMM}} = \underset{\lambda_n \in \Lambda}{\operatorname*{argmin}}\left\lbrace \left[\widetilde{\mathbf G}_n\begin{pmatrix} \lambda_n \\ \lambda^2_n \end{pmatrix} - \widetilde{\mathbf g}_n \right]^\top \left[\widetilde{\mathbf G}_n\begin{pmatrix} \lambda_n \\ \lambda^2_n \end{pmatrix} - \widetilde{\mathbf g}_n\right]\right\rbrace = \underset{\lambda_n\in \Lambda}{\operatorname*{argmin}}\left[\mathbf m_n(\lambda_n,\widetilde{\boldsymbol\delta}_{n, \textrm{S2SLS}})^\top \mathbf m_n(\lambda_n,\widetilde{\boldsymbol\delta}_{n, \textrm{S2SLS}})\right], \] where $\boldsymbol\Upsilon^{\lambda\lambda}_n = \mathbf I_2$. This estimator is consistent but not efficient. For efficiency we need to replace $\boldsymbol\Upsilon^{\lambda\lambda}_n$ by the variance-covariance matrix of the sample moments. Furthermore, the expression above can be interpreted as a nonlinear least squares system of equations. The initial estimate is thus obtained as a solution of the above system.

Now, we need to define the expression for the matrices $\mathbf A_{n,s}$. Drukker et al. (2013) suggest, for the homoskedastic case, the following expressions: \[ \begin{aligned} \mathbf A_{1n} & = \upsilon \left[\mathbf M^\top_n\mathbf M_n - \frac{1}{n}\operatorname{tr}\left(\mathbf M^\top_n\mathbf M_n\right)\mathbf I_n\right],\\ \mathbf A_{2n} & = \mathbf M_n, \end{aligned} \] where $\upsilon$ is the scaling factor needed to obtain the same estimator of Kelejian and Prucha (1998); Kelejian and Prucha (1999).

On the other hand, when heteroskedasticity is assumed, Kelejian and Prucha (2010) recommend the following expressions: \[ \begin{aligned} \mathbf A_{1n} & = \mathbf M^\top_n\mathbf M_n - \operatorname{Diag}(\mathbf M^\top_n\mathbf M_n), \\ \mathbf A_{2n} & = \mathbf M_n \end{aligned} \]

6.6.4.3 Step 1c: Efficient GMM estimator of $\lambda_0$ based on S2SLS residuals

The efficient GMM estimator of $\lambda_0$ is a weighted nonlinear least squares estimator. Specifically, this estimator is $\widetilde{\lambda}_n$ where: \[ \widetilde{\lambda}_{n, \textrm{EGMM}} = \underset{\lambda_n \in \Lambda}{\operatorname*{argmin}} \left[\mathbf m_n(\lambda_n,\widetilde{\boldsymbol\delta}_n)^\top\widetilde{\boldsymbol\Psi}^{-1}_n\mathbf m_n(\lambda_n,\widetilde{\boldsymbol\delta}_n) \right], \tag{6.79}\] and where the weighting matrix is $\widetilde{\boldsymbol\Psi}^{-1}_n$, where $\boldsymbol\Psi_n$ is the variance of the moment conditions $\mathbf m(\lambda_0,\widetilde{\boldsymbol\delta}_n)$.

Let $\widetilde{\mathbf u}_n = \mathbf y - \mathbf Z\widetilde{\boldsymbol\delta}_{n, \textrm{S2SLS}}$. The matrix $\widetilde{\boldsymbol\Psi}^{-1}_n= \widetilde{\boldsymbol\Psi}^{-1}_n (\breve{\lambda}_{n,\textrm{GMM}})$ is defined as follows. Let $\widetilde{\boldsymbol\Psi}_n = \left[\widehat{\Psi}_{rs}\right]_{r,s = 1, 2}$ with \[ \widetilde{\Psi}_{rs} = (2n)^{-1}\operatorname{tr}\left[(\mathbf A_{r, n} + \mathbf A_{r, n}^\top)\widetilde{\boldsymbol\Sigma}_n(\mathbf A_{s, n} + \mathbf A_{s, n}^\top)\widetilde{\boldsymbol\Sigma}_n\right] + n^{-1}\widetilde{\mathbf a}_{r, n}^\top\widetilde{\boldsymbol\Sigma}_n\widetilde{\mathbf a}_{s, n}, \tag{6.80}\] where: \[ \begin{aligned} \widetilde{\boldsymbol\Sigma}_n & = \operatorname{Diag}\left(\widetilde{\epsilon}_{in}^2\right), \\ \widetilde{\boldsymbol\varepsilon}_n & = \left(\mathbf I_n - \breve{\lambda}_{n, \textrm{GMM}}\mathbf M_n\right)\widetilde{\mathbf u}_n, \\ \widetilde{\mathbf a}_{r,n} & = \left(\mathbf I_n - \breve{\lambda}_{n, \textrm{GMM}}\mathbf M_n^\top\right)^{-1}\mathbf H_n\widetilde{\mathbf P}_n\widetilde{\boldsymbol\alpha}_{r,n}, \\ \widetilde{\boldsymbol\alpha}_{r,n} & = - n^{-1}\left[\mathbf Z^\top_n\left(\mathbf I_n - \breve{\lambda}_{n, \textrm{GMM}}\mathbf M_n^\top\right)(\mathbf A_{r, n} + \mathbf A_{r, n}^\top)\left(\mathbf I_n - \breve{\lambda}_{n, \textrm{GMM}}\mathbf M_n\right)\widetilde{\mathbf u}_n\right] \\ \widetilde{\mathbf P}_n & = \left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H_n^\top\mathbf Z_n \right)\left[\left(\frac{1}{n}\mathbf H^\top\mathbf Z_n\right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_n\right)\right]^{-1} \end{aligned} \]

It is important to note that this step is not necessary since the previous estimator of $\lambda$ is already consistent.

6.6.4.4 Step 2a: GS2SLS Estimator

Using $\breve{\lambda}_{ogmm}$ from step 1c (or the consistent estimator from step 1b) in the transformed model we have: \[ \widehat{\boldsymbol\delta}_n(\widetilde{\lambda}_{n, \textrm{EGMM}}) = \left[\widehat{\mathbf Z}_{s,n}^\top(\widetilde{\lambda}_{n, \textrm{EGMM}}) \mathbf Z(\widetilde{\lambda}_{n, \textrm{EGMM}})\right]^{-1} \widehat{\mathbf Z}_{s, n}^\top(\widetilde{\lambda}_{n, \textrm{EGMM}}) \mathbf y_{s, n}(\widetilde{\lambda}_{n, \textrm{EGMM}}) \] where \[ \begin{aligned} \mathbf y_{s, n}(\widetilde{\lambda}_{n, \textrm{EGMM}}) & = \mathbf y_n - \widetilde{\lambda}_{n, \textrm{EGMM}}\mathbf M_n\mathbf y_n, \\ \mathbf Z_{s, n}(\widetilde{\lambda}_{n, \textrm{EGMM}}) & = \mathbf Z_n - \widetilde{\lambda}_{n, \textrm{EGMM}}\mathbf M_n\mathbf Z_n, \\ \widehat{\mathbf Z}_{s, n}(\widetilde{\lambda}_{n, \textrm{EGMM}}) & = \mathbf P_{H, n}\mathbf Z_{s, n}(\widetilde{\lambda}_{n, \textrm{EGMM}}),\\ \mathbf P_{H, n} & = \mathbf H_n\left(\mathbf H^\top_n\mathbf H_n\right)^{-1}\mathbf H^\top_n. \end{aligned} \]

6.6.4.5 Step 2b: Efficient GMM estimator of $\lambda$ using GS2SLS residuals

In this last step, the efficient GMM estimator of $\lambda_0$ based on the GS2SLS residuals is obtained by minimizing the following expression: \[ \widehat{\lambda}_n = \underset{\lambda_n \in \Lambda}{\operatorname*{argmin}}\left\lbrace \left[\widehat{\mathbf G}_n \begin{pmatrix} \lambda_n \\ \lambda^2_n \end{pmatrix} - \widehat{\mathbf g}_n \right]^\top (\widehat{\boldsymbol\Psi}_n^{\widehat{\lambda}\widehat{\lambda}})^{-1} \left[\widehat{\mathbf G}_n\begin{pmatrix} \lambda_n \\ \lambda^2_n \end{pmatrix} - \widehat{\mathbf g}_n \right]\right\rbrace \] where $\widehat{\boldsymbol\Psi}_n^{\widehat{\lambda}\widehat{\lambda}}$ is an estimator for the variance-covariance matrix of the (normalized) sample moment vector based on the GS2SLS residuals. This estimator differs for the cases of homoskedastic and heteroskedastic errors.

Note that the residuals are $\widehat{\mathbf u}_n = \mathbf y_n-\mathbf Z_n\widehat{\boldsymbol\delta}_n(\widetilde{\lambda}_{n, \textrm{EGMM}})$. For the heteroskedastic case the $r, s$ (with $r,s = 1,2$) element of $\widehat{\boldsymbol\Psi}^{\widehat{\lambda}\widehat{\lambda}}$ is given by: \[ \widehat{\boldsymbol\Psi}^{\widehat{\lambda}\widehat{\lambda}}_{rs} = (2n)^{-1}\operatorname{tr}\left[\left(\mathbf A_{n,r} + \mathbf A_{n, r}^\top\right)\widehat{\boldsymbol\Sigma}_n\left(\mathbf A_{n, s} + \mathbf A_{n, s}^\top\right)\widehat{\boldsymbol\Sigma}\right] + n^{-1}\widehat{\mathbf a}_{n, r}^\top\widehat{\boldsymbol\Sigma}_n\widehat{\mathbf a}_{n, s}^\top, \tag{6.81}\] where: \[ \begin{aligned} \widehat{\boldsymbol\Sigma}_n & = \operatorname{Diag}\left(\widetilde{\epsilon}_{in}^2\right), \\ \widehat{\boldsymbol\varepsilon}_n & = \left(\mathbf I_n - \widetilde{\lambda}_{n, \textrm{EGMM}}\mathbf M_n\right)\widehat{\mathbf u}_n, \\ \widehat{\mathbf a}_{r,n} & = \mathbf H_n\widehat{\mathbf P}_n\widetilde{\boldsymbol\alpha}_{r,n}, \\ \widehat{\boldsymbol\alpha}_{r,n} & = - n^{-1}\left[\mathbf Z^\top_n\left(\mathbf I_n - \widetilde{\lambda}_{n, \textrm{EGMM}}\mathbf M_n^\top\right)(\mathbf A_{r, n} + \mathbf A_{r, n}^\top)\left(\mathbf I_n - \widetilde{\lambda}_{n, \textrm{EGMM}}\mathbf M_n\right)\widehat{\mathbf u}_n\right] \\ \widehat{\mathbf P}_n & = \left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H_n^\top\mathbf Z_{n, s} \right)\left[\left(\frac{1}{n}\mathbf H^\top\mathbf Z_{n, s}\right)^\top\left(\frac{1}{n}\mathbf H^\top_n\mathbf H_n\right)^{-1}\left(\frac{1}{n}\mathbf H^\top_n\mathbf Z_{n, s}\right)\right]^{-1} \end{aligned} \]

6.6.5 Asymptotic Distribution

The following theorem provides the joint limiting distribution of $\widehat{\lambda}_n$ and $\widehat{\boldsymbol\delta}_n$ following Arraiz et al. (2010).

Theorem 6.9 (Joint asymptotic distribution of the GS2SLS and efficient GMM estimators) Under the preceding assumptions, $\widehat{\lambda}_n$ is efficient among the class of GMM estimators based on GS2SLS residuals, and \[ \begin{pmatrix} \sqrt{n}\left(\widehat{\boldsymbol\delta}_n - \boldsymbol\delta_0\right)\\ \sqrt{n}\left(\widehat{\lambda}_n - \lambda_0\right) \end{pmatrix} \xrightarrow{d} \mathcal N\left[\mathbf 0, \operatorname{plim}_{n\to\infty}\widehat{\boldsymbol\Omega}_n(\widehat{\lambda}_n)\right], \] where $\operatorname{plim}_{n\to\infty}\widehat{\boldsymbol\Omega}_n(\widehat{\lambda}_n)$ is a positive definite matrix.

The estimator $\widehat{\boldsymbol\Omega}_n$ is given by \[ \widehat{\boldsymbol\Omega}_n = \begin{pmatrix} \widehat{\mathbf P}_n^\top & \mathbf 0 \\ \mathbf 0 & \left(\widehat{\mathbf J}_n^\top\widehat{\boldsymbol\Psi}_n^{-1}\widehat{\mathbf J}_n\right)^{-1}\widehat{\mathbf J}^\top_n\widehat{\boldsymbol\Psi}^{-1}_n \end{pmatrix} \widehat{\boldsymbol\Psi}_0 \begin{pmatrix} \widehat{\mathbf P}_n & \mathbf 0 \\ \mathbf 0 & \widehat{\boldsymbol\Psi}^{-1}_n\widehat{\mathbf J}_n\left(\widehat{\mathbf J}_n^\top\widehat{\boldsymbol\Psi}_n^{-1}\widehat{\mathbf J}_n\right)^{-1} \end{pmatrix} \tag{6.82}\] with \[ \begin{aligned} \widehat{\boldsymbol\Psi}_0 &= \begin{pmatrix} \widehat{\boldsymbol\Psi}_{n, \boldsymbol\delta\boldsymbol\delta} & \widehat{\boldsymbol\Psi}_{n, \boldsymbol\delta\lambda} \\ \widehat{\boldsymbol\Psi}_{n, \boldsymbol\delta\lambda}^\top & \widehat{\boldsymbol\Psi}_n \end{pmatrix}, \\ \widehat{\boldsymbol\Psi}_{n} & = \begin{pmatrix} \widehat{\boldsymbol\Psi}^{\widehat{\lambda}\widehat{\lambda}}_{n, 11} & \widehat{\boldsymbol\Psi}^{\widehat{\lambda}\widehat{\lambda}}_{n, 12} \\ \widehat{\boldsymbol\Psi}^{\widehat{\lambda}\widehat{\lambda}}_{n, 12} & \widehat{\boldsymbol\Psi}^{\widehat{\lambda}\widehat{\lambda}}_{n, 22} \end{pmatrix}, \\ \widehat{\boldsymbol\Psi}_{n, \boldsymbol\delta\boldsymbol\delta} & = n^{-1}\mathbf H_n^\top\widehat{\boldsymbol\Sigma}_n\mathbf H_n, \\ \widehat{\boldsymbol\Psi}_{n, \boldsymbol\delta\lambda} & = n^{-1}\mathbf H_n^\top\widehat{\boldsymbol\Sigma}_n\left[\widehat{\mathbf a}_{n,1}, \widehat{\mathbf a}_{n,2}\right],\\ \widehat{\mathbf J}_n & = \widehat{\mathbf G}_n\begin{pmatrix} 1 \\ 2 \widehat{\lambda}_n \end{pmatrix}, \\ \widehat{\mathbf G}_n & = \begin{pmatrix} 2n^{-1}\widehat{\mathbf u}_{n, s}^\top \mathbf A_{n, 1}\widehat{\mathbf u}_n & -n^{-1}\widehat{\mathbf u}_{n, s}^\top\mathbf A_{n, 1}\widehat{\mathbf u}_{n, s} \\ n^{-1}\widehat{\mathbf u}_{n, s}^\top\left(\mathbf A_{n, 2} + \mathbf A_{n, 2}^\top\right)\widehat{\mathbf u}_n & - n^{-1}\widehat{\mathbf u}_{n, s}^\top\mathbf A_{n, 2}\widehat{\mathbf u}_{n, s} \end{pmatrix} \end{aligned} \]

6.7 GMM Estimator for the SAC Model

Consider the SAC model \[ \begin{aligned} \mathbf y_n & = \rho_0\mathbf W_n\mathbf y_n + \mathbf X_n\boldsymbol\beta_0 + \mathbf u_n, \\ \mathbf u_n & = \lambda_0\mathbf M_n\mathbf u_n + \boldsymbol\varepsilon_n, \end{aligned} \] where $\epsilon_{ni}, i = 1, \ldots, n$ of $\boldsymbol\varepsilon_n$ are i.i.d. $(0, \sigma^2_0)$ with zero mean and variance $\sigma_0^2$. Let $\boldsymbol\theta = (\lambda, \rho, \boldsymbol\beta^\top)^\top$, and denote at the true $\boldsymbol\theta = \boldsymbol\theta_0$ \[ \begin{aligned} \mathbf S_0 & = \mathbf I_n - \rho_0\mathbf W_n, \\ \mathbf R_0 & = \mathbf I_n - \lambda_0 \mathbf M_n, \\ \boldsymbol\varepsilon_n & = \mathbf R_0\mathbf u_n. \end{aligned} \]

The reduced form equation of this model can be written as \[ \begin{aligned} \mathbf y_n & = \mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0 + \mathbf S_0^{-1}\mathbf R_0^{-1}\boldsymbol\varepsilon_n, \\ & = \mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0 + \mathbf F_0^{-1}\boldsymbol\varepsilon_n, \end{aligned} \] where $\mathbf F_0 = \mathbf R_0\mathbf S_0$. The empirical moment conditions are \[ \mathbf g_n(\boldsymbol\theta) = \begin{pmatrix} \left(\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top\\ \vdots \\ \left(\mathbf P_{mn}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right)^\top \\ \mathbf H_n^\top \end{pmatrix}\boldsymbol\varepsilon_n(\boldsymbol\theta) = \begin{pmatrix} \left(\mathbf P_{1n}\mathbf R_n(\lambda)\mathbf u_n(\boldsymbol\theta)\right)^\top\\ \vdots \\ \left(\mathbf P_{mn}\mathbf R_n(\lambda)\mathbf u_n(\boldsymbol\theta)\right)^\top \\ \mathbf H_n^\top \end{pmatrix}\mathbf R_n(\lambda)\mathbf u_n(\boldsymbol\theta), \] where $\mathbf P_{jn}, j = 1, \ldots, m$ are chosen from $\mathcal P_{1n}$, the IV matrix $\mathbf H_n$ is constructed from $\mathbf X_n, \mathbf W_n$ and $\mathbf M_n$.

6.8 Exercises

Exercise 6.1 Consider the following model: \[ \begin{aligned} \mathbf y & = \mathbf X\boldsymbol\beta + \mathbf u \\ \mathbf u & = \lambda\mathbf W\boldsymbol\varepsilon + \boldsymbol\varepsilon \end{aligned} \] where $\left|\lambda\right| < 1$, $\boldsymbol\varepsilon$ has zero mean and variance $\sigma^2\mathbf I_n$, respectively. Determine moment equations for a GMM approach you would use to estimate $\lambda$ and $\sigma^2$. (Hint: This model is known as the spatial moving average model for the error term).

Exercise 6.2 Consider the following model: \[ \begin{aligned} \mathbf y = \mathbf X\boldsymbol\beta + \rho_1\mathbf W_1\mathbf y + \rho_2\mathbf W_2\mathbf y+ \boldsymbol\varepsilon \end{aligned} \] where $\boldsymbol\varepsilon$ has zero mean and variance $\sigma^2\mathbf I_n$, respectively, and $\mathbf W_1$ and $\mathbf W_2$ are observed exogenous weighting matrices. Suggest an instrumental variable estimation procedure for this model which accounts for the endogeneity of $\mathbf W_1\mathbf y$ and $\mathbf W_2\mathbf y$.

Exercise 6.3 Consider the following model: \[ \begin{aligned} \mathbf y & = \mathbf X\boldsymbol\beta + \rho_1\mathbf W_1\mathbf y + \rho_2\mathbf W_2\mathbf y+ \mathbf u \\ \mathbf u & = \lambda\mathbf M\mathbf u + \boldsymbol\varepsilon \end{aligned} \] where $\boldsymbol\varepsilon$ has zero mean and variance $\sigma^2\mathbf I_n$, respectively, and $\mathbf W_1$, $\mathbf W_2$ and $\mathbf M$ are observed exogenous weighting matrices. Suggest an instrumental variable estimation procedure for this model which accounts for the endogeneity of $\mathbf W_1\mathbf y$ and $\mathbf W_2\mathbf y$, as well as for the spatially correlated term.

6.9 Technical Appendix: Asymptotic Properties

6.9.1 Asymptotic Properties of BS2SLSE

The following algebraic identities are used repeatedly in the asymptotic expansions below. We state them explicitly for later reference.

Lemma 6.2 (Difference identities for the spatial filter and its resolvent) Let \[ \mathbf S_n(\rho)\equiv \mathbf I_n-\rho \mathbf W_n, \qquad \mathbf G_n(\rho)\equiv \mathbf W_n\mathbf S_n(\rho)^{-1}, \] and fix $\rho_0$ such that $\mathbf S_n(\rho_0)$ is nonsingular. Define \[ \mathbf S_{0n}\equiv \mathbf S_n(\rho_0), \qquad \mathbf G_{0n}\equiv \mathbf G_n(\rho_0)=\mathbf W_n\mathbf S_{0n}^{-1}. \] Then, for any $\rho$ such that $\mathbf S_n(\rho)$ is nonsingular, \[ \mathbf S_n(\rho)^{-1}-\mathbf S_{0n}^{-1} = (\rho-\rho_0)\mathbf S_n(\rho)^{-1}\mathbf W_n\mathbf S_{0n}^{-1}. \tag{6.83}\]

\[ \mathbf G_n(\rho)-\mathbf G_{0n} = (\rho-\rho_0)\mathbf G_n(\rho)\mathbf G_{0n}. \tag{6.84}\] Moreover, \[ \mathbf G_n(\rho)-\mathbf G_{0n} = (\rho-\rho_0)\mathbf G_{0n}^2 + (\rho-\rho_0)^2\mathbf G_n(\rho)\mathbf G_{0n}^2. \tag{6.85}\]

Proof. Proof of Lemma Lemma 6.2. Using the general matrix identity \[ \mathbf A^{-1}-\mathbf B^{-1}=\mathbf A^{-1}(\mathbf B-\mathbf A)\mathbf B^{-1}, \] with $\mathbf A=\mathbf S_n(\rho)$ and $\mathbf B=\mathbf S_{0n}$, we obtain \[ \mathbf S_n(\rho)^{-1}-\mathbf S_{0n}^{-1} = \mathbf S_n(\rho)^{-1}\bigl(\mathbf S_{0n}-\mathbf S_n(\rho)\bigr)\mathbf S_{0n}^{-1}. \] Since \[ \mathbf S_{0n}-\mathbf S_n(\rho) = (\rho-\rho_0)\mathbf W_n, \] it follows that \[ \mathbf S_n(\rho)^{-1}-\mathbf S_{0n}^{-1} = (\rho-\rho_0)\mathbf S_n(\rho)^{-1}\mathbf W_n\mathbf S_{0n}^{-1}, \] which proves Equation 6.83.

Premultiplying by $\mathbf W_n$ gives \[ \mathbf G_n(\rho)-\mathbf G_{0n} = \mathbf W_n\bigl(\mathbf S_n(\rho)^{-1}-\mathbf S_{0n}^{-1}\bigr) = (\rho-\rho_0)\mathbf W_n\mathbf S_n(\rho)^{-1}\mathbf W_n\mathbf S_{0n}^{-1}, \] and therefore \[ \mathbf G_n(\rho)-\mathbf G_{0n} = (\rho-\rho_0)\mathbf G_n(\rho)\mathbf G_{0n}, \] which proves Equation 6.84.

Finally, since \[ \mathbf G_n(\rho)=\mathbf G_{0n}+\bigl(\mathbf G_n(\rho)-\mathbf G_{0n}\bigr), \] substituting this into Equation 6.84 yields \[ \mathbf G_n(\rho)-\mathbf G_{0n} = (\rho-\rho_0)\mathbf G_{0n}^2 + (\rho-\rho_0)\bigl(\mathbf G_n(\rho)-\mathbf G_{0n}\bigr)\mathbf G_{0n}. \] Applying Equation 6.84 again to the last term gives \[ (\rho-\rho_0)\bigl(\mathbf G_n(\rho)-\mathbf G_{0n}\bigr)\mathbf G_{0n} = (\rho-\rho_0)^2\mathbf G_n(\rho)\mathbf G_{0n}^2, \] which proves Equation 6.85.

Proof. Proof of Theorem 6.2. The argument follows the logic of Lee (2003), but we write out all steps in the SLM notation used in this chapter.

Step 0: setup and an exact decomposition. Recall the SLM $\mathbf y_n=\mathbf X_n\boldsymbol\beta_0+\rho_0\mathbf W_n\mathbf y_n+\boldsymbol\varepsilon_n$, $\mathbf S_n(\rho)\equiv \mathbf I_n-\rho\mathbf W_n$, $\mathbf S_{0n}\equiv \mathbf S_n(\rho_0)$. For notational clarity, we make the dependence on $n$ explicit and define $\mathbf G_n(\rho)\equiv \mathbf W_n\mathbf S_n(\rho)^{-1}$, $\mathbf G_{0n}\equiv \mathbf G_n(\rho_0)=\mathbf W_n\mathbf S_{0n}^{-1}$. Also define $\mathbf Z_n\equiv [\mathbf X_n,\ \mathbf W_n\mathbf y_n]$. Since $\mathbf y_n=\mathbf S_{0n}^{-1}\mathbf X_n\boldsymbol\beta_0+\mathbf S_{0n}^{-1}\boldsymbol\varepsilon_n$, we have \[ \mathbf W_n\mathbf y_n = \mathbf W_n\mathbf S_{0n}^{-1}\mathbf X_n\boldsymbol\beta_0+\mathbf W_n\mathbf S_{0n}^{-1}\boldsymbol\varepsilon_n = \mathbf G_{0n}\mathbf X_n\boldsymbol\beta_0+\mathbf G_{0n}\boldsymbol\varepsilon_n. \tag{6.86}\]

Therefore \[ \mathbf Z_n = \underbrace{\bigl[\mathbf X_n,\ \mathbf G_{0n}\mathbf X_n\boldsymbol\beta_0\bigr]}_{\mathbf H_n^*} + \underbrace{\bigl[\mathbf 0,\ \mathbf G_{0n}\boldsymbol\varepsilon_n\bigr]}_{\mathbf R_n}, \tag{6.87}\] and we can write $\mathbf Z_n=\mathbf H_n^*+\mathbf R_n$. The feasible optimal-instrument matrix is $\widetilde{\mathbf H}_n^* = \bigl[\mathbf X_n,\ \mathbf G_n(\widetilde\rho_n)\mathbf X_n\widetilde{\boldsymbol\beta}_n\bigr]$, and the BS2SLSE is $\widehat{\boldsymbol\delta}_n = \bigl(\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\bigr)^{-1}\widetilde{\mathbf H}_n^{*\top}\mathbf y_n$. Since $\mathbf y_n=\mathbf Z_n\boldsymbol\delta_0+\boldsymbol\varepsilon_n$, it follows that \[ \begin{aligned} \widehat{\boldsymbol\delta}_n-\boldsymbol\delta_0 &= \bigl(\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\bigr)^{-1}\widetilde{\mathbf H}_n^{*\top}(\mathbf Z_n\boldsymbol\delta_0+\boldsymbol\varepsilon_n)-\boldsymbol\delta_0 \nonumber\\ &= \bigl(\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\bigr)^{-1}\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n. \end{aligned} \tag{6.88}\]

Multiplying by $\sqrt n$, \[ \sqrt n(\widehat{\boldsymbol\delta}_n-\boldsymbol\delta_0) = \Bigl(\frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\Bigr)^{-1} \Bigl(\frac1{\sqrt n}\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n\Bigr). \tag{6.89}\]

Step 1: preliminary bounds for $\mathbf H_n^*$, $\widetilde{\mathbf H}_n^*$, and their difference. We will use the following results repeatedly:

Step 1.1: row-sum bounds imply entrywise bounds. If a square matrix $\mathbf A_n=(a_{ij,n})$ has row sums uniformly bounded in absolute value, i.e. $\sup_n\sup_i\sum_{j=1}^n |a_{ij,n}| \le C$, then every entry is uniformly bounded because $|a_{ij,n}| \le \sum_{\ell=1}^n |a_{i\ell,n}| \le C$.

Step 1.2: products preserve bounded row sums. See Lemma Lemma 3.6.

Step 1.3: the second column of $\mathbf H_n^*$ is uniformly bounded. Write $\mathbf g_n^*\equiv \mathbf G_{0n}\mathbf X_n\boldsymbol\beta_0$. Its $i$-th element is $g_{i,n}^* = \sum_{j=1}^n [\mathbf G_{0n}]_{ij}\,\mathbf x_{j,n}^\top\boldsymbol\beta_0$. By Assumption 9(d), the elements of $\mathbf X_n$ are uniformly bounded, so there exists a constant $C_X<\infty$ such that $\sup_{j,n} |\mathbf x_{j,n}^\top\boldsymbol\beta_0| \le C_X\|\boldsymbol\beta_0\|$. By Assumption 9(c), the row sums of $\mathbf G_{0n}=\mathbf W_n\mathbf S_{0n}^{-1}$ are uniformly bounded. Hence there exists $C_G<\infty$ such that $\sup_n\sup_i \sum_{j=1}^n |[\mathbf G_{0n}]_{ij}| \le C_G$. Therefore \[ |g_{i,n}^*| \le \sum_{j=1}^n |[\mathbf G_{0n}]_{ij}|\,|\mathbf x_{j,n}^\top\boldsymbol\beta_0| \le C_G C_X\|\boldsymbol\beta_0\|, \] uniformly in $i$ and $n$. Thus every element of $\mathbf H_n^*=[\mathbf X_n,\mathbf g_n^*]$ is uniformly bounded.

Step 1.4: the second column of $\widetilde{\mathbf H}_n^*$ is bounded in probability. Define $\widetilde{\mathbf g}_n^* \equiv \mathbf G_n(\widetilde\rho_n)\mathbf X_n\widetilde{\boldsymbol\beta}_n$, and $\mathbf G_n(\rho)\equiv \mathbf W_n\mathbf S_n(\rho)^{-1}$. By Lemma Lemma 6.1, applied with the maximum row-sum norm, there exists an open neighborhood $\mathcal B$ of $\rho_0$ and a constant $C_{\mathcal B}<\infty$ such that \[ \sup_n\sup_{\rho\in\mathcal B} \sup_{1\le i\le n} \sum_{j=1}^n \big|[\mathbf S_n(\rho)^{-1}]_{ij}\big| \le C_{\mathcal B}. \]

Also, by Assumption 9(c), there exists a constant $C_W<\infty$ such that \[ \sup_n \sup_{1\le i\le n}\sum_{j=1}^n |w_{n,ij}| \le C_W. \]

We now show that the row sums of $\mathbf G_n(\rho)=\mathbf W_n\mathbf S_n(\rho)^{-1}$ are uniformly bounded for $\rho\in\mathcal B$. Fix $i$. Then \[ \begin{aligned} \sum_{j=1}^n |[\mathbf G_n(\rho)]_{ij}| &= \sum_{j=1}^n \left|\sum_{\ell=1}^n w_{n,i\ell}[\mathbf S_n(\rho)^{-1}]_{\ell j}\right|, \\ &\le \sum_{j=1}^n \sum_{\ell=1}^n |w_{n,i\ell}|\,|[\mathbf S_n(\rho)^{-1}]_{\ell j}| \mbox{by triangle inequality}, \\ &= \sum_{\ell=1}^n |w_{n,i\ell}| \sum_{j=1}^n |[\mathbf S_n(\rho)^{-1}]_{\ell j}| \\ &\le C_{\mathcal B}\sum_{\ell=1}^n |w_{n,i\ell}| \\ &\le C_{\mathcal B} C_W. \end{aligned} \]

Hence \[ \sup_n\sup_{\rho\in\mathcal B}\sup_i \sum_{j=1}^n |[\mathbf G_n(\rho)]_{ij}| \le C_{\mathcal B}C_W. \tag{6.90}\]

Next, because $\widetilde\rho_n\xrightarrow{p}\rho_0$, we have $\Pr(\widetilde\rho_n\in\mathcal B)\to 1$. Also, $\widetilde{\boldsymbol\beta}_n\xrightarrow{p}\boldsymbol\beta_0$, so $\widetilde{\boldsymbol\beta}_n=O_p(1)$.

We now control the factor $\sup_j |\mathbf x_{j,n}^\top \widetilde{\boldsymbol\beta}_n|$. By Assumption 9(d), the elements of $\mathbf X_n$ are uniformly bounded, so there exists a constant $C_X<\infty$ such that $\sup_{n}\sup_{1\le j\le n}\sup_{1\le \ell\le k}|x_{j\ell,n}|\le C_X$. Therefore, for every $j$, \[ |\mathbf x_{j,n}^\top \widetilde{\boldsymbol\beta}_n| = \left|\sum_{\ell=1}^k x_{j\ell,n}\widetilde\beta_{\ell,n}\right| \le \sum_{\ell=1}^k |x_{j\ell,n}|\,|\widetilde\beta_{\ell,n}| \le C_X \sum_{\ell=1}^k |\widetilde\beta_{\ell,n}| = C_X\|\widetilde{\boldsymbol\beta}_n\|_1. \]

Taking the supremum over $j$, \[ \sup_j |\mathbf x_{j,n}^\top \widetilde{\boldsymbol\beta}_n| \le C_X\|\widetilde{\boldsymbol\beta}_n\|_1 = O_p(1), \] because $k$ is fixed and $\widetilde{\boldsymbol\beta}_n=O_p(1)$. Now work on the event $\{\widetilde\rho_n\in\mathcal B\}$. For each $i$, \[ \begin{aligned} |\widetilde g_{i,n}^*| &= \left|\sum_{j=1}^n [\mathbf G_n(\widetilde\rho_n)]_{ij}\,\mathbf x_{j,n}^\top\widetilde{\boldsymbol\beta}_n\right|, \\ &\le \sum_{j=1}^n |[\mathbf G_n(\widetilde\rho_n)]_{ij}|\, |\mathbf x_{j,n}^\top\widetilde{\boldsymbol\beta}_n|, \\ &\le \left(\sum_{j=1}^n |[\mathbf G_n(\widetilde\rho_n)]_{ij}|\right) \left(\sup_j |\mathbf x_{j,n}^\top\widetilde{\boldsymbol\beta}_n|\right). \end{aligned} \]

Taking the supremum over $i$, and using Equation 6.90, \[ \sup_i |\widetilde g_{i,n}^*| \le C_{\mathcal B}C_W \, \sup_j |\mathbf x_{j,n}^\top\widetilde{\boldsymbol\beta}_n| = O_p(1) \qquad \text{on } \{\widetilde\rho_n\in\mathcal B\}. \]

It remains to pass from this eventwise bound to an unconditional $O_p(1)$ result. Let $\varepsilon>0$. Since $\sup_j |\mathbf x_{j,n}^\top \widetilde{\boldsymbol\beta}_n| = O_p(1)$, by Markov’s inequality there exists $M_\varepsilon<\infty$ such that, for all sufficiently large $n$, $\Pr\!\left( \sup_j |\mathbf x_{j,n}^\top \widetilde{\boldsymbol\beta}_n| > M_\varepsilon \right) < \varepsilon$. Hence, for all sufficiently large $n$, \[ \begin{aligned} \Pr\!\left( \sup_i |\widetilde g_{i,n}^*| > C_{\mathcal B}C_W M_\varepsilon \right) &\le \Pr(\widetilde\rho_n\notin\mathcal B) + \Pr\!\left( \sup_j |\mathbf x_{j,n}^\top \widetilde{\boldsymbol\beta}_n| > M_\varepsilon \right). \end{aligned} \] The first probability tends to zero, and the second is smaller than $\varepsilon$ for all sufficiently large $n$. Therefore \[ \sup_i |\widetilde g_{i,n}^*| = O_p(1). \]

Since the elements of $\mathbf X_n$ are uniformly bounded by Assumption 9(d), it follows that every element of $\widetilde{\mathbf H}_n^*=[\mathbf X_n,\widetilde{\mathbf g}_n^*]$ is uniformly bounded in probability.

Step 1.5: the feasible and infeasible optimal instruments are uniformly close entrywise. Write \[ \widetilde{\mathbf H}_n^*-\mathbf H_n^* = [\mathbf 0,\ \mathbf D_n], \qquad \mathbf D_n\equiv \widetilde{\mathbf g}_n^*-\mathbf g_n^*. \tag{6.91}\]

Decompose $\mathbf D_n$ as \[ \mathbf D_n = \underbrace{\bigl(\mathbf G_n(\widetilde\rho_n)-\mathbf G_{0n}\bigr)\mathbf X_n\boldsymbol\beta_0}_{\mathbf D_{1n}} + \underbrace{\mathbf G_n(\widetilde\rho_n)\mathbf X_n(\widetilde{\boldsymbol\beta}_n-\boldsymbol\beta_0)}_{\mathbf D_{2n}}. \tag{6.92}\]

We first control $\mathbf G_n(\widetilde\rho_n)-\mathbf G_{0n}$. Using Lemma Lemma 6.2, we obtain \[ \mathbf G_n(\widetilde\rho_n)-\mathbf G_{0n} = (\widetilde\rho_n-\rho_0)\mathbf G_n(\widetilde\rho_n)\mathbf G_{0n}. \tag{6.93}\]

Now fix $i$. Because $\mathbf G_n(\widetilde\rho_n)$ and $\mathbf G_{0n}$ have uniformly bounded row sums (in probability for the first one, deterministically for the second), Step 1.2 implies that the product $\mathbf G_n(\widetilde\rho_n)\mathbf G_{0n}$ has uniformly bounded row sums in probability. Since $\mathbf X_n\boldsymbol\beta_0$ has uniformly bounded entries, Equation 6.93 yields \[ \sup_i |D_{1i,n}| \le |\widetilde\rho_n-\rho_0| \sup_i\sum_{j=1}^n |[\mathbf G_n(\widetilde\rho_n)\mathbf G_{0n}]_{ij}| \sup_j |\mathbf x_{j,n}^\top\boldsymbol\beta_0| = o_p(1). \]

For the second term in Equation 6.92, since $\mathbf G_n(\widetilde\rho_n)$ has row sums uniformly bounded in probability and the entries of $\mathbf X_n(\widetilde{\boldsymbol\beta}_n-\boldsymbol\beta_0)$ satisfy $\sup_j |\mathbf x_{j,n}^\top(\widetilde{\boldsymbol\beta}_n-\boldsymbol\beta_0)| \le C_X\|\widetilde{\boldsymbol\beta}_n-\boldsymbol\beta_0\| =o_p(1)$, we obtain \[ \sup_i |D_{2i,n}| \le \sup_i\sum_{j=1}^n |[\mathbf G_n(\widetilde\rho_n)]_{ij}| \sup_j |\mathbf x_{j,n}^\top(\widetilde{\boldsymbol\beta}_n-\boldsymbol\beta_0)| = o_p(1). \]

Combining the two bounds, \[ \sup_{1\le i\le n}|D_{i,n}|=o_p(1). \tag{6.94}\]

This implies three consequences that we use repeatedly: \[ \frac1n\mathbf D_n^\top\mathbf D_n = \frac1n\sum_{i=1}^n D_{i,n}^2 \le \sup_i D_{i,n}^2 = o_p(1). \tag{6.95}\]

\[ \frac1n\mathbf D_n^\top\mathbf X_n =o_p(1). \tag{6.96}\]

\[ \frac1n\mathbf D_n^\top\mathbf g_n^* =o_p(1). \tag{6.97}\]

To see Equation 6.96, note that each entry of $n^{-1}\mathbf D_n^\top\mathbf X_n$ is bounded in absolute value by $\sup_i |D_{i,n}| \cdot \sup_{i,j}|x_{ij,n}|$, which is $o_p(1)$. The proof of Equation 6.97 is identical because $\mathbf g_n^*$ is uniformly bounded by Step 1.3.

Step 2: show $\frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\xrightarrow{p} \mathbf J^*$. Using $\mathbf Z_n=\mathbf H_n^*+\mathbf R_n$ from Equation 6.87, \[ \frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n = \frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf H_n^* + \frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf R_n. \tag{6.98}\]

We treat both terms separately.

Step 2.1: the term $\frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf H_n^*$. By Equation 6.91, $\widetilde{\mathbf H}_n^*=\mathbf H_n^*+[\mathbf 0,\mathbf D_n]$. Hence \[ \begin{aligned} \frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf H_n^* &= \frac1n\mathbf H_n^{*\top}\mathbf H_n^* + \frac1n[\mathbf 0,\mathbf D_n]^\top\mathbf H_n^*, \nonumber\\ &= \frac1n\mathbf H_n^{*\top}\mathbf H_n^* + \frac1n \begin{bmatrix} \mathbf 0\\ \mathbf D_n^\top\mathbf X_n \quad \mathbf D_n^\top\mathbf g_n^* \end{bmatrix}. \end{aligned} \tag{6.99}\]

By Assumption 9(e), $\frac1n\mathbf H_n^{*\top}\mathbf H_n^* \to \mathbf J^*$. By Equation 6.96 and Equation 6.97, the second matrix on the right-hand side of Equation 6.99 is $o_p(1)$. Therefore \[ \frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf H_n^* \xrightarrow{p} \mathbf J^*. \tag{6.100}\]

Step 2.2: the term $\frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf R_n$. Recall $\mathbf R_n=[\mathbf 0,\ \mathbf G_{0n}\boldsymbol\varepsilon_n]$. Thus only the second column of $\mathbf R_n$ is nonzero. First consider the deterministic part: \[ \frac1n\mathbf H_n^{*\top}\mathbf R_n = \frac1n \begin{bmatrix} \mathbf X_n^\top\\ \mathbf g_n^{*\top} \end{bmatrix} \mathbf G_{0n}\boldsymbol\varepsilon_n. \]

We show that each component is $o_p(1)$. For a fixed column $j$ of $\mathbf X_n$, write \[ \frac1n\mathbf x_{j,n}^\top \mathbf G_{0n}\boldsymbol\varepsilon_n = \frac1n\sum_{m=1}^n a_{jm,n}\epsilon_{m,n}, \qquad a_{jm,n}\equiv \sum_{i=1}^n x_{ij,n}[\mathbf G_{0n}]_{im}. \]

Because the entries of $\mathbf X_n$ are uniformly bounded and the column sums of $\mathbf G_{0n}$ are uniformly bounded by Assumption 9(c), the coefficients $a_{jm,n}$ are uniformly bounded: \[ |a_{jm,n}| \le \sup_{i,j}|x_{ij,n}|\sum_{i=1}^n |[\mathbf G_{0n}]_{im}| \le C. \]

Hence \[ \mathbb E\!\left(\frac1n\sum_{m=1}^n a_{jm,n}\epsilon_{m,n}\right)=0, \qquad \operatorname{Var}\!\left(\frac1n\sum_{m=1}^n a_{jm,n}\epsilon_{m,n}\right) = \frac{\sigma_\epsilon^2}{n^2}\sum_{m=1}^n a_{jm,n}^2 \le \frac{C}{n}. \]

Therefore by Chebyshev’s inequality, \[ \frac1n\mathbf x_{j,n}^\top \mathbf G_{0n}\boldsymbol\varepsilon_n=o_p(1). \]

The same argument applies to the last component: \[ \frac1n\mathbf g_n^{*\top}\mathbf G_{0n}\boldsymbol\varepsilon_n = \frac1n\sum_{m=1}^n b_{m,n}\epsilon_{m,n}, \qquad b_{m,n}\equiv \sum_{i=1}^n g_{i,n}^*[\mathbf G_{0n}]_{im}. \]

Since $\sup_i|g_{i,n}^*|\le C$ by Step 1.3 and the column sums of $\mathbf G_{0n}$ are uniformly bounded, \[ |b_{m,n}| \le \sup_i|g_{i,n}^*|\sum_{i=1}^n |[\mathbf G_{0n}]_{im}| \le C. \]

Hence \[ \frac1n\mathbf g_n^{*\top}\mathbf G_{0n}\boldsymbol\varepsilon_n=o_p(1). \]

We conclude that \[ \frac1n\mathbf H_n^{*\top}\mathbf R_n=o_p(1). \tag{6.101}\]

Now consider the feasible correction: \[ \frac1n[\mathbf 0,\mathbf D_n]^\top\mathbf R_n = \frac1n \begin{bmatrix} \mathbf 0\\ \mathbf D_n^\top \mathbf G_{0n}\boldsymbol\varepsilon_n \end{bmatrix}. \]

By Cauchy–Schwarz, \[ \left|\frac1n\mathbf D_n^\top\mathbf G_{0n}\boldsymbol\varepsilon_n\right| \le \left(\frac1n\mathbf D_n^\top\mathbf D_n\right)^{1/2} \left(\frac1n\boldsymbol\varepsilon_n^\top\mathbf G_{0n}^\top\mathbf G_{0n}\boldsymbol\varepsilon_n\right)^{1/2}. \tag{6.102}\]

The first factor is $o_p(1)$ by Equation 6.95. It remains to show that the second factor is $O_p(1)$. Observe that \[ \mathbb E\!\left(\frac1n\boldsymbol\varepsilon_n^\top\mathbf G_{0n}^\top\mathbf G_{0n}\boldsymbol\varepsilon_n\right) = \frac{\sigma_\epsilon^2}{n}\operatorname{tr}(\mathbf G_{0n}^\top\mathbf G_{0n}) = \frac{\sigma_\epsilon^2}{n}\sum_{i=1}^n\sum_{j=1}^n [\mathbf G_{0n}]_{ij}^2. \]

Because the entries of $\mathbf G_{0n}$ are uniformly bounded (Step 1.1) and its row sums are uniformly bounded, we have \[ \sum_{i=1}^n\sum_{j=1}^n [\mathbf G_{0n}]_{ij}^2 \le \Bigl(\sup_{i,j}|[\mathbf G_{0n}]_{ij}|\Bigr) \sum_{i=1}^n\sum_{j=1}^n |[\mathbf G_{0n}]_{ij}| \le C\cdot Cn = O(n). \]

Hence \[ \mathbb E\!\left(\frac1n\boldsymbol\varepsilon_n^\top\mathbf G_{0n}^\top\mathbf G_{0n}\boldsymbol\varepsilon_n\right)=O(1), \] so by Markov’s inequality, \[ \frac1n\boldsymbol\varepsilon_n^\top\mathbf G_{0n}^\top\mathbf G_{0n}\boldsymbol\varepsilon_n=O_p(1). \]

Therefore Equation 6.102 implies \[ \frac1n[\mathbf 0,\mathbf D_n]^\top\mathbf R_n=o_p(1). \tag{6.103}\]

Combining Equation 6.101 and Equation 6.103, \[ \frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf R_n=o_p(1). \tag{6.104}\]

Finally, Equation 6.98, Equation 6.100, and Equation 6.104 give \[ \frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n \xrightarrow{p} \mathbf J^*. \tag{6.105}\] Since $\mathbf J^*$ is nonsingular by Assumption 9(e), continuity of inversion implies \[ \Bigl(\frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\Bigr)^{-1} \xrightarrow{p} \mathbf J^{*-1}, \qquad\text{hence}\qquad \Bigl\|\Bigl(\frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\Bigr)^{-1}\Bigr\|=O_p(1). \tag{6.106}\]

Step 3: consistency. From Equation 6.88, it remains to prove $\frac1n\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n=o_p(1)$. Write \[ \frac1n\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n = \frac1n\mathbf H_n^{*\top}\boldsymbol\varepsilon_n + \frac1n[\mathbf 0,\mathbf D_n]^\top\boldsymbol\varepsilon_n. \]

For the first term, every component is $o_p(1)$. For the $j$-th element of the $k$-vector $n^{-1}\mathbf X_n^\top\boldsymbol\varepsilon_n$, \[ \frac1n\mathbf x_{j,n}^\top\boldsymbol\varepsilon_n = \frac1n\sum_{i=1}^n x_{ij,n}\epsilon_{i,n}, \] which has mean zero and variance \[ \operatorname{Var}\!\left(\frac1n\sum_{i=1}^n x_{ij,n}\epsilon_{i,n}\right) = \frac{\sigma_\epsilon^2}{n^2}\sum_{i=1}^n x_{ij,n}^2 \le \frac{C}{n}. \]

Hence $n^{-1}\mathbf X_n^\top\boldsymbol\varepsilon_n=o_p(1)$. Similarly, \[ \frac1n\mathbf g_n^{*\top}\boldsymbol\varepsilon_n = \frac1n\sum_{i=1}^n g_{i,n}^*\epsilon_{i,n}, \] has mean zero and variance \[ \operatorname{Var}\!\left(\frac1n\sum_{i=1}^n g_{i,n}^*\epsilon_{i,n}\right) = \frac{\sigma_\epsilon^2}{n^2}\sum_{i=1}^n (g_{i,n}^*)^2 \le \frac{C}{n}, \] because $\sup_i|g_{i,n}^*|\le C$. Therefore \[ \frac1n\mathbf H_n^{*\top}\boldsymbol\varepsilon_n=o_p(1). \tag{6.107}\]

For the second term, \[ \left|\frac1n\mathbf D_n^\top\boldsymbol\varepsilon_n\right| \le \left(\frac1n\mathbf D_n^\top\mathbf D_n\right)^{1/2} \left(\frac1n\boldsymbol\varepsilon_n^\top\boldsymbol\varepsilon_n\right)^{1/2} \] by Cauchy–Schwarz. The first factor is $o_p(1)$ by Equation 6.95. The second factor is $O_p(1)$ because \[ \frac1n\boldsymbol\varepsilon_n^\top\boldsymbol\varepsilon_n=\frac1n\sum_{i=1}^n \epsilon_{i,n}^2 \xrightarrow{p} \sigma_\epsilon^2 \] by the LLN under Assumption 9(a). Hence \[ \frac1n[\mathbf 0,\mathbf D_n]^\top\boldsymbol\varepsilon_n=o_p(1). \tag{6.108}\]

Combining Equation 6.107 and Equation 6.108, \[ \frac1n\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n=o_p(1). \]

Now Equation 6.88 and Equation 6.106 imply \[ \widehat{\boldsymbol\delta}_n-\boldsymbol\delta_0 = \Bigl(\frac1n\widetilde{\mathbf H}_n^{*\top}\mathbf Z_n\Bigr)^{-1} \Bigl(\frac1n\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n\Bigr) =o_p(1). \] Thus $\widehat{\boldsymbol\delta}_n\xrightarrow{p} \boldsymbol\delta_0$.

Step 4: asymptotic normality. Start from the exact identity Equation 6.89. By Equation 6.106, it is enough to derive the limit of $\frac1{\sqrt n}\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n$. Decompose: \[ \frac1{\sqrt n}\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n = \frac1{\sqrt n}\mathbf H_n^{*\top}\boldsymbol\varepsilon_n + \frac1{\sqrt n}[\mathbf 0,\mathbf D_n]^\top\boldsymbol\varepsilon_n. \tag{6.109}\]

Step 4.1: CLT for $\frac1{\sqrt n}\mathbf H_n^{*\top}\boldsymbol\varepsilon_n$. At this point, one could proceed in two equivalent ways. First, Theorem 3.21 applies directly. Indeed, take $\mathbf v_n=\boldsymbol\varepsilon_n$, $\mathbf A_n=\mathbf H_n^*$. By Step 1.3, the entries of $\mathbf H_n^*$ are uniformly bounded, and by Step 4 the matrix $\frac1n\mathbf H_n^{*\top}\mathbf H_n^* \to \mathbf J^*$, where $\mathbf J^*$ is finite and nonsingular. Therefore, Theorem 3.21 immediately gives \[ \frac1{\sqrt n}\mathbf H_n^{*\top}\boldsymbol\varepsilon_n \xrightarrow{d} \mathcal N(\mathbf 0,\sigma_\epsilon^2\mathbf J^*). \tag{6.110}\]

However, since the argument is short and illustrates the use of triangular-array CLTs, we also present it using Theorem 3.21 together with the Cramér–Wold device. Let $\mathbf a\in\mathbb R^{k+1}$ be fixed and nonzero. Then \[ \mathbf a^\top\frac1{\sqrt n}\mathbf H_n^{*\top}\boldsymbol\varepsilon_n = \frac1{\sqrt n}\sum_{i=1}^n c_{i,n}\epsilon_{i,n}, \qquad c_{i,n}\equiv \mathbf a^\top \mathbf h_{i,n}^*, \] where $\mathbf h_{i,n}^{*\top}$ is the $i$-th row of $\mathbf H_n^*$.

Define \[ X_{ni}\equiv \frac{c_{i,n}\epsilon_{i,n}}{\sqrt n}, \qquad i=1,\ldots,n. \]

Then, for each fixed $n$, the variables $X_{n1},\ldots,X_{nn}$ are independent because $\epsilon_{1,n},\ldots,\epsilon_{n,n}$ are independent by Assumption 9(a). Moreover, $\mathbb E(X_{ni})=0$, and $\operatorname{Var}(X_{ni})=\frac{c_{i,n}^2\sigma_\epsilon^2}{n}$. By Step 1.3, every entry of $\mathbf H_n^*$ is uniformly bounded. Since $\mathbf a$ is fixed, there exists a constant $C(\mathbf a)<\infty$ such that $\sup_{i,n}|c_{i,n}| \le C(\mathbf a)$. Therefore, \[ \sup_{i,n}\mathbb E|X_{ni}|^4 = \sup_{i,n}\frac{|c_{i,n}|^4}{n^2}\mathbb E|\epsilon_{i,n}|^4 \le \frac{C}{n^2}, \] for some finite constant $C$, because the fourth moments of $\epsilon_{i,n}$ are finite by Assumption 9(a).

Next, let $B_n^2\equiv \sum_{i=1}^n \operatorname{Var}(X_{ni})$. Then \[ \begin{aligned} B_n^2 &= \sum_{i=1}^n \frac{c_{i,n}^2\sigma_\epsilon^2}{n} = \frac{\sigma_\epsilon^2}{n}\sum_{i=1}^n c_{i,n}^2 \\ &= \sigma_\epsilon^2\, \mathbf a^\top\Bigl(\frac1n\mathbf H_n^{*\top}\mathbf H_n^*\Bigr)\mathbf a \to \sigma_\epsilon^2\,\mathbf a^\top\mathbf J^*\mathbf a. \end{aligned} \] Because $\mathbf J^*$ is nonsingular, $\mathbf a^\top\mathbf J^*\mathbf a>0$ for every nonzero $\mathbf a$. Hence $B_n^2\to B^2$ for some $B^2>0$.

We now verify Lyapunov’s condition with $\delta=2$, as stated in Theorem 3.21. Since $\mathbb E(X_{ni})=0$, \[ \frac{1}{B_n^4}\sum_{i=1}^n \mathbb E|X_{ni}|^4 = \frac{1}{B_n^4} \sum_{i=1}^n \frac{|c_{i,n}|^4}{n^2}\mathbb E|\epsilon_{i,n}|^4. \] Using the uniform bound on $|c_{i,n}|$ and the finiteness of the fourth moments, \[ \sum_{i=1}^n \mathbb E|X_{ni}|^4 \le \frac{C}{n^2}\cdot n = O(n^{-1}) \to 0. \] Since $B_n^2\to B^2>0$, we also have $B_n^4\to B^4>0$, and therefore \[ \frac{1}{B_n^4}\sum_{i=1}^n \mathbb E|X_{ni}|^4 \to 0. \] Thus the Lyapunov condition of Theorem 3.21 holds, and \[ \sum_{i=1}^n X_{ni} = \mathbf a^\top\frac1{\sqrt n}\mathbf H_n^{*\top}\boldsymbol\varepsilon_n \xrightarrow{d} \mathcal N\!\left(0,\ \sigma_\epsilon^2\,\mathbf a^\top\mathbf J^*\mathbf a\right). \]

Since this holds for every fixed $\mathbf a\in\mathbb R^{k+1}$, the Cramér–Wold device yields \[ \frac1{\sqrt n}\mathbf H_n^{*\top}\boldsymbol\varepsilon_n \xrightarrow{d} \mathcal N\!\left(\mathbf 0,\ \sigma_\epsilon^2\mathbf J^*\right). \tag{6.111}\]

Step 4.2: the remainder term is $o_p(1)$. Using Cauchy–Schwarz, \[ \left| \frac1{\sqrt n}\mathbf D_n^\top\boldsymbol\varepsilon_n \right| \le \left(\frac1n\mathbf D_n^\top\mathbf D_n\right)^{1/2} \left(\frac1n\boldsymbol\varepsilon_n^\top\boldsymbol\varepsilon_n\right)^{1/2}. \] The first factor is $o_p(1)$ by Equation 6.95, and the second factor is $O_p(1)$ by the LLN. Hence \[ \frac1{\sqrt n}[\mathbf 0,\mathbf D_n]^\top\boldsymbol\varepsilon_n=o_p(1). \tag{6.112}\]

Step 4.3: combine pieces. From Equation 6.109, Equation 6.111, and Equation 6.112, \[ \frac1{\sqrt n}\widetilde{\mathbf H}_n^{*\top}\boldsymbol\varepsilon_n \xrightarrow{d} \mathcal N\!\left(\mathbf 0,\ \sigma_\epsilon^2\mathbf J^*\right). \tag{6.113}\] Finally, Equation 6.89, Equation 6.106, and Equation 6.113 imply by Slutsky’s theorem that \[ \sqrt n(\widehat{\boldsymbol\delta}_n-\boldsymbol\delta_0) \xrightarrow{d} \mathcal N\!\left(\mathbf 0,\ \mathbf J^{*-1}(\sigma_\epsilon^2\mathbf J^*)\mathbf J^{*-1}\right) = \mathcal N\!\left(\mathbf 0,\ \sigma_\epsilon^2\mathbf J^{*-1}\right). \] This is exactly Equation 6.34.

6.9.2 Asymptotic Properties of GMME for SLM

6.9.2.1 Consistency of GMM estimator

Proof. Proof (consistency) for the GMM estimator of the SLM in Theorem 6.3. Step 0 (notation and reduced form). Let \[ \mathbf S_n(\rho)\equiv \mathbf I_n-\rho\mathbf W_n, \qquad \mathbf S_0\equiv \mathbf S_n(\rho_0), \qquad \mathbf G_0\equiv \mathbf W_n\mathbf S_0^{-1}. \] Let $\boldsymbol\varepsilon_n\equiv \boldsymbol\varepsilon_n(\boldsymbol\theta_0)$ denote the innovation vector, so that \[ \mathbf y_n=\mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0+\mathbf S_0^{-1}\boldsymbol\varepsilon_n, \qquad \mathbb E(\boldsymbol\varepsilon_n)=\mathbf 0, \qquad \mathbb E(\boldsymbol\varepsilon_n\boldsymbol\varepsilon_n^\top)=\sigma_0^2\mathbf I_n. \] For any $\boldsymbol\theta=(\rho,\boldsymbol\beta^\top)^\top$, define \[ \boldsymbol\varepsilon_n(\boldsymbol\theta)\equiv \mathbf S_n(\rho)\mathbf y_n-\mathbf X_n\boldsymbol\beta. \]

Step 1 (two basic algebraic identities). Because \[ \mathbf S_n(\rho)=\mathbf S_0+(\rho_0-\rho)\mathbf W_n, \] we have \[ \mathbf S_n(\rho)\mathbf S_0^{-1} = \mathbf I_n+(\rho_0-\rho)\mathbf W_n\mathbf S_0^{-1} = \mathbf I_n+(\rho_0-\rho)\mathbf G_0. \tag{6.114}\] Substituting the reduced form of $\mathbf y_n$ into $\boldsymbol\varepsilon_n(\boldsymbol\theta)$ gives \[ \begin{aligned} \boldsymbol\varepsilon_n(\boldsymbol\theta) &= \mathbf S_n(\rho)\mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0-\mathbf X_n\boldsymbol\beta +\mathbf S_n(\rho)\mathbf S_0^{-1}\boldsymbol\varepsilon_n \nonumber\\ &\equiv \mathbf d_n(\boldsymbol\theta)+\mathbf S_n(\rho)\mathbf S_0^{-1}\boldsymbol\varepsilon_n, \end{aligned} \tag{6.115}\] where \[ \mathbf d_n(\boldsymbol\theta)\equiv \mathbf S_n(\rho)\mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0-\mathbf X_n\boldsymbol\beta. \] Using Equation 6.114, \[ \mathbf d_n(\boldsymbol\theta) = \mathbf X_n(\boldsymbol\beta_0-\boldsymbol\beta)+(\rho_0-\rho)\mathbf G_0\mathbf X_n\boldsymbol\beta_0. \tag{6.116}\]

Step 2 (population moment map). Let \[ \mathbf g_n(\boldsymbol\theta)\equiv \begin{pmatrix} \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\\[2mm] \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}\boldsymbol\varepsilon_n(\boldsymbol\theta)\\[2mm] \mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) \end{pmatrix}, \qquad \bar{\mathbf Q}_n(\boldsymbol\theta)\equiv \frac{1}{n}\mathbb E[\mathbf g_n(\boldsymbol\theta)]. \] Assume $\operatorname{tr}(\mathbf P_{jn})=0$ for $j=1,2$.

Step 2(a): linear block. From Equation 6.115, \[ \mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta) = \mathbf H_n^\top\mathbf d_n(\boldsymbol\theta)+\mathbf H_n^\top\mathbf S_n(\rho)\mathbf S_0^{-1}\boldsymbol\varepsilon_n. \] Taking expectations and using $\mathbb E(\boldsymbol\varepsilon_n)=\mathbf 0$, \[ \mathbb E[\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)] = \mathbf H_n^\top\mathbf d_n(\boldsymbol\theta). \tag{6.117}\]

Step 2(b): quadratic block. Fix $j\in\{1,2\}$. Using Equation 6.115, \[ \begin{aligned} \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta) &= \mathbf d_n(\boldsymbol\theta)^\top\mathbf P_{jn}\mathbf d_n(\boldsymbol\theta) +2\,\mathbf d_n(\boldsymbol\theta)^\top\mathbf P_{jn}\mathbf S_n(\rho)\mathbf S_0^{-1}\boldsymbol\varepsilon_n \\ &\quad +\boldsymbol\varepsilon_n^\top \mathbf S_0^{-1\top}\mathbf S_n(\rho)^\top\mathbf P_{jn}\mathbf S_n(\rho)\mathbf S_0^{-1}\boldsymbol\varepsilon_n. \end{aligned} \] The middle term has expectation zero because $\mathbf d_n(\boldsymbol\theta)$ is nonrandom and $\mathbb E(\boldsymbol\varepsilon_n)=\mathbf 0$. Hence \[ \mathbb E\!\left[\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right] = \mathbf d_n(\boldsymbol\theta)^\top\mathbf P_{jn}\mathbf d_n(\boldsymbol\theta) + \mathbb E\!\left[\boldsymbol\varepsilon_n^\top\mathbf B_{jn}(\rho)\boldsymbol\varepsilon_n\right], \] where \[ \mathbf B_{jn}(\rho)\equiv \mathbf S_0^{-1\top}\mathbf S_n(\rho)^\top\mathbf P_{jn}\mathbf S_n(\rho)\mathbf S_0^{-1}. \] Using the standard trace identity, \[ \mathbb E(\boldsymbol\varepsilon_n^\top \mathbf B \boldsymbol\varepsilon_n) = \operatorname{tr}\!\bigl(\mathbf B\,\mathbb E(\boldsymbol\varepsilon_n\boldsymbol\varepsilon_n^\top)\bigr) = \sigma_0^2\operatorname{tr}(\mathbf B), \] we obtain \[ \mathbb E\!\left[\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta)\right] = \mathbf d_n(\boldsymbol\theta)^\top\mathbf P_{jn}\mathbf d_n(\boldsymbol\theta) +\sigma_0^2\operatorname{tr}\!\left(\mathbf B_{jn}(\rho)\right). \tag{6.118}\]

Step 2(c): make the $\rho$-dependence explicit. Let $a\equiv \rho_0-\rho$. By Equation 6.114, \[ \mathbf S_n(\rho)\mathbf S_0^{-1}=\mathbf I_n+a\mathbf G_0. \] Hence \[ \mathbf B_{jn}(\rho) = (\mathbf I_n+a\mathbf G_0)^\top\mathbf P_{jn}(\mathbf I_n+a\mathbf G_0). \] Expanding the trace, \[ \begin{aligned} \operatorname{tr}(\mathbf B_{jn}(\rho)) &= \operatorname{tr}(\mathbf P_{jn}) +a\,\operatorname{tr}(\mathbf P_{jn}\mathbf G_0) +a\,\operatorname{tr}(\mathbf G_0^\top\mathbf P_{jn}) +a^2\operatorname{tr}(\mathbf G_0^\top\mathbf P_{jn}\mathbf G_0). \end{aligned} \] Since $\operatorname{tr}(\mathbf P_{jn})=0$ and $\mathbf P_{jn}^s\equiv \mathbf P_{jn}+\mathbf P_{jn}^\top$, \[ \operatorname{tr}(\mathbf B_{jn}(\rho)) = a\,\operatorname{tr}(\mathbf P_{jn}^s\mathbf G_0) +a^2\operatorname{tr}(\mathbf G_0^\top\mathbf P_{jn}\mathbf G_0). \tag{6.119}\]

Step 2(d): identification. By Equation 6.116, Equation 6.117, Equation 6.118, and Equation 6.119, each component of $\bar{\mathbf Q}_n(\boldsymbol\theta)$ is a polynomial in $(\rho,\boldsymbol\beta)$ with deterministic coefficients determined by the limits in Assumption 10. By the identification part of that assumption, the limiting scaled population moments have a unique zero at $\boldsymbol\theta_0$. Thus, \[ \lim_{n\to\infty}\bar{\mathbf Q}_n(\boldsymbol\theta)=\mathbf 0 \quad\Longrightarrow\quad \boldsymbol\theta=\boldsymbol\theta_0. \]

Step 3 (uniform convergence of the normalized moments). Define \[ \bar{\mathbf g}_n(\boldsymbol\theta)\equiv \frac{1}{n}\mathbf g_n(\boldsymbol\theta). \] We prove \[ \sup_{\boldsymbol\theta\in\Theta} \left\| \bar{\mathbf g}_n(\boldsymbol\theta)-\bar{\mathbf Q}_n(\boldsymbol\theta) \right\| \xrightarrow{p} 0. \tag{6.120}\]

Step 3(a): linear block. From Equation 6.115 and Equation 6.114, \[ \frac1n\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)-\frac1n\mathbb E[\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)] = \frac1n\mathbf H_n^\top(\mathbf I_n+a\mathbf G_0)\boldsymbol\varepsilon_n, \qquad a=\rho_0-\rho. \] Therefore \[ \sup_{\rho\in\Theta_\rho} \left\| \frac1n\mathbf H_n^\top(\mathbf I_n+a\mathbf G_0)\boldsymbol\varepsilon_n \right\| \le \left\|\frac1n\mathbf H_n^\top\boldsymbol\varepsilon_n\right\| + \sup_{\rho\in\Theta_\rho}|a| \left\|\frac1n\mathbf H_n^\top\mathbf G_0\boldsymbol\varepsilon_n\right\|. \] Because $\Theta_\rho$ is bounded, $\sup_{\rho\in\Theta_\rho}|a|<\infty$. By Assumption 10(c)–(d), the entries of $\mathbf H_n$ are uniformly bounded and $\mathbf G_0$ is uniformly bounded in row and column sums. Hence the rows of $\mathbf H_n^\top$ and $\mathbf H_n^\top\mathbf G_0$ have uniformly bounded entries. Applying Lee’s Lemma A.4 componentwise, \[ \frac1n\mathbf H_n^\top\boldsymbol\varepsilon_n=o_p(1), \qquad \frac1n\mathbf H_n^\top\mathbf G_0\boldsymbol\varepsilon_n=o_p(1). \] Thus \[ \sup_{\rho\in\Theta_\rho} \left\| \frac1n\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)-\frac1n\mathbb E[\mathbf H_n^\top\boldsymbol\varepsilon_n(\boldsymbol\theta)] \right\| =o_p(1). \tag{6.121}\]

Step 3(b): quadratic block. Fix $j\in\{1,2\}$ and define \[ R_{jn}(\boldsymbol\theta) \equiv \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta) - \mathbb E[\boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n(\boldsymbol\theta)]. \] We show \[ \sup_{\boldsymbol\theta\in\Theta}\left|\frac1n R_{jn}(\boldsymbol\theta)\right|=o_p(1). \tag{6.122}\]

Using Equation 6.115 and Equation 6.114, \[ \boldsymbol\varepsilon_n(\boldsymbol\theta)=\mathbf d_n(\boldsymbol\theta)+(\mathbf I_n+a\mathbf G_0)\boldsymbol\varepsilon_n, \qquad a=\rho_0-\rho. \] Hence \[ R_{jn}(\boldsymbol\theta)=2\,\ell_{jn}(\boldsymbol\theta)+u_{jn}(\boldsymbol\theta), \] where \[ \ell_{jn}(\boldsymbol\theta) = \mathbf d_n(\boldsymbol\theta)^\top\mathbf P_{jn}(\mathbf I_n+a\mathbf G_0)\boldsymbol\varepsilon_n, \] and \[ u_{jn}(\boldsymbol\theta) = \boldsymbol\varepsilon_n^\top(\mathbf I_n+a\mathbf G_0)^\top\mathbf P_{jn}(\mathbf I_n+a\mathbf G_0)\boldsymbol\varepsilon_n - \mathbb E\!\left[\boldsymbol\varepsilon_n^\top(\mathbf I_n+a\mathbf G_0)^\top\mathbf P_{jn}(\mathbf I_n+a\mathbf G_0)\boldsymbol\varepsilon_n\right]. \]

Linear part. Using Equation 6.116, \[ \mathbf d_n(\boldsymbol\theta)=\mathbf X_n(\boldsymbol\beta_0-\boldsymbol\beta)+a\,\mathbf G_0\mathbf X_n\boldsymbol\beta_0. \] Substituting, \[ \begin{aligned} \ell_{jn}(\boldsymbol\theta) &= (\boldsymbol\beta_0-\boldsymbol\beta)^\top\mathbf X_n^\top\mathbf P_{jn}\boldsymbol\varepsilon_n +a(\boldsymbol\beta_0-\boldsymbol\beta)^\top\mathbf X_n^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n \\ &\quad +a(\mathbf X_n\boldsymbol\beta_0)^\top\mathbf G_0^\top\mathbf P_{jn}\boldsymbol\varepsilon_n +a^2(\mathbf X_n\boldsymbol\beta_0)^\top\mathbf G_0^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n. \end{aligned} \] The dependence on $(\rho,\boldsymbol\beta)$ appears only through the coefficients \[ (\boldsymbol\beta_0-\boldsymbol\beta),\qquad a(\boldsymbol\beta_0-\boldsymbol\beta),\qquad a,\qquad a^2, \] which are uniformly bounded because $\Theta$ is bounded. Now consider the four random linear forms: \[ \mathbf X_n^\top\mathbf P_{jn}\boldsymbol\varepsilon_n,\qquad \mathbf X_n^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n,\qquad (\mathbf G_0\mathbf X_n)^\top\mathbf P_{jn}\boldsymbol\varepsilon_n,\qquad (\mathbf G_0\mathbf X_n)^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n. \] Since $\mathbf X_n$ has uniformly bounded entries and $\mathbf P_{jn}$, $\mathbf G_0$ are uniformly bounded in row and column sums, each corresponding coefficient matrix has uniformly bounded rows or columns. Therefore, by Lee’s Lemma A.4, each of these linear forms divided by $n$ is $o_p(1)$. Since their coefficients are uniformly bounded over $\Theta$, it follows that \[ \sup_{\boldsymbol\theta\in\Theta}\left|\frac1n \ell_{jn}(\boldsymbol\theta)\right|=o_p(1). \tag{6.123}\]

Quadratic part. Let \[ \mathbf C_{jn}(a)\equiv (\mathbf I_n+a\mathbf G_0)^\top\mathbf P_{jn}(\mathbf I_n+a\mathbf G_0). \] Then \[ u_{jn}(\boldsymbol\theta)=\boldsymbol\varepsilon_n^\top\mathbf C_{jn}(a)\boldsymbol\varepsilon_n-\mathbb E[\boldsymbol\varepsilon_n^\top\mathbf C_{jn}(a)\boldsymbol\varepsilon_n]. \] Expanding $\mathbf C_{jn}(a)$, \[ \mathbf C_{jn}(a) = \mathbf P_{jn} +a\,\mathbf G_0^\top\mathbf P_{jn} +a\,\mathbf P_{jn}\mathbf G_0 +a^2\mathbf G_0^\top\mathbf P_{jn}\mathbf G_0. \] Hence \[ \begin{aligned} u_{jn}(\boldsymbol\theta) &= \Bigl(\boldsymbol\varepsilon_n^\top\mathbf P_{jn}\boldsymbol\varepsilon_n-\mathbb E[\boldsymbol\varepsilon_n^\top\mathbf P_{jn}\boldsymbol\varepsilon_n]\Bigr) \\ &\quad +a\Bigl(\boldsymbol\varepsilon_n^\top\mathbf G_0^\top\mathbf P_{jn}\boldsymbol\varepsilon_n-\mathbb E[\boldsymbol\varepsilon_n^\top\mathbf G_0^\top\mathbf P_{jn}\boldsymbol\varepsilon_n]\Bigr) \\ &\quad +a\Bigl(\boldsymbol\varepsilon_n^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n-\mathbb E[\boldsymbol\varepsilon_n^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n]\Bigr) \\ &\quad +a^2\Bigl(\boldsymbol\varepsilon_n^\top\mathbf G_0^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n-\mathbb E[\boldsymbol\varepsilon_n^\top\mathbf G_0^\top\mathbf P_{jn}\mathbf G_0\boldsymbol\varepsilon_n]\Bigr). \end{aligned} \] Again, the coefficients $1,a,a,a^2$ are uniformly bounded over $\Theta$. Moreover, the matrices \[ \mathbf P_{jn},\qquad \mathbf G_0^\top\mathbf P_{jn},\qquad \mathbf P_{jn}\mathbf G_0,\qquad \mathbf G_0^\top\mathbf P_{jn}\mathbf G_0 \] are uniformly bounded in row and column sums. Therefore, Lee’s Lemma A.3 applies to each centered quadratic form and yields \[ \frac1n\Bigl(\boldsymbol\varepsilon_n^\top\mathbf A_n\boldsymbol\varepsilon_n-\mathbb E[\boldsymbol\varepsilon_n^\top\mathbf A_n\boldsymbol\varepsilon_n]\Bigr)=o_p(1) \] for each such matrix sequence $\{\mathbf A_n\}$. Since the coefficients are uniformly bounded, \[ \sup_{\boldsymbol\theta\in\Theta}\left|\frac1n u_{jn}(\boldsymbol\theta)\right|=o_p(1). \tag{6.124}\]

Combining Equation 6.123 and Equation 6.124 proves Equation 6.122.

Step 3(c): conclude the moment ULLN. Stacking Equation 6.121 and Equation 6.122 for $j=1,2$, we obtain \[ \sup_{\boldsymbol\theta\in\Theta} \left\| \bar{\mathbf g}_n(\boldsymbol\theta)-\bar{\mathbf Q}_n(\boldsymbol\theta) \right\| =o_p(1). \]

Step 4 (uniform convergence of the criterion function). Define the normalized sample criterion \[ Q_n(\boldsymbol\theta)\equiv \bar{\mathbf g}_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_n \bar{\mathbf g}_n(\boldsymbol\theta). \] Let \[ Q(\boldsymbol\theta)\equiv \lim_{n\to\infty}\bar{\mathbf Q}_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_0 \bar{\mathbf Q}_n(\boldsymbol\theta). \] We show \[ \sup_{\boldsymbol\theta\in\Theta}|Q_n(\boldsymbol\theta)-Q(\boldsymbol\theta)|\xrightarrow{p} 0. \]

First, \[ Q_n(\boldsymbol\theta)-\bar{\mathbf Q}_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_n \bar{\mathbf Q}_n(\boldsymbol\theta) = \bigl(\bar{\mathbf g}_n(\boldsymbol\theta)-\bar{\mathbf Q}_n(\boldsymbol\theta)\bigr)^\top \boldsymbol\Upsilon_n \bar{\mathbf g}_n(\boldsymbol\theta) + \bar{\mathbf Q}_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_n \bigl(\bar{\mathbf g}_n(\boldsymbol\theta)-\bar{\mathbf Q}_n(\boldsymbol\theta)\bigr). \] Since $\boldsymbol\Upsilon_n\to\boldsymbol\Upsilon_0$, the matrices $\boldsymbol\Upsilon_n$ are stochastically bounded. Also, $\sup_{\boldsymbol\theta\in\Theta}\|\bar{\mathbf g}_n(\boldsymbol\theta)\|=O_p(1)$, because \[ \sup_{\boldsymbol\theta\in\Theta}\|\bar{\mathbf g}_n(\boldsymbol\theta)\| \le \sup_{\boldsymbol\theta\in\Theta}\|\bar{\mathbf g}_n(\boldsymbol\theta)-\bar{\mathbf Q}_n(\boldsymbol\theta)\| + \sup_{\boldsymbol\theta\in\Theta}\|\bar{\mathbf Q}_n(\boldsymbol\theta)\|, \] the first term is $o_p(1)$ from Step 3, and the second is $O(1)$ because each component of $\bar{\mathbf Q}_n(\boldsymbol\theta)$ is a polynomial in $(\rho,\boldsymbol\beta)$ with bounded coefficients and $\Theta$ is bounded. Therefore, \[ \sup_{\boldsymbol\theta\in\Theta} \left| Q_n(\boldsymbol\theta)-\bar{\mathbf Q}_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_n \bar{\mathbf Q}_n(\boldsymbol\theta) \right| =o_p(1). \]

Second, because $\boldsymbol\Upsilon_n\to\boldsymbol\Upsilon_0$ and $\bar{\mathbf Q}_n(\boldsymbol\theta)$ converges uniformly to its deterministic limit on the bounded parameter space, \[ \sup_{\boldsymbol\theta\in\Theta} \left| \bar{\mathbf Q}_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_n \bar{\mathbf Q}_n(\boldsymbol\theta)-Q(\boldsymbol\theta) \right| \to 0. \] Combining these two displays gives \[ \sup_{\boldsymbol\theta\in\Theta}|Q_n(\boldsymbol\theta)-Q(\boldsymbol\theta)|\xrightarrow{p} 0. \]

Step 5 (extremum consistency). By Step 2(d), $Q(\boldsymbol\theta)$ has a unique minimizer at $\boldsymbol\theta_0$. Hence, by the argmin theorem for extremum estimators, \[ \widehat{\boldsymbol\theta}_n\xrightarrow{p} \boldsymbol\theta_0. \]

6.9.2.2 Asymptotic distribution of GMM estimator

Proof. Proof of asymptotic normality for the GMM estimator of the SLM. Let \[ Q_n(\boldsymbol\theta)\equiv \mathbf g_n(\boldsymbol\theta)^\top \boldsymbol\Upsilon_n \mathbf g_n(\boldsymbol\theta), \qquad \mathbf G_n(\boldsymbol\theta)\equiv \frac{\partial \mathbf g_n(\boldsymbol\theta)}{\partial \boldsymbol\theta^\top}. \] Assume $\widehat{\boldsymbol\theta}_n\in\operatorname{int}(\Theta)$. Then the first-order conditions are \[ \frac{\partial Q_n(\widehat{\boldsymbol\theta}_n)}{\partial \boldsymbol\theta} = 2\,\mathbf G_n(\widehat{\boldsymbol\theta}_n)^\top \boldsymbol\Upsilon_n \mathbf g_n(\widehat{\boldsymbol\theta}_n) = \mathbf 0. \tag{6.125}\]

Step 1 (Taylor expansion and generic linearization). Taylor-expand $\mathbf g_n(\widehat{\boldsymbol\theta}_n)$ around $\boldsymbol\theta_0$: \[ \mathbf g_n(\widehat{\boldsymbol\theta}_n) = \mathbf g_n(\boldsymbol\theta_0)+\mathbf G_n(\bar{\boldsymbol\theta}_n)(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0), \] for some $\bar{\boldsymbol\theta}_n$ on the line segment between $\widehat{\boldsymbol\theta}_n$ and $\boldsymbol\theta_0$. Substitute into Equation 6.125: \[ \mathbf G_n(\widehat{\boldsymbol\theta}_n)^\top\boldsymbol\Upsilon_n \Big\{ \mathbf g_n(\boldsymbol\theta_0)+\mathbf G_n(\bar{\boldsymbol\theta}_n)(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0) \Big\} = \mathbf 0. \] Hence \[ \widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0 = - \Big[ \mathbf G_n(\widehat{\boldsymbol\theta}_n)^\top\boldsymbol\Upsilon_n\mathbf G_n(\bar{\boldsymbol\theta}_n) \Big]^{-1} \mathbf G_n(\widehat{\boldsymbol\theta}_n)^\top\boldsymbol\Upsilon_n\mathbf g_n(\boldsymbol\theta_0). \tag{6.126}\] Multiply by $\sqrt n$ and divide the Jacobians by $n$: \[ \sqrt n(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0) = - \Big[ \Big(\tfrac1n\mathbf G_n(\widehat{\boldsymbol\theta}_n)^\top\Big) \boldsymbol\Upsilon_n \Big(\tfrac1n\mathbf G_n(\bar{\boldsymbol\theta}_n)\Big) \Big]^{-1} \Big(\tfrac1n\mathbf G_n(\widehat{\boldsymbol\theta}_n)^\top\Big) \boldsymbol\Upsilon_n \Big(\tfrac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0)\Big). \tag{6.127}\]

Step 2 (exact Jacobian formula). Since \[ \boldsymbol\varepsilon_n(\boldsymbol\theta)=\mathbf S_n(\rho)\mathbf y_n-\mathbf X_n\boldsymbol\beta, \qquad \frac{\partial \boldsymbol\varepsilon_n(\boldsymbol\theta)}{\partial \boldsymbol\theta^\top} = \begin{pmatrix} -\mathbf W_n\mathbf y_n & -\mathbf X_n \end{pmatrix}, \] and for any square matrix $\mathbf P$, \[ \frac{\partial}{\partial \boldsymbol\theta^\top}\bigl(\boldsymbol\varepsilon^\top\mathbf P\boldsymbol\varepsilon\bigr) = \boldsymbol\varepsilon^\top(\mathbf P+\mathbf P^\top)\frac{\partial \boldsymbol\varepsilon}{\partial \boldsymbol\theta^\top} = \boldsymbol\varepsilon^\top\mathbf P^s\frac{\partial \boldsymbol\varepsilon}{\partial \boldsymbol\theta^\top}, \] we get \[ \mathbf G_n(\boldsymbol\theta) = - \begin{pmatrix} \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}^s\mathbf W_n\mathbf y_n & \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{1n}^s\mathbf X_n\\ \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}^s\mathbf W_n\mathbf y_n & \boldsymbol\varepsilon_n(\boldsymbol\theta)^\top\mathbf P_{2n}^s\mathbf X_n\\ \mathbf H_n^\top\mathbf W_n\mathbf y_n & \mathbf H_n^\top\mathbf X_n \end{pmatrix}. \tag{6.128}\]

Step 3 (limit of the normalized Jacobian at $\boldsymbol\theta_0$). At $\boldsymbol\theta_0$, \[ \mathbf y_n=\mathbf S_0^{-1}\mathbf X_n\boldsymbol\beta_0+\mathbf S_0^{-1}\boldsymbol\varepsilon_n, \qquad \mathbf W_n\mathbf y_n=\mathbf G_0\mathbf X_n\boldsymbol\beta_0+\mathbf G_0\boldsymbol\varepsilon_n. \]

Fix $j\in\{1,2\}$. Then \[ \begin{aligned} \frac1n\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf W_n\mathbf y_n &= \frac1n\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf G_0\mathbf X_n\boldsymbol\beta_0 + \frac1n\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf G_0\boldsymbol\varepsilon_n. \end{aligned} \] The first term is a linear form. Since $\mathbf P_{jn}^s\mathbf G_0\mathbf X_n$ has uniformly bounded entries under the maintained boundedness assumptions, Lee’s Lemma A.4 gives \[ \frac1n\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf G_0\mathbf X_n\boldsymbol\beta_0=o_p(1). \] For the second term, use Lee’s Lemma A.3: \[ \frac1n\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf G_0\boldsymbol\varepsilon_n - \frac1n\mathbb E\!\left[\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf G_0\boldsymbol\varepsilon_n\right] =o_p(1). \] By Equation 3.18, \[ \mathbb E\!\left[\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf G_0\boldsymbol\varepsilon_n\right] = \sigma_0^2\operatorname{tr}(\mathbf P_{jn}^s\mathbf G_0). \] Hence \[ \frac1n\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf W_n\mathbf y_n = \frac{\sigma_0^2}{n}\operatorname{tr}(\mathbf P_{jn}^s\mathbf G_0)+o_p(1). \tag{6.129}\]

Similarly, \[ \frac1n\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf X_n=o_p(1) \] by Lemma A.4, because it is a linear form in $\boldsymbol\varepsilon_n$ with uniformly bounded coefficients.

For the last row of Equation 6.128, \[ \frac1n\mathbf H_n^\top\mathbf W_n\mathbf y_n = \frac1n\mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 + \frac1n\mathbf H_n^\top\mathbf G_0\boldsymbol\varepsilon_n. \] The second term is $o_p(1)$ by Lemma A.4, so \[ \frac1n\mathbf H_n^\top\mathbf W_n\mathbf y_n = \frac1n\mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0+o_p(1). \] Also, \[ \frac1n\mathbf H_n^\top\mathbf X_n \to \lim_{n\to\infty}\frac1n\mathbf H_n^\top\mathbf X_n. \]

Collecting all blocks, \[ \frac1n\mathbf G_n(\boldsymbol\theta_0) = -\frac1n\mathbf D_n+o_p(1), \tag{6.130}\] where \[ \mathbf D_n = \begin{pmatrix} \sigma_0^2\operatorname{tr}(\mathbf P_{1n}^s\mathbf G_0) & \mathbf 0\\ \sigma_0^2\operatorname{tr}(\mathbf P_{2n}^s\mathbf G_0) & \mathbf 0\\ \mathbf H_n^\top\mathbf G_0\mathbf X_n\boldsymbol\beta_0 & \mathbf H_n^\top\mathbf X_n \end{pmatrix}. \] This is exactly the matrix appearing in Lee’s Proposition 1.

Step 4 (replace $\boldsymbol\theta_0$ by $\widehat{\boldsymbol\theta}_n$ and $\bar{\boldsymbol\theta}_n$). We now justify the step that Lee compresses heavily.

Each entry of $(1/n)\mathbf G_n(\boldsymbol\theta)$ is a polynomial in $(\rho,\boldsymbol\beta)$ whose random coefficients are linear or quadratic forms of the type handled in Steps 4 and 3 above. Therefore, the same Lemma A.3/Lemma A.4 arguments used in the consistency proof imply that $(1/n)\mathbf G_n(\boldsymbol\theta)$ is uniformly stochastically equicontinuous on $\Theta$: \[ \sup_{\boldsymbol\theta\in\Theta}\left\| \frac1n\mathbf G_n(\boldsymbol\theta)-\frac1n\mathbb E[\mathbf G_n(\boldsymbol\theta)] \right\|=o_p(1). \] Since $\widehat{\boldsymbol\theta}_n\xrightarrow{p}\boldsymbol\theta_0$ and $\bar{\boldsymbol\theta}_n\xrightarrow{p}\boldsymbol\theta_0$, and since the deterministic map $(1/n)\mathbb E[\mathbf G_n(\boldsymbol\theta)]$ is continuous in $\boldsymbol\theta$ on the bounded parameter space, it follows that \[ \frac1n\mathbf G_n(\widehat{\boldsymbol\theta}_n)-\frac1n\mathbf G_n(\boldsymbol\theta_0)=o_p(1), \qquad \frac1n\mathbf G_n(\bar{\boldsymbol\theta}_n)-\frac1n\mathbf G_n(\boldsymbol\theta_0)=o_p(1). \] Hence, by Equation 6.130, \[ \frac1n\mathbf G_n(\widehat{\boldsymbol\theta}_n)= -\frac1n\mathbf D_n+o_p(1), \qquad \frac1n\mathbf G_n(\bar{\boldsymbol\theta}_n)= -\frac1n\mathbf D_n+o_p(1). \tag{6.131}\]

Step 5 (CLT for $(1/\sqrt n)\mathbf g_n(\boldsymbol\theta_0)$). Because $\operatorname{tr}(\mathbf P_{jn})=0$ and $\mathbb E(\boldsymbol\varepsilon_n)=\mathbf 0$, \[ \mathbb E[\mathbf g_n(\boldsymbol\theta_0)]=\mathbf 0. \] Now fix any constant vector \[ \mathbf c= (c_1,c_2,\mathbf c_3^\top)^\top. \] Then \[ \frac1{\sqrt n}\mathbf c^\top\mathbf g_n(\boldsymbol\theta_0) = \frac1{\sqrt n}\boldsymbol\varepsilon_n^\top\mathbf A_n\boldsymbol\varepsilon_n + \frac1{\sqrt n}\mathbf c_3^\top\mathbf H_n^\top\boldsymbol\varepsilon_n, \qquad \mathbf A_n\equiv c_1\mathbf P_{1n}+c_2\mathbf P_{2n}. \] So every scalar projection of $(1/\sqrt n)\mathbf g_n(\boldsymbol\theta_0)$ is a sum of one quadratic form and one linear form in i.i.d. innovations. Its variance is exactly the corresponding projection of $(1/n)\boldsymbol\Omega_n$, where $\boldsymbol\Omega_n=\operatorname{Var}(\mathbf g_n(\boldsymbol\theta_0))$; Lee’s Lemma A.2 gives the covariance formulas for quadratic blocks, and Lemmas A.3–A.4 give the required orders. Therefore, by the CLT for linear-quadratic forms invoked in Lee’s Proposition 1 and Cramér–Wold, \[ \frac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0) \xrightarrow{d} \mathcal N\!\left(\mathbf 0,\lim_{n\to\infty}\frac1n\boldsymbol\Omega_n\right). \tag{6.132}\]

Step 6 (Slutsky and final covariance formula). Combine Equation 6.127, Equation 6.131, and Equation 6.132. The minus signs in the Jacobians cancel, and we obtain \[ \sqrt n(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0) = \Big[ \Big(\tfrac1n\mathbf D_n^\top\Big)\boldsymbol\Upsilon_0\Big(\tfrac1n\mathbf D_n\Big) \Big]^{-1} \Big(\tfrac1n\mathbf D_n^\top\Big)\boldsymbol\Upsilon_0 \Big(\tfrac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0)\Big) +o_p(1). \] Therefore, \[ \sqrt n(\widehat{\boldsymbol\theta}_n-\boldsymbol\theta_0) \xrightarrow{d} \mathcal N(\mathbf 0,\boldsymbol\Sigma_0), \] with \[ \boldsymbol\Sigma_0= \Big[ \Big(\tfrac1n\mathbf D_n^\top\Big)\boldsymbol\Upsilon_0\Big(\tfrac1n\mathbf D_n\Big) \Big]^{-1} \Big(\tfrac1n\mathbf D_n^\top\Big)\boldsymbol\Upsilon_0 \Big(\tfrac1n\boldsymbol\Omega_n\Big) \boldsymbol\Upsilon_0\Big(\tfrac1n\mathbf D_n\Big) \Big[ \Big(\tfrac1n\mathbf D_n^\top\Big)\boldsymbol\Upsilon_0\Big(\tfrac1n\mathbf D_n\Big) \Big]^{-1}. \] This is exactly Lee’s Proposition 1 covariance formula.

6.9.3 Asymptotic Properties of OGMME for SLM

Proof. Proof of Theorem 6.4. Step 1 (normalization of the weighting matrix). Let \[ m\equiv 2+k_x, \qquad p\equiv k+1, \qquad \mathbf S_n\equiv \frac1n\boldsymbol\Omega_n, \qquad \widehat{\mathbf S}_n\equiv \frac1n\widehat{\boldsymbol\Omega}_n. \] By assumption, \[ \mathbf S_n\to \mathbf S_0, \qquad \widehat{\mathbf S}_n^{-1}-\mathbf S_n^{-1}=o_p(1), \] with $\mathbf S_0$ nonsingular. Since \[ \widehat{\boldsymbol\Omega}_n^{-1} = \frac1n\,\widehat{\mathbf S}_n^{-1}, \] the objective \[ \mathbf g_n(\boldsymbol\theta)^\top \widehat{\boldsymbol\Omega}_n^{-1}\mathbf g_n(\boldsymbol\theta) = \frac1n\,\mathbf g_n(\boldsymbol\theta)^\top \widehat{\mathbf S}_n^{-1}\mathbf g_n(\boldsymbol\theta) \] has the same minimizer as \[ \widetilde Q_n(\boldsymbol\theta) \equiv \frac1n\,\mathbf g_n(\boldsymbol\theta)^\top \widehat{\mathbf S}_n^{-1}\mathbf g_n(\boldsymbol\theta). \] Hence, for the proof it is convenient to work with the normalized criterion $\widetilde Q_n(\boldsymbol\theta)$.

Step 2 (consistency of the feasible OGMM estimator). Define the infeasible normalized objective \[ Q_n^*(\boldsymbol\theta) \equiv \frac1n\,\mathbf g_n(\boldsymbol\theta)^\top \mathbf S_n^{-1}\mathbf g_n(\boldsymbol\theta). \] Since $\mathbf S_n^{-1}$ is nonrandom and converges to $\mathbf S_0^{-1}$, the consistency result from Theorem 6.3 applies directly to the infeasible objective: \[ \sup_{\boldsymbol\theta\in\Theta}|Q_n^*(\boldsymbol\theta)-Q^*(\boldsymbol\theta)|\xrightarrow{p} 0, \] where \[ Q^*(\boldsymbol\theta) = \lim_{n\to\infty} \frac1n\,\mathbb E[\mathbf g_n(\boldsymbol\theta)]^\top \mathbf S_0^{-1} \frac1n\,\mathbb E[\mathbf g_n(\boldsymbol\theta)]. \] By the identification conditions already established, $Q^*(\boldsymbol\theta)$ has a unique minimizer at $\boldsymbol\theta_0$.

It remains to compare the feasible and infeasible criteria: \[ \widetilde Q_n(\boldsymbol\theta)-Q_n^*(\boldsymbol\theta) = \frac1n\,\mathbf g_n(\boldsymbol\theta)^\top\bigl(\widehat{\mathbf S}_n^{-1}-\mathbf S_n^{-1}\bigr)\mathbf g_n(\boldsymbol\theta). \] Using the Euclidean norm, \[ \left| \widetilde Q_n(\boldsymbol\theta)-Q_n^*(\boldsymbol\theta) \right| \le \left(\frac1n\|\mathbf g_n(\boldsymbol\theta)\|\right)^2 \left\| \widehat{\mathbf S}_n^{-1}-\mathbf S_n^{-1} \right\|. \] Therefore, \[ \sup_{\boldsymbol\theta\in\Theta} \left| \widetilde Q_n(\boldsymbol\theta)-Q_n^*(\boldsymbol\theta) \right| \le \sup_{\boldsymbol\theta\in\Theta}\left(\frac1n\|\mathbf g_n(\boldsymbol\theta)\|\right)^2 \left\| \widehat{\mathbf S}_n^{-1}-\mathbf S_n^{-1} \right\|. \tag{6.133}\]

We now justify that \[ \sup_{\boldsymbol\theta\in\Theta}\frac1n\|\mathbf g_n(\boldsymbol\theta)\|=O_p(1). \] From the consistency proof of Theorem 6.3, \[ \sup_{\boldsymbol\theta\in\Theta} \left\| \frac1n\bigl(\mathbf g_n(\boldsymbol\theta)-\mathbb E[\mathbf g_n(\boldsymbol\theta)]\bigr) \right\| =o_p(1). \] Hence it suffices to show that \[ \sup_{\boldsymbol\theta\in\Theta} \left\| \frac1n\mathbb E[\mathbf g_n(\boldsymbol\theta)] \right\| =O(1). \] But the components of $(1/n)\mathbb E[\mathbf g_n(\boldsymbol\theta)]$ are explicit polynomials in $(\rho_0-\rho)$ and $(\boldsymbol\beta_0-\boldsymbol\beta)$; see the previous identification proof. Each coefficient is an $O(1)$ object by Assumption 10 and Lee’s Lemma A.1, and the parameter space $\Theta$ is bounded. Therefore \[ \sup_{\boldsymbol\theta\in\Theta} \left\| \frac1n\mathbb E[\mathbf g_n(\boldsymbol\theta)] \right\| =O(1), \] which proves \[ \sup_{\boldsymbol\theta\in\Theta}\frac1n\|\mathbf g_n(\boldsymbol\theta)\|=O_p(1). \]

Returning to Equation 6.133, assumption (2) implies \[ \sup_{\boldsymbol\theta\in\Theta} \left| \widetilde Q_n(\boldsymbol\theta)-Q_n^*(\boldsymbol\theta) \right| =o_p(1). \] Therefore, \[ \sup_{\boldsymbol\theta\in\Theta} \left| \widetilde Q_n(\boldsymbol\theta)-Q^*(\boldsymbol\theta) \right| \xrightarrow{p} 0. \] Since $Q^*(\boldsymbol\theta)$ is uniquely minimized at $\boldsymbol\theta_0$, the argmin theorem yields \[ \widehat{\boldsymbol\theta}_{o,n}\xrightarrow{p} \boldsymbol\theta_0. \]

Step 3 (first-order condition). Let \[ \mathbf G_n(\boldsymbol\theta)\equiv \frac{\partial \mathbf g_n(\boldsymbol\theta)}{\partial \boldsymbol\theta^\top}. \] For an interior minimizer, \[ \mathbf 0 = \frac{\partial \widetilde Q_n(\widehat{\boldsymbol\theta}_{o,n})}{\partial \boldsymbol\theta} = \frac{2}{n}\, \mathbf G_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\mathbf S}_n^{-1}\, \mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}). \] Equivalently, \[ \mathbf G_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\mathbf S}_n^{-1}\, \mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) = \mathbf 0. \tag{6.134}\]

Step 4 (Taylor expansion of the moment vector). By the mean-value theorem, for some $\bar{\boldsymbol\theta}_n$ on the line segment between $\widehat{\boldsymbol\theta}_{o,n}$ and $\boldsymbol\theta_0$, \[ \mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) = \mathbf g_n(\boldsymbol\theta_0) + \mathbf G_n(\bar{\boldsymbol\theta}_n)(\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0). \tag{6.135}\] Substituting Equation 6.135 into Equation 6.134 gives \[ \mathbf G_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\mathbf S}_n^{-1} \Bigl\{ \mathbf g_n(\boldsymbol\theta_0)+\mathbf G_n(\bar{\boldsymbol\theta}_n)(\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0) \Bigr\} = \mathbf 0. \] Rearranging, \[ \Bigl[ \mathbf G_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\mathbf S}_n^{-1}\mathbf G_n(\bar{\boldsymbol\theta}_n) \Bigr] (\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0) = - \mathbf G_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\mathbf S}_n^{-1}\mathbf g_n(\boldsymbol\theta_0). \] Multiplying by $\sqrt n$ and normalizing the Jacobians by $n$, \[ \sqrt n(\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0) = - \Bigl[ \Bigl(\tfrac1n\mathbf G_n(\widehat{\boldsymbol\theta}_{o,n})\Bigr)^\top \widehat{\mathbf S}_n^{-1} \Bigl(\tfrac1n\mathbf G_n(\bar{\boldsymbol\theta}_n)\Bigr) \Bigr]^{-1} \Bigl(\tfrac1n\mathbf G_n(\widehat{\boldsymbol\theta}_{o,n})\Bigr)^\top \widehat{\mathbf S}_n^{-1} \Bigl(\tfrac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0)\Bigr). \tag{6.136}\]

Step 5 (limits of the Jacobian terms). From the previous GMM theorem, with \[ \mathbf D_n \equiv \frac{\partial\mathbb E[\mathbf g_n(\boldsymbol\theta_0)]}{\partial\boldsymbol\theta^\top}, \] we have \[ \frac1n\mathbf G_n(\boldsymbol\theta_0) = -\frac1n\mathbf D_n+o_p(1). \] Since $\widehat{\boldsymbol\theta}_{o,n}\xrightarrow{p}\boldsymbol\theta_0$ and $\bar{\boldsymbol\theta}_n\xrightarrow{p}\boldsymbol\theta_0$, the same continuity and stochastic equicontinuity argument used in the previous proof yields \[ \frac1n\mathbf G_n(\widehat{\boldsymbol\theta}_{o,n}) = -\frac1n\mathbf D_n+o_p(1), \qquad \frac1n\mathbf G_n(\bar{\boldsymbol\theta}_n) = -\frac1n\mathbf D_n+o_p(1). \] Also, \[ \widehat{\mathbf S}_n^{-1} = \mathbf S_n^{-1}+o_p(1) \to \mathbf S_0^{-1}. \]

Step 6 (CLT for the moments). By the CLT for the linear-quadratic moment vector established in the proof of Theorem 6.3, \[ \frac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0) \xrightarrow{d} \mathcal N(\mathbf 0,\mathbf S_0), \qquad \mathbf S_0=\lim_{n\to\infty}\frac1n\boldsymbol\Omega_n. \]

Step 7 (asymptotic normality). Substitute the limits from Steps 5 and 6 into Equation 6.136. The two minus signs in the Jacobians cancel, so \[ \sqrt n(\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0) = \Bigl[ \Bigl(\tfrac1n\mathbf D_n^\top\Bigr)\mathbf S_n^{-1}\Bigl(\tfrac1n\mathbf D_n\Bigr) \Bigr]^{-1} \Bigl(\tfrac1n\mathbf D_n^\top\Bigr)\mathbf S_n^{-1} \Bigl(\tfrac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0)\Bigr) +o_p(1). \] Hence \[ \sqrt n(\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0) \xrightarrow{d} \mathcal N\!\left( \mathbf 0,\, (\mathbf D_0^\top\mathbf S_0^{-1}\mathbf D_0)^{-1} \right), \] where \[ \mathbf D_0=\lim_{n\to\infty}\frac1n\mathbf D_n. \] Finally, since $\mathbf S_n=\boldsymbol\Omega_n/n$, \[ (\mathbf D_0^\top\mathbf S_0^{-1}\mathbf D_0)^{-1} = \left[ \lim_{n\to\infty} \frac1n\mathbf D_n^\top \boldsymbol\Omega_n^{-1}\mathbf D_n \right]^{-1}, \] which is Lee’s equation (3.3).

Step 8 (overidentification test). Define the deterministic matrix \[ \mathbf A_n \equiv \mathbf I_m - \Bigl(\frac1n\mathbf D_n\Bigr) \Bigl[ \Bigl(\frac1n\mathbf D_n^\top\Bigr)\mathbf S_n^{-1}\Bigl(\frac1n\mathbf D_n\Bigr) \Bigr]^{-1} \Bigl(\frac1n\mathbf D_n^\top\Bigr)\mathbf S_n^{-1}. \] By Taylor expansion, \[ \frac1{\sqrt n}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) = \frac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0) + \Bigl(\frac1n\mathbf G_n(\bar{\boldsymbol\theta}_n)\Bigr)\sqrt n(\widehat{\boldsymbol\theta}_{o,n}-\boldsymbol\theta_0). \] Substituting the asymptotic linear representation from Step 7, \[ \frac1{\sqrt n}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) = \mathbf A_n\frac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0)+o_p(1). \] Therefore, \[ \begin{aligned} \mathbf g_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\boldsymbol\Omega}_n^{-1}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) &= \Bigl(\frac1{\sqrt n}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n})\Bigr)^\top \widehat{\mathbf S}_n^{-1} \Bigl(\frac1{\sqrt n}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n})\Bigr) \\ &= \Bigl(\frac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0)\Bigr)^\top \mathbf A_n^\top \mathbf S_n^{-1}\mathbf A_n \Bigl(\frac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0)\Bigr) +o_p(1). \end{aligned} \] Now prewhiten the moment vector: \[ \mathbf Z_n \equiv \mathbf S_n^{-1/2}\frac1{\sqrt n}\mathbf g_n(\boldsymbol\theta_0) \xrightarrow{d} \mathcal N(\mathbf 0,\mathbf I_m). \] Then \[ \mathbf g_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\boldsymbol\Omega}_n^{-1}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) = \mathbf Z_n^\top \mathbf M_n \mathbf Z_n +o_p(1), \] where \[ \mathbf M_n = \mathbf I_m - \mathbf S_n^{-1/2}\Bigl(\frac1n\mathbf D_n\Bigr) \Bigl[ \Bigl(\frac1n\mathbf D_n^\top\Bigr)\mathbf S_n^{-1}\Bigl(\frac1n\mathbf D_n\Bigr) \Bigr]^{-1} \Bigl(\frac1n\mathbf D_n^\top\Bigr)\mathbf S_n^{-1/2}. \] The matrix $\mathbf M_n$ is symmetric and idempotent, and its rank is \[ \operatorname{rank}(\mathbf M_n)=m-p. \] Therefore, in the limit, \[ \mathbf Z_n^\top \mathbf M_n \mathbf Z_n \xrightarrow{d} \chi^2_{m-p}. \] Thus \[ \mathbf g_n(\widehat{\boldsymbol\theta}_{o,n})^\top \widehat{\boldsymbol\Omega}_n^{-1}\mathbf g_n(\widehat{\boldsymbol\theta}_{o,n}) \xrightarrow{d} \chi^2_{m-p}, \] which is Lee’s equation (3.4).

6.9.4 Asymptotic Distribution of GMME for SEM Model

Let \[ \mathbf R_n(\lambda)=\mathbf I_n-\lambda\mathbf M_n, \qquad \mathbf R_{0n}=\mathbf R_n(\lambda_0), \qquad \mathbf Q_{0n}=\mathbf M_n\mathbf R_{0n}^{-1}. \] Then \[ \begin{aligned} \mathbf R_n(\lambda)\mathbf R_{0n}^{-1} &= (\mathbf R_{0n}+(\lambda_0-\lambda)\mathbf M_n) \mathbf R_{0n}^{-1} \\ &= \mathbf I_n+(\lambda_0-\lambda)\mathbf Q_{0n}. \end{aligned} \tag{6.137}\]

The error term for the SEM model can therefore be expressed as: \[ \begin{aligned} \boldsymbol\varepsilon_n(\boldsymbol\theta) & = \left(\mathbf I_n - \lambda\mathbf M_n\right)\mathbf u_n, \\ & = \mathbf R_n\left(\mathbf y_n - \mathbf X\boldsymbol\beta\right), \\ & = \mathbf R_n\left(\mathbf X_n\boldsymbol\beta_0 + \mathbf R_0^{-1}\boldsymbol\varepsilon_n - \mathbf X\boldsymbol\beta\right),\quad \mbox{replacing $\mathbf y_n$} \\ & = \mathbf R_n\mathbf X_n\boldsymbol\beta_0 + \mathbf R_n\mathbf R_{0n}^{-1}\boldsymbol\varepsilon_n - \mathbf R_n\mathbf X_n\boldsymbol\beta, \\ & = \mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) + \mathbf R_n\mathbf R_{0n}^{-1}\boldsymbol\varepsilon_n, \\ & = \mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) + \left(\mathbf I_n + (\lambda_0 - \lambda)\mathbf Q_{0n}\right)\boldsymbol\varepsilon_n, \quad \mbox{using @eq-iv-gmm-sem-resolvent-identity} \\ & = \mathbf R_n\mathbf d_n(\boldsymbol\beta) + \left(\mathbf I_n + (\lambda_0 - \lambda)\mathbf Q_{0n}\right)\boldsymbol\varepsilon_n, \end{aligned} \] where $\mathbf Q_{0n}=\mathbf M_n\mathbf R_{0n}^{-1}$.

We next find $\mathbf D_n$. For the first element of the Jacobian, we note that $\mathbf R_n = \mathbf R_{0n} + (\lambda_0 - \lambda)\mathbf M_n$: \[ \frac{1}{n}\mathbf X_n^\top\mathbf R_n\mathbf X_n \xrightarrow{p} \frac{1}{n}\mathbf X_n^\top\mathbf R_0\mathbf X_n +\frac{1}{n}(\lambda_0 - \lambda)\mathbf X_n^\top\mathbf M_n\mathbf X_n. \]

In addition, \[ \begin{aligned} \frac{1}{n}\mathbf X_n^\top\mathbf M_n\mathbf u_n & = \frac{1}{n}\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\boldsymbol\varepsilon_n, \\ & = \frac{1}{n}\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\left[\mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) + \left(\mathbf I_n + (\lambda_0 - \lambda)\mathbf Q_{0n}\right)\boldsymbol\varepsilon_n\right] \\ & = \frac{1}{n}\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) + \frac{1}{n}(\lambda_0 - \lambda)\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\mathbf Q_{0n}\boldsymbol\varepsilon_n, \\ & \xrightarrow{p} \frac{1}{n}\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) + \frac{1}{n}(\lambda_0 - \lambda)\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\mathbf Q_0\mathbb E(\boldsymbol\varepsilon_n), \\ & \xrightarrow{p} \frac{1}{n}\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta). \end{aligned} \]

Similarly, \[ \begin{aligned} \frac{1}{n}\boldsymbol\varepsilon_n^\top \mathbf P_{jn}^s\mathbf M_n\mathbf u_n & = \frac{1}{n}\left(\mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) + \mathbf R_n\mathbf R_{0n}^{-1}\boldsymbol\varepsilon_n\right)^\top\mathbf P_{jn}^s\mathbf M_n\mathbf u_n, \\ & = \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top \mathbf P_{jn}^s\mathbf M_n\mathbf u_n + \frac{1}{n}\boldsymbol\varepsilon_n^\top\mathbf R_0^{-1\top}\mathbf R_n\mathbf P_{jn}^s\mathbf M_n\mathbf u_n. \end{aligned} \]

For the first element \[ \begin{aligned} \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top \mathbf P_{jn}^s\mathbf M_n\mathbf u_n & = \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top \mathbf P_{jn}^s\mathbf M_n\mathbf R_0^{-1}\boldsymbol\varepsilon_n,\\ & \xrightarrow{p} \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top \mathbf P_{jn}^s\mathbf M_n\mathbf R_0^{-1}\mathbb E(\boldsymbol\varepsilon_n), \\ & \xrightarrow{p} \mathbf 0. \end{aligned} \]

For the second element \[ \begin{aligned} \frac{1}{n}\boldsymbol\varepsilon_n^\top\mathbf R_0^{-1\top}\mathbf R_n\mathbf P_{jn}^s\mathbf M_n\mathbf u_n & = \frac{1}{n}\boldsymbol\varepsilon_n^\top\mathbf R_0^{-1\top}\mathbf R_n\mathbf P_{jn}^s\mathbf M_n\mathbf R_0^{-1}\boldsymbol\varepsilon_n, \\ = & \frac{1}{n}\boldsymbol\varepsilon_n^\top\left[\mathbf I_n + (\lambda_0-\lambda)\mathbf Q_0\right]^\top\mathbf P_{jn}^s\mathbf M_n\mathbf R_0^{-1}\boldsymbol\varepsilon_n, \\ & = \frac{1}{n}\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf Q_{0n}\boldsymbol\varepsilon_n + (\lambda_0 - \lambda)\frac{1}{n}\boldsymbol\varepsilon_n^\top\mathbf Q_0^\top\mathbf P_{jn}^s\mathbf Q_{0n}\boldsymbol\varepsilon_n, \\ & \xrightarrow{p} \frac{1}{n}\mathbb E\left(\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf Q_{0n}\boldsymbol\varepsilon_n\right) + \frac{1}{n} (\lambda_0 - \lambda) \mathbb E\left(\boldsymbol\varepsilon_n^\top\mathbf Q_0^\top\mathbf P_{jn}^s\mathbf Q_{0n}\boldsymbol\varepsilon_n \right),\\ & \xrightarrow{p} \sigma_0^2 \operatorname{tr}\left(\mathbf P_{jn}^{s}\mathbf Q_{0n}\right) + (\lambda_0- \lambda)\frac{\sigma_0^2}{n}\operatorname{tr}\left(\mathbf Q_0^\top\mathbf P_{jn}^s\mathbf G_0\right). \end{aligned} \]

Now: \[ \begin{aligned} \frac{1}{n}\boldsymbol\varepsilon_n^\top\mathbf P_{jn}^s\mathbf R_n\mathbf X_n & = \frac{1}{n}\left(\mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) + \mathbf R_n\mathbf R_{0n}^{-1}\boldsymbol\varepsilon_n\right)^\top\mathbf P_{jn}^s\mathbf R_n\mathbf X_n, \\ & = \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top\mathbf P_{jn}^s\mathbf R_n\mathbf X_n + \frac{1}{n}\boldsymbol\varepsilon_n^\top\left(\mathbf R_n\mathbf R_{0n}^{-1}\right)^\top\mathbf P_{jn}^s\mathbf R_n\mathbf X_n, \\ & = \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top\mathbf P_{jn}^s\mathbf R_n\mathbf X_n + \frac{1}{n}\boldsymbol\varepsilon_n^\top\left(\mathbf I_n + (\lambda_0 - \lambda)\mathbf Q_{0n}\right)^\top\mathbf P_{jn}^s\mathbf R_n\mathbf X_n, \\ & \xrightarrow{p} \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top\mathbf P_{jn}^s\mathbf R_n\mathbf X_n. \end{aligned} \]

As a result, \[ \begin{aligned} \frac{1}{n}\frac{\partial \mathbb E\left[\mathbf g_n(\boldsymbol\theta)\right]}{\partial \boldsymbol\theta^\top} & = \begin{pmatrix} \frac{1}{n}\mathbf X_n^\top\mathbf R_0\mathbf X_n +\frac{1}{n}(\lambda_0 - \lambda)\mathbf X_n^\top\mathbf M_n\mathbf X_n & \frac{1}{n}\mathbf X_n^\top\mathbf M_n\mathbf R_0^{-1}\mathbf R_n\mathbf X_n(\boldsymbol\beta_0 - \boldsymbol\beta) \\ \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top\mathbf P_{1n}^s\mathbf R_n\mathbf X_n & \sigma_0^2 \operatorname{tr}\left(\mathbf P_{jn}^{s}\mathbf Q_{0n}\right) + (\lambda_0- \lambda)\frac{\sigma_0^2}{n}\operatorname{tr}\left(\mathbf G_0^\top\mathbf P_{jn}^s\mathbf Q_{0n}\right) \\ \frac{1}{n}(\boldsymbol\beta_0 - \boldsymbol\beta)^\top\mathbf X_n^\top\mathbf R_n^\top\mathbf P_{2n}^s\mathbf R_n\mathbf X_n & \sigma_0^2 \operatorname{tr}\left(\mathbf P_{jn}^{s}\mathbf Q_{0n}\right) + (\lambda_0- \lambda)\frac{\sigma_0^2}{n}\operatorname{tr}\left(\mathbf G_0^\top\mathbf P_{jn}^s\mathbf Q_{0n}\right) \end{pmatrix} \end{aligned} \]

At $\boldsymbol\theta = \boldsymbol\theta_0$, \[ \underset{(k + 2)\times (k + 1)}{\mathbf D_n} = \begin{pmatrix} \underset{k\times k}{\mathbf X_n^\top\mathbf R_0\mathbf X_n} & \underset{k\times 1}{\mathbf 0} \\ \underset{1 \times k}{\mathbf 0} & \underset{1\times 1}{\sigma_0^2 \operatorname{tr}\left(\mathbf P_{1n}^{s}\mathbf Q_{0n}\right)}\\ \underset{1\times k}{\mathbf 0} & \underset{1\times 1}{\sigma_0^2 \operatorname{tr}\left(\mathbf P_{2n}^{s}\mathbf Q_{0n}\right)} \end{pmatrix} \]

6.9.5 Proof of Theorem 3 in Kelejian and Prucha (1998)

Recall that the GS2SLS is given by:

\[ \widehat{\boldsymbol\delta}_n = \left[\widehat{\mathbf Z}_s(\lambda)^\top\widehat{\mathbf Z}_s(\lambda)\right]^{-1}\widehat{\mathbf Z}_s(\lambda)^\top\mathbf y_s(\lambda) \]

Whereas, the FGS2SLS is given by:

\[ \widehat{\boldsymbol\delta}_{F, n} = \left[\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\mathbf y_s(\widehat{\lambda}) \] where

\[ \begin{aligned} \widehat{\mathbf Z}_s(\widehat{\lambda}_n) & = \mathbf P_{H_n}\mathbf Z_s(\widehat{\lambda}_n) \\ \mathbf Z_s(\widehat{\lambda}_n) & = \mathbf Z_n - \widehat{\lambda}_n\mathbf M_n\mathbf Z_n \\ \mathbf y_s(\widehat{\lambda}_n) & = \mathbf y_n - \widehat{\lambda}_n\mathbf M_n\mathbf y_n \\ \widehat{\mathbf Z}_s(\widehat{\lambda}_n) & = \left(\mathbf X_n - \widehat{\lambda}_n\mathbf M_n\mathbf X_n, \mathbf W_n\mathbf y_n - \widehat{\widehat{\lambda}_n\mathbf M_n}\mathbf W_n\mathbf y_n\right) \\ \widehat{\widehat{\lambda}_n\mathbf M_n}\mathbf W_n\mathbf y_n & = \mathbf P_{H_n}\left(\mathbf W_n\mathbf y_n - \widehat{\lambda}_n\mathbf M_n\mathbf W_n\mathbf y_n\right). \end{aligned} \]

The sampling error is:

\[ \widehat{\boldsymbol\delta}_{F, n} - \boldsymbol\delta = \left[\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\mathbf u_s(\widehat{\lambda}_n) \] where:

\[ \begin{aligned} \mathbf u_s(\widehat{\lambda}_n) & = (\mathbf I - \widehat{\lambda}_n)\mathbf u \\ & = (\mathbf I - \widehat{\lambda}_n)\mathbf u + \boldsymbol\varepsilon_n - \boldsymbol\varepsilon_n \\ & = \boldsymbol\varepsilon_n + (\mathbf I - \widehat{\lambda}_n\mathbf M_n)\mathbf u - (\mathbf I - \lambda\mathbf M_n)\mathbf u \\ & = \boldsymbol\varepsilon_n + \mathbf u - \widehat{\lambda}_n\mathbf M_n\mathbf u - \mathbf u + \lambda\mathbf M_n\mathbf u \\ & = \boldsymbol\varepsilon_n - \left(\widehat{\lambda}_n- \lambda\right)\mathbf M_n\mathbf u_n \end{aligned} \]

Then:

\[ \begin{aligned} \widehat{\boldsymbol\delta}_{F, n} - \boldsymbol\delta & = \left[\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\left[\boldsymbol\varepsilon_n - \left(\widehat{\lambda}_n- \lambda\right)\mathbf M_n\mathbf u_n\right] \\ & = \left[\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\boldsymbol\varepsilon_n - \left[\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\left(\widehat{\lambda}_n- \lambda\right)\mathbf M_n\mathbf u_n \\ & = \left[\frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\boldsymbol\varepsilon_n - \left[\frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\left(\widehat{\lambda}_n- \lambda\right)\frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\mathbf M_n\mathbf u_n \\ \sqrt{n} (\widehat{\boldsymbol\delta}_{F, n} - \boldsymbol\delta) & = \left[\frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\frac{1}{\sqrt{n}}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\boldsymbol\varepsilon_n - \left[\frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\left(\widehat{\lambda}_n- \lambda\right)\frac{1}{\sqrt{n}}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\mathbf M_n\mathbf u_n \end{aligned} \]

By consistency $\widehat{\lambda}_n- \lambda = o_p(1)$. Now, we need to show that:

\[ \frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\xrightarrow{p} \frac{1}{n}\widehat{\mathbf Z}_s(\lambda)^\top\widehat{\mathbf Z}_s(\lambda) = \bar{\mathbf Q} \tag{6.138}\]

\[ \frac{1}{\sqrt{n}}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\boldsymbol\varepsilon_n\xrightarrow{d} \mathcal N(\mathbf 0, \sigma^2_{\epsilon}\bar{\mathbf Q}), \tag{6.139}\]

\[ \left(\widehat{\lambda}_n- \lambda\right)\frac{1}{\sqrt{n}}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\mathbf M_n\mathbf u_n \xrightarrow{p} 0 \tag{6.140}\] where:

\[ \bar{\mathbf Q} = \left[\mathbf Q_{HZ} - \lambda \mathbf Q_{mHZ}\right]^\top \mathbf Q_{HH}^{-1}\left[\mathbf Q_{HZ} - \lambda \mathbf Q_{mHZ}\right] \] is finite and nonsingular. For Equation 6.138, note that:

\[ \begin{aligned} \frac{1}{n}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda}) & = \frac{1}{n}\left(\mathbf Z_n - \widehat{\lambda}_n\mathbf M_n\mathbf Z_n\right)^\top\mathbf P_{H_n}\left(\mathbf Z_n - \widehat{\lambda}_n\mathbf M_n\mathbf Z_n\right) \\ & = \frac{1}{n}\left(\mathbf Z_n - \widehat{\lambda}_n\mathbf M_n\mathbf Z_n\right)^\top\mathbf H_n\left(\mathbf H_n^\top\mathbf H_n\right)^{-1}\mathbf H_n^\top\left(\mathbf Z_n - \widehat{\lambda}_n\mathbf M_n\mathbf Z_n\right) \\ & = \left(\underbrace{\frac{1}{n}\mathbf Z_n^\top \mathbf H_n}_{\xrightarrow{p} \mathbf Q_{HZ}^\top} - \underbrace{\widehat{\lambda}_n}_{\xrightarrow{p} \lambda}\frac{1}{n}\underbrace{\mathbf Z_n^\top\mathbf M_n^\top\mathbf H_n}_{\xrightarrow{p} \mathbf Q_{HMZ}^\top}\right)\underbrace{\left(\frac{1}{n}\mathbf H_n^\top\mathbf H_n\right)^{-1}}_{\to \mathbf Q_{HH}^{-1}}\underbrace{\left(\frac{1}{n}\mathbf H_n^\top\mathbf Z_n - \widehat{\lambda}_n\frac{1}{n}\mathbf H_n^\top\mathbf M_n\mathbf Z_n\right)}_{\xrightarrow{p} \mathbf Q_{HZ}-\lambda\mathbf Q_{HMZ}} \end{aligned} \]

For Equation 6.139, note that:

\[ \begin{aligned} \frac{1}{\sqrt{n}}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\boldsymbol\varepsilon_n & = \frac{1}{\sqrt{n}}\left(\mathbf Z_n -\widehat{\lambda}_n\mathbf M_n\mathbf Z_n\right)^\top\mathbf P_{H_n}\boldsymbol\varepsilon \\ & = \left(\underbrace{\frac{1}{n}\mathbf Z_n^\top \mathbf H_n}_{\xrightarrow{p} \mathbf Q_{HZ}^\top} - \underbrace{\widehat{\lambda}_n}_{\xrightarrow{p} \lambda}\frac{1}{n}\underbrace{\mathbf Z_n^\top\mathbf M_n^\top\mathbf H_n}_{\xrightarrow{p} \mathbf Q_{HMZ}^\top}\right)\underbrace{\left(\frac{1}{n}\mathbf H_n^\top\mathbf H_n\right)^{-1}}_{\to \mathbf Q_{HH}^{-1}}\underbrace{\frac{1}{\sqrt{n}}\mathbf H_n^\top\boldsymbol\varepsilon_n}_{\xrightarrow{d} \mathcal N(\mathbf 0, \sigma^2_{\epsilon}\mathbf Q_{HH})} \end{aligned} \]

For Equation 6.140 note that:

\[ \left(\widehat{\lambda}_n- \lambda\right)\frac{1}{\sqrt{n}}\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\mathbf M_n\mathbf u_n = \underbrace{\left(\widehat{\lambda}_n- \lambda\right)}_{o_p(1)}\left(\underbrace{\frac{1}{n}\mathbf Z_n^\top \mathbf H_n}_{\xrightarrow{p} \mathbf Q_{HZ}^\top} - \underbrace{\widehat{\lambda}_n}_{\xrightarrow{p} \lambda}\frac{1}{n}\underbrace{\mathbf Z_n^\top\mathbf M_n^\top\mathbf H_n}_{\xrightarrow{p} \mathbf Q_{HMZ}^\top}\right)\underbrace{\left(\frac{1}{n}\mathbf H_n^\top\mathbf H_n\right)^{-1}}_{\to \mathbf Q_{HH}^{-1}}\frac{1}{\sqrt{n}}\mathbf H_n^\top\mathbf M_n\mathbf u_n \]

Note that $\mathbb E\left(n^{-1/2}\mathbf H_n^\top\mathbf M_n\mathbf u_n\right) = 0$ and $\mathbb E\left(n^{-1}\mathbf H_n^\top\mathbf M_n\mathbf u_n\mathbf u^\top_n\mathbf M_n^\top\mathbf H^\top_n\right) = n^{-1} \mathbf H_n^\top\mathbf M_n\boldsymbol\Sigma_{u_n}\mathbf M_n^\top\mathbf H^\top_n$, whose elements are bounded, where

\[ \boldsymbol\Sigma_{u_n} = \sigma^2_{\epsilon}\left(\mathbf I - \lambda\mathbf M_n\right)^{-1}\left(\mathbf I - \lambda\mathbf M_n^\top\right)^{-1} \]

Then $\frac{1}{\sqrt{n}}\mathbf H_n^\top\mathbf M_n\mathbf u_n = O_p(1)$ and finally

\[ \sqrt{n} (\widehat{\boldsymbol\delta}_{F, n} - \boldsymbol\delta)\xrightarrow{d} \mathcal N(\mathbf 0, \sigma^2_{\epsilon}\bar{\mathbf Q}^{-1}) \]

The small sample approximation is

\[ \widehat{\boldsymbol\delta}_{F, n} \sim \mathcal N\left(\boldsymbol\delta, \widehat{\sigma}^2\left[\widehat{\mathbf Z}_s(\widehat{\lambda})^\top\widehat{\mathbf Z}_s(\widehat{\lambda})\right]^{-1}\right) \] where:

\[ \widehat{\sigma}^2 = \widehat{\boldsymbol\varepsilon}^\top\widehat{\boldsymbol\varepsilon} / n \]

and $\widehat{\boldsymbol\varepsilon} = \mathbf y_s(\widehat{\lambda}) - \mathbf Z_{s}(\widehat{\lambda})\widehat{\boldsymbol\delta}_{F}$.

Anselin, Luc. 1988. Spatial Econometrics: Methods and Models. Vol. 4. Springer.

Arraiz, Irani, David M Drukker, Harry H Kelejian, and Ingmar R Prucha. 2010. “A Spatial Cliff-Ord-Type Model with Heteroskedastic Innovations: Small and Large Sample Results.” Journal of Regional Science 50 (2): 592–614.

Das, Debabrata, Harry H Kelejian, and Ingmar R Prucha. 2003. “Finite Sample Properties of Estimators of Spatial Autoregressive Models with Autoregressive Disturbances.” Papers in Regional Science 82 (1): 1–26.

Drukker, David M, Peter Egger, and Ingmar R Prucha. 2013. “On Two-step Estimation of a Spatial Autoregressive Model with Autoregressive Disturbances and Endogenous Regressors.” Econometric Reviews 32 (5-6): 686–733.

Hansen, Lars Peter. 1982. “Large Sample Properties of Generalized Method of Moments Estimators.” Econometrica: Journal of the Econometric Society, 1029–54.

Kelejian, Harry H., Ingmar R. Prucha, and Yevgeny Yuzefovich. 2004. “Instrumental Variable Estimation of a Spatial Autoregressive Model with Autoregressive Disturbances: Large and Small Sample Results.” In Spatial and Spatiotemporal Econometrics, edited by J. P. Lesage and R. K Pace. Emerald Group Publishing Limited. https://doi.org/10.1016/S0731-9053(04)18005-5.

Kelejian, Harry H, and Ingmar R Prucha. 1998. “A Generalized Spatial Two-Stage Least Squares Procedure for Estimating a Spatial Autoregressive Model with Autoregressive Disturbances.” The Journal of Real Estate Finance and Economics 17 (1): 99–121.

Kelejian, Harry H, and Ingmar R Prucha. 1999. “A Generalized Moments Estimator for the Autoregressive Parameter in a Spatial Model.” International Economic Review 40 (2): 509–33.

Kelejian, Harry H, and Ingmar R Prucha. 2010. “Specification and Estimation of Spatial Autoregressive Models with Autoregressive and Heteroskedastic Disturbances.” Journal of Econometrics 157 (1): 53–67.

Kelejian, Harry, and Gianfranco Piras. 2017. Spatial Econometrics. Academic Press.

Lee, Lung-fei. 2001. “Generalized Method of Moments Estimation of Spatial Autoregressive Processes.” Unpublished Manuscript.

Lee, Lung-fei. 2003. “Best Spatial Two-Stage Least Squares Estimators for a Spatial Autoregressive Model with Autoregressive Disturbances.” Econometric Reviews 22 (4): 307–35.

Lee, Lung-fei. 2007. “GMM and 2SLS Estimation of Mixed Regressive, Spatial Autoregressive Models.” Journal of Econometrics 137 (2): 489–514.

Lee, Lung-fei, and Xiaodong Liu. 2010. “Efficient GMM Estimation of High Order Spatial Autoregressive Models with Autoregressive Disturbances.” Econometric Theory 26 (1): 187–230.

Lin, Xu, and Lung-fei Lee. 2010. “GMM Estimation of Spatial Autoregressive Models with Unknown Heteroskedasticity.” Journal of Econometrics 157 (1): 34–52.

Liu, Xiaodong, Lung-fei Lee, and Christopher R Bollinger. 2010. “An Efficient GMM Estimator of Spatial Autoregressive Models.” Journal of Econometrics 159 (2): 303–19.

Liu, Xiaodong, and Paulo Saraiva. 2015. “GMM Estimation of SAR Models with Endogenous Regressors.” Regional Science and Urban Economics 55: 68–79.

Prucha, IngmarR. 2014. “Instrumental Variables/Method of Moments Estimation.” In Handbook of Regional Science, edited by Manfred M. Fischer and Peter Nijkamp. Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-23430-9_90.

In particular, Kelejian and Prucha (1998) introduce S2SLS as the first step of their generalized S2SLS procedure.↩︎
In Kelejian et al. (2004), a similar approach is outlined where the matrix inverse is replaced by the power expansion. This yield an instruments matrix as $\mathbf H =\left[\mathbf X, \mathbf W\left(\sum_{l = 1}^{\infty}\rho_0^{l}\mathbf W^l\right)\mathbf X\boldsymbol\beta\right]$. In practice, the power expansion must be truncated at a finite value to ensure feasibility.↩︎
It is important to note that Lee (2003) provides the BS2SLS estimator in the context of the SAC model. Thus, his estimator is termed Best Generalized Spatial Two Stage Least Squares (BGS2SLS) estimator.↩︎
This assumption is stronger than Assumption 6 in Lee (2003). Lee (2003) shows consistency under the assumption that $\widetilde{\rho}_n$ is $n^\gamma$-consistent for some $\gamma>0$.↩︎
For any matrices $\mathbf A$ and $\mathbf B$, the generalized Schwarz inequality states that: \[ \mathbf A^\top\mathbf B\left(\mathbf B^\top\mathbf B\right)^{-1}\mathbf B^\top\mathbf A\leq \mathbf A^\top\mathbf A. \] This follows from the fact that the projection matrix $\mathbf B\left(\mathbf B^\top\mathbf B\right)^{-1}\mathbf B^\top$ is positive semi-definite and represents an orthogonal projection onto the column space of $\mathbf B$, which cannot increase the norm of any vector.↩︎
Spatially lagged variables are denoted by bar superscripts in the articles. For clarity, we use the $L$ subscript throughout. That is, a first order spatial lag of $\mathbf y_n$, $\mathbf W_n\mathbf y_n$, is denoted by $\mathbf y_L$. Higher order spatial lags are symbolized by adding additional $L$ subscripts.↩︎
Note that we are assuming that $\lambda^2$ is a new parameter.↩︎
They argue that this estimator is more efficient than the OLS estimator. However, both estimator are consistent. See Theorem 2 in (Kelejian and Prucha 1998).↩︎
For a more formal proof with any consistent estimate see Lemma C.1 in Kelejian and Prucha (2010).↩︎

6.1 A Review of GMM

6.1.1 Model Specification and Moment Conditions

6.1.1.1 Linear (IV-type) moments

6.1.1.2 Quadratic (spatially distinctive) moments

6.1.2 One-Step and Two-Step Constructions

6.1.3 Asymptotic Distribution of the One-Step GMM Estimator

6.1.4 Asymptotic Distribution of the Two-Step GMM Estimator

6.2 Spatial Two-Stage Least Squares Estimation of the SLM

6.2.1 Instruments in the Spatial Context

6.2.2 Defining the S2SLS Estimator

6.2.3 Consistency of S2SLS Estimator

6.2.4 Asymptotic Distribution of S2SLS Estimator

6.2.5 Best S2SLS Estimator

6.3 GMM Estimator of SLM

6.3.1 GMM Estimator Under Homoskedasticity

6.3.1.1 Identification strategy

6.3.2 OGMM Estimator

6.3.3 Best GMM estimator

6.3.3.1 Selection from \(\mathcal P_{2n}\)

6.3.3.2 Assuming Normality

6.3.4 S2SLS Estimator as GMM Estimator

6.4 Feasible Generalized Least Squares Estimation of the SEM

6.4.1 Generalized Least Squares Estimator

6.4.2 Moment Conditions

6.4.3 Feasible Generalized Least Squares Estimator

6.5 GMM Estimator for SEM

6.5.1 GMM Estimator Under Homoskedasticity

6.6 Feasible Generalized Spatial Two-Stage Least Squares Estimation of the SAC Model

6.6.1 Intuition Behind the Procedure

6.6.2 Moment Conditions Revised

6.6.3 Assumptions

6.6.4 Estimators and Estimation Procedure in a Nutshell

6.6.4.1 Step 1a: S2SLS estimator

6.6.4.2 Step 1b: Initial GMM estimator of \(\lambda\) based on S2SLS residuals

6.6.4.3 Step 1c: Efficient GMM estimator of \(\lambda_0\) based on S2SLS residuals

6.6.4.4 Step 2a: GS2SLS Estimator

6.6.4.5 Step 2b: Efficient GMM estimator of \(\lambda\) using GS2SLS residuals

6.6.5 Asymptotic Distribution

6.7 GMM Estimator for the SAC Model

6.8 Exercises

6.9 Technical Appendix: Asymptotic Properties

6.9.1 Asymptotic Properties of BS2SLSE

6.9.2 Asymptotic Properties of GMME for SLM

6.9.2.1 Consistency of GMM estimator

6.9.2.2 Asymptotic distribution of GMM estimator

6.9.3 Asymptotic Properties of OGMME for SLM

6.9.4 Asymptotic Distribution of GMME for SEM Model

6.9.5 Proof of Theorem 3 in Kelejian and Prucha (1998)