3.

Non-linear Models
3.1 Introduction
In previous chapters, we considered linear regression models, i.e., models that are linear in
the parameters. Such models can be represented by the general linear regression model

Linear regression models, as we have seen, include not only first-order models in p – 1 predictor
variables but also more complex models. For instance, a polynomial regression model in one or
more predictor variables is linear in the parameters, such as the following model in two predictor
variables with linear, quadratic, and interaction terms:

By definition, a linear regression model is one that is linear in its parameters or coefficients.

When linear regression model fails to describe a situation, we try to explore whether there is any
other relationship exists between the response variable and the explanatory variables.
There are occasions, however, when an empirically indicated or a theoretically justified nonlinear
regression model is more appropriate. For example, growth from birth to maturity in human
subjects typically is nonlinear in nature, characterized by rapid growth shortly after birth,
pronounced growth during puberty, and a leveling off sometime before adulthood. In another
example, dose-response relationships tend to be nonlinear with little or no change in response for
low dose levels of a drug, followed by rapid S-shaped changes occurring in the more active dose
region, and finally with dose response leveling off as it reaches a saturated level.

A regression model is called nonlinear model, if the derivatives of the model with respect to the
model parameters depends on one or more parameters. This definition is essential to distinguish
nonlinear from curvilinear regression. A regression model is not necessarily nonlinear if the
graphed regression trend is curved. A polynomial model appears curved when y is plotted against
x. It is, however, not a nonlinear model.

Model that is linear in the parameters as well as the variables is a linear regression model and so
is a model that is linear in the parameters but nonlinear in the variables. On the other hand, if a
model is nonlinear in the parameters it is a nonlinear (in-the-parameter) regression model whether

1
the variables of such a model are linear or not. In non-linear models at least one parameter appears
nonlinearly.
The following forms of the models are considered as examples of nonlinear regression models
1. Gompertz: Y  EXP ( EXP (  X ))

2. Logistic regression: Y 
1  EXP (  X )

3. Poisson Regression Model: 𝑌 = 𝑒𝑥𝑝(𝛼+𝛽𝑥𝑖 )

4. Exponential Regression Model:

  X 
5. Morgan-Mercer-Flodin: Y 
 X
6. 𝑌 = 𝑒𝑥𝑝(𝜃1 +𝜃1 𝑥+𝜀)

7.
An important difference of nonlinear regression models is that the number of regression parameters
is not necessarily directly related to the number of independent variables in the model. In linear
regression models, if there are p − 1 explanatory variables in the model, then there are p regression
coefficients in the model.
3.2 Intrinsically Linear and Intrinsically Non-linear Regression Models
Non-linear regression models can be classified into two groups according to whether they can or
cannot be made linear with respect to the parameters to be estimated. However, one has to be
careful here, for some models may look nonlinear in the parameters but are inherently or
intrinsically linear because with suitable transformation they can be made linear-in-the-parameter
regression models.
1. Intrinsically Linear Models:
Nonlinear response functions that can be linearized by a transformation are sometimes called
intrinsically linear response functions. Suitable transformations of data can frequently (not

2
always) be found that will reduce a theoretical nonlinear model to a linear form. These
transformations are said to be Linearizable and comprise a class of functions that may exist either
occur in practice or may themselves provide reasonable approximations to functions that occur in
practice. Linearizing may require transforming both the independent and dependent variable.
 For instance, the exponential response function 𝑌 = 𝑒𝑥𝑝(𝜃1 +𝜃1 𝑥+𝜀) which can be
transformed in a model which is linear in parameter just by taking logarithms to the base e
and we shall call the model Intrinsically Linear – that is, an intrinsically linear model is
one that can be made linear by a transformation of parameters.
However, the least squares estimates of the parameters will not in general be equivalent to the non-
linear parameter estimates in the original model in this case. The reason is that – in the original
non-linear model least squares implies minimization of the sum of squared residuals on y, whereas
in the transformed model, we are minimizing the sum of squared residual on “transformed
response” (that is, ln y in our present case).
 The simple logistic model is
𝒆𝒙𝒑(𝜶+𝜷𝒙𝒊)
𝝅(𝒙𝒊 ) =
𝟏 + 𝒆𝒙𝒑(𝜶+𝜷𝒙𝒊 )
𝜋(𝑥𝑖 )
logit[𝜋(𝑥𝑖 )] = 𝑙𝑜𝑔 ( ) = 𝛼 + 𝛽𝑥𝑖
1 − 𝜋(𝑥𝑖 )
 Simple Poisson regression model
𝑌𝑖 = 𝑒𝑥𝑝(𝛼+𝛽𝑥𝑖 )
2. Intrinsically Non-linear Models: The second category of nonlinear models contains models
which are nonlinear in the parameters and which cannot be made linear in the parameters after
a transformation. Models cannot be linearized in the parameters, they are called intrinsically
nonlinear regression models. Not all functions are linearizable, nor in some cases, it is
desirable to transform to linearity.
For example
𝜃1
 𝑌=𝜃 [𝑒𝑥𝑝−𝜃2 𝑥 − 𝑒 −𝜃1 𝑥 ] + 𝜀 however, is impossible to convert into a form –
1 +𝜃2

linear in parameters, which we call Intrinsically Nonlinear.

 Nonlinear exponential model in one independent variable with an additive error term
is

3
 3.3 Estimation of Regression Parameters
Estimation of the parameters of a nonlinear regression model is usually carried out by the method
of least squares or the method of maximum likelihood, just as for linear regression models. Also
as in linear regression, both of these methods of estimation yield the same parameter estimates
when the error terms in nonlinear regression model are independent normal with constant variance.
Unlike linear regression, it is usually not possible to find analytical expressions for the least squares
and maximum likelihood estimators for nonlinear regression models. Instead, numerical search
procedures must be used with both of these estimation procedures, requiring intensive
computations. The analysis of nonlinear regression models is therefore usually carried out by
utilizing standard computer software programs.