sensors

Article
Prediction of Driver’s Intention of Lane Change by
Augmenting Sensor Information Using Machine
Learning Techniques
Il-Hwan Kim 1 , Jae-Hwan Bong 2 , Jooyoung Park 3, * and Shinsuk Park 2, *
1 Hyundai Motor Company, Hwaseong-si 18280, Korea; kihwan2@[Link]
2 Department of Mechanical Engineering, Korea University, Seoul 02841, Korea; delitian@[Link]
3 Department of Control and Instrumentation Engineering, Korea University, Sejong 30019, Korea
* Correspondences: parkj@[Link] (J.P.); drsspark@[Link] (S.P.); Tel.: +82-44-860-1444 (J.P.);
+82-2-3290-3373 (S.P.)

Academic Editor: Vittorio M. N. Passaro

Received: 26 April 2017; Accepted: 7 June 2017; Published: 10 June 2017

Abstract: Driver assistance systems have become a major safety feature of modern passenger vehicles.
The advanced driver assistance system (ADAS) is one of the active safety systems to improve the
vehicle control performance and, thus, the safety of the driver and the passengers. To use the ADAS
for lane change control, rapid and correct detection of the driver’s intention is essential. This study
proposes a novel preprocessing algorithm for the ADAS to improve the accuracy in classifying the
driver’s intention for lane change by augmenting basic measurements from conventional on-board
sensors. The information on the vehicle states and the road surface condition is augmented by using
an artificial neural network (ANN) models, and the augmented information is fed to a support vector
machine (SVM) to detect the driver’s intention with high accuracy. The feasibility of the developed
algorithm was tested through driving simulator experiments. The results show that the classification
accuracy for the driver’s intention can be improved by providing an SVM model with sufficient
driving information augmented by using ANN models of vehicle dynamics.

Keywords: advanced driver assistance system (ADAS); lane change; driver’s intention; artificial
neural network (ANN); support vector machine (SVM)

1. Introduction
As the number of vehicles increases worldwide, the traffic situation becomes increasingly
complicated in terms of safety. The automotive industry has been developing various safety
technologies, and driver assistance systems, such as headway distance control, automatic braking
system and evasive steering system, have become one of the major features of a vehicle for the safety of
the driver and passengers. As an active safety system, the advanced driver assistance system (ADAS)
has been developed to assist the driver for improved safety and better vehicle control. The ADAS
equipped with advanced sensors and intelligent video systems is designed to alert the driver to
potential traffic hazards or to take over control of the vehicle to avoid impending collisions and
accidents. The ADAS is activated when the predetermined conditions for the driver’s operation and
the state of the vehicle are met. In conventional ADAS, a threshold is set for driver’s control input, such
as the steering wheel angle, the steering wheel angular velocity, or the pedal position. If the driver’s
control input is greater than the predetermined threshold, the ADAS is activated. In the activation of
ADAS, however, there can be conflicting situations where the intervention of the ADAS can interfere
with the driver’s intention of operation. Correct prediction of driver’s intention is an essential part to
determine whether the ADAS should engage to override the driver’s control inputs [1].

Sensors 2017, 17, 1350; doi:10.3390/s17061350 [Link]/journal/sensors

Sensors 2017, 17, 1350 2 of 18

For the most time during driving, the driver is required to maneuver the steering wheel, and
the lane change maneuver is one of the main causes of road traffic accidents [2]. It was reported that
the percentage of fatal accidents related to lane change increased from 18% in 2005 to 23.6% in 2014,
while the total number of fatal crash in the U.S. gradually decreased from 50,000 to 38,000 during the
same period [3]. ADAS technologies, such as Lane Support Systems, Lane Keeping Assistance System
(LKAS) and Lane Departure Warning System (LDWS), enable automated lane control. The lane change
control of the ADAS is based on the driver’s control input and surrounding traffic situation. With the
current technologies of the ADAS, however, there are possibilities of unwanted lane change against
the driver’s intention, which may lead to a situation that endangers the safety of the driver’s vehicle
and its surrounding vehicles.
To alleviate the risk of misjudging the driver’s intention, many studies have attempted to
incorporate machine learning techniques to identify the driver’s intention for lane change control with
the ADAS [4–10]. Machine learning has proven its utility in estimation, classification and prediction
of system behaviors. For identification of the driver’s intention, in particular, many researchers have
investigated classification techniques, such as Hidden Markov Model (HMM), Support Vector Machine
(SVM), and Bayesian network. Kuge et al. (2000) [4] developed an HMM-based steering behavior
model for emergency lane change, normal lane change, and lane keeping. They reported that the
classification accuracy of the model was higher than 98.3%. Jin et al. (2011) [5] developed an algorithm
for lane change recognition using the steering wheel angle and the angular velocity as input data
to a HMM model. With their method, the classification accuracies for lane change left (LCL), lane
change right (LCR) and lane keeping (LK) were 84%, 88% and 94%, respectively. Tran et al. (2015) [6]
investigated the performance of a HMM-based system with two different sets of inputs: one only
with the driver’s control input (steering wheel angle and gas and brake pedal positions) and the other
both with the driver’s control input and with the vehicle states (velocity, acceleration and yaw rate).
It was confirmed that with the driver’s control input and the vehicle states the HMM model shows far
superior performance in terms of classification time and accuracy. Mandalia and Salvucci (2005) [7]
compared by experiment the overlapping window method with the non-overlapping window method.
The accuracy of the overlapping method was about 1.2% higher than the non-overlapping method.
Aoude et al. (2011) [8] compared SVM- and HMM-based methods in classification of law-abiding
and violating drivers. They reported that the SVM-based method has higher accuracies that the
HMM-based method in most cases. Kumar et al. (2013) [9] proposed a machine learning algorithm
that combines SVM and Bayesian filter. Relevance vector machine (RVM) is an SVM-based Bayesian
inference model for probabilistic classification. Morris and Doshi (2011) [10] introduced a RVM model
that is capable of classifying the driver’s intention within 3 s before an actual lane change happens.
Liu et al. (2010) [11] employed the parallel Bayesian network (PBN) to identify the driver’s lane
change behavior. They reported that the PBN model can reduce the response time and error rate.
Schubert et al. (2010) [12] developed a classification technique for lane change maneuvers by using
camera vision and a radar sensor. In other studies, the Bayesian network was employed to classify the
driver intention [13–17].
To classify the driver’s intention at a high level of accuracy, abundant information on the vehicle
states should be provided to the machine learning algorithms. The studies mentioned above employed
rather expensive sensors to measure various vehicle states, such as the lateral velocity, the heading
angle, the side slip angle and the lateral position, to identify the driver’s intention for lane change.
Those sensors, however, are impractical to be used in commercial passenger vehicles. Recently, many
commercial vehicles are being equipped with on-board sensors to provide basic measurements, such as
the steering wheel angle, the yaw rate, the longitudinal and lateral accelerations, and the wheel speed,
at an affordable cost [18]. While the on-board sensors are unable to provide the ADAS algorithm with
sufficient information on the vehicle states, the vehicle states other than the direct measurements from
the on-board sensors may be estimated using machine learning techniques based on the measured data.
Along with the vehicle states, the road condition such as the friction coefficient of the road surface is
Sensors 2017, 17, 1350 3 of 18

an important factor to be considered to classify the driver’s intention for lane change, and it can also
be estimated from the vehicle states measured by the on-board sensors [19,20].
While complex vehicle dynamics models with nonlinear differential equations can be used in
augmenting the information on the vehicle states and the road surface condition, real-time computation
of numerical integration requires great computation power [21]. For fast real-time computation, purely
numerical models with better computational efficiency, such as artificial neural network (ANN), were
suggested rather than physical mathematical models of vehicle dynamics [22,23]. An ANN model of
vehicle dynamics is suitable for real-time information augmentation, since it only requires summation
and product operations of matrices, rather than time-consuming numerical integration of nonlinear
differential equations.
In this study, we propose a novel preprocessing algorithm as a practical solution to improving
the accuracy of the ADAS in determining the driver’s intention for lane change by augmenting basic
measurements from conventional on-board sensors. The inputs to the algorithm include the measured
data from the on-board sensors and the augmented vehicle states along with the road surface condition
estimated from the measured data. The vehicle states and the road surface condition are estimated by
using ANN models that simulate nonlinear dynamics of the vehicle and the interaction between the
tires and the road. The ANN models trained by the data obtained from a driving simulator provide
augmented information on the vehicle states and the road condition based on the limited information
from the on-board sensors. The augmented information from the ANN models along with the direct
sensor measurements is then fed to an SVM mode to classify the driver’s intention for lane change.
The effectiveness of the proposed algorithm was verified through driving simulator experiments,
and the experimental results show that the classification accuracy for the driver’s intention can be
improved by providing an SVM model with sufficient driving information augmented by using ANN
models of vehicle dynamics and vehicle-road interaction.
This paper is organized as follows: Section 2 illustrates the preprocessing algorithm for the
ADAS developed in this study. Section 3 describes the driving simulator experiments to evaluate
the performance for the proposed algorithm, and Section 4 presents the experimental results. Finally,
Section 5 contains conclusions and future work directions.

2. Classification of Driver’s Intention for Lane Change Using Augmented Sensor Information
This section describes the preprocessing algorithm for the ADAS, which detects the driver’s
intention for lane change based on the augmented information on the road surface condition and the
vehicle states. Figure 1 illustrates the schematic diagram of the algorithm. The augmented information
is estimated from the basic measurements acquired from the on-board sensors commonly equipped on
commercial passenger vehicles. The on-board sensors provide the basic measurements of the vehicle
states and the driver’s control inputs: the measured vehicle states include the longitudinal and lateral
accelerations, the yaw rate, and the wheel speed, while the measured driver’s control inputs include
the steering wheel angle and the throttle position. Based on the sensor measurements, the algorithm
estimates the road surface condition (non-slippery or slippery). The estimated condition of the road
surface, along with the sensor measurements, is then used to estimate the vehicle states that cannot be
measured by the on-board sensors. The vehicle states augmented by estimation include the lateral
velocity, the side slip angle, the lateral tire force, the roll rate, the suspension spring compression, and
the heading direction (Figure 2). The augmented information on the vehicle states is then provided to
the algorithm to classify the driver’s intention for lane change. The identified driver’s intention is used
to determine whether to activate the ADAS and override the driver’s control inputs for lane change.
Sensors 2017, 17, 1350 4 of 18
Sensors 2017, 17, 1350 4 of 18

The vehicle state estimation module augments the vehicle states by using an ANN model
The vehicle state estimation module augments the vehicle states by using an ANN model
representation
Sensors
of vehicle dynamics. The driver intention detection module identifies the driver’s
2017, 17, 1350 4 of 18
representation of vehicle dynamics. The driver intention detection module identifies the driver’s
intention for lane change by using an SVM model with the augmented information as inputs.
intention for lane change by using an SVM model with the augmented information as inputs.

Figure 1.
Figure 1. A
A schematic
schematic diagram of the
diagram of the system
system developed
developed for
for driver
driver intention
intention classification.
classification.
Figure 1. A schematic diagram of the system developed for driver intention classification.

Figure 2. Vehicle states measured by on-board sensors and estimated by ANN models.
Figure
Figure 2.
2. Vehicle
Vehicle states
states measured by on-board
measured by on-board sensors
sensors and
and estimated
estimated by
by ANN
ANN models.
models.
2.1. ANN Models for Road Condition Classification and Vehicle State Estimation
2.1. ANN Models for Road Condition Classification and Vehicle State Estimation
The preprocessing
ANN-based modelsalgorithm
are used forforthe
theroad
ADAS consistsclassification
condition of three main modules
module and as
theillustrated in
vehicle state
FigureANN-based
1: the road models
condition areclassification
used for themodule,
road condition
the classification
vehicle state module
estimation and the
module andvehicle
the state
driver
estimation module of the preprocessing algorithm for the ADAS. Artificial neural network (ANN)
estimation
intention modulemodule.
detection of the preprocessing
Theapproach algorithm
road condition for the ADAS.
classification moduleArtificial neuralwhether
determines networkthe (ANN)
road
is a computational learning inspired by how biological neural networks learn from
is a computational
surface condition learning approach inspired by how biological neural networks learn from
experiences. SinceisANN
non-slippery or slippery
can effectively by nonlinear
solve using an ANN-based
problems ofpatternvehiclerecognition
dynamics,technique.
an ANN
experiences.
The Since ANN can effectively solve nonlinear problems ofanvehicle dynamics, an ANN
model can be a pertinent solution to augmenting sensor information that is insufficientrepresentation
vehicle state estimation module augments the vehicle states by using ANN model to determine
model
of candynamics.
be a pertinent solution to augmenting sensor information
identifiesthat
theisdriver’s
insufficient to determine
thevehicle
driver’s intention The driver
for lane intention
change. detection module intention for lane
the driver’s
change by intention
using an SVMfor lane
model change.
with the augmented information as inputs.
The basic structure of the three-layered ANN is illustrated in Figure 3, where the network
The basic structure of the three-layered ANN is illustrated in Figure 3, where the network
consists of an input layer, a hidden layer, and an output layer. Each node in the hidden layer and
2.1. ANNofModels
consists an inputfor Road
layer,Condition
a hiddenClassification
layer, and anandoutput
Vehicle layer.
State Estimation
Each node in the hidden layer and
the output layer has an activation function, which defines the output of that node given its own
the output layer has
ANN-based modelsan activation
are used for function,
the road which defines the outputmodule
of that and
node given its state
own
input. The type of the activation function cancondition
be chosen classification
properly based on thethe vehicle
purpose of the
input. The
estimation type of the activation function can be chosen properly based on the purpose of thea
network. Inmodule
learning of phase,
the preprocessing
the weighted algorithm for the
connections [Link]
between Artificial neural
of the network
network (ANN) is
are adjusted.
network. In learning phase, the weighted connections between nodes of the network
computational learning approach inspired by how biological neural networks learn from experiences. are adjusted.
Since ANN can effectively solve nonlinear problems of vehicle dynamics, an ANN model can be a
Sensors 2017, 17, 1350 5 of 18

pertinent solution to augmenting sensor information that is insufficient to determine the driver’s
intention for lane change.
The basic structure of the three-layered ANN is illustrated in Figure 3, where the network consists
of an input layer, a hidden layer, and an output layer. Each node in the hidden layer and the output
layer has an activation function, which defines the output of that node given its own input. The type
of the activation function can be chosen properly based on the purpose of the network. In learning
Sensors the
phase, 2017,weighted
17, 1350 connections between nodes of the network are adjusted. 5 of 18

[Link]
Figure Basicnetwork
networkarchitecture
architectureofofthree-layered
three-layeredANN.
ANN.

[Link]
2.1.1. RoadCondition
ConditionClassification
ClassificationModule
Module
Table1 summarizes
Table 1 summarizes thethe friction
friction coefficients
coefficients of four
of four roadroad conditions:
conditions: dry asphalt,
dry asphalt, gravel,
gravel, wet,
wet, and
and snowy [24,25]. In this study, dry asphalt and gavel are grouped as the non-slippery
snowy [24,25]. In this study, dry asphalt and gavel are grouped as the non-slippery road condition, road
condition,
and wet andandsnowywet are
andgrouped
snowy as arethe
grouped
slipperyasroad
the slippery road
condition. Thecondition. The road
road condition condition
classification
classification
module module
classifies classifies
the road surfacethe road surface
conditions conditions
into the intonon-slippery
two classes: the two classes: non-slippery and
and slippery.
slippery.
Table 1. Road friction coefficient and road surface condition.
Table 1. Road friction coefficient and road surface condition.
Road Surface Conditions Friction Coefficients
Road Surface Conditions Friction Coefficients
Dry Asphalt 0.8
Dry Asphalt 0.8
Gravel 0.6
Gravel
Wet 0.4 0.6
Wet
Snowy 0.3 0.4
Snowy 0.3
The road condition classification module is designed to activate when the throttle signal is
The since
detected, road condition classification
the identification of themodule is designed
road friction to activate
coefficient when
is easier the throttle
during signaloris
acceleration
detected, since
deceleration than the identification
during constant-speedof the road The
driving. friction coefficient
signals from theison-board
easier during
sensorsacceleration
from the timeor
the driver step one’s foot on the acceleration pedal to the time when the driver takes one’s footfrom
deceleration than during constant-speed driving. The signals from the on-board sensors the
off the
time the driver step one’s foot on the
pedal are used to determine the road condition. acceleration pedal to the time when the driver takes one’s foot
off the
Thepedal
road are used toclassification
condition determine the road condition.
module has the structure of the three-layered ANN with the
softmax activation function in the output [Link]
The road condition classification module Thethe structure of model
three-layered the three-layered
was employed ANN with on
based the
softmax activation function in the output layer. The three-layered model was
the guideline suggested by Panchal et al. (2011) [26]. The input, hidden, and output layers have six, employed based on
the guideline
thirty, and twosuggested by PanchalAt
nodes, respectively. et the
al. (2011)
output[26].
node,Thethe
input, hidden,
softmax and output
activation layers
function havethe
yields six,
thirty, and two nodes, respectively. At the output node, the softmax activation
probability values of the classification represented by the node. For the nodes in the hidden layer, the function yields the
probability
bipolar sigmoidvalues of the classification
is commonly used as the represented
activation by the node.
function. For training
In the the nodes in the
phase of hidden layer,
the module,
the bipolar sigmoid is commonly used as the activation function. In
the performance index defined by cross-entropy is minimized, and in the testing phase, the class the training phase of the
module, the performance index defined by cross-entropy is minimized, and in
labels are determined by applying the one-hot-encoding to the output probability values. The detailed the testing phase, the
class labels are determined by applying the one-hot-encoding to the output probability values. The
detailed architecture of the neural network is shown in Figure 4. In the input layer, the six nodes are
for the signals from the on-board sensors (the longitudinal and lateral accelerations, the yaw rate,
the wheel speed, the steering wheel angle and the throttle position).
Sensors 2017, 17, 1350 6 of 18

architecture of the neural network is shown in Figure 4. In the input layer, the six nodes are for the
signals from the on-board sensors (the longitudinal and lateral accelerations, the yaw rate, the wheel
speed,2017,
Sensors the 17,
steering
1350 wheel angle and the throttle position). 6 of 18

Figure
Figure 4.
4. Artificial
Artificialneural
neural network
network model
model for
for road
road condition
condition classification
classification module.
module.

2.1.2.
2.1.2. Vehicle
Vehicle StateState Estimation
Estimation Module Module
The
The vehicle
vehicle state
state estimation
estimation module module estimates
estimates the the vehicle
vehicle states
states basedbased on on the
the datadata from
from the the
on-board sensors and the road condition classification
on-board sensors and the road condition classification module. This module uses the NARX (nonlinear module. This module uses the NARX
(nonlinear
autoregressive autoregressive
with exogenous with input)
exogenousneuralinput) network,neural which network,
is a type which is a type
of recurrent of recurrent
neural network
neural network particularly
particularly useful for time series analysis. useful for time series analysis.
The NARX model
The NARX modelisisknown knowntotobebe anan effective
effective tooltool
for for
timetimeseries series prediction
prediction comparedcompared with
with other
other feedforward ANN models, since it enables to relate the
feedforward ANN models, since it enables to relate the current value of a time series to the past values current value of a time series to the
past
of the values of the and
time series timethe series and the inputs
exogenous exogenous inputs [27,28].
[27,28].
The
The mathematical form of the NARX model is
mathematical form of the NARX model is given
given as as follows:
follows:
( ) = [ ( − 1), ( − 2); ( − 1), ( − 2), ( − 3); ( − 1), ( − 2), (
(1)
ŷ(k ) = f [y(k − 1), y(k− − 3);
2); u… ; ( − 1), ( − 2), ( − 3)]
1 ( k − 1), u1 ( k − 2), u1 ( k − 3); u2 ( k − 1), u2 ( k − 2), u2 ( k − 3); . . . ; u5 ( k − 1), u5 ( k − 2), u5 ( k − 3)] (1)
where ( ) ∈ ℝ and ( ) ∈ ℝ denote the inputs and output of the model at discrete time step k,
respectively.
where u(k) ∈The filterŷorders
R and (k) ∈ R fordenote
input the ( )inputs
and output
and output ( ) of arethe model = 3 and = 2, respectively.
at discrete time step k,
(∙) is a nonlinear
respectively. function
The filter orderswithfor universal
input uapproximation
(k) and outputcapability. ŷ(k ) are duThe = 3nonlinear
and dy =function (∙) of
2, respectively.
the
f (·) is a nonlinear function with universal approximation capability. The nonlinear function f (·) of
NARX model plays an important role to model nonlinear relations among the vehicle states of
vehicle
the NARX dynamics.
model plays an important role to model nonlinear relations among the vehicle states of
vehicle There are two modes for the NARX model: series-parallel (SP) mode and parallel (P) mode. SP
dynamics.
modeThere is mainly
are two used for single
modes for the step prediction
NARX model: or series-parallel
short term prediction (SP) mode since andtheparallel
values (P) from the
mode.
previous
SP mode step are inserted
is mainly used forassingle an input step vector
prediction for the prediction
or short at the nextsince
term prediction [Link] P mode
valueshas froma
feedback
the previous loopstep structure and theas
are inserted estimated
an inputoutput vectorvalues
for theare includedatasthe
prediction an next
[Link] P of
modenetwork,
has a
and
feedback its performance
loop structure is better
and thethan SP mode
estimated outputin multi-step
values areor mid-and-long
included as an input termvector
prediction tasks
of network,
[29–31]. Combinations
and its performance of thethan
is better twoSP modes
modecan be used for
in multi-step ortraining
mid-and-long and testing of the neural
term prediction tasksnetwork
[29–31].
[31–35]. For the vehicle state estimation module, P mode
Combinations of the two modes can be used for training and testing of the neural network [31–35]. was used for training and testing of the
For
module,
the vehicle sincestatetheestimation
module ismodule, designed to carry
P mode was out long
used forterm prediction.
training and testing of the module, since the
module Theisstructure
designedoftothe carryNARX neural
out long network
term in P mode is shown in Figure 5. The NARX neural
prediction.
network The structure of the NARX neural network structure.
has a multi input-single out (MISO) in P mode To yield in
is shown sixFigure
vehicle 5. states
The NARX (the lateral
neural
velocity,
network the has side
a multislipinput-single
angle, the lateral tire force,
out (MISO) the roll
structure. rate, the
To yield suspension
six vehicle statesspring compression,
(the lateral velocity,
and
the side the heading
slip angle, direction),
the lateral six tire
separate
force,NARXthe roll models
rate, the aresuspension
required. Each spring NARX model consists
compression, and theof
the input layer, the output layer, and the hidden layer. The
heading direction), six separate NARX models are required. Each NARX model consists of the input input, hidden and output layers have 17,
10
layer,andthe one nodes,
output respectively.
layer, and the hidden In the layer.
hiddenThe layer,
input,thehidden
bipolarand sigmoid
outputfunctionlayers have is used17, 10as and
the
activation function. For the output layer, a linear activation
one nodes, respectively. In the hidden layer, the bipolar sigmoid function is used as function is employed for the activation
activation
function. Five measurements from the on-board sensors (the longitudinal and lateral accelerations,
the yaw rate, the wheel speed, and the steering wheel angle) and the estimated output are used as
inputs to the nodes in the input layer. For each of the five measurements from the on-board sensors,
the values at two time steps and one time step before the present time and the present time are used
( = 3). For the estimated input, the estimation at one step before the present time and the
presentation are used ( = 2). These form the total of 17 inputs. For the hidden layer, 10 nodes
Sensors 2017, 17, 1350 7 of 18

function. For the output layer, a linear activation function is employed for the activation function.
Five measurements from the on-board sensors (the longitudinal and lateral accelerations, the yaw rate,
the wheel speed,
Sensors 2017, and the steering wheel angle) and the estimated output are used as inputs
17, 1350 7 of 18to the
nodes in the input layer. For each of the five measurements from the on-board sensors, the values
were
at two timeused afterand
steps havingonetested the number
time step beforeofthenodes fromtime
present five toand20 nodes. The output
the present time layer has one
are used (du = 3).
node that yields one of the six vehicle states.
For the estimated input, the estimation at one step before the present time and the presentation are
Sensors 2017, 17, 1350 7 of 18
used (dy = 2). These form the total of 17 inputs. For the hidden layer, 10 nodes were used after having
tested the number of nodes
were used fromtested
after having five to
the 20 nodes.
number The from
of nodes output layer
five to has one
20 nodes. nodelayer
The output thathas
yields
one one of the
node that yields one of the six vehicle states.
six vehicle states.

Figure 5. NARX Neural network model for estimation of vehicle state parameters (z is the unit
Figure 5. NARX −1 is the unit
time Neural
delay). network model for estimation of vehicle state parameters (z
Figure
time delay). 5. NARX Neural network model for estimation of vehicle state parameters (z is the unit
time delay) .
2.2. Driver Intention Detection Module
TheDetection
2.2. Driver Intention driver intention
Moduledetection module classifies the driver’s intention for lane change based on
2.2. Driverthe
Intention Detection
augmented Module
information from the on-board sensors and the vehicle state estimation module.
The driver Thisintention detection
module employs module
a support classifies
vector machine (SVM) the driver’s
model intention
for classification. SVMforclassification
lane change based
The driver intention detection module classifies the driver’s intention for lane change based on
is known to have good generalization abilities. For example, binary classification using SVM can
on the augmented
the augmented information from the on-board sensors and the vehicle state estimation module.
find theinformation from the
optimal hyper-plane thaton-board
maximizes sensors and the
the separation vehicle
margin betweenstate
twoestimation
classes. Whenmodule.
This This
module employs
module employs
dealing a support
with a support vector
non-separable data,machine
vector machine
SVM (SVM)
utilizes (SVM) modelmapfor
model
the feature for classification.
φ toclassification.
transform a low SVM
SVM classification is
classification
dimensional
known to have
is known togood
inputhave generalization
space
good abilities.
into ageneralization
feature space higherFor
of aabilities. example,
dimension
For wherebinary
example, linear classification
classification
binary is moreusing
classification feasibleSVM
using as
SVM can find
can
illustrated in Figure 6.
the optimal hyper-plane that maximizes the separation margin between two classes. When dealing
find the optimal hyper-plane that maximizes the separation margin between two classes. When
with dealing with non-separable
non-separable data, SVM data, SVM
utilizes theutilizes
featurethe map feature
φ to map φ to transform
transform a low dimensional
a low dimensional input space
into ainput space
feature into of
space a feature
a higher space of a higher
dimension dimension
where linearwhere linear classification
classification is moreas
is more feasible feasible as
illustrated in
illustrated
Figure 6. in Figure 6.

Figure 6. Use of feature map for non-separable problem.

Figure 6. [Link]
Figure Useofoffeature
feature map fornon-separable
map for non-separable problem.
problem.
Sensors 2017, 17, 1350 8 of 18

The problem of finding the optimal hyper plane can be formulated as the constrained optimization
problem by introducing the slack variable ξ of the so-called soft-margin method, as follows:

N
1
Minimize J (ω, ξ ) = kωk2 + C ∑ ξ i
2 i =1 (2)
 
T
Subject to ti ω Φ( xi ) + b ≥ 1 − ξ i , i = 1, . . . , Nξ i ≥ 0 , i = 1, . . . , N

By introducing
Sensors 2017,the Lagrange multipliers and applying the Karush–Kuhn–Tucker (KKT)
17, 1350 8 ofconditions,
18

Equation (7) becomes the following Lagrange dual problem:

The problem of finding the optimal hyper plane can be formulated as the constrained
optimization problem by introducing the slack variable ξ of the so-called soft-margin method, as
N
follows: 1 N N
∑ αi − ∑ ∑ αi α j ti t j K


Maximize L(α) =
f xi , x j
1 2 i =1 j =1
Minimize (i =,1 ) = ‖ω‖ + (3)
2
N (2)
Subject
Subject ∑ α( i tiΦ(= 0,) +0 )≤≥α1i −≤ C,
to to
≥ 0 , = 1, … ,
, =i 1,
=…1,
, ..., N
i =1
By introducing the Lagrange multipliers and applying the Karush–Kuhn–Tucker (KKT)
conditions,

Equation (7) becomes the following Lagrange dual problem:
where K xi , x j is the kernel function defined by the inner-product of two feature vectors φ(xi ) and
1
Maximize ((8)) =
φ(x j ). The dual problem of Equation can be−solved by utilizing
( , )the KKT condition to yield the
2
optimal values for αi and b. With the acquired constants αi and b, the decision function (3)is defined
as follows: Subject to = 0, 0 ≤ ≤ C, = 1, … ,!
N
where ( , ) is the kernel x ) = sgn
f (function ∑
defined by αi ythe ( x, xi ) + b of two feature vectors φ( ) and
i K inner-product (4)
φ( ). The dual problem of Equation (8) can i =1be solved by utilizing the KKT condition to yield the
optimal values for
With the input x, the and . With the acquired constants
decision function yields the binary and , the decision function is
value, either positive or
defined as follows:
negative, to classify two classes. ( This )= can
(∑ be extended
( , )+ ) to multi-class classification,(4) such as
one-against-all classification.
With the input , the decision function yields the binary value, either positive or negative, to
Figure 7classify
showstwo theclasses. This can
schematic be extended
diagram of the to multi-class
driver classification,
intention detection such as one-against-all
module. The 11 inputs to
classification.
the driver intention detection module includes five measurements from the on-board sensors and six
Figure 7 shows the schematic diagram of the driver intention detection module. The 11 inputs
vehicle states
to from theintention
the driver vehicledetection
states estimation
module includes module. For signalfrom
five measurements extraction, overlapping
the on-board sensors and sliding
windows aresixapplied
vehicle to the from
states 11 input signals.
the vehicle It was
states reported
estimation thatFor
module. thesignal
overlapping
extraction,sliding window has
overlapping
sliding windows
better classification abilityare applied
than to the 11 input signals.
the non-overlapping It was window
sliding reported that[7].theThe
overlapping
windowsliding
size and the
window has better classification ability than the non-overlapping sliding window [7]. The window
window slide size used in the module are 0.5 s and 0.2 s, respectively.
size and the window slide size used in the module are 0.5 s and 0.2 s, respectively.

Figure 7. Operation procedure of driver intention recognition using SVM.

Figure 7. Operation procedure of driver intention recognition using SVM.
For feature extraction, the average and the variance of the windowed signals are calculated,
and the principal component analysis (PCA) is performed on the windowed signals. The feature
For feature extraction, the average and the variance of the windowed signals are calculated, and
sets obtained from feature extraction are fed to the pre-trained SVM to classify the driver’s intention
the principalforcomponent analysis
lane change into (PCA)
three classes: laneischange
performed onlane
left (LCL), thechange
windowed signals.
right (LCR), The
and lane feature sets
keeping
obtained from
(LK).feature extraction are fed to the pre-trained SVM to classify the driver’s intention for
lane change intoThisthreeis classes:
a multi-class
laneclassification
change leftproblem with change
(LCL), lane three classes.
rightOne-against-all
(LCR), and lane method is
keeping (LK).
employed for the multi-class SVM, and K-fold cross validation is used for the performance
 Sensors 2017, 17, 1350 9 of 18

evaluation of the driver intention detection module, which is known to be an effective method to
deal with a small set of data. For the kernels for training, the quadratic and Gaussian kernels are
Sensors 2017, 17, 1350 9 of 18
used for non-slippery and slippery road conditions, respectively.

3. Driving
This Simulation
is a multi-class Experiments
classification problem with three classes. One-against-all method is employed
for the multi-class SVM, andalgorithm
The preprocessing K-fold cross validation
for the is used for
ADAS described in the performance
Section evaluationinofa the
2 was implemented
PC-based driving simulator (Figure 8). The driving simulator is controlled by
driver intention detection module, which is known to be an effective method to deal with PreScan software ver. set
a small
7.3 (TASS International, Rijswijk, The Netherlands), CarSim software (Mechanical Simulation
of data. For the kernels for training, the quadratic and Gaussian kernels are used for non-slippery and
Corporation,
slippery Ann Arbor,
road conditions, MI, USA), and Simulink (MathWorks, Natick, MA, USA). PreScan is used
respectively.
as a physics-based simulation platform, and CarSim is used for simulation of vehicle dynamics. The
vehicle
3. Driving used in theExperiments
Simulation simulator was front-wheel drive. The data from the driving simulator can be
collected at the rate of 500 Hz, which is the maximum sampling frequency with PreScan and
The preprocessing
CarSim software. algorithm for the ADAS described in Section 2 was implemented in a PC-based
driving simulator
A human (Figure
subject8).(27-year-old
The driving simulator
male with sixisyears
controlled by PreScan
of driving software
experience) ver. 7.3 (TASS
was instructed to
performRijswijk,
International, three maneuvers of lane change
The Netherlands), control:
CarSim lane change
software left (LCL),
(Mechanical lane change
Simulation right (LCR)Ann
Corporation,
Arbor, and
MI, lane
USA),keeping (LK). The(MathWorks,
and Simulink subject was asked to drive
Natick, MA, within the speedisrange
USA). PreScan used 30–80 km/h on a
as a physics-based
simulation platform, and CarSim is used for simulation of vehicle dynamics. The vehicleplaced
one-way road with three lanes of the width of 3.5 m. The lanes were separated by cones used inat the
the interval of 50 m.
simulator was front-wheel drive. The data from the driving simulator can be collected at the rate of
The three modules of the algorithm were trained using the data collected from the driving
500 Hz, which is the maximum sampling frequency with PreScan and CarSim software.
simulator. The driving simulator was also used to test the performance of the trained modules.

(a) (b)
Figure 8. Setup for driving simulator experiments: (a) schematic diagram of driving simulator; and
Figure 8. Setup for driving simulator experiments: (a) schematic diagram of driving simulator; and (b)
(b) setup of Steering wheel and pedal for obtaining driving data.
setup of Steering wheel and pedal for obtaining driving data.
3.1. Training of Road Condition Classification Module
A human subject
To train the(27-year-old
ANN modelmale of thewith
roadsixcondition
years of classification
driving experience)
module,wasthe instructed to perform
driving simulation
three maneuvers
was performed of lane
underchange control:
four different lane
road change
surface left (LCL),
conditions lane change
(dry asphalt, gavel, right (LCR)
wet, and andaslane
snowy)
keeping (LK). The subject was asked to drive within the speed range 30–80 km/h on
listed in Table 1. The four road conditions are grouped into two classes (non-slippery for dry a one-way road
with three
asphalt and gavel; slippery for wet and snowy). The module was trained by using a total of 120,000 m.
lanes of the width of 3.5 m. The lanes were separated by cones placed at the interval of 50
datathree
The sets with 60,000 of
modules forthe
the non-slippery
algorithm were road condition and 60,000
trained using for the
the data slippery road
collected fromcondition.
the driving
simulator. The driving simulator was also used to test the performance of the trained modules.
3.2. Training of Vehicle State Estimation Module
3.1. Training of Road
The Condition
vehicle Classification
state estimation Module
module was trained under two different road conditions
To(non-slippery
train the ANN and slippery). For each road condition, six NARX models yield six vehicle states.
model of the road condition classification module, the driving simulation was
Thus, the vehicle state estimation module is composed of 12 sub-modules.
performed under four different road surface conditions (dry asphalt, gavel, wet, and snowy) as listed
To train each sub-module, the input data from the on-board sensors (five measurements) and
in Table 1. The four road conditions are grouped into two classes (non-slippery for dry asphalt and
the target data from the driving simulator (six vehicle states) were provided to train the vehicle
gavel; slippery for wetmodule
state estimation and snowy). The
by using themodule was trained by
Levenberg–Marquart using a total of algorithm.
back-propagation 120,000 data sets with
It should
60,000 for the non-slippery road condition and 60,000 for the slippery road condition.

3.2. Training of Vehicle State Estimation Module

The vehicle state estimation module was trained under two different road conditions (non-slippery
and slippery). For each road condition, six NARX models yield six vehicle states. Thus, the vehicle
state estimation module is composed of 12 sub-modules.
To train each sub-module, the input data from the on-board sensors (five measurements) and
the target data from the driving simulator (six vehicle states) were provided to train the vehicle state
Sensors 2017, 17, 1350 10 of 18

estimation module by using the Levenberg–Marquart back-propagation algorithm. It should be noted

that all 12 sub-modules have the same structure and training data sets, while the target data sets are
all different.
For the training of the module, we used 80,000 data sets with 40,000 for the non-slippery road
condition and 40,000 for the slippery road condition, which was sampled at 500 Hz. For the two
sub-modules to estimate the heading direction in the non-slippery and slippery road conditions, the
on-board sensor data, down-sampled at 50 Hz, were used for training, and the total number of the
training data sets was 3000 (1500 for the non-slippery road and 1500 for the slippery road).

3.3. Training of Driver Intention Detection Module

The driver intention detection module was trained using the input data from the on-board sensors
and the vehicle state estimation module and the target data of the driver’s intention of lane change
obtained from the questionnaire with the human subject performing the driving simulation. From the
windowed signals, the features (the average, the variance, and the principal components) of the signals
were extracted to be used as the input to the SVM module. The SVM module was trained to classify the
three intentions of the driver labeled as LCL, LCR and LK by using 581 data sets for the non-slippery
road and 550 data sets for the slippery road.
The classification performance of the SVM can be improved by selecting the optimal combination
of the input signals, rather than using all the available input data [36,37]. We tested six combinations
of input signals to the SVM, as listed in Table 2, and compared the classification abilities of the
combinations. In the table, the vehicle states estimated by ANN are marked by boldface.

Table 2. Combinations of Input Signals.

Feature Set Combinations of Input Signals

1 Yaw rate, Longitudinal acceleration, Lateral acceleration, Steering wheel angle, Wheel speed
Yaw rate, Longitudinal acceleration, Lateral acceleration, Steering wheel angle, Wheel speed,
2
Lateral velocity, Roll rate
Yaw rate, Longitudinal acceleration, Lateral acceleration, Steering wheel angle, Wheel speed,
3
sideslip angle, Lateral tire force, Spring compression
Yaw rate, Longitudinal acceleration, Lateral acceleration, Steering wheel angle, Wheel speed,
4
Lateral velocity, Roll rate, sideslip angle, Lateral tire force, Spring compression, Heading
Yaw rate, Lateral acceleration, Steering wheel angle, Lateral velocity, Roll rate, sideslip angle,
5
Lateral tire force, Spring compression, Heading
6 Yaw rate, Lateral acceleration, Steering wheel angle, Lateral velocity, Roll rate, Heading

4. Experimental Results
The performance of the three of the three modules of the preprocessing algorithm of the ADAS
was tested through driving simulation experiments. The three modules were tested with the driving
simulation data that is different from the data used to train the modules. This section presents the
experimental results of the road condition classification, the vehicle state estimation and the driver
intention detection modules.

4.1. Classification of Road Condition

The road condition classification determines the road surface based on the on-board sensor signals
during the time when the throttle is on. Figure 9 shows the throttle position and the road surface
condition estimated while the throttle is on. As shown in the Figure 9, the module correctly classified
the road condition (slippery) with a high level of confidence (100%, 62.7%, and 100%). Table 3 lists
the results from 52 test trials. Out of 52 trials, there were 51 correct classifications with only one
misclassification (highlighted in grey in the Table 3). By using the measurements from the on-board
sensors, the module can identify the road surface condition with fairly high accuracy of 98%.
Sensors 2017, 17, 1350 11 of 18
Sensors 2017, 17, 1350 11 of 18

Sensors 2017, 17, 1350 11 of 18

Figure 9. Classification of the road condition while the throttle is on.

Figure 9. Classification of the road condition while the throttle is on.
Table 3. Result of classification test depending on friction coefficient.
Table 3. Result of classification test depending on friction coefficient.
1 2 3 4 5 6 7 8 9 10 11 12 13 Rate
Dry
NS
Asphalt 1 NS 2 Figure
NS
3 [Link] NS
5 ofNS
Classification
4 6 roadNS
the NS whileNS
7condition
8 10NS is on.
9the throttle 11NS 12 NS 13 NS Rate13/13
87.9% 69.6% 79.8% 75.9% 96.0% 83.2% 57.9% 77.5% 94.2% 77.1% 63.9% 96.7% 96.5%
(NS)
Dry
Gravel NS NS NS NS Table NSNS 3. Result
NSNS ofNS NS NS testNS
NS
classification NS NSNS onNS
depending NS
frictionNS NS NSNS NSNS NS NS
coefficient. 13/13
(NS)Asphalt 92.5% 70.5%
87.9% 69.6%93.0%
79.8%71.0%
75.9% 67.0%
96.0% 53.3%
83.2% 53.6% 75.7% 94.2%
57.9% 77.5% 88.4% 77.1%
75.9% 63.9%
79.9% 96.7%
94.7% 96.5%100%13/13
(NS) S 1 S2 S3 S4 S5 6
S 7S 8S 9S 10 S 11 S 12 S 13 S Rate
Wet Dry
(S) 13/13
Gravel58.1%
Asphalt
NSNS 93.4%
NSNS 91.5%NS 53.6%
NS NSNS 82.3%NS 100%
NS NS
NS 87.8%
NS
NS 100%
NS
NS 92.9%
NS
NS 95.6%
NS
NS 96.2% NS
NS
NS NS 95.7% NS
NS65.4% 13/13
Snowy S
87.9%
(NS) 100% 92.5%
S
69.6% S
79.8%
70.5%100%
S
75.9%
93.0%76.0%
S
96.0% S
67.0% 83.2%
71.0% 80.8%
S
53.3% 57.9%
53.6% 77.5%
NS
75.7% 94.2%S
88.4% 77.1%
75.9%
S 63.9%
79.9%
S
94.7% 96.5%S 13/13
96.7%S 100% 12/13
(S)(NS) 62.7% 100% 72.8% 56.1% 73.4% 77.0% 95.0% 100% 100%
Gravel NS NS NS NS NS NS NS NS NS NS NS NS NS
Wet S S S S S S S S S S S S S 13/13
(NS) 92.5% 70.5% 93.0% 71.0% 67.0% 53.3% 53.6% 75.7% 88.4% 75.9% 79.9% 94.7% 100% 13/13
4.2.(S)Estimation
S58.1% of93.4% 91.5%
S VehicleS State 53.6% 82.3%
Parameters
S S 100%
S 87.8%
S 100%
S 92.9%
S 95.6%
S 96.2%
S 95.7%
S 65.4%
S
Wet (S) 13/13
Snowy 58.1%S 93.4% S 91.5% S 53.6%
S 82.3%
S 100%
S 87.8%
S 100%
NS 92.9%
S 95.6%
S 96.2%
S 95.7%
S 65.4%
S
Snowy The Svehicle S statesSestimated
S by Sthe trained
S vehicle
S state
NS estimation
S S moduleS are Scompared S 12/13
with
(S) 100%
(S)
100% 62.7%
62.7% 100%
100% 76.0%
76.0% 80.8%
80.8% 100%
100% 72.8%
72.8% 56.1%
56.1% 73.4%
73.4% 77.0%
77.0%
95.0%
95.0%
100%
100%
100%
100%
12/13
those from the on-board sensors of the driving simulator under four different road surface
conditions in Figures 10–15. As shown in Figures 10–15, the estimated lateral velocity, side slip
4.2. Estimation
4.2. Estimation of Vehicle
of Vehicle State
State Parameters
Parameters
angle, tire lateral force, roll rate, suspension spring compression and heading direction represented
The The lines
by dotted
vehicle vehicle
arestates
states closeestimated
to thoseby
estimated by
from the
the thetrained vehicle
on-board
trained state
sensors
vehicle stateestimation
of module
the driving
estimation are compared
simulator
module withby with
represented
are compared
those from the on-board sensors of the driving simulator under four different road surface
thosesolid
fromlines.
the on-board sensors of the driving simulator under four different road surface conditions
conditions in Figures 10–15. As shown in Figures 10–15, the estimated lateral velocity, side slip
in Figures 10–15. As shown
angle, tire lateral in Figures
force, roll 10–15, the
rate, suspension estimated
spring lateral
compression andvelocity, side sliprepresented
heading direction angle, tire lateral
force, roll rate, suspension
by dotted lines are closespring compression
to those from the on-boardand heading
sensors ofdirection represented
the driving by dottedbylines are
simulator represented
close tosolid
those from the on-board sensors of the driving simulator represented by solid lines.
lines.

Figure 10. Estimated Lateral Velocity depending on road surface condition: (a) Dry asphalt; (b)
Gravel; (c) Wet; and (d) Snow.

Figure Figure 10. Estimated

10. Estimated Lateral
Lateral Velocity
Velocity depending
depending on road
on road surface
surface condition:(a)
condition: (a)Dry
Dryasphalt;
asphalt; (b)
(b) Gravel;
Gravel; (c) Wet; and (d) Snow.
(c) Wet; and (d) Snow.
Sensors 2017, 17, 1350 12 of 18
Sensors 2017, 17, 1350 12 of 18
Sensors 2017, 17, 1350 12 of 18
Sensors 2017, 17, 1350 12 of 18

Figure Figure 11. Estimated

11. Estimated Sideangle
Side slip slip depending
angle depending on surface
on road road surface condition:
condition: (a) asphalt;
(a) Dry Dry asphalt;
(b) Gravel;
Figure 11. (c)
(b) Gravel; Estimated
Wet; andSide slip angle depending on road surface condition: (a) Dry asphalt;
(d) Snowy.
(c) Wet;Figure
and (d)[Link].
Estimated Side slip angle depending on road surface condition: (a) Dry asphalt;
(b) Gravel; (c) Wet; and (d) Snowy.
(b) Gravel; (c) Wet; and (d) Snowy.

Figure 12. Estimated Lateral Tire Force depending on road surface condition: (a) Dry asphalt; (b)
Figure Figure
Gravel; 12. Estimated
(c) Wet;
12. Estimated and Lateral
(d)
[Link]
Tire Forcedepending
Force depending on
onroad
roadsurface condition:
surface (a) Dry
condition: (a)asphalt; (b)
Dry asphalt; (b)
Figure 12.
Gravel; Estimated
(c) Wet; Lateral
and (d) [Link] Force depending on road surface condition: (a) Dry asphalt; (b)
Gravel;Gravel;
(c) Wet;
(c)and
Wet;(d)
andSnowy.
(d) Snowy.

Figure 13. Estimated Roll rate depending on road surface condition: (a) Dry asphalt; (b) Gravel; (c)
Wet; and (d) Snowy.
 Sensors 2017, 17, 1350 13 of 18
Sensors 2017, 17, 1350 13 of 18
Sensors 2017, 17, 135013. Estimated Roll rate depending on road surface condition: (a) Dry asphalt; (b) Gravel; (c)
Figure 13 of 18
Figure 13.(d)
Wet; and Estimated
Snowy. Roll rate depending on road surface condition: (a) Dry asphalt; (b) Gravel; (c)
Wet; and (d) Snowy.

Figure 14. Estimated

14. Estimated
Figure Figure Suspension
Suspension SpringCompression
Spring Compression depending
dependingon on
road surface
road condition:
surface (a) Dry (a) Dry
condition:
asphalt;14.
(b)Estimated
Gravel; (c)Suspension Spring
Wet; and (d) Compression depending on road surface condition: (a) Dry
Snowy.
asphalt;asphalt;
(b) Gravel; (c) Wet;
(b) Gravel; andand
(c) Wet; (d)(d)
Snowy.
Snowy.

Figure 15. Estimated Heading (Yaw) depending on road surface condition: (a) Dry asphalt; (b)
Figure Figure 15. Estimated
15. Estimated
Gravel; (c) Wet; (d)Heading
Heading
and (Yaw)
(Yaw)
Snowy. depending
depending on road
on road surface
surface condition:(a)
condition: (a)Dry
Dryasphalt;
asphalt; (b)
(b) Gravel;
Gravel; (c) Wet; and (d) Snowy.
(c) Wet; and (d) Snowy.
The errors between the estimated vehicle states and the measured vehicle states were
The errors
evaluated by the between
root mean thesquare
estimated vehicle and
error (RMSE) statestheand the measured
normalized vehicleerror
mean square states were
(NMSE)
Theevaluated
errors by the
between
given as follows [38]:root mean
the square
estimated error
vehicle (RMSE)
states andand the
thenormalized
measured mean square
vehicle error
states (NMSE)
were evaluated
by the given
root as follows
mean [38]: error (RMSE) and the normalized mean square error (NMSE) given as
square RMSE = ∑ ( − ) (5)
follows [38]: RMSE = ∑ ( − ) (5)
2
∑u=1( − )2
v
NMSE = ∑ u =1( 1 − )2
N (6)
=1( − )(2yi − ŷi )
NMSE ==∑t
RMSE ∑ =1(N − )∑ 2 (6) (5)
Table 4 lists RMSE and NMSE under four different
i =1 road surface conditions. The errors were
Table from
evaluated 4 liststhe
RMSE anddata
driving NMSE underfor
collected four
thedifferent
duration road
of 100surface
s. The conditions.
results showThe thaterrors were
the orders
evaluated from the driving data collected for the duration
N of 1002 s. The results show that the orders
of errors for NMSE range from 10 to 10 for the
−3 −1
∑i =
six yi − ŷi states
1 (vehicle ) in the four road conditions. The
of errorsstate
vehicle for NMSE range module
estimation
NMSE
from 10−3represents
to 10−1 for=the
the N
six vehicle vehicle
nonlinear states
2 indynamics
the four roadwithconditions.
high levelThe of
(6)
vehicle state estimation module represents the∑inonlinear
=1 ( yi − y )vehicle dynamics with high level of
accuracy.
accuracy.
Table 4 lists RMSE and NMSE under four different road surface conditions. The errors were
evaluated from the driving data collected for the duration of 100 s. The results show that the orders
of errors for NMSE range from 10−3 to 10−1 for the six vehicle states in the four road conditions.
The vehicle state estimation module represents the nonlinear vehicle dynamics with high level
of accuracy.
Sensors 2017, 17, 1350 14 of 18

Table 4. Range of data, RMSE and NMSE in each case.

Road Condition Data RMSE NMSE Order (NMSE)

Dry asphalt −0.6~0.6 0.0148 0.0139
Gravel −0.6~0.6 0.0229 0.0195
Lateral Velocity
Wet −0.6~0.6 0.0218 0.0082 10−2
Snowy −0.6~0.6 0.0236 0.0077
Dry asphalt −0.6~0.6 0.0133 0.0123
Gravel −0.6~0.6 0.0233 0.0167
Side Slip Angle
Wet −0.9~0.9 0.0374 0.0142 10−2
Snowy −0.9~0.9 0.0403 0.0097
Dry asphalt −3000~2000 34.6 0.00089
Gravel −3000~2000 84.6 0.0066
Lateral Tire Force
Wet −2500~2000 38.2 0.0021 10−3
Snowy −2000~2000 27.5 0.0021
Dry asphalt −8~8 0.332 0.0304
Gravel −6~6 0.363 0.0475
Roll rate
Wet −5~5 0.353 0.0774 10−1
Snowy −4~4 0.304 0.1096
Dry asphalt 50~85 0.708 0.0145
Gravel 50~85 0.962 0.0303
Spring Compression
Wet 60~80 0.594 0.0192 10−2
Snowy 60~80 0.698 0.0479
Dry asphalt −15~15 1.05 0.0226
Gravel −15~15 0.962 0.0154
Heading
Wet −15~15 0.551 0.008 10−2
Snowy −15~15 0.545 0.0166

4.3. Detection of Driver Intention

The driver’s intention estimated by the trained SVM was compared with the driver’s true intention
for lane change. The SVM was trained with six feature sets with six different combinations of the
input signals (Table 2). Table 5 compares the accuracy rates with the six feature sets under four road
surface conditions.

Table 5. Detection accuracy in four different road conditions.

Set 1 (%) Set 2 (%) Set 3 (%) Set 4 (%) Set 5 (%) Set 6 (%)
(a) Dry Asphalt
LCL 70.51 71.79 65.38 88.46 91.03 91.03
LK 96.30 95.06 95.68 96.91 96.91 96.91
LCR 67.14 74.29 75.71 91.43 90.00 91.43
(b) Gravel
LCL 66.15 72.31 56.92 90.77 92.30 92.30
LK 95.57 96.20 95.57 96.20 96.84 96.84
LCR 56.96 68.35 64.56 89.87 89.87 91.14
(c) Wet
LCL 54.29 67.14 60.00 92.86 92.86 92.86
LK 97.14 97.71 97.71 97.71 97.71 97.14
LCR 60.66 73.77 70.49 90.16 90.16 90.16
(d) Snowy
LCL 62.26 62.26 52.83 90.57 90.57 90.57
LK 97.84 97.84 97.84 97.30 97.30 97.30
LCR 71.43 73.21 75.00 89.29 89.29 91.07
 Sensors 2017, 17, 1350 15 of 18

The results show that the feature set with the yaw rate, the longitudinal acceleration, the
steering wheel angle, the lateral velocity, the roll rate and the heading direction (Set 6) shows the
highest accuracy rate of detection in any road surface condition. It should be noted that the
accuracy
Sensors 2017, 17,of detection is better than that of the feature set with all the available input signals (Set 4)15 of 18
1350
and that of the feature set with on-board sensor measurements only (Set 1). Sets 4–6 with the
heading direction show much higher accuracy rates than Sets 1–3 without the heading direction.
The results
Thus, show that
the heading the feature
direction set with
is an input theofyaw
signal rate,importance
major the longitudinal acceleration,
for identifying the steering
the driver’s
wheel angle, the lateral
intention for lane change. velocity, the roll rate and the heading direction (Set 6) shows the highest
accuracyFigurerate of 16detection
shows typical laneroad
in any change maneuvers
surface on dry
condition. It asphalt
should and the driver’s
be noted that theintentions
accuracy of
classified
detection by thethan
is better driver
thatintention detection
of the feature setmodule
with allusing the optimal
the available feature
input set (Set
signals (Set6).4)Figure 16a of the
and that
plots
feature setthe steering
with wheelsensor
on-board angle during lane change
measurements maneuvers.
only Figure
(Set 1). Sets 4–616b compares
with the driver’s
the heading true show
direction
intention (solid line) and the detected intention by the module (marked by o). In Figure 16, LCR, LK,
much higher accuracy rates than Sets 1–3 without the heading direction. Thus, the heading direction is
and LCL are labeled as −1, 0, and 1, respectively.
an input signal of major importance for identifying the driver’s intention for lane change.
As can be seen in Figure 16, the module correctly detected the driver’s intention for lane
Figure 16 shows typical lane change maneuvers on dry asphalt and the driver’s intentions
change, while there are time delays before correct detections. The delay is mainly attributed to the
classified
updateby theofdriver
rate intention
detection, which isdetection
dependent module using the
on the window optimal
slide feature
size (0.2 s). Theset (Set 6).
module Figure 16a
requires
plotsa the
time longer than 0.2 s to determine the driver’s intention from sufficient information on the true
steering wheel angle during lane change maneuvers. Figure 16b compares the driver’s
intention
pattern (solid
of laneline) and maneuver.
change the detected intentionitby
In addition, the module
appears that this(marked
time delayby mainly
o). In Figure 16, LCR,
contributes to LK,
and the
LCL errors listed in as
are labeled −1,5.0, and 1, respectively.
Table

Figure Lane
16. 16.
Figure change
Lane changemaneuvers
maneuversand
and driver’s intention:
driver’s intention: (a)(a) steering
steering wheel
wheel angle;
angle; anddriving
and (b) (b) driving
statestate
(−1(−1 is LCR,
is LCR, 0 isLK,
0 is LK,11isisLCL).
LCL).

Table 6 lists the average time to correctly detect the driver’s intention from the onset of the
As can lane
driver’s be seen in Figure
change 16, the
maneuver module
under correctly
four different detected
road surfacethe driver’sThe
conditions. intention
resultsfor lane
show change,
that
while
thethere are time
average time delays before
delays for LCLcorrect detections.
and LCR range fromThe0.4delay is mainly
to 0.45 s, whileattributed toisthe
that for LK update rate
between
of detection, which is dependent on the window slide size (0.2 s). The module requires a time longer
than 0.2 s to determine the driver’s intention from sufficient information on the pattern of lane change
maneuver. In addition, it appears that this time delay mainly contributes to the errors listed in Table 5.
Table 6 lists the average time to correctly detect the driver’s intention from the onset of the driver’s
lane change maneuver under four different road surface conditions. The results show that the average
time delays for LCL and LCR range from 0.4 to 0.45 s, while that for LK is between 0.146 and 0.222 s.
The experimental results demonstrate that the trained driver intention detection module can identify
the driver’s intention accurately and quickly.
Sensors 2017, 17, 1350 16 of 18

Table 6. Average time delay for correct detection.

Driver Maneuver Dry Asphalt Gravel Wet Snowy

LCL 0.45 s 0.4 s 0.4 s 0.4 s
LK 0.15 s 0.222 s 0.182 s 0.146 s
LCR 0.45 s 0.433 s 0.433 s 0.4 s

5. Conclusions
In this study, we propose a novel method to classify the driver’s intention for lane change, based
on measured and estimated information on the driver’s control inputs, the vehicle states, and the road
condition. By using machine learning-based estimation techniques, the road surface condition and the
extra vehicle states are augmented from the measured data obtained from the basic on-board sensors,
which are commonly equipped on recent passenger vehicles.
For the classification of the driver’s intention, an SVM-based model is employed. As the inputs to
the SVM model, the road surface condition and the extra vehicle states are estimated by ANN-based
models that are trained a priori using the dynamics simulation data from a driving simulator.
The augmented information estimated by the ANN models (the friction coefficient of the road surface,
the lateral velocity, the side slip angle, the lateral tire force, the roll rate, the spring compression, and
the heading direction) is essential to capture the dynamic situations of the driving vehicle.
The effectiveness of the proposed method was tested using driving simulation data. The results
demonstrate that the driver’s intention for lane change can be detected more accurately using both
measured and estimated data than using only measured data. The simulation results also show that the
classification accuracy is the highest with the yaw rate, the lateral acceleration, the steering wheel angle,
the lateral velocity, the roll rate and the heading direction as the inputs to the SVM module, rather
than with all the available information from measurement and estimation as the inputs. Among the
estimated vehicle states, the heading direction, the lateral velocity, and the roll rate appear to play an
important role to improve the classification accuracy of the SVM model. The classification accuracy
with the augmented information was higher than 90% in any road surface condition.
The proposed method can be utilized as a preprocessing algorithm for the ADAS by accurately
and effectively detecting the driver’s intention for lane change. The developed algorithm can allow
us to replace expensive sensors with economical on-board sensor algorithms. Due to its superior
computational efficiency to numerical integration, the ANN-based vehicle dynamics model can be
more effective than complex differential equation-based approaches. The proposed method can also
be applicable to analysis of the driver's driving pattern. Based on the analysis, the ADAS may be
adaptively activated according to the driver’s driving pattern.
For our future works, we are planning to implement the developed algorithm on actual vehicle
systems with multiple human subjects with different characteristics. In addition, we will investigate
other advanced machine learning algorithms, such as long short-term memory (LSTM), for different
types of driver intentions. Another important issue with driving safety is human factors. It was
reported that about 90% of vehicle accidents were caused by human errors [39]. We also plan to study
algorithms to differentiate the driver’s true intention from the driver’s erroneous maneuver.

Acknowledgments: This work was supported by the Agency for Defense Development (ADD) under the contract
UD140073ID. The work of Il-hwan Kim was supported by Hyundai Motor Company.
Author Contributions: Il-Hwan Kim, Jooyoung Park, and Shinsuk Park conceived and designed the system.
Il-Hwan Kim and Jae-Hwan Bong performed the experiments and analyzed the experimental data. Finally,
Il-Hwan Kim wrote the paper with the help of Jooyoung Park and Shinsuk Park.
Conflicts of Interest: The authors declare no conflict of interest.
Sensors 2017, 17, 1350 17 of 18

References
1. Hou, H.; Jin, L.; Niu, Q.; Sun, Y.; Lu, M. Driver intention recognition method using continuous hidden
markov model. Int. J. Comput. Intell. Syst. 2011, 4, 386–393. [CrossRef]
2. Tomar, R.S.; Verma, S.; Tomar, G.S. Prediction of lane change trajectories through neural network.
In Proceedings of the 2010 International Conference on Computational Intelligence and Communication
Networks, Bhopal, India, 26–28 November 2010; pp. 249–253.
3. FARS Encyclopedia. Vehicles Involved in Single- and Two-Vehicle Fatal Crashes by Vehicle Maneuver.
National Highway Traffic Safety Administration. Available online: [Link]
Vehicles/[Link] (accessed on 15 September 2016).
4. Kuge, N.; Yamamura, T.; Shimoyama, O.; Liu, A. A Driver Behavior Recognition Method Based on a Driver Model
Framework; Delphi Automotive Systems: Gillingham, UK, March 2000.
5. Jin, L.S.; Hou, H.J.; Jiang, Y.Y. Driver intention recognition based on continuous hidden markov model.
In Proceedings of the International Conference on Transportation, Mechanical, and Electrical Engineering
(TMEE), Changchun, China, 16–18 December 2011.
6. Tran, D.; Sheng, W.; Liu, L.; Liu, M. A Hidden Markov Model based driver intention prediction system.
In Proceedings of the 2015 IEEE International Conference on Cyber Technology in Automation, Control, and
Intelligent Systems (CYBER), Shenyang, China, 8–12 June 2015; pp. 115–120.
7. Mandalia, H.M.; Salvucci, M.D.D. Using support vector machines for lane-change detection. In Proceedings
of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, USA, 26–30 October 2015;
Volume 49, pp. 1965–1969.
8. Aoude, G.S.; Desaraju, V.R.; Stephens, L.H.; How, J.P. Behavior classification algorithms at intersections
and validation using naturalistic data. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV),
Baden-Baden, Germany, 5–9 June 2011; pp. 601–606.
9. Kumar, P.; Perrollaz, M.; Lefevre, S.; Laugier, C. Learning-based approach for online lane change intention
prediction. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Gold Coast City, Australia,
23–26 June 2013; pp. 797–802.
10. Morris, B.; Doshi, A.; Trivedi, M. Lane change intent prediction for driver assistance: On-road design
and evaluation. In Proceedings of the IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany,
5–9 June 2011; pp. 895–901.
11. Liu, L.; Xu, G.Q.; Song, Z. Driver lane changing behavior analysis based on parallel bayesian networks.
In Proceedings of the 2010 Sixth International Conference on Natural Computation, Yantai, China,
10–12 August 2010; Volume 3, pp. 1232–1237.
12. Schubert, R.; Schulze, K.; Wanielik, G. Situation assessment for automatic lane-change maneuvers. IEEE Trans.
Intell. Trans. Syst. 2010, 11, 607–616. [CrossRef]
13. Lefèvre, S.; Laugier, C.; Ibañez-Guzmán, J. Exploiting map information for driver intention estimation at
road intersections. In Proceedings of the Intelligent Vehicles Symposium (IV), Baden-Baden, Germany,
5–9 June 2011; pp. 583–588.
14. Kumagai, T.; Akamatsu, M. Prediction of human driving behavior using dynamic bayesian networks.
IEICE Trans. Inf. Syst. 2006, 89, 857–860. [CrossRef]
15. McCall, J.C.; Trivedi, M.M.; Wipf, D.; Rao, B. Lane change intent analysis using robust operators and sparse
bayesian learning. IEEE Trans. Intell. Transp. Syst. 2007, 8, 431–440. [CrossRef]
16. Gindele, T.; Brechtel, S.; Dillmann, R. A probabilistic model for estimating driver behaviors and vehicle
trajectories in traffic environments. In Proceedings of the 13th International IEEE Conference on Intelligent
Transportation Systems (ITSC), Funchal, Portugal, 19–22 September 2013; pp. 1066–1071.
17. Lefèvre, S.; Laugier, C.; Ibañez-Guzmán, J. Risk assessment at road intersections: Comparing intention
and expectation. In Proceedings of the 2012 IEEE Intelligent Vehicles Symposium (IV), Madrid, Spain,
3–7 June 2012; pp. 165–171.
18. Turnip, A.; Fakhrurroja, H. Estimation of the wheel-ground contact tire forces using extended Kalman filter.
Int. J. Instrum. Sci. 2013, 2, 34–40.
19. Castillo Aguilar, J.J.; Cabrera Carrillo, J.A.; Guerra Fernández, A.J.; Carabias Acosta, E. Robust road condition
detection system using in-vehicle standard sensors. Sensors 2015, 15, 32056–32078. [CrossRef] [PubMed]
Sensors 2017, 17, 1350 18 of 18

20. Zareian, A.; Azadi, S.; Kazemi, R. Estimation of road frictio coefficient using extended Kalman filter, recursive
least square, and neural network. Proc. Inst. Mech. Eng. Part K 2016, 230, 52–68.
21. Mikelsons, L.; Brandt, T.; Schramm, D. Real-time vehicle dynamics using equation-based reduction
techniques. In IUTAM Symposium on Dynamics Modeling and Interaction Control in Virtual and Real Environments;
Springer: Dordrecht, The Netherlands, 2011; pp. 91–98.
22. Doumiati, M.; Victorino, A.; Charara, A.; Lechner, D. A method to estimate the lateral tire force and the
sideslip angle of a vehicle: Experimental validation. In Proceedings of the 2010 American Control Conference,
Baltimore, MD, USA, 30 June–2 July 2010; pp. 6936–6942.
23. Guarneri, P.; Rocca, G.; Gobbi, M. A neural-network-based model for the dynamic simulation of the
tire/suspension system while traversing road irregularities. IEEE Trans. Neural Netw. 2008, 19, 1549–1563.
[CrossRef] [PubMed]
24. Elvik, R.; Vaa, T.; Erke, A.; Sorensen, M. The Handbook of Road Safety Measures; Emerald Group Publishing:
Bingley, UK, 2009; pp. 363–368.
25. Engineering Dynamics Corporation. EDVAP Program Manual; Engineering Dynamics Corporation:
Beaverton, OR, USA, 1994.
26. Panchal, G.; Ganatra, A.; Kosta, Y.P.; Panchal, D. Behaviour analysis of multilayer perceptrons with multiple
hidden neurons and hidden layers. Int. J. Comput. Theory Eng. 2011, 3, 332–337. [CrossRef]
27. Diaconescu, E. The use of NARX neural networks to predict chaotic time series. WSEAS Trans. Comput. Res.
2008, 3, 182–191.
28. Hagan, M.T.; Demuth, H.B.; Beale, M.H.; De Jesús, O. Neural Network Design; PWS Publishing Company:
Boston, MA, USA, 1996; Volume 20.
29. Kargar, M.J.; Charsoghi, D.K. Predicting annual electricity consumption in Iran using artificial neural
networks (NARX). Indian J. Sci. Res. 2014, 5, 231–242.
30. Lin, T.N.; Giles, C.L.; Horne, B.G.; Kung, S.Y. A delay damage model selection algorithm for NARX neural
networks. IEEE Trans. Signal Process. 1997, 45, 2719–2730.
31. Jiang, C.; Song, F. Sunspot forecasting by using chaotic time-series analysis and NARX network. J. Comput.
2011, 6, 1424–1429. [CrossRef]
32. Choudhary, I.; Assaleh, K.; AlHamaydeh, M. Nonlinear AutoRegressive eXogenous Artificial Neural
Networks for predicting buckling restrained braces force. In Proceedings of the 2012 8th International
Symposium on Mechatronics and Its Applications (ISMA), Sharjah, UAE, 10–12 April 2012; pp. 1–5.
33. Xie, H.; Tang, H.; Liao, Y.H. Time series prediction based on NARX neural networks: An advanced approach.
In Proceedings of the 2009 International Conference on Machine Learning and Cybernetics, Baoding, China,
12–15 July 2009; pp. 1275–1279.
34. Abdulkadir, S.J.; Yong, S.P. Empirical analysis of parallel-NARX recurrent network for long-term chaotic
financial forecasting. In Proceedings of the 2014 International Conference on Computer and Information
Sciences (ICCOINS), Kuala Lumpur, Malaysia, 3–5 June 2014; pp. 1–6.
35. Menezes, J.M.P.; Barreto, G.A. Long-term time series prediction with the NARX network: An empirical
evaluation. Neurocomputing 2008, 71, 3335–3343. [CrossRef]
36. Myung, I.J.; Pitt, M.A. Applying Occam’s razor in modeling cognition: A Bayesian approach. Psychonol. Bull.
Rev. 1997, 4, 79–95. [CrossRef]
37. Berndt, H.; Emmert, J.; Dietmayer, K. Continuous driver intention recognition with hidden markov models.
In Proceedings of the 2008 11th International IEEE Conference on Intelligent Transportation Systems, Beijing,
China, 12–15 October 2008; pp. 1189–1194.
38. Ardalani-Farsa, M.; Zolfaghari, S. Chaotic time series prediction with residual analysis method using hybrid
Elman–NARX neural networks. Neurocomputing 2010, 73, 2540–2553. [CrossRef]
39. AlertDriving Magazine. Human Error Accounts for 90% of Road Accidents. Available
online: [Link]
90-road-accidents (accessed on 30 September 2016).

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license ([Link]