行政院國家科學委員會專題研究計畫 成果報告

網路學習之形成性評量與學習診斷工具發展與研究(第 3
年)
研究成果報告(完整版)

計 畫 類 別 ： 個別型
計 畫 編 號 ： NSC 97-2628-S-004-001-MY3
執 行 期 間 ： 99 年 08 月 01 日至 100 年 07 月 31 日
執 行 單 位 ： 國立政治大學圖書資訊與檔案學研究所

計 畫 主 持 人 ： 陳志銘

計畫參與人員： 碩士班研究生-兼任助理人員：陳勇汀
碩士班研究生-兼任助理人員：黃浚維
碩士班研究生-兼任助理人員：謝綸
碩士班研究生-兼任助理人員：吳志豪
碩士班研究生-兼任助理人員：范蔚敏
碩士班研究生-兼任助理人員：林上資

報 告 附 件 ： 出席國際會議研究心得報告及發表論文

處 理 方 式 ： 本計畫涉及專利或其他智慧財產權，1 年後可公開查詢

中 華 民 國 100 年 10 月 17 日
行政院國家科學委員會專題研究計畫成果報告
網路學習之形成性評量與學習診斷工具發展與研究
計畫主持人：陳志銘 教授
計畫執行單位：國立政治大學圖書資訊與檔案學研究所
計畫編號：97-2628-S-004-001-MY3
計畫執行期間：97 年 8 月 1 日~100 年 7 月 31 日

摘要
網路學習近年來蓬勃發展，相對於網路學習的不斷創新與突破，有效的學習評量與回饋
機制卻一直是網路學習系統較弱的一環。傳統的總結性評量旨在評估學習者最終的學習成
效，卻無法衡量學習者的學習歷程；透過形成性評量，教學者可以在學習者的學習過程中，
觀察學生學習歷程中的學習行為，並給予回饋，或針對學生的盲點進行補救。資訊技術的進
步，讓教師可透過學習系統，在不干擾學習者的情形下記錄學習者的學習歷程檔案，進而利
用資料探勘技術，將學習者的學習歷程轉換成具有教育意義的資訊，協助教師進行評量，更
能即時讓教師對學習者的行為予以督促及回應，增進學習者學習成效。本研究提出一個形成
性評量機制，整合包括因素分析、模糊聚類、灰關聯分析、模糊關聯規則分析和模糊推論等
資料探勘方法，對學習歷程檔案進行分析，探勘關鍵的形成性評量規則，應用於網路學習環
境中，協助教師與學生進行學習成效回饋互動。本研究所提出之學習規則探勘過程可以基於
學習歷程檔案找出主要影響學習成效的關鍵性學習因素，來預估學生的學習成效，提供學習
者即時回饋；而對學習者而言，可藉由評量系統的即時回饋，調整自己的學習狀況，達最佳
學習狀態。實驗結果顯示，本研究所發展之基於學習歷程檔案之形成性評量非常接近總結性
評量結果，此外，具有形成性評量支援之網路學習系統更能提升學習者的學習成效及學習動
機。
關鍵字：形成性評量、網路學習、學習歷程檔案、學習因素分析、資料探勘

Abstract
Current trends clearly indicate that online learning is gaining in importance, but there is still a
lack of an effective learning performance assessment mechanism for e-learning systems. Learning
performance assessment aims to evaluate what learners learnt during the learning process.
Traditional summative evaluation only considers final learning outcomes without concerning
learning processes of learners. With the evolution of learning technology, the use of learning
portfolios in a web-based learning environment can be beneficially applied to record the procedure
of the learning, evaluate the learning performances of learners, and produce feedback information to
learners in ways that enable them to learn better. Accordingly, this study proposed a mobile
formative assessment tool using data mining techniques, which involves six computational
intelligence theories, i.e., statistic correlation analysis, fuzzy clustering analysis, grey relational
analysis, K-means clustering scheme, fuzzy association rule mining and fuzzy inference, in order to
identify the key formative assessment rules based on the web-based learning portfolios of an
individual learner for the performance promotion of web-based learning. In other words, the

1
proposed scheme can help teachers to precisely assess the learning performance of individual
learner utilizing only the learning portfolios in a web-based learning environment. Therefore,
teachers can devote themselves to teaching and designing courseware since they save a lot of time
in evaluating learning performance. More significantly, teachers could understand the main factors
influencing learning performance in a web-based learning environment according to the obtained
interpretable learning performance assessment rules. Experimental results indicated that the
evaluation results of the proposed scheme are very close to those of summative assessment results.
Moreover, the factor analysis provides the benefit in terms of obtaining simpler and clear learning
performance assessment rules. Furthermore, the proposed learning feedback with formative
assessment could obviously promote the learning performances and interests of learners.
Keywords: Formative Assessment, Web-based Learning, Web-based Learning Portfolio, Learning
Factor Analysis, Data Mining
1. Introduction

In the past years, more and more assisted learning tools on e-learning systems were developed
due to the network and the computer popularization. Learning performance assessment approaches
are essential in the web-based learning field, owing to the rapid growth of e-learning systems
globally and lack of assisted learning performance assessment tools for assessing web-based
learning process. Gagnés’ research on the internal process of learning has indicated that the
complete learning process should assess learning performance (Gagn é s, 1997). Learning
performance evaluation instruments can generally break down into two broad categories:
summative and formative (Torrance & Pryor, 1998; Margaret, 2003). Summative evaluation is
generally carried out towards the end of a course (Nuhfer, 1996). It stands in contrast to formative
evaluation, which is provided while the course is ongoing so as to permit improvements (Scriven,
1967; Tessmer, 1993). The purpose of summative assessments is to make a judgment of learner
competency after an instructional phase is complete. Conversely, the use of formative assessment
helps teachers to obtain feedback about how well learners are learning and particular difficulties
they might be having. In other words, the formative assessment can help teachers to gather
information and to use the information as feedback to modify their teaching strategies and to
improve learners’ learning outcomes.
A web-based learning portfolio can be collected, stored and managed automatically by
computers when learners interact with an e-learning platform. Consequently, the learning portfolios
not only provide true and rich information for reflecting and assessing the true performances and
achievements of learners, but it also help learners to engage in meaningful learning. Therefore,
learning performance assessment using a web-based learning portfolio has received significant
attention recently (Lankes, 1995; Rahkila & Karjalainen, 1999). Lankes (1995) stated that
implementing computer-based learner assessment portfolios is an innovatively educational
innovation owing to its ability not only offers an authentic demonstration of accomplishments, but
also enables learners to take responsibility for their completed tasks. Several studies emphasized
that the learning portfolio assessment is supported by the cognitive–constructive theory of learning
(Rahkila & Karjalainen, 1999; Bruner, 1996). Wang et al. (2003) found that learning behavior
information, commonly referred to as a learning portfolio, can help teachers to understand why a
learner obtained a high or low grade.
However, developing a precise learning performance assessment scheme using web-based
learning portfolio is a challenging task for web-based learning systems. Data mining had been
considered as an appropriate method of knowledge discovery to excavate the implicit information
(Margaret, 2003). Thus, this study presents a data mining approach that integrates six computational
intelligence schemes, i.e., the statistic correlation analysis (Johnson & Wichern, 1988), fuzzy
clustering algorithm (Gath & Geva 1989), the grey relational analysis (Deng, 1989; Chen, Chang, &

2
Liao, 2000), K-means clustering scheme (Krishna & Murty, 1999), fuzzy association rule (Delgado
et al., 2003; Hong, Kuo & Chi, 1999) and fuzzy inference (Lin & Lee, 1996), to evaluate on-line
learning behavior and learning performance. The six computational intelligence schemes were
employed to perform independence and importance analysis of the considered learning factors for
extracting the key factors that contribute to learning performance and to discover the useful fuzzy
association rules relating to the learning performance assessment.
Based the proposed assessment method, the results can help teachers to perform precise
formative assessments according to the learning portfolios of individual learners gathered from a
web-based learning system and mobile formative assessment tool implemented on PDA for teachers.
The inferred learning performance can be applied as a reference guide for teachers and as learning
feedback for learners. Through the feedback mechanism, every learner can understand his/her
current learning status and make suitable learning adjustments. Hence, every learner is able to play
a more active role in his/her learning (Black, 2001). Additionally, teachers can observe the main
factors influencing learning performance in a web-based learning environment from mobile
formative assessment tool according to the interpretable learning performance assessment rules.
Moreover, teachers can adjust their teaching strategies based on these main learning factors
influencing the learning performance. In the meanwhile, since teachers save much time in
evaluating learning, they can devote more time to teaching and designing courseware. Experimental
results indicated that the evaluation results of the proposed formative assessment scheme are very
close to those of summative assessment results and the proposed factor analysis scheme can
simplify the learning performance assessment rules. Moreover, the experimental results also
revealed that the learning feedback of formative assessment is very helpful to assist web-based
learning, enhance significantly the learners’ learning achievements and promote their learning
interests.

2. System Design
Learning assessment is typically the most appropriate process to evaluate the learning
performance and teaching effects regardless of the traditional classroom or web-based learning
environment. This section aims to present the proposed formative assessment system and scheme.
First, the system architecture is presented in Section 2.1. Next, the considered learning portfolio in
the learning profile database is explained in Section 2.2. Finally, the proposed formative assessment
approach based on learning portfolios is detailed in Section 2.3.

2.1 System Architecture

The personalized e-learning system (PELS) based on Item Response Theory was presented for
adaptive learning services of individual learners in our previous study (Chen, Lee & Chen, 2005;
Chen, Liu & Chang, 2006). Figure 1 shows the learning interface of the personalized e-learning
system. However, the PELS mainly focuses on performing adaptive learning. The learning
performance assessment is lacked feature in this system. In this study, the PELS is extended to
include the learning assessment & feedback module and the teacher formative assessment module
for assisting learning performance assessment and learning feedback using the gathered learning
portfolios of individual learners. The PELS can automatically gather the useful learning portfolios
of individual learners for the learning performance assessment during learning processes. The
proposed personalized e-learning system with formative assessment mechanism is shown as Fig. 2.
The primary functions of the extended formative assessment modules will be described in Sections
2.1.1 and 2.1.2, respectively.

3
 Figure 1. The learning interface on PELS

Figure 2. The system architecture of the proposed personalized e-learning system with
formative assessment mechanism

Based on the system architecture, the details of system operation procedure are described and
summarized as follows:
Step 1 Learners login the system through the learning interface agent by the legal learners’
accounts.
Step 2 After a learner logins the system, the learning interface agent will check whether his account
stored in the user account database.
Step 3 If the learner has already owned a registered account, the system will get his learning profile
from the user profile database for personalized learning services.

4
Step 4 The system guides the learner to perform the personalized courseware learning based on the
course materials stored in the courseware database and the user profile information stored in
the user profile database.
Step 5 The teacher uses PDA to assess the learning states of individual learners during the learning
process, including the attendance statuses, question & answer responses for teacher
questions, concentration degree on learning, and learning comments for individual learners.
All learning records will be stored in the user profile database through wireless network
communication.
Step 6 The learning side formative assessment and feedback agent gets the learning portfolios from
the user profile database and analyze the key formative learning assessment rules.
Step 7 The learning side formative assessment and feedback agent stores those discovered
formative learning performance assessment rules into the learning rule database for inferring
learning performances of individual learners.
Step 8 The learning side formative assessment and feedback agent predicts the learning
performances of individual learners according to the learning portfolios of individual
learners and conveys the evaluating results to the learners in order to assist learning
reflections and adjust learning strategies.
Step 9 The teacher can also get the discovered learning performance assessment rules from the
learning rules database through PDA. Based on the results, the teacher can adjust his\her
teaching strategies. The user then returns to Step 4 for the next learning cycle or logs out,
terminating the learning process.

2.1.1 Teacher formative assessment module

Formative assessment is a new trend of assessment wherever in the classroom or on the web
learning environment. In the study, formative assessment was conducted in a classroom with
computer-assisted learning environment and mainly focused on evaluating the learning situations of
each individual leaner. That is, teachers have to evaluate learning processes of each individual
learner based on several considered formative learning factors mentioned later, such that learners
are able to get better learning performance by reflecting their learning strategies during learning
processes. By contrast, teachers also need to know the learning situations of the whole class, and
what the class needs work on. Learning feedback from formative assessment outcome can aid
learners to improve their learning performances and facilitate teachers to adjust their teaching
strategies. Therefore, the teacher side formative assessment agent aims to assist teachers to record
learners learning processes, such as the attendance statuses, concentration degree, question &
answer response and learning comments to the learners, and provide a friendly interface to aid
teachers to view learners’ learning statuses on the PELS.

2.1.2 Learning assessment and feedback module

Nicol and McFarlane-Dick (2006) indicated that formative assessment with feedback
information provides simultaneously benefit to learners and lecturers. Feedback enables learners to
restructure their understanding/skills and to build more powerful ideas and capabilities. Ramaprasad
and Sadler (Ramaprasad, 1983; Sadler, 1989) also explained that feedback given as part of
formative assessment helps learners to become aware of any gaps that exist between their desired
learning goal and their current knowledge. Learning, assessment and feedback form the learning
cycle improving teaching and learning. In the personalized e-learning system, all learners’
interaction with learning system can be recorded automatically in the web learning portfolios.
Therefore, how to turn the huge amount of learning portfolios data into meaning educational
information and how to give effective feedbacks for teachers and online learners are two concerned
issues in the study. To solve these two problems, the statistical theory and data mining techniques

5
are employed to analyze learners’ online learning behaviors based on the web learning portfolios.
In the learning assessment and feedback module, the learning side formative assessment and
feedback agent is applied to mine the main learning factors influencing learning performance based
on the web learning portfolios and the learning records in the classroom with computer-assisted
learning environment from teacher. Moreover, the agent was employed to produce the results of the
formative learning assessment. Learning feedback is not only used to predict learners’ learning
performance in the learning procedure, but also provides benefits in terms of teachers’ teaching and
learners’ learning. The agent aims to simultaneously provide assessment feedback to the learners
who were performing learning activities on the PELS and the teachers with PDA mobile formative
assessment tool who were performing teaching activities on the PELS.

2.2 Considered Learning Portfolio in the User Profile Database

This section describes the learning portfolio information collected by PELS and PDA for the
proposed learning performance assessment approach. The ten gathered learning factors are
described in detail as follows:

2.2.1 Learning factors gathered by PELS

(1) Reading rate of course materials (RR)
The reading rate of course materials is defined as the rate of studying course materials in a
course unit, and the notation RR is employed to represent the learning factor.
(2) Total accumulated reading time of all learned course materials (RT)
The total accumulated reading time of each learner is calculated by summing up the reading
time of all learned course materials on the PELS system, and the notation RT is used to represent
the learning factor in this study.
(3) Learner ability evaluated by PELS (LA)
After studying the recommended courseware, the PELS (Chen, Lee & Chen, 2005; Chen, Liu
& Chang, 2006) can dynamically estimate a learner’s ability according to the Item Response Theory
(Baker, 1992) by collecting the replied responses of the learner to the randomly selected testing
questions in the learned course unit. The range of learner’s ability and difficulty parameter of
courseware is limited from -3 to +3. The notation LA is used to represent the learning factor of
learner ability in this paper.
(4) Correct response rate of randomly selecting testing questions (CR)
After a learner has studied the recommended course material, the PELS tests the learner on his
understanding by randomly selecting a relevant question from the testing item database. The rate of
correct responses to test questions helps to determine the learner’s degree of understanding for all
learned courseware, and the notation CR is used to represent the learning factor in this paper.
(5) Effort level of studying course materials (EL)
Since each course material in the PELS system is assigned a required minimum reading time
by course experts based on the courseware content, the effort level is defined as the actual reading
time compared with the required minimum reading time for the learned courseware, and the
notation EL is used to represent the learning factor in this study.
(6) Final test grade (GRADE)

6
 This study measures the final test grade through the summative assessment scheme of
fixed-length testing examination after the entire learning process is completed, and the notation
GRADE is used to represent the learning factor in this paper.

2.2.2 Learning factors gathered by PDA

(1) Attendance rate (AR)
The attendance rate is defined as the rate of participating in courseware learning in a course
unit, and the notation AR is employed to represent the learning factor.
(2) Accumulated score of question and answer (QA)
The teacher side formative assessment agent defines various scores for the different qualities
of question and answer responses. Teachers can give different scores that were predefined by the
system as 1 point, 3 points and 5 points according to the qualities of learner question or answer
responses. The accumulated score of question and answer responses represents the level of active
interaction with teachers, and the notation QA is used to represent the learning factor in this study.
(3) Concentration degree (CD)
If a learner does not concentrate on the learning activity during learning processes, the teacher
can give him/her –1 point, -3 points and –5 points according to the degree of his/her distraction,
respectively. The summation of the score assessed by the teacher can be viewed as the distraction
degree. Therefore, the concentration degree is an inverse score according to the score of the
distraction degree and the notation CD is used to represent the learning factor.
(4) Accumulated score based on teacher’s comments (SC)
The teacher side formative assessment agent implemented on PDA provides an interface for
teachers to edit some default comments before performing learning activities. In the study, the
positive comments and negative comments will get 3 points and –3 points, respectively. The teacher
gives a comment score based on assessing each learner’s learning behaviors during learning
processes and the total score of teacher’s comments will be accumulated automatically. The
notation SC is used to represent the learning factor.

2.3 The Proposed Formative Assessment Approach Based on Web-based

Learning Portfolios
2.3.1 The flowchart of formative assessment
Figure 3 shows the entire flowchart of the proposed learning performance assessment scheme.
Initially, the factor analysis procedure attempts to identify the main learning factors, which are
independent and important factors affecting the final learning outcome. To identify the
independence between learning factors, a statistical correlation analysis and the fuzzy clustering
method are primarily employed, and then the grey relational analysis is used to measure the
importance of factors. Next, the K-means clustering algorithm (Krishna & Murty, 1999) is used to
logically determine the fuzzy membership function based on real data distribution of learning
portfolios for the fuzzy association mining, and hence discover valuable fuzzy knowledge rules for
learning performance assessment. The K-means clustering algorithm is a very popular, simple,
useful, and unsupervised clustering method (Chinrungrueng & Sequin, 1995; Sarkar,
Yegnanarayana & Khemani, 1997; Lee, Baek & Sung, 1997; Krishna & Murty, 1999) based on the
Euclidean distance measure for numerous engineering and scientific disciplines such as image
segmentation, patterns reorganization and data mining (Wu & Huang, 2006). Based on the reasons,
this study employed the K-means clustering algorithm to logically determine the membership
functions used in the fuzzy association rule mining. After the fuzzy association rule mining
identifies the fuzzy rules for learning performance assessment, the fuzzy inference is employed to
grade the learning performance for learners. The following sections give details for the proposed

7
learning performance assessment scheme for individual learners.

Figure 3. The flowchart of learning performance assessment

2.3.2 Learning factor analysis

First of all, factor analysis is an essential and important step towards finding key learning
factors affecting learning performance assessment (Wang & Kuo, 2004). Before mining the learning
performance assessment rules from the learning portfolios, the independent and important learning
factors have to be first decided. Learning factor analysis is critical for the proposed learning
performance assessment scheme owing to it can enhance the learning evaluation efficiency and
accuracy with respect to filtering out some factors that are dependent and less important to learning
performance from the learning portfolios (Chen, Chen & Liu, 2007). The purpose of learning factor
analysis aims to find the minimum number of learning factors that can represent complete
information of the learning process as possible. Fig. 4 shows the proposed flowchart of the learning
factor analysis.

8
 Given
Learning
Factors

Measure of
Independence

Exist
Y Dependent N
Factors?

Dependent
Factors
Independent
Factors Exist
Measure of
Evaluate Redundant N Finish
Importance Factors?
Determination

Y
Remove
Input
Insignificant Remove
Factors Factors
Redundant
Factors

Figure 4. The learning factor analysis flowchart

2.3.2.1 Learning factor dependence analysis using the fuzzy clustering method
To consider the independence of factors is because they can represent the key learning factor
better than considering all learning factors (Wang & Kuo, 2004). Therefore, identifying the
independence of learning factors is also the necessary condition of mining key formative
assessment rules from the learning portfolios in the web-based learning system. Learning factor
dependence analysis not only improves the efficiency of data mining due to reducing the
complexity of the procedure, but also facilitates the teacher to understand the simple and clear
learning performance assessment rules. To analyze the dependence among the learning factors, the
statistic theory and fuzzy set theory are employed, which involves the correlation analysis, the
coefficient of determination and fuzzy clustering analysis.
Statistic theory is a conventional method broadly used to investigate the relationship between
factors. From the statistic point of view, the correlation is generally used to define whether one
factor is related to another. Therefore, the independence among the learning factors can be
measured by their pairwise correlation coefficient (Johnson & Wichern, 1988). Moreover, fuzzy
clustering analysis algorithm is then employed to cluster those learning factors based on their
similarities (Behounek & Cintula, 2005). Since fuzzy theory was proposed by Zadeh (1996) in 1965,
fuzzy clustering analysis algorithm had been successfully implemented in taxonomy, feature
analysis, pattern recognition, image processing, medicine, geology and neural network (Bezdek,
1981; Hoppner et al., 1999; Yang, 1993). Unlike most of the traditional clustering algorithms, fuzzy
clustering analysis accepts the fact that every data belongs to all clusters with different degrees of
membership between 0 and 1 to every datum (Kennedy, Price & Susanto, 1997). For such a reason,
the learning factors belong to different clusters with different degrees and the dependence of
learning factors can then be determined accordingly. The learning factors dependence analysis will
be elaborated as following steps:
Step 1. Determining the referred and comparative sequences

9
 The first step for adopting statistical correlation for factor dependence analysis is to determine
the original referred sequence and comparative sequence. The final test grade is regarded as the
referred sequence, and the rest of nine learning factors mentioned-above are treated as the
comparative sequences.
Step 2. Calculating the correlation coefficient
Correlation analysis is a statistical technique that can evaluate whether and how strongly pairs
of learning factors are related by computing the pairwise correlation coefficient ri, j of the learning
factors xi and x j . The formula is defined as following (Johnson & Wichern, 1988):
Cov ( xi , x j )
ri , j  (1)
Var ( xi )Var ( x j )
where Cov( xi , x j ) represents the covariance of the learning factors xi and x j , Var ( xi ) and
respectively stand for standard deviations of the learning factors xi and x j .
Var ( x j )
The correlation coefficient always takes a value between -1 and 1, with 1 or -1 indicating
perfect correlation. A positive correlation indicates a positive relation between the learning factors,
while a negative correlation indicates a negative relation between the learning factors. A correlation
value close to 0 indicates no relation between the learning factors.
Step 3. Fuzzifying the correlation coefficient
If the learning factors xi and x j are independent, then ri, j is 0. However, it is difficult to
define any two factors are completely independent or dependent. Based on the fuzzy theory
(Behounek & Cintula, 2005), the relation of two factors can be viewed as “degree of dependence”
by fuzzifying the correlation coefficient of the learning factors xi and x j within the range
between 0 to 1. In this work, the membership function  A~ (r ) was employed to fuzzify the
correlation coefficients computed in Step 2, and formulated as the following:
 b 2 2
b  a a  r ,0  r  a
 A~ ( r )   (2)
b
 b a 2  (1  r ) 2 , a  r  1
 a

where a, b  0,1 , r  ri , j .
Step 4. Constructing the fuzzy relation matrix
A fuzzy relation matrix A~i can be constructed to represent the level of dependence between
each pair of learning factors. The notation  A~ (r ) in fuzzy relation matrix is the fuzzy degree
~
computed in Step 3. The Ai is represented as following:
 1 12 13  1n 
 1  23   2 n 
 21 (3)
A    31  32 1  3n 
 
      
  n1    1 
Step 5. Clustering the learning factors
For each fuzzy relation matrix, a hierarchical clustering method (Liao, 2001) with a chosen
 -cut value for identifying dependence level is used to detect the dependent learning factors.
Different  -cut values of dependence levels form different number of clusters. To optimize the
clustering result, the fuzzy correlation coefficients within the same cluster are expected as high as
possible, but the ones between different clusters are expected as low as possible. Thus, the cost
function, which integrates maximizing the fuzzy correlation coefficients in the same cluster and

10
minimizing the fuzzy correlation coefficients among different clusters, was employed to determine
best appropriate number of clusters under considering differentα-cuts in the employed fuzzy
clustering analysis scheme. In other words, the used cost function aims to consider that the learning
factors within the same cluster should be as similar as possible, but the learning factors of different
clusters should be as dissimilar as possible.
Step 6: Computing the coefficient of determination
After the dependence of learning factors clustered in the same cluster is detected, the
dependent learning factors with the lower degree of importance should be removed. The factor with
higher coefficient of determination indicates that the factor contains the larger amount of
information to contribute the prediction of the learning performance (i.e. final test grade). Therefore,
the dependent factors with the lower degree of importance will be removed because of their
unimportance, whereas factors with the larger degree of determination are remained. The coefficient
of determination of each dependent learning factor is computed as follows:
SS yy  SSE
rxy2   1
SSE (4)
SS yy SS yy

SSE   ( y  yˆ )
i i
2
(5)
  ( y  y)
2
SS yy i (6)
2
where rxy represents the coefficient of determination of the learning factors x against the
learning performance y (i.e. final test grade), SSE measures the deviation of the observation of the
learning performance y with their predicted values ŷ i , SSyy measures the deviation of the
observation of the learning performance y with the mean y , and ŷ i is the predicted value of the
regression curve.

2.3.2.2 Learning factor importance analysis using the grey relational analysis
The relationships between the learners’ learning performance and the learning factors affecting
the learning performance are usually unclear or the relational information is incomplete. It could be
difficult to analyze important learning factors related to the learning performance by the traditional
statistical methods or machine learning methods. This is because these methods usually require a
large amount of complete samples or data must follow a certain statistical distribution, but the grey
relational analysis (GRA) method (Deng, 1982) in grey system theory throws emphasis on the
problem of “small-sized data samples, poor information and uncertainty” which cannot be handled
by traditional statistics (Kui, 2005; Qinbao, Martin & Carolyn, 2005). The main reason of using the
GRA is the consideration of the reference sequence. The grey relational analysis (GRA) proposed
by Professor Deng, which ranks sub-factors with each main factor by degree of relevance using the
grey relational grade (Deng, 1989; Chen, Chang, & Liao, 2000). The importance of a single data
sequence can be explored by the grey relational grade.

2.3.3 Fuzzy association rule mining

Hong et al. study (1999) presented the fuzzy association rule to overcome a defect in Boolean
association rules for handling quantitative transaction data. To identify the large fuzzy grids for the
fuzzy association rule mining, the transaction data with quantitative value must be first transformed
into fuzzy degrees. To determine the fuzzy membership function logically, the K-means clustering
algorithm (Krishna & Murty, 1999) is used to determine the centers of the triangle fuzzy
membership functions automatically according to the data distribution of each learning factor in the
learning portfolios for the fuzzy association rules mining herein. Additionally, this study employed
Hong’s fuzzy association rule scheme (Hong et al., 1999) to discover whether the fuzzy knowledge
rules are related to the learning performance from the learning portfolios. The employed fuzzy

11
association rule mining procedures for mining formative assessment rules has been proposed in our
previous study (Chen, Chen & Liu, 2007).

2.3.4 Fuzzy inference for the learning performance assessment

The discovered fuzzy production rules are formed by IF-THEN rules. A defuzzification
strategy aims to convert the outcome of fuzzy inference into a crisp value. In the fuzzy set theory,
the center of gravity (COG) (Lin & Lee, 1996), which is most widely used defuzzification scheme,
is utilized to obtain the crisp value of a learner’s learning performance.

3. Experiments
In this section, the performance of the proposed formative assessment scheme is first
evaluated in Section 3.1. Section 3.2 explains the discovered learning performance assessment
rules in the actual teaching scene. Next, the developed formative assessment tools for both the
learner and teacher sides are introduced in Section 3.3. Section 3.4 describes the experimental
design for evaluating the promotion of the learning performance with learning performance
assessment feedback. Section 3.5 illustrates the evaluation results of learning performance.
Finally, a discussion is drawn out for the proposed learning performance assessment scheme.

3.1 Evaluation Results of the Proposed Learning Performance Assessment

Scheme
To verify the quality of the discovered learning performance assessment rules, the learning
portfolio gathered from 583 third-grade students of Taipei County Jee-May Elementary School, who
were invited to participate in the learning activity of the “Fractions” course unit in elementary
school mathematics, was first used to identify learning performance fuzzy rules affecting their
learning outcomes. In the experiment, the learning records of 400 out of 583 learners were used as
training data to extract learning performance assessment rules, and the other learning records served
as testing data to verify the accuracy of the discovered learning performance assessment rules. The
experimental results are described as follows.

3.1.1 Learning factor analysis

3.1.1.1 Learning factor independence analysis

Initially, Table 1 displays the computing correlation coefficients between the seven considered
learning factors presented in our previous study (Chen, Chen & Liu, 2007). Then, the correlation
coefficients of learning factors are fuzzified for the fuzzy clustering analysis. Table 2 lists the result
of fuzzy clustering analysis with different  -cut values. When the  -cut value is set 0.79, the best
clustering result {{RR, RT}{LA, CR}{PN, AS}{EL}} with maximum value of clustering cost
function mentioned-above is obtained. Next, the coefficients of determination of those dependent
learning factors in the clusters {RR, RT}, {LA, CR} and {PN, AS} are measured, and the
dependent learning factors are filtered out except the one with largest coefficient of determination.
Table 3 shows the learning factors{RT}、{LA} and { PN } can be removed due to relatively lower
coefficients of determination to the learning performance assessment. Thus, these four learning
factors {RR}、{CR}、{AS} and {EL} are served as significant learning factors after performing the
factor dependence analysis.

12
 Table 1. The correlation coefficients between seven considered learning factors

Learning Factors RR RT LA CR PN AS EL
RR 1 -0.705 0.256 0.495 0.269 0.381 0.309
RT -0.705 1 -0.242 -0.421 -0.278 -0.406 -0.240
LA 0.256 -0.242 1 0.859 0.276 0.323 -0.027
CR 0.495 -0.421 0.859 1 0.286 0.357 0.102
PN 0.269 -0.278 0.276 0.286 1 0.667 -0.033
AS 0.381 -0.406 0.323 0.357 0.667 1 -0.078
EL 0.309 -0.240 -0.027 0.102 -0.033 -0.078 1

Table 2. The fuzzy clustering results with different  -cut values

 -cut Value Clustering Results

Dependence in Independence
Total
the Same Cluster between Clusters
0.69 {RR, RT, LA, CR, PN, AS, EL} 14.012 0 14.012
0.70 {RR, RT, LA, CR, PN, AS}{EL} 13.223 5.211 18.434
0.78 {RR, RT, LA, CR}{PN, AS}{EL} 10.645 10.633 21.279
0.79 {RR, RT}{LA, CR}{PN, AS}{EL} 9.232 13.219 *22.451
0.90 {RR, RT}{LA, CR}{PN}{AS}{EL} 8.564 13.552 22.116
0.92 {RR, RT}{LA}{CR}{PN}{AS}{EL} 7.705 13.693 21.397
0.93 {RR}{RT}{LA}{CR}{PN}{AS}{EL} 7.000 13.988 20.988

Table 3. The coefficients of determination of dependent learning factors

*{RR} {RT} {LA} *{CR} {PN} *{AS}
Learning factors
The Coefficient of 0.168 0.090 0.281 0.459 0.032 0.063
Determination

3.1.1.2 Learning factor importance analysis

In the previous subsection, the four independent learning factors {RR}、{CR}、{AS} and {EL}
are extracted based on the proposed learning factor analysis scheme. Next, these four learning
factors are further measured by the grey relational analysis in order to figure out the degree of
importance. Table 4 lists the grey relational grades between the referred sequence and various
comparative sequences based on the ranking order of the grey relational grades. The results indicate
that the top three considered learning factors are highly relevant to the final test score since their
grey relational grades are greater than the threshold of 0.5. Thus, the three learning factors were
preserved to perform the fuzzy association rule mining and detect useful fuzzy rules for learning
performance assessment, but the learning factor AS was eliminated since it has lowest relevance to
the final test score among four considered learning factors.

13
 Table 4. The grey relational grades between the referred sequence and comparative one

Factor Analysis Item Grey Relational Grade

 (GRADE, CR) 0.8594
 (GRADE, EL) 0.7027
 (GRADE, RR) 0.5883
 (GRADE, AS ) 0.4757

3.1.2 The discovered fuzzy rules for learning performance assessment

To explain the discovered fuzzy association rules for learning performance assessment, the
simplified representation notations VH, H, M, L and VL were employed to represent “very high”,
“high”, “moderate”, “low” and “very low” for fuzzy rules, respectively. After performing the fuzzy
association rule mining procedure, fourteen fuzzy learning rules are discovered in this study. Table
5 displays the entire discovered fuzzy rules that can be employed to assess the learning performance
of learners with various grade levels in the proposed learning performance assessment scheme.
These interpretable fuzzy rules for the learning performance assessment of various grade levels are
valuable to teachers to understand which learning factors influence the learning performance in a
web-based learning environment.
Table 5. The discovered learning performative assessment rules with the satisfied minium
fuzzy support, confidence, and certainty factor

Support (Min Confidence (Min CF(Min

Fuzzy Rule
Sup=0.2) Conf=0.5) CF=0.5)
1 CR_L  GRADE_L 0.407075 0.882605 0.565105
2 RR_VL  GRADE_L 0.330171 0.899318 0.627017
3 CR_L ∩ RR_VL  GRADE_L 0.202906 0.979183 0.922883
4 CR_M  GRADE_M 0.468819 0.913054 0.637235
5 RR_VL  GRADE_M 0.341713 0.921658 0.673133
6 CR_M ∩ RR_VL  GRADE_M 0.259632 0.955261 0.813337
7 CR_H  GRADE_H 0.514702 0.911135 0.616762
8 RR_L  GRADE_H 0.286783 0.926365 0.682442
9 CR_H ∩ RR_L  GRADE_H 0.238686 0.964525 0.847008
10 CR_H ∩ EL_VH  GRADE_H 0.217541 0.889984 0.525543
11 CR_VH  GRADE_VH 0.68382 0.956338 0.685182
12 RR_H  GRADE_VH 0.257372 0.967798 0.767812
13 CR_VH ∩ RR_H  GRADE_VH 0.22981 0.979792 0.854291
14 CR_VH ∩ EL_VH  GRADE_VH 0.400021 0.960917 0.718194

3.1.3 Evaluating accuracy rate of learning performance assessment

14
 To measure the accuracy rate of learning performance assessment for the proposed method,
two methods were used to measure the predicted learning performance. First, the  5 point method
was used to evaluate the accuracy rate of the predicted learning performance. That is, if the
difference between the predicted learning score and the actual final test score was in the range −5 to
+5, then the predicted result is served as correct; otherwise, the predicted result is incorrect.
Additionally, the score level method was used to evaluate the accuracy rate of the predicted learning
performance. In this evaluation method, the each learner is assessed according to one of five score
levels based on the mapping membership degrees of learning factor GRADE. For example, the
score level of a learner with a final test score of 85.16 was set to GRADE.M, because the linguistic
term of GRADE.M has the largest mapping membership degree among the other linguistic terms of
the learning factor GRADE.
Figure 5 illustrates the prediction accuracy rates of 183 testing data under various
combinations of learning factors for the two proposed accuracy evaluation methods. The
experimental results show that the proposed dependence and importance analysis scheme of
learning factor can help the proposed learning performance assessment scheme to identify the key
learning factors. As mentioned early, when the more sets of dependent learning factors, such as {CR,
LA}, {RR, RT} and {PN, EL}, are included, the lower accuracy rate is displayed in Figs. 5. For
example, when the dependent learning factor set {CR, LA} is considered to perform the learning
performance assessment, the predicted accuracy rate is lower than those of only considering the
learning factor {CR}. According to the results, the independent learning factors to each other
facilitate the learning performance assessment.
In addition, the experimental results revealed as Fig. 5 show that the proposed grey relational
analysis scheme can help the proposed learning performance assessment scheme to identify the
significant learning factors. When the learning factors with low grey relational grades, such as PN,
AS and RT, were considered to perform the learning performance assessment, the predicted
accuracy rate will descend. Conversely, if only the learning factor with the largest grey relational
grad was considered, then the proposed method obtained the highest prediction accuracy rate.
validity SCORE LEVEL validity ±5POINT

1
0.9
0.732
0.8
0.672 0.689
accurancy rate

0.7
0.6
0.459
0.585
0.5
0.508 0.525
0.4
0.23 0.23 0.257
0.3 0.399

0.2
0.208 0.208 0.24
0.1
0
CR-LA-EL- CR-LA-EL- CR-LA-EL- CR-LA-EL- CR-LA-EL CR-LA CR
RR-RT -AS- RR-RT -AS RR-RT RR
PN

learning factors

Figure 5. The predicted accuracy rates of 183 testing data under considering various
combinations of learning factors for the two proposed accuracy evaluation methods

3.2 The Discovered Learning Performance Assessment Rules in the Actual

15
 Teaching Scene
In the actual teaching experiment for verifying how the feedback of learning performance
assessment outcomes affects the learning performance assessment, 69 third-grade students of Taipei
County Jee-May Elementary School, who had majored in the “Fractions” course unit in elementary
school mathematics, were invited to test this system. The learning portfolios with 9 considered
learning factors gathered from PELS and PDA-based formative assessment tool were used to mine
the learning performance assessment rules. After performing factor analysis, the learning factors
{CR}, {RT}, {AR}, and {CD} were identified as key learning factors influencing learning
performance. All of the 40 discovered learning performance assessment rules are listed in Table 6.
Table 6. The discovered learning performance assessment rules in the actual teaching
experiment

Support (Min Confidence (Min

Fuzzy Rule CF(Min CF=0.5)
Sup=0.2) Conf=0.5)
1 CR_L  GRADE_L 0.69233 0.967124 0.764938
2 CD_M  GRADE_L 0.546624 0.985402 0.895627
3 RT_H  GRADE_L 0.446809 0.966771 0.762413
4 CR_L ∩ CD_M  GRADE_L 0.518934 1 1
5 CR_L ∩ RT_H  GRADE_L 0.446809 0.966771 0.762413
6 CR_L ∩ AR_H  GRADE_L 0.69233 0.967124 0.764938
7 CR_M ∩ RT_H  GRADE_L 0.273413 1 1
8 CD_M ∩ AR_H  GRADE_L 0.546624 0.985402 0.895627
9 RT_H ∩ AR_H  GRADE_L 0.446809 0.966771 0.762413
10 CR_L ∩ CD_M ∩ RT_H  GRADE_L 0.273413 1 1
11 CR_L ∩ CD_M ∩ AR_H  GRADE_L 0.518934 1 1
12 CR_L ∩ RT_H ∩ AR_H  GRADE_L 0.446809 0.966771 0.762413
13 CD_M ∩ RT_H ∩ AR_H  GRADE_L 0.273413 1 1
14 CR_L ∩ CD_M ∩ RT_H ∩ AR_H  GRADE_L 0.273413 1 1
15 CR_M  GRADE_M 0.530424 0.930341 0.713427
16 RT_M  GRADE_M 0.380117 0.900504 0.590679
17 CR_M ∩ CD_VH  GRADE_M 0.241811 0.948051 0.786285
18 CR_M ∩ RT_M  GRADE_M 0.285976 0.935708 0.735505
19 CR_M ∩ AR_H  GRADE_M 0.530424 0.930341 0.713427
20 RT_M ∩ AR_H  GRADE_M 0.380117 0.900504 0.590679
21 CR_M ∩ CD_VH∩AR_H  GRADE_M 0.241811 0.948051 0.786285
22 CR_M ∩ RT_M ∩ AR_H  GRADE_M 0.285976 0.935708 0.735505
23 CR_ H  GRADE_H 0.555071 0.916925 0.613411
24 CD_ VH  GRADE_H 0.383544 0.916144 0.609776
25 RT_ M  GRADE_H 0.264558 0.917956 0.618208

16
 26 CR_ H∩ CD_VH  GRADE_H 0.264974 0.96196 0.822979
27 CR_ H∩ RT_M  GRADE_H 0.215576 0.943127 0.735342
28 CR_ H∩ AR_H  GRADE_H 0.555071 0.916925 0.613411
29 CD_ VH∩ AR_H  GRADE_H 0.383544 0.916144 0.609776
30 RT_ M∩ AR_H  GRADE_H 0.264558 0.917956 0.618208
31 CR_ H∩ CD_VH ∩ AR_H  GRADE_H 0.264974 0.96196 0.822979
32 CR_ H∩ RT_M ∩ AR_H  GRADE_H 0.215576 0.943127 0.735342
33 CR_ VH  GRADE_VH 0.556584 0.967693 0.825014
34 RT_ VL  GRADE_VH 0.386925 0.930153 0.621686
35 CR_ VH ∩ CD_H  GRADE_VH 0.266175 0.964644 0.808503
36 CR_ VH ∩ RT _VL  GRADE_VH 0.360335 0.960131 0.784057
37 CR_ VH ∩ AR_H  GRADE_VH 0.556584 0.967693 0.825014
38 RT_ VL ∩ AR_H  GRADE_VH 0.386925 0.930153 0.621686
39 CR_ VH∩CD_H∩AR_H  GRADE_VH 0.266175 0.964644 0.808503
40 CR_VH∩RT_VL∩AR_H  GRADE_VH 0.360335 0.960131 0.784057

3.3 The Implemented Learning Performance Assessment Tools

3.3.1 The teacher side formative assessment tool
Figures 6(a) thru 6(k) display the proposed teacher formative assessment tool implemented on
the PDA to support teachers to perform formative assessment in a computer classroom with Internet
for supporting web-based learning. Before using the formative assessment tool, teachers have to
register their user accounts and passwords in advance. Figure 6(a) shows the user login interface.
After a user logins the mobile assessment tool by legal accounts, the user menu with two functions
is displayed as Fig. 6(b) for teachers. The first function is to view the detailed learning statuses of
individual learners on the PELS, and the second function is to perform the formative assessment in
the classroom with computer-assisted learning.
In Fig. 6(b), if the second function is selected, then teachers can assess learning statuses of
individual learners including attendance statuses, concentration degree, question & answer
responses, and learning comments. Figure 6(c) shows the interface for assessing the attendance
statuses of learners. Figure 6(d) displays the interface for recording learners’ question & answer
responses during learning processes. Figure 6(e) shows the interface that can assist teachers to
evaluate the concentration degrees of individual learners. In Fig. 6(f), several default comments
about learners’ performances can be selected for assessing the learning statuses of individual
learners. In addition, these default comments can also be edited by teachers. Figure 6(g) illustrates
the interface for editing learning comments as positive or negative. In other words, each teacher can
freely create his/ her own comments that appropriately characterize his/her learners.
If the first button shown as Fig. 6(b) is selected, then teachers can view the comprehensive
degree of each courseware for individual learners through the interface shown as Fig. 6(h). Figure
6(i) displays the interface for viewing the variances of learners’ learning abilities during learning
processes. Furthermore, each learner has to take a test after finishing the learning process on the
PELS, and then the tests results are stored in the user profile database. Figure 6(j) displays the
interface for viewing learners’ test scores. The interface makes teachers understand learners’
learning effects on the PELS and this function is also helpful to judge whether remedy learning is

17
needed. Finally, the learning portfolios based on the considered learning factors gathered by PELS
system and PDA-based teacher assessment tool were utilized to mine learner’s learning
performance assessment rules. Figure 6(k) shows the discovered learning rules influencing learners’
learning performances. In conclusion, the proposed teacher side formative assessment tool offers
assessing information related to learners’ learning statuses on the PELS as well as provides a
friendly user interface for assessing learners’ learning processes by PDA. Using the tool, teachers
not only can monitor learners’ learning statuses, but also can know how to guide more effectively
learning.

(a) User login interface (b) Menu of learning performance assessment

tool

(c) The interface for assessing the attendance (d) The interface for recording learners’
statuses of learners Q & A responses

18
 (e) The interface for recording learners’ (f) The interface for giving learners’ learning
concentration degree during learning comments
processes

(g) The interface for editing learning (h) The interface for viewing the difficulty
comments as positive or negative levels of courseware from learners’ feedback
responses

(i) The interface for viewing the variances of (j) The interface for viewing learners’ test
learners’ learning abilities assessed by the scores
proposed PELS during learning processes

19
 (k) The interface for viewing the discovered learning performance assessment rules
Figure 6. The implemented mobile formative assessment tool for teachers

3.3.2 The learner side formative assessment tool

Figures 7(a) and 7(b) present the feedback interface of learning performance assessment for
individual learners on the PELS. In Fig. 7(a), the learning feedback assessed by the teacher side
formative assessment tool can help each learner to adjust his/her learning strategy during learning
processes. In actual teaching scene, the information of learning feedback was first explained by the
teacher to confirm that each learner understood the feedback information meaning. Initially, the
system inferred the final learning score according to the discovered formative assessment fuzzy
rules automatically. The inferred score was conveyed to individual learner, thus reminding the
learner with low learning score to study hard in the next learning stage. Next, the learning behaviors
in the classroom with computer-assisted learning including the attendance rate, accumulated score
of question and answer responses, accumulated score of teacher’s comments were showed to
individual learner through the feedback interface on PELS. In this work, these learning portfolios
recorded by teacher through PDA are integrated with learning portfolios gathered by PELS to
progress the formative assessment. These formative assessment outcomes are helpful to illustrate
the learning pictures of the learning process. Besides, learner ability and correct response rate of
randomly selecting testing questions were also important messages for each learner. Each learner
can use the information to view his/her final learning states and make his/her learning better. Finally,
Fig. 7(b) displays the learning rules associated with five grade levels involving “very high score”,
“high score”, “moderate score”, “low score” and “very low score”. Learners could select different
grade levels to look at related learning rules and find out key learning factors influencing the
learning performance.

20
(a) Learning feedback about learning behavior of individual learners from teacher formative
assessment

(b) The discovered learning performance rules

Figure 7. The implemented feedback interface for the discovered learning performance
assessment rules on PELS

3.4 Experimental Design

The participants of this study were recruited from two intact three-grade classes of Taipei
County Jee-May Elementary School. There are 35 learners in the experimental group and 34
learners in the control group, ranging in age from 9 to 11 years old. The experimental group
received a two-week mathematical courseware learning in a computer classroom using PELS with
learning feedback of formative assessment during learning processes. In contrast with the
experimental group, the control group received the same mathematical courseware by PELS
without learning feedback of formative assessment during learning processes.
The pretest-posttest nonequivalent group design of quasi-experimental method was adopted for
investigating the promotion of learning performance in the study. Its purpose was to assess whether
learning feedback of formative assessment contributes a significant positive effect on the promotion
of learners’ learning performances and learning attitude. The duration of this study lasted two weeks.

21
Table 7 displays the procedures of quasi-experimental design method for assessing learning
performance of both the participating groups in the study. Both the learning modes perform the
pretest and posttest for comparing the difference of learning performance before and after learning.
Table 7. The procedures of quasi-experimental design method

Pre-test Treatment Post-test

O1 X O3
O2 C O4

O1 and O 2 represent the pre-test processes; O3 and O 4 are the post-test processes; X and
C stand for the experimental and control groups performed by learning mode with and
without learning feedback of formative assessment, respectively.
In the experiment, the teacher first detailed the system operation procedures for all
participators in the first hour, and then all participators logged in the system to perform the learning
process. Figure 8 exhibits the actual teaching scene at Taipei County Jee-May Elementary School in
the experiment.

Figure 8. The actual teaching scene at Taipei County Jee-May Elementary School

3.5 System Evaluation

Three evaluating procedures including a pre-test, post-test, and questionnaire were performed
to assess the learning outcomes for the proposed system. That is, the system evaluation in the
study includes two parts which are the learning performance promotion of learners and
questionnaire results from learner’s feedback responses.

3.5.1 Learning performance evaluation

Table 8 displays the comparison result of learning performance for both the learning modes.
The results reveal that 54.29% learners who learnt by the proposed personalized e-learning system
with formative assessment feedback have progressive score, but only 52.94% learners who learnt by
the proposed personalized e-learning system without formative assessment feedback have
progressive score. Additionally, Fig. 9 indicates that most learners have obvious progress in score
according to the score distributions of the experimental group. In contrast with the experimental
group, Fig.10 shows that the progressive score of the control group is not so obvious like the

22
experimental group. Besides, the information is beneficial to further analyzing the learning
performance by statistics analysis method.
Table 8. Comparison result of learning performance for both the participating groups
conducting different learning modes
The Experimental Group
Learning Mode The Control Group Performing
Performing Learning
Learning without Formative
with Formative
Comparison Item Assessment Feedback
Assessment Feedback
Number of learners 35 34
Number of learners with progress score 19 (54.29%) 18 (52.94%)
Number of learners with retrogression score 6 (17.14%) 8(23.53%)
Number of learners with constant score 10 (28.57%) 8 (23.53%)

PreTest PostTest
100
90
80
70
Score

60
50
40
30
20
10
0

1 4 7 10 13 16 19 22 25 28 31 34
Learner

Figure 9. The score distributions of the experimental group with formative assessment
feedback for both the pre-test and post-test

PreTest PostTest
100
90
80
70
60
Score

50
40
30
20
10
0

1 5 8 11 14 17 20 23 26 30 33 36
Learner

Figure 10. The score distributions of the control group without formative assessment feedback
for both the pre-test and post-test
In the work, the independent samples t-test and matched-pairs t-tests were used to analyze
whether the experimental group with formative assessment feedback or the control group without
formative assessment feedback provides benefits in terms of learning performance promotion based
on pretest and posttest scores. First, the gathered pre-test and post-test scores were analyzed using

23
the independent samples t-test. The SPSS analysis result of pre-test is presented in Table 9. The
results indicate that the mean and standard deviation of the experimental group on the pre-test score
is 86.86 and 9.858 points, respectively. The mean and standard deviation of the control group on the
pre-test score is 86.03 and 13.969 points. Since these two participating groups do not reach
significant difference on the pre-test score (t=-.285, p = .776 > 0.05), the mathematics abilities of
two participating groups in the “Fraction” unit can be viewed as equivalent before performing the
experiment process.
Table 9. The independent samples T-test of pre-test between two participating groups
(a) Group statistics

Group N Mean Std. Deviation Std. Error Mean

Pretest Exp. 35 86.86 9.858 1.666
Ctrl. 34 86.03 13.969 2.396

(b) Independent samples t-test

t-test for Equality of Means

95% Confidence Interval
Std. Error of the Difference
t df Sig. (2-tailed) Mean Difference Difference Lower Upper
Pre-Test .285 67 .776 .828 2.904 -4.968 6.624

Next, the SPSS analysis result of post-test is presented in Table 10. The post-test of two
participating groups were also analyzed using independent-samples t-test and the mean scores of the
experimental and control groups on the post-test score are 91.86 and 87.94, respectively. The
t-test result (t=1.417, p=. 163 > 0.05) shows that two participating groups do not reach significant
difference on the post-test score. Thus, this study further compared the pre-test and post-test for
each group using the paired-samples t-test. Table 11 shows the result of the paired samples t-test of
the experimental group. In the experimental result, the difference of the mean scores of pre-test and
post-test is -5 and the results of paired-samples t-test reach the significant level (t= -3.3, p=. 002 <
0.05). In other words, the promotion of learning performance in the experimental group is
significant and the mean score increases 5 points.
Table 10. The independent samples t-test of post-test between two groups
(a) Group statistics

Group N Mean Std. Deviation Std. Error Mean

Posttest Exp. 35 91.86 6.869 1.161
Ctrl. 34 87.94 14.622 2.508

(b) Independent samples t-test

p p

t-test for Equality of Means

95% Confidence Interval
Std. Error of the Difference
t df Sig. (2-tailed) Mean Difference Difference Lower Upper
Post-Test 1.417 46.587 .163 3.916 2.763 -1.645 9.477

24
 Table 11. The paired samples t-test of the experimental group
(a)Paired samples statistics

Mean N Std. Deviation Std. Error Mean

Pair 1 Pretest 86.86 35 9.858 1.666
Posttest 91.86 35 6.869 1.161

(b)Paired samples t-test

Paired Samples Test

Paired Differences
95% Confidence
Std. Interval of the
Std. Error Difference Sig.
Mean Deviation Mean Lower Upper t df (2-tailed)
Pair 1 Pretest-
Posttest -5.000 8.911 1.506 -8.061 -1.939 -3.3 34 .002

Next, the pre-test and post-test scores of the control group are also assessed using the
paired-samples t-test. Table 12 displays the result of the paired samples t-test of the control group.
In the experimental result, the difference of the mean scores between pre-test and post-test is -1.91
and the result of paired-samples t-test (t=-1.169, p=. 251 > 0.05) shows that the control group does
not achieve significant difference after performing learning mode without formative assessment
feedback. Besides, the testing score in the experimental group is 5 points higher than that in the
control group. This result can prove that the learning performance of the learning mode with
formative assessment feedback is superior to the learning mode without formative assessment
feedback.
Table 12. The paired samples t-test of the control group
(a) Paired samples statistics
Mean N Std. Deviation Std. Error Mean
Pair 1 Pretes
t 86.03 34 13.969 2.396
Postte
st 87.94 34 14.622 2.508

(b) Paired samples t-test

Paired Samples Test

Paired Differences
95% Confidence
Std. Interval of the
Std. Error Difference Sig.
Mean Deviation Mean Lower Upper t df (2-tailed)
Pair 1 Pretest-
Posttest -1.91 9.536 1.635 -5.239 1.415 -1.169 33 .251

Moreover, to investigate whether the proposed learning modes with and without formative
assessment feedback provide different learning performances for learners with various mathematics
abilities, the learners of each class in both the participating groups were divided into three groups
based on their pre-test scores. The learners whose pre-test scores are above 27% in each class are
viewed as high score group, and the learners whose pre-test scores are below 27% are viewed as
low score group. The remaining learners are viewed as moderate score group. Similarly, the
paired-samples t-test was employed to analyze these three groups with different learning abilities in
both the experimental and control groups, respectively. Table 13 shows the result of the
paired-samples t-test. In the experimental group, the progress scores of the learners with high,
moderate and low score are –2.00, 5.00 and 12.778, respectively. Besides, the t- test results of the

25
three different score groups of the experimental group (i.e. the high score group: t=-1.309, p=.223 >
0.05; the moderate score group: t=-3.464, p=.003 < 0.05; the low score group: t=-3.507, p=.008 <
0.05) show that the score progress of the moderate and low score groups reaches significant level
after performing the learning mode with formative assessment feedback, but the high score group
does not reach significant level. Nevertheless, all the t-test results of the three different score groups
in the control group do not reach significant level. In conclusion, the proposed learning mode with
formative assessment feedback indeed surpasses the learning mode without formative assessment
feedback because it can direct learners to adjust their learning states based on immediate learning
performance result during learning processes.
Table 13. The paired samples t-test result of three groups with various learning abilities for
both the experimental and control groups
Statistics Pre-test Post-test Paired Difference
Learning Std. Mean
Std. Std. t Sig.
abilities Class Mean deviatio Mean (pretest-post
deviation deviation
n test)
High score 306(Exp.) 96.50 2.415 94.50 4.972 2.000 4.830 1.309 .223
group 305 (Ctrl.) 97.27 2.611 93.64 6.360 3.636 5.954 2.025 .070

Moderate 306(Exp.) 88.13 2.500 93.13 5.439 -5.000 5.774 -3.464 .003*
score group
305 (Ctrl.) 90.00 .000 94.17 5.845 -4.167 5.845 -1.746 .141
Low score 306(Exp.) 73.89 9.280 86.67 8.660 -12.778 10.929 -3.507 .008*
group
305 (Ctrl.) 77.35 15.012 82.06 18.205 -4.706 11.106 -1.747 .100

3.5.2 Questionnaire analysis

To evaluate learners’ satisfaction degree for the learning mode and learning system, a
questionnaire which involves twenty-four questions distinguished five various question types was
designed to measure whether the provided services in the PELS with formative assessment
feedback satisfy the real requirements of most learners. The five question types contain the
personal information about learner’s learning experience using the computer, the convenience of the
system operation, the learners’ learning attitude towards using the proposed learning system, the
benefits related to the proposed learning mode with formative learning feedback, and the
improvement of learner’s mathematics abilities and confidence after using the proposed learning
mode. Table 14 gives a summarization of the descriptions of question types. Totally, there are
35 learners in the experimental group and 34 learners in the control group to participate in the
experiment and they were invited to fill out this questionnaire after attending the two weeks’
learning activity. The evaluation results of satisfaction degree are listed in Table 15. To
conveniently observe the evaluating results, the investigation results of “strongly agreed” and
“agreed” are merged as “approved”, and the investigation results of “strongly disagreed” and
“disagreed” are merged as “disapproved”.

26
 Table 14. The descriptions of question types
The number of
Question Type Description
questions

Personal Information 3 To get the personal information about learners who attend the
learning activity
System Operation 5 Questions related to the user interface and the content of
learning materials
Learning Attitude 6 To investigate whether the system can enhance learners’
learning motivation or interests or not
Learning Mode 6 Questions related to the proposed PELS system with formative
assessment feedback responses for individual learners
Learning Performance 4 To explore whether the learning mode can promote their
learning achievements and confidence or not

The investigation results of the personal information are listed in Table 15(a) and it indicates
94.29% learners of the experimental group and 94.12% learners of the control group have
computers at home. Additionally, 97.14% learners of the experimental group and 94.12% learners
of the control group like to use computers, but only about a half of learners of both the groups
respectively used the learning system through the Internet. From Table 15(b), the satisfaction
degrees of “approved” of system operation of the experimental group are 84% and that of the
control group are 77.65%. Moreover, Table 15(c) specifies the 84.29% learners of the experimental
group and 76.47% learners of the control group agreed that the proposed learning system can
promote their learning motivation. Finally, in terms of the learning mode and learning performance,
the results are summarized in Tables 15(d) and (e). Most learners in two participating groups agreed
that the learning records and learning feedback provide benefit in terms of the promotion of the
learning performance. 84.97% learners of the experimental group and 72.06% learners of the
control group believed that the proposed learning system can improve their confidence in learning
and get good grades in the learning.
Table 15. The satisfaction evaluation results of questionnaire
(a) The investigation results of the personal information
The Number of Learners
Question Type Question The Experiment Group The Control Group
Yes No Yes No
33 2 32 2
Do you have any computer at your home?
94.29% 5.71% 94.12% 5.88%
Personal 34 1 32 2
Do you like to use the computer?
Information 97.14% 2.86% 94.12% 5.88%
15 20 17 17
Have you ever used any web site for learning?
42.86% 57.14% 50% 50%

(b) The investigation results of the system operation

Satisfaction Degree
Question
Question The Experimental Group The Control Group
Type
Strongly No Strongly Strongly No Strongly
Agreed Disagreed Agreed Disagreed
Agreed Opinion Disagreed Agreed Opinion Disagreed

27
 I think that the PELS 21 8 6 0 0 23 5 5 1 0
learning system
provides a friendly user
interface 60% 22.86% 17.14% 0% 0% 67.65% 14.71% 14.71% 2.94% 0%

The flash learning

materials make me 21 10 3 0 1 23 6 4 1 0
understand the fraction
unit of mathematics
course more easily 60% 28.57% 8.57% 0% 2.86% 67.65% 17.65% 11.76% 2.94% 0%
It is interesting to me to
operate and learn 20 5 9 0 1 23 4 3 0 4
System mathematics actively
on the PELS learning 57.14% 14.29% 25.71% 0% 2.86% 67.65% 11.76% 8.82% 0% 11.76%
Operation system

I think the PELS 23 10 2 0 0 22 3 7 0 2

system is an excellent
learning tool to assist
mathematics learning 65.71% 28.57% 5.71% 0% 0% 64.71% 8.82% 20.59% 0% 5.89%

I agree that learning

through the Internet is 18 11 5 0 1 20 3 5 3 3
very convenient
because I can perform
mathematics learning at
any time and place 51.43% 31.43% 14.29% 0% 2.86% 58.82% 8.82% 14.71% 8.82% 8.82%

Average 84% 14.28% 1.72% 77.65% 14.12% 8.23%

(c) The investigation results of the learning attitude

Satisfaction Degree
Question
Question The Experimental Group The Control Group
Type
Strongly No Strongly Strongly No Strongly
Agreed Disagreed Agreed Disagreed
Agreed Opinion Disagreed Agreed Opinion Disagreed
Learning
Learning by the PELS 18 11 5 0 1 20 6 4 1 3
Attitude learning system can
promote my learning
interest 51.43% 31.43% 14.29% 0% 2.86% 58.82% 17.65% 11.76% 2.94% 8.82%

I agree that using the

PELS system to learn 21 8 5 0 1 23 5 3 1 2
mathematics is a very
interesting learning
mode 60% 22.86% 14.29% 0% 2.86% 67.65% 14.71% 8.82% 2.94% 5.88%

I feel happy when I learn 19 10 5 0 1 19 7 3 2 3

mathematics by the
PELS learning system 54.29% 28.57% 14.29% 0% 2.86% 55.88% 20.59% 8.82% 5.88% 8.82%

28
 I think that using the
PELS learning system to 18 9 5 0 3 23 2 5 2 2
perform mathematics
learning can attract me
to concentrate on the
mathematics course 51.43% 25.71% 14.29% 0% 8.57% 67.65% 5.88% 14.71% 5.88% 5.88%
To learn mathematics by
the PELS learning 20 13 2 0 0 24 2 5 0 3
system is a cheerful
learning experience to 57.14% 37.14% 5.71% 0% 0% 70.59% 5.88% 14.71% 0% 8.82%
me

I feel it is easy to learn 18 12 4 0 1 23 2 7 0 2

mathematics after using
the PELS system 51.43% 34.29% 11.43% 0% 2.86% 67.65% 5.88% 20.59% 0% 5.88%

Average 84.29% 12.38% 3.34% 76.47% 13.24% 10.29%

(d) The investigation results of learning mode

Satisfaction Degree
Question
Question The Experimental Group The Control Group
Type
Strongly No Strongly Strongly No Strongly
Agreed Disagreed Agreed Disagreed
Agreed Opinion Disagreed Agreed Opinion Disagreed
I agree that my teacher
Learning
record my learning
20 11 4 0 0 20 4 5 3 2
Mode process including the
frequency of proposing
questions, the
concentration degree, 57.14% 31.43% 11.43% 0% 0% 58.82% 11.76% 14.71% 8.82% 5.88%
etc.
The records of learning
behaviors in the 19 13 2 0 1 16 4 10 2 2
classroom can represent
a part of my learning 54.29% 37.14% 5.71% 0% 2.86% 47.06% 11.76% 29.41% 5.88% 5.88%
performance
The PELS learning
system with formative 16 13 6 0 0
assessment feedback is --- --- --- --- ---
very helpful to my
mathematics learning 45.71% 37.14% 17.14% 0% 0%

The PELS learning

system with formative 21 7 6 0 1
assessment feedback can
--- --- --- --- ---
enhance my learning
motivation, thus let me
60% 20% 17.14% 0% 2.86%
learn better
I expect that my teacher
can often interact with 21 4 7 0 2
me during my --- --- --- --- ---
mathematics learning 61.76% 11.76% 20.59% 0% 5.88%
process

29
 I think I can learn better
if my teacher gives some 19 8 4 2 1
suggestions related to
--- --- --- --- ---
my learning statuses
during the learning
55.88% 23.53% 11.76% 5.88% 2.94%
process

(e) The investigation results of learning performance

Satisfaction Degree
Question
Question Experimental Group Control Group
Type
Strongly No Strongly Strongly No Strongly
Agreed Disagreed Agreed Disagreed
Agreed Opinion Disagreed Agreed Opinion Disagreed
The PELS learning
system can promote 19 10 6 0 0 20 6 4 3 1
the learning effect
and my learning
achievement. 54.29% 28.57% 17.14% 0% 0% 58.82% 17.65% 11.76% 8.82% 2.94%
I think that using the
PELS learning
18 11 4 1 1 21 3 8 1 1
system can
effectively promote
my mathematics
ability in a short 51.43% 31.43% 11.43% 2.86% 2.86% 61.76% 8.82% 23.53% 2.94% 2.94%
Learning time.
Performance The PELS learning
system increases my 19 12 4 0 0 21 1 9 1 2
confidence of
learning 54.29% 34.29% 11.43% 0% 0% 61.76% 2.94% 26.47% 2.94% 5.88%
mathematics.
I satisfied my
learning 22 9 2 1 1 23 3 6 0 2
performance on the
PELS learning
system. 62.86% 25.71% 5.71% 2.86% 2.86% 67.65% 8.82% 17.65% 0% 5.88%

Average 84.97% 11.43% 2.86% 72.06% 19.85% 8.09%

3.6 Discussion
The experimental results show that the mathematics abilities of most participators are
promoted after using the proposed learning mode with learning feedback of formative assessment
during learning processes. In addition, the result of questionnaires indicates that they agreed that
their mathematics abilities were promoted and they like to learn mathematics using the proposed
learning mode. Moreover, the designed user interfaces and system functions were satisfied to most
learners. Furthermore, the proposed formative assessment scheme has some disadvantages that need
to be further improved. One problem is how to promote the correct rate of the gathered learning
portfolios, especially for effective reading time. The correct rate of learning portfolios influences
the results of the proposed formative assessment rule mining scheme and noisy data of learning
portfolios have to be filtered out before conducting learning performance assessment fuzzy mining.
Another topic is that the clustering method of key learning factors affects the fuzzy membership
functions used in the employed fuzzy association rules mining. In this work, the K-means clustering

30
algorithm was used in this study and it is especially sensitive to initial clustering centers. Therefore,
the other more excellent cluster methods can be considered to determine the fuzzy membership
functions for the employed fuzzy association rule mining scheme in the future. The other issue is
that how to validate the discovered learning performance assessment fuzzy rules. The learning rules
obtained from the formative assessment scheme are helpful to learners and teachers, but proposing
an effective mechanism to validate the quality of the learning performance assessment rules is
urgently needed. Finally, some disadvantages of the proposed system with regard to system
functions from the learners’ feedbacks can be improved in our future work.

4. Conclusion
The goal of this study is respectively to develop the formative assessment tools, which contain
the proposed key learning factor analysis and learning performance assessment rule mining schemes
to discover simplified and key fuzzy learning rules for evaluating the learning performance of
individual learners, on the PELS for learners and on the PDA for teachers based on the gathered
learning portfolios. The proposed method can help teachers to perform precise formative
assessment according to the learning portfolios of individual learners in a computer classroom with
Internet assisted web-based learning environment. The inferred learning performance assessment
rules can be applied as a teaching reference guide for teachers and as learning outcome feedback for
learners. Through the feedback mechanism, every learner can understand his/her current learning
status and make suitable learning strategy adjustments during learning processes. Hence, this
mechanism enables learners to become active learners with self-examining their own learning
behavior or outcomes. Additionally, teachers can determine the main factors influencing learning
performance in a web-based learning environment according to the interpretable learning
performance assessment rules. Therefore, teachers can modify their teaching strategies according to
these main factors affecting the learning performance. In the meanwhile, since teachers save much
time in evaluating learning, they can devote more time to teaching and designing courseware. The
experimental results indicated that the evaluation results of the proposed formative assessment
scheme are very close to those of summative assessment results and the proposed factor analysis
scheme can simplify the learning performance assessment rules. Moreover, most learners agreed
that the learning feedback of formative assessment is very helpful to assist mathematics learning,
enhance significantly the learners’ mathematics abilities and promote their learning interests.

References
Baker, F. B. (1992). Item response theory: parameter estimation techniques. New York: Marcel
Dekker.
Behounek L. & Cintula P. (2005). Fuzzy class theory. Fuzzy Sets Systems, 154, 34–55.
Bezdek, J. C. (1981). Pattern recognition with fuzzy objective function algorithms. Plenum Press.
Black, P. (2001). Dreams, strategies and systems: portraits of assessment past, present and future.
Assessment in Education: Principles, Policy & Practice, 8 (1), 65-85.
Bruner J. S. (1996). Toward a theory of instruction. Cambridge, MA, USA: Harvard University
Press.
Chen, C. M., Chen, Y. Y., & Liu, C. Y. (2007). Learning performance assessment approach using
web-based learning portfolios for e-learning systems, IEEE Transactions on Systems, Man and
Cybernetics-Part C (to appear).
Chen, C. M., Lee, H. M., & Chen, Y. H. (2005). Personalized e-learning system using item

31
 response theory. Computers & Education, 44 (3), 237-255.
Chen, C. M., Liu, C. Y., & Chang, M. H. (2006). Personalized curriculum sequencing using
modified item response theory for web-based instruction. Expert Systems with Applications, 30
(2), 378-396.
Chen, F. S., Chang, T. C., Liao, H. H. (2000). The application of the grey relation analysis on
teacher appraisal. IEEE International Conference on Systems, Man, and Cybernetics, 5,
3366-3371.
Chinrungrueng, C., & Sequin, C. (1995). Optimal adaptive k-means algorithm with dynamic
adjustment of learning rate. IEEE Transactions on Neural Networks, 8(5), 729-743.
Delgado, M., Martín, N., Sánchez, D., & Vila, M. A. (2003). Fuzzy association rules: general model
and applications. IEEE Transactions on Fuzzy Systems, 11(2), 214-225.
Deng, J. L. (1982), Control problems of grey systems, Systems and Control Letters, 5, 288-294.
Deng, J. L. (1989), Introduction to Grey System, The Journal of Grey System, 1, 1-24.
Gagnés, R, (1997). The conditions of learning and theory of instruction. New York: Holt, Reinehart
& Winston.
Gath, I. & Geva, A.B. (1989). Unsupervised optimal fuzzy clustering. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 7, 773-781.
Hong, T. P., Kuo, C. S., Chi, S. C. (1999). Mining association rules from quantitative data.
Intelligent data analysis, 3, 363-376.
Hoppner, F., Klawonn, F., Kruse, R. & Runkler, T. (1999). Fuzzy cluster analysis-methods for
classification, data analysis and image recognition. New York: John Wiley and Sons.
Johnson, R. A., & Wichern, D. W. (1998). Applied multivariate statistical analysis. Englewood
Cliffs, NJ: Prentice-Hall.
Kennedy, D. R., Price, J. W. & Susanto , S. (1997). A preliminary study of a fuzzy clustering and
assignment problem-based cell formation algorithms. Proceedings International Conference on
Manufacturing Automation.
Krishna, K. & Murty, M. (1999) Genetic k-means algorithm. IEEE Transactions on Systems, Man
and Cybernetics - Part B: Cybernetics, 29, 433-439.
Kui, C. (2005). Feature relevance learning in content-based image retrieval using GRA. 11th
International Multimedia Modelling Conference (MMM'05), 304-309.
Lankes, A. M. D. (1995). Electronic portfolios: a new idea in assessment, (ERIC Digest
EDO-IR-95-9), Available April 11, 2006 at http://searcheric.org/digests/ed390377.html.
Lee, D., Baek, S., & Sung, K. (1997). Modified k-means algorithm for vector quantizer design.
IEEE Signal Processing Letters, 4(1), 2-4.
Liao T.W. (2001). Classification and coding approaches to part family formation under a fuzzy
environment. Fuzzy Sets and Systems, 122, 425-441.
Lin, C. T., George Lee, C. S. (1996). A neuro-fuzzy synergism to intelligent systems. Singapore:
Prentice Hall.

32
Margaret, H. Dunham (2003). Data mining: introductory and advanced topics. Kentucky: Prentice
Hall.
Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: a
model and seven principles of good feedback practice. Studies in Higher Education, 31(2).
Nuhfer, E. B. (1996). The place of formative evaluations in assessment and ways to reap their
benefits. Journal of Geoscience Education, 44(4), 385-394.
Qinbao, S., Martin, S., & Carolyn, M. (2005). Using grey relational analysis to predict software
effort with small data sets. 11th IEEE International Software Metrics Symposium (METRICS
2005), 1530-1435.
Rahkila, M., & Karjalainen M. (1999). Evaluation of learning in computer based education using
log systems. 29th ASEE/IEEE Frontiers in Education Conference, 8, 12A3/16-12A3/21.
Ramaprasad, A. (1983). On the definition of feedback. Behavioral Science, 28(1), 4-13.
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional
Science, 18(2), 119-144.
Sarkar M, Yegnanarayana B., & Khemani D. (1997). A clustering algorithm using an evolutionary
programming-based approach, Pattern Recognition Letters, 18, 975-986.
Scriven, M. (1967). The methodology of evaluation. In Stake, R. E. (Ed ). Curriculum Evaluation.
121-132. Chicago, IL: Rand MoNallg.
Tessmer, M. (1993). Planning and conducting formative evaluations: Improving the quality of
education and training. London: Kogan Page.
Torrance, H., & Pryor, J. (1998). Investigating formative assessment: teaching, learning and
assessment in the classroom, Buckingham: Open University Press.
Wang, H. F., & Kuo, C. Y. (2004). Factor analysis in data mining. Journal of Computers &
Mathematics with Applications.
Wang, W., Weng, J. F., Su, J. M., & Tseng, S. S. (2003). Learning portfolio analysis and mining in
SCORM compliant environment. The 34th ASEE/IEEE Frontiers in Education Conference,
TC2-17-TC2-34.
Wu, J. L., & Huang, C. H. (2006). A study of clustering algorithm based on central symmetry. The
first conference on Intelligent Living Technology.
Yang, M. S. (1993). A survey of fuzzy clustering. Mathematical and Computer Modeling, 18(11),
1-16.
Zadeh, L. (1965). Fuzzy sets. Information and Control, 8(3):338-353.

33
 國科會補助專題研究計畫項下出席國際學術會議心得報告
日期： 100 年 6 月 15 日
計畫編號 NSC 97－2628－S－004－001－MY3
計畫名稱 網路學習之形成性評量與學習診斷工具發展與研究
出國人員 服務機構 國立政治大學圖書資訊與檔案學
陳志銘
姓名 及職稱 研究所 教授
100 年 5 月 29 日
會議時間 至 100 年 5 月 會議地點 中國杭州市
31 日
(中文)2011 全球華人計算機教育應用大會
會議名稱
(英文)The Global Chinese Conference on Computer in Education
發表論文 (中文)數位圖書館支援問題導向學習
題目 (英文) Effective Problem-based Learning Supported by Digital Library

一、參加會議經過
此次參與全球華人計算機教育應用大會係與國立台灣師範大學科技學院院長洪欽銘教授一起成行，委
由天喜旅行社代為訂購機票及安排住宿事宜，經由旅行社安排，個人與洪教授於 5 月 28 上午由台北松
山機場搭乘復興航空班機飛往中國杭州蕭山國際機場，途中巧遇台北教育大學數位科技設計學系范丙
林教授亦帶領研究生多人參與本次會議，因此就一起搭乘他們預先安排機場接機的小巴士前往杭州市
區，並入住之江飯店，該飯店為大會報到與註冊地點，在稍作休息整理行囊後即展開為期將四天的參
加會議及參訪行程。會議結束後，個人於當地時間 5 月 31 日下午由杭州蕭山國際機場搭乘復興航空班
機飛抵台北松山國際機場返國，過程相當順利且圓滿。
二、與會心得
(一)深入了解全球華人聚焦之數位學習研究現況
此次大會由浙江大學主辦，會議地點位於浙江大學紫金港校區國際會議中心(蒙民傳樓)，並由國際電
機電子工程學會(IEEE)浙江分會贊助，大會也於第一天晚上安排歡迎酒宴，並安排了四場重要的
Keynote Speech，其中我國國立台灣科技大學數位學習與教育研究所的黃國禛教授也是受邀在大會進行
專題演講的學者之一，他發表有關於「行動與無所不在學習的研究趨勢與應用」令人印象非常深刻，
個人認為行動與無所不在學習研究在數位學習的應用潛力非常大，例如行動學習結合無線網路感測技
術、手機行動學習等，在這場演講的啟發下，未來個人決定繼續深入於手機行動學習相關系統的開發
與學習應用研究。會議發表論文中，中國大陸發表的論文佔有極高的比例，這或許是主辦國的因素，
但也可以看出中國大陸也逐漸重視數位學習領域的研究。此外，這次兩岸三地前來參與會議的人數相
當多，可說盛況空前，參加會議人士討論熱絡，堪稱一次學術界的盛會，內容涵蓋極廣，本人參加會
議所獲取之新知對個人之未來研究幫助極大。
(二)結交兩岸三地學者
經由參加會議吸收許多新知識，並於 Coffee Break 時間，結交許多國際學者，有的係邀稿，有的係吸收
會員，有的純粹作學術交流，也有的則閒話家常。在會議其間也遇到許多中國大陸浙江大學、南京大
學及北京師範大學的教授，可能大家語言及文化背景具有不少交集，很自然就熟悉與熱絡起來，除了
藉此瞭解中國大陸這幾年在數位學習領域發展的現況外，也瞭解進幾年中國大陸高等教育及研究發展
近況，大家也交換名片相約於會後繼續保持密切聯繫，並相約參加明年於台灣舉辦的 2012 全球華人計
算機教育應用大會，收穫良多。
(三)體驗杭州市文化及風土民情
個人已累積多次出國發表論文經驗，此次與台灣師範大學科技學院院長洪欽銘教授一同與會發表論
文，也藉由此次會議的空閒時間，參訪杭州西湖十景、浙江大學、靈隱寺、六和塔等名勝，尤其對於
杭州西湖十景中的蘇堤春曉，以及張藝謀的導演印象西湖特別印象深刻，也親自體驗杭州市的飲食文
化，是一次成功且難得的經驗，真是不枉此行，個人覺得獲益良多。
三、考察參觀活動(無是項活動者略)
無
四、建議
國際間在數位學習領域的研究進步快速，也已經發展到某種成熟的階段，極需有一些更開創性的研究
議題發展，此次會議有相關於華語文數位學習、腦神經科學與學習等議題被廣泛討論。近幾年來全世
界掀起了一股華語文學習熱潮，因此如何透過數位學習發展更多支援有效華語文學習的研究，個人認
為這是一個非常具有發展潛力的研究方向，我們應該掌握自己在華語文的優勢，未來成為主導發展華
語文數位學習理論與應用的學術重鎮，更藉由產學媒合方式促使國內學術界與業界結合，發展華語文
數位學習產業。此外，目前腦神經科學的研究隨著生理感測儀器的成熟發展，而有了進一步結合數位
學習的新契機，例如利用腦波(EEG)偵測儀器針對學習者的認知歷程進行研究，以及設計大腦訓練之遊
戲與課程等研究，個人認為未來將成為熱門研究領域，國內學者應可積極朝此方向發展。
五、攜回資料名稱及內容
此次參加會議，大會準備了大會議程表、論文集、光碟及許多明年相關國際研討會的 Call for Paper
資料，個人也攜回台灣作為明年參加會議的參考。例如 2012 年的 The Global Chinese Conference on
Computer in Education 、2012 IEEE International Conference on Advanced Learning Technology 及 2011
年 IEEE International Conference on Systems, Man, and Cybernetics 等，這些即將舉辦的國際研討會將
作為個人明年出國發表論文的目標。另外個人也攜回研討會出版的論文集及光碟一片，許多論文對
於個人未來的研究極具參考價值。
 Acceptance Notification

Mar. 26th , 2011

Dear Author,

Congratulations! It is a great pleasure for us to inform you that your paper:

Paper ID: AP32628

Author: Chih-Ming Chen, Chin-Ming Hong
Paper Title: Effective Problem-based Learning Supported by Digital Library

has been accepted for presentation at the 15th Global Chinese Conference on Computers in Education
(GCCCE2011).

Please finish all registration procedures before April 30th,2011 by the Registration Instructions.

In the past 14 years, GCCCE has been held respectively in Guangzhou, Hong Kong, Macau, Singapore, Chung-
Li, Beijing, Nanjing, Hawaii, Michigan and Taipei. It has now been recognized as a major gathering for all the
experts around the world. You are warmly welcome to participate in GCCCE2011 in the gorgeous city—
Hangzhou, China!

For more information, please visit our website: http://www.gccce.org

Thanks for your support again!
Best regards!

GCCCE2011 Organizing Committee

Zhejiang University, China

Effective Problem-based Learning Supported by Digital
Library
Ching-Pu Chuang Chih-Ming Chen Chin-Ming Hong Yu-Ju Lin
Department of Applied Graduate Institute of Library, Department of Applied Graduate Institute of Library,
Electronics Technology, Information and Archival Electronics Technology, Information and Archival
National Taiwan Normal Studies, National Chengchi National Taiwan Normal Studies, National Chengchi
University, University, University, University,
[email protected] [email protected] [email protected] [email protected]

Abstract—This study mainly assesses the differences between scaffold that, in a timely manner, offers learning support for
learning performance and the satisfaction of learners who use solving a target problem in a designed problem-oriented
digital resources in Taiwan libraries’ history digital library learning process, and integrates curriculum, learning resources,
(organized digital resources) and the Google search engine ICT, and real situations. Teachers can support learners
(unorganized digital resources) in problem-solving learning for the
investigating questions, and learners can share their research
same subject via the problem-based learning (PBL) mode. The
study explores the advantages and characteristics of using digital results with others via the collection and their analysis of
archives to support PBL and offers suggestions that are helpful information. This not only enhances learner motivation and
when using digital archives to support e-Learning. deep learning, but also improves the chances of people to
interact, solve problems, and use ICTs via the Internet
Keywords-Digital archive, digital library, e-Learning
environment. In the PBL model, however, only teachers digest
supported by digital archives, problem-based learning
and provide resources, and learners seek resources in the vast
Internet world. That is, this model has some critical problems
I. INTRODUCTION
that need to be solved. First, learners must expend effort in
Via digitization and the development of networks, library seeking, digesting, and ordering resources. Second, learners can
processes and services have changed markedly. Not only has easily become lost in hyperspace when seeking resources on the
convenience increased, but digitization has proved useful for Internet. Finally, after finding suitable resources, students have
long-term preservation of information [1]. This new form of problems to justice quality of resource. In a manner, digital
storing information uses a large amount of information libraries, which are authoritative, organized and effective, have
technology, and has facilitated the development of so-called potential benefits that support PBL.
digital libraries with digital collections. A digital library has This study presents an innovative learning mode that
powerful and efficient functionalities for content management integrates the PBL mode and digital libraries that have
(acquisition, storage, indexing, access, and maintenance), organized digital resources to overcome these problems. Based
considerable metadata for content enrichment and structuring, on PBL with the support of organized digital resources in
as well as services for effective content searches, access, digital libraries, this study investigates whether students process
annotation, filtering, and dissemination [2]. Due to the richness knowledge better in PBL when using organized digital
of structured digital collections, the use of digital repositories resources from digital libraries than when using unorganized
for educational purposes has garnered the attention of digital resources. This study uses resources from organized
researchers in the fields of computer science, library science digital collections stored in the Taiwan libraries’ history digital
and education [3][4][5][6]. The content in digital collections is library (TLHDL) and unorganized digital resources accessed
approved, organized, completed by experts as valuable through the Google search engine to determine whether
resources for e-Learning. Hence, how to support e-Learning organized and structured digital resources have more benefits to
using resources in digital collections is an issue that warrants learners than unorganized digital resources. This research offers
in-depth research. Saeed [3] indicated that digital library an in-depth understanding of digital libraries and digital
services are an essential component of a quality e-Learning collections that support e-Learning, and helps to clarify further
system, and the growth in e-Learning, in which education is the position of digital libraries or digital collections in current
delivered and supported through computer networks, has raised e-Learning environments. Research results can serve as a
new research issues for library services. reference guide when developing digital libraries or digital
Problem-based learning (PBL) is a learning model in which resources that support e-Learning, and increase the value of
students solve problems associated with learning using online digital collections.
resources [7]. Notably, PBL provides learners with a learning
II. PROBLEM-BASED LEARNING SUPPORTED BY TLHDL. If necessary, learners can directly remit the data stored
DIGITAL LIBRARY in the ‘Notebook’ to the PBL system to organize their reports
for the solving the target problem during the practical problem-
This section describes the proposed learning environment
solving stage.
that integrates PBL with the TLHDL to support effective PBL.
Next, this study briefly explains how the TLHDL provides a
well-structured and organized IA to support PBL. Figure 3
A. The proposed problem-based learning procedure shows the homepage of the TLHDL. The homepage lists all
archived digital collections associated with Taiwan’s library
For problem-solving processes, Amiable [8] proposed a history from 1989 to the present, and simultaneously provides
learning procedure that has five stages: (1) task presentation; (2) metadata and full-text search tools with basic and advanced
preparation; (3) idea generation; (4) idea validation; and, (5) search functionalities. Figure 4 presents the content display and
outcome assessment. West [9] proposed a cyclical model that assisted navigation bar for showing digital content related to
has four stages: (1) recognition, (2) initiation, (3) subject content. Figure 5 shows the designed information
implementation, and (4) stabilization. Isaksen, Treffinger and architecture of the TLHDL for an archived digital item. To stop
Dorval [10], in their book, Creative Approaches to Problem learners for losing information in hyperspace, the designed
Solving, introduced numerous problem-solving concepts and navigation bar directs learners such that they know where they
methods. This study adopts their concepts and methods and, in are in the TLHDL. Moreover, a second navigation bar shows
considering learning task characteristics, proposes a PBL the digital items related to the current digital item a learner is
procedure that has four major learning stages when solving a reading. Thus, it can direct learners to other content. To
problem: (1) identifying the problem and situation; (2) improve the efficiency of reading digital content, an internal
designing the problem-solving method; (3) solving the problem; bookmarker helps learners find a target paragraph rapidly in an
and, (4) reflecting on the process and result. Furthermore, these archived digital item presented on a Web page. Showing the
four stages were further summarized as a “cognition-action- files related to a digital item is convenient for learners when
reflection” (i.e., knowing, doing, and thinking) mental process. downloading additional information to be read later.
Figure 1 displays the proposed cognition-action-reflection
mental process and the corresponding instructions and learning
activities supplied by the teacher and learners for solving a
problem utilizing digital resources in the TLHDL.
Teacher Learners

Device questions to guide Task 1 Identify the problem and Think and answer the
learners to think situation questions
Cognition

Direct learners to use Use resources to find the

arranged resources Task 2 Design the method
problem-solving method
Action
Ask learners to solve the Execute the determined
. Task 3 Solve the problem
target problem problem-solving method

Device questions to guide Task 4 Reflect on the process and

result
Reflect on the questions Figure 2. The user interface that integrates PBL with Taiwan libraries’ history
learners to reflect
Reflection digital library
Figure 1. The proposed “cognition-action-reflection” mental process for
Search
problem-solving learning interface

B. Integrating the problem-based learning system with Taiwan

The list of
libraries’ history digital library to support problem–solving digital
collections
learning
To retrieve digital resources in the TLHDL, this study
combines the PBL system with the TLHDL into an integrated
learning interface that supports PBL. Figure 2 shows the user
interface combining PBL and the TLHDL. In this user interface,
the system menu provides some assistive learning tools,
including a discussion board, online chat room, message center,
and guidance wizard, to support PBL. Furthermore, the PBL
system also uses the ‘Notebook’ function to assist learners in
Figure
organizing and storing their data obtained by searching the 3. The homepage of the developed Taiwan libraries’ history digital library
 to students from the other departments, these students have
more opportunities to learn about topics related to libraries.
Moreover, freshmen were chosen for this experiment because
this study attempted to avoid the effect of prior knowledge on
Picture Second the final experimental result. Since these freshmen are just
navigation
bar getting involved in this field, they tend to be less familiar with
library history and other related knowledge than senior students;
Content
hence, this study assumes that these freshmen in the library
science field have an interest in library history.
In addition, this study requires each student to finish a
report on the topic entitled “the development of Taiwan’s
libraries from the reconstruction period to the present” at home.
The purpose is to avoid mutual interference among learners and
Figure 4. The content display and assisted navigation bar for showing related provide learners with a need to login onto the PBL learning
digital items with current content
platform anytime and anywhere. Importantly, to analyze
effectively the effects of organized and unorganized resources
Second
on student performance during PBL, this study set up a control
Navigation
bar
navigation
bar mechanism that did not allow control students to enter the
TLHDL. That is, the TLHDL has a login interface and only
allows experimental group learners with a password to search
Related for information for solving a problem assigned by the instructor.
files

Internal
IV. EXPERIMENTAL ALALYSIS
bookmarker

This study attempts to determine whether a difference exists

Content
in learning performance when organized and unorganized
resources are used as learning support with the same learning
model and explorative topic in the proposed PBL processes.
A. Statistical analysis of the influence of organized and
Figure 5. The designed information architecture in the developed Taiwan
libraries’ history digital library for an archived digital item unorganized resources on overall learning performance in the
proposed PBL processes
III. EXPERIMENTAL DESIGN
(1) Descriptive statistical data analysis
This study compares and explores the differences in student
learning processes and performance when using digital This study first analyzes the descriptive statistics (pretest
resources organized differently in a PBL environment. In total, and posttest scores) for the experimental and control groups.
43 learners from the Department of Information and Library The pretest scores represent learner prior knowledge related to
Science at Tamkang University participated in the experiment. library history; thus, it can be obtained from the PBL process at
These learners were assigned to two groups. The experimental the cognitive (i.e., knowing) learning stage. The posttest score
group (n=23) used organized and arranged digital archival is the average score for a learner’s report at the action and
resources (organized resources), while the control group (n=20) reflection (i.e., doing and thinking) stage. TABLE I presents the
used open resources not arranged or organized by experts that pretest and posttest data. The difference in average pretest
can be searched using any general search engine (unorganized scores for the experimental group and control group is 0.138,
resources). This study adopts the statistical method of which is a small value. However, after learning, the difference
quantitative analysis to analyze experimental results. In addition in average scores is 1.513; the posttest score for the
to collecting the instructor evaluation and learner questionnaire experimental group is slightly higher than that of the control
responses, this study observed student learning processes. group.
Through this deliberate procedure, this study hopes to make an
accurate comparison and analyze the value of digital archival
resources in PBL.
The reason why this study chose 43 library science students
as the experimental group is because these students are studying
library sciences and are interested in library history. Compared
 resources the other group used. Based on the average value of
TABLE I. THE DESCRIPTIVE STATISTICAL DATA OF LEARNING
satisfactory degree, learners in experimental group had a higher
PERFORMANCE IN PRETEST AND POSTTEST FOR THE LEARNERS degree of satisfaction with the completeness, reliability, and
WHO UNDERWENT PROBLEM-BASED LEARNING PROCESSES applicability of data and overall resources than the control
SUPPORTED BY DIGITAL RESOURCES WITH DIFFERENT group. However, for speed of accessing information and the
ORGANIZATIONS
presentation of information, the degree of satisfaction for the
Number Pretest Posttest control group was higher than that of the experimental group.
Group of
Learners Mean
Std.
Mean
Std. To understand further whether the degree of satisfaction of
Deviation Deviation the experimental group was significantly different from that of
Experimental
Group
23 83.5362 0.903 85.913 1.15771 the control group, this study analyzed via a questionnaire by
Control independent sample t-test. Except for the presentation of
20 83.65 1.33 84.3995 1.01505 information (classification/list) and completeness of data
Group
(retrieving result), significant differences exist in speed of
To determine whether the two different resources accessing information, reliability of data (retrieving result),
influenced learning performance, this study applied the applicability of data (retrieving result) and overall resources
interdependent sample t-test to pretest and posttest scores for between the two groups. Analytical results also indicate that
the experimental group and control group, and to learning learners have a higher degree of satisfaction in terms of
performance for the two groups. reliability and applicability of resources and with overall
(2) The interdependent t-test applied to pretest and posttest resources when using organized resources (i.e., resources from
scores for the experimental group and control group the TLHDL) than when using unorganized resources. However,
when using unorganized resources (i.e. resources accessed
The interdependent t-test results show that the learning through Google), learners have a higher degree of satisfaction
performance of the experimental and control groups was with the speed in accessing information than TLHDL with
significantly different (Sig.=0.000, 0.017<.05) based on pretest organized resources.
and posttest scores. However, if a relatively more strict
statistical test is applied, the experimental group has more C. Statistical analysis of the influence of organized and
significant difference than the control group (Sig.=0.000<.01). unorganized resource on individual learning performance in the
Therefore, this study confirms that the experimental group, three proposed PBL mental processes
which used organized resources to support learning, has better To determine whether a difference in learning performance
learning performance than the control group. Notably, the two exists between the two groups while performing the three
resources had obvious effects on PBL learning performance. proposed PBL mental processes to solve the same problem, this
(3) Analysis of covariance based on pretest and posttest scores study applied the independent sample t-test. Analytical results
show that the difference between the experimental and control
groups increased in the ‘action of second half phase’ learning
To determine whether the different resources influenced
stage. No significant difference existed in the cognition learning
learning performance, this study applied analysis of covariance
stage between the two groups; however, this begins changing in
to the learning effect for the two groups. The analytical result
the action and reflection learning stages and shows great
shows that the control variable has an independent influence on
difference in the ‘action of second half phase’ learning stage.
the observational variable. Moreover, the average sum of
This finding demonstrates that compared to learners conducting
squares for the experimental group and control group is 25.886;
PBL with unorganized resources, those receiving organized
the contrasted F-test and salience values are 24.899 and 0.000,
support performed significantly better in the action and
respectively. This analytical result reveals that two groups using
reflection learning stages, especially in the action stage. This
unorganized or organized resources receive significant
shows that organized resources help learners solve problems.
difference on learning performance, and the experimental group
is superior to the control group in terms of learning V. CONCLUSIONS
performance.
The primary purpose of this study is to discuss and compare
B. Analysis of learner satisfaction with organized and differences in learning progress and the effects of differently
unorganized resources organized digital resources in PBL. First of all, the study
confirmed that digital archival resources facilitate better
After using different digital resources during different
learning than open digital resources stored on the Internet in
learning stages in the proposed PBL mode, learners filled out
supporting problem-based learning. Secondly, problem-based
questionnaires about their degree of satisfaction with different
learning supported by digital archival resources was rated
resources. We assumed learners were not aware of the
higher in terms of learning satisfaction from learners than
problem-based learning supported by open digital resources on
the Internet. Moreover, digital archival resources provide more
learning benefits than open resources on the Internet,
particularly during the action learning phase in the proposed
problem-based learning procedures. Importantly, digital
archives should improve searching functions to increase search
performance while supporting problem-based learning.
REFERENCES
[1] K. Russell, “CEDARS: long term access and usability of
digital resources–the digital preservation conundrum,”
Ariadne (Web version）, 18, 1998. Retrieved September
1, 2010, from http://www.ariadne.ac.uk/issue18/cedars/
[2] M. Fuchs, C. Muscogiuri, C. Niederée, C., and M.
Hemmje, “Digital libraries in knowledge management: an
e-learning case study,” International Journal on Digital
Libraries, vol. 4, issue 1, pp. 31-35.
[3] R. S. Saeed, “How digital libraries can support e-
learning,” The Electronic Library, vol. 24, no. 3, pp. 389-
401, 2006.
[4] H. C. Chu, G. J. Hwang, S. X. Huang, and T. T. Wu, “A
knowledge engineering approach to developing e-
libraries for mobile learning,” The Electronic Library, vol.
26, no. 3, pp. 303-317, 2008.
[5] G. Marchionini, and H. Maurer, (1995), “The roles of
digital libraries in teaching and learning,”
Communications of the ACM, vol. 38, no. 4, pp. 67-75,
1995.
[6] B. B. Marshall, H. Chen, R. Shen, E. A. Fox, E.A.
“Moving digital libraries into the student learning space:
The GetSmart experience,” ACM Journal of Educational
Resources in Computing, vol. 2, no. 1, pp. 1-20, 2006.
[7] M. P. Chen, and J. C. Yen, “Exploring the effects of
design perspective on web-based learning performance,”
The ED-MEDIA 2003-World Conference on Educational
Multimedia, Hypermedia & Telecommunications,
Honolulu, Hawaii, USA, June 23-28, 2003.
[8] T. M. Amabile, “A model of creativity and innovation in
organizations,” Research in Organizational Behavior, vol.
10, 123-167, 1988.
[9] W. W. West, Student perceptions of teaching effectiveness,
Texas: Texas A&M University, 1990.
[10] S. G. Isaksen, D. J. Treffinger, and K. B. Dorval, K. B.,
Creative approaches to problem solving: A framework for
change (2nd. Ed.), Williamsville, NY: Creative Problem
Solving Group- Buffalo, 2000.
 國科會補助計畫衍生研發成果推廣資料表
日期:2011/10/15

計畫名稱: 網路學習之形成性評量與學習診斷工具發展與研究
國科會補助計畫 計畫主持人: 陳志銘
計畫編號: 97-2628-S-004-001-MY3 學門領域: 資訊教育一電腦輔助教學

無研發成果推廣資料
 97 年度專題研究計畫研究成果彙整表
計畫主持人：陳志銘 計畫編號：97-2628-S-004-001-MY3
計畫名稱：網路學習之形成性評量與學習診斷工具發展與研究
量化 備註 （ 質 化 說 明 ： 如 數 個
實際已達 預期總達 本計畫 計 畫 共 同 成 果、成 果 列 為
成果項目 成數（被接 成數(含實 實際貢 單位 該 期 刊 之 封 面 故 事 ...等 ）
受或已發 際已達成 獻百分
表） 數) 比
期刊論文 0 0 100%
研究報告/技術報
0 0 100% 篇
論文著作 告
研討會論文 0 0 100%
專書 0 0 100%
申請中件數 0 0 100%
專利 件
國內 已獲得件數 0 0 100%
件數 0 0 100% 件
技術移轉
權利金 0 0 100% 千元
碩士生 0 0 100%
參與計畫人力 博士生 0 0 100%
人次
（本國籍） 博士後研究員 0 0 100%
專任助理 0 0 100%
1.Chih-Ming Chen*, and
Ming-Chuan Chen, ’Mobile
Formative Assessment Tool
Based on Data Mining
Techniques for Supporting
Web-based Learning, ’
Computers &amp； Education,
vol. 52, issue 1, pp.
256-273, 2009. [SSCI, SCI,
EI 收錄] (國科會計畫編號
國外 期刊論文 2 2 100%
NSC97-2628-S-004-001-MY3)
篇 2.Chih-Ming Chen*, ’
論文著作 Personalized E-learning
System with Self-Regulated
Learning Mechanisms for
Promoting Learning
Performance, ’ Expert
Systems with Applications,
vol. 36, pp. 8816-8829,
2009. [SSCI, SCI, EI 收錄]
研究報告/技術報
3 3 100%
告
研討會論文 6 6 100%
專書 0 0 100% 章/本
 申請中件數 0 0 100%
專利 件
已獲得件數 0 0 100%
件數 0 0 100% 件
技術移轉
權利金 0 0 100% 千元
碩士生 6 6 100%
參與計畫人力 博士生 0 0 100%
人次
（外國籍） 博士後研究員 0 0 100%
專任助理 0 0 100%
1.榮獲 99 年度行政院國科會吳大猷先生紀念獎。
其他成果
( 無 法 以 量 化 表 達 2.榮獲國立政治大學 97、98、99 研究優良獎。
之成果如辦理學術 3.榮獲行政院 99、100 年度獎勵特殊優秀人才獎助。
活動、獲得獎項、
重要國際合作、研
究成果國際影響力
及其他協助產業技
術發展之具體效益
事項等，請以文字
敘述填列。)

成果項目 量化 名稱或內容性質簡述
測驗工具(含質性與量性) 1 形成性評量模糊規則
課程/模組 0
科 電腦及網路系統或工具 具支援自律與形成性評量學習機制之個人
教 化網路教學系統(Personalized E-learning
處 1 System with Self-regulated Learning and
計 Fprmative Assessment Mechanism, v 1.0)
畫
2.
加
教材 1 國小數學分數單元多媒體教材共計 34 單元
填
項 舉辦之活動/競賽 0
目 研討會/工作坊 0
電子報、網站 0
計畫成果推廣之參與（閱聽）人數 0
 國科會補助專題研究計畫成果報告自評表

請就研究內容與原計畫相符程度、達成預期目標情況、研究成果之學術或應用價
值（簡要敘述成果所代表之意義、價值、影響或進一步發展之可能性）、是否適
合在學術期刊發表或申請專利、主要發現或其他有關價值等，作一綜合評估。

1. 請就研究內容與原計畫相符程度、達成預期目標情況作一綜合評估
■達成目標
□未達成目標（請說明，以 100 字為限）
□實驗失敗
□因故實驗中斷
□其他原因
說明：
2. 研究成果在學術期刊發表或申請專利等情形：
論文：■已發表 □未發表之文稿 □撰寫中 □無
專利：□已獲得 □申請中 ■無
技轉：□已技轉 □洽談中 ■無
其他：（以 100 字為限）
研究成果分別 Expert Systems with Applications[SSCI, SCI, EI 收錄]及 Computers
&amp； Education[SSCI, SCI, EI 收錄]各發表一篇期刊論文。
3. 請依學術成就、技術創新、社會影響等方面，評估研究成果之學術或應用價
值（簡要敘述成果所代表之意義、價值、影響或進一步發展之可能性）（以
500 字為限）
利用學習者在網路學習過程中學習系統所記錄之學習者日誌檔案，利用資料探勘的方法歸
納出學習成效評量的模糊規則，這些用於評量的模糊規則可以作為教師瞭解影響學習的因
素外，也可以利用這些形成性的評量規則在未實施實際測驗的情形之下，正確的預測學生
學習成效。在形成性評量領域為非常具有創新及突破的方法，目前發表的兩篇論文在 ISI
Web of Knowledge 的引用率正穩定增加中。