CHAPTER 12 EVALUATION
Pauline Robinson, in his books, defines evaluation as the discovery of the value for some
purposes. Other clearer definition is proposed by Alderson and Murphy who write that
evaluation is concerned with describing what is there and placing some value on what is found.
Evaluation
ESP has developed in response to certain pressures. Developments in the theoretical
bases of language teaching indicated a need to pay more attention to the individual learner. The
world of commerce and technology were producing host of people with specific language
learning needs at that time. A demand was generated as a result for courses which would equip
particular learners with necessary skills to carry out particular tasks in English.
Like any other language teaching course, ESP has requirements which are brought sharply into
focus by the fact that ESP has specified objectives. ESP learners and sponsors are investors in
ESP course and they want to see a return on their investment of time or money. The managers
of the ESP course are accountable to these investors. This accountability has produced a
demand for more and better evaluation procedures. Two levels of evaluation have thus been
brought into prominence:
A. Learner Assessment. In any language course, there is a need to assess students’ performance
at any strategic point. i.e. at start or at end. Learner Assessment has great importance in ESP
because it deals with the ability to perform particular communicative tasks. The result of this
kind of evaluation enables sponsors, learners and teachers to decide whether and how much
tuition is required.
B. Course Evaluation Levels of Evaluation. Evaluating ESP course helps to establish whether it
is meeting its aims or not. Course evaluation also plays a useful social role, by showing the
various parties involved (teachers, learners, sponsors etc.) that their views are important. There
are four aspects of ESP course evaluation to be considered: What should be evaluated? How
can ESP courses be evaluated? Who should be involved in the evaluation? When (and how
often) should evaluation take place?
These two forms of evaluation are not always distinct. Evaluation f learner reflects not
just learners’ performance but to some extent the effectiveness r otherwise of the course too.
Evaluation of the learners is unlikely to indicate exactly where a fault lies, but it will at least
indicate the existence of a fault somewhere. More price diagnostic evaluation can then be used
to trace the fault.
In this sense, then, both course and learner evaluations have a similar function in
providing feedback on the ESP course. However, each type of evaluations also has other
purposes and procedures. Thus, while bearing in mind the similarity of role a feedback, we shall
consider those two kinds of evaluation separately. Placement Tests
The aim of placement tests is to determine the learners’ state of knowledge before the
ESP begins. Placement tests are used to place the learners in the ESP course most suited to
student needs. If a learner is already proficient in the skills required, no further tuition is
required. The placement tests is diagnostic indicating how far and in what ways the learner falls
short of the proficiency level. A good placement test should reveal positive factors. It should
show not just what the learner lacks but also what potential for learning can be exploited in ESP
course.
Achievement tests
Achievement test is the least problematic, because it is usually internal to the course and does
not have to conform to external influences. It is should reflect the nature and content of the
course itself (Alderson and Hughes, 1981). This kind of test, usually ESP teacher is most likely
to have to construct. In constructing a good ESP achievement test you should follow the basic
principles as you would for constructing any test. For example:
a. Test what you can assume the learners have learn, but not necessarily the same as what you
have taught.
b. The test should test what you actually want to test.
c. Avoid bias in the test.
.Example of Achievement Test
TEST
The diagram below shows the flow of coolant through a refrigeration system. Study the diagram
and then write a description of how the coolant circulates through the system.
The features of this test that make it a suitable means of evaluating the learner’s understanding
of the materials are:
a. It involves production as well as understanding.
b. It is an integrated task rather than a set of discrete-points test focusing on particular sub-skills,
for example, ‘linking clauses’.
c. The content is of a similar nature to that of the unit: it is concerned with the movement of a
fluid through an enclosed system.
d. The subject matter is probably already known to the learners.
e. It does not require knowledge of subject-specific vocabulary of the refrigerator system is also
given.
f. It tests written production.
The characteristics of learner assessment:
1. In any language course, there is a need to assess students’ performance at any strategic points.
2. Learner assessment has a great importance in ESP because it deals with the ability to perform
particular communicative task.
3. The results of this kind of evaluation enable sponsors, learners and teachers to decide whether
how much tuition is required.
4. The evaluation of the learners reflects not just the learners’ performance but to some extent the
affectiveness or otherwise of the course too.
The facility to assess proficiency is, therefore, central to the whole concept of ESP. in spite of
the importance, which ESP should logically give to assessment of students’ performance; there
is a general lack of discussion or guidance on ESP testing. In ESP, there are 3 types assessment:
1. Placements test: these are used to place the learners in ESP course most suited to their needs.
The placement normally comes at the beginning of the course.
2. Achievement test: these test how well the learners are keeping up with the syllabus and can be
administrated anytime through the course.
3. Proficiency test: these assess whether or not the students can cope with the demands of a
particular situation. For example, study at university or reading technical manual.
These different types of test do not necessarily vary in terms of content. Their differ in terms of
their initial function. There is a general lack of discussion or guidance on ESP testing:
1. The lack does not imply that there are no tests available in ESP.
2. There is no shortage of available examinations in ESP.
3. Proficiency Test
Davies and West (1984) in the introduction to their Guide to English Language
Examinations, identify the primary purpose of language testing in the eighties as 'proficiency
testing designed to assess whether candidates will be able to perform the language tasks
required of them'. Such tests, they say, are primarily criterion-referenced. There is
no pass/fail distinction, but rather a scale of degrees of proficiency in the task. An example of
such a scale is that used for the British Council's EL TS test which is used to assess a
candidate's ability to study at an English-medium institution of higher education:
The move towards proficiency testing fits very neatly with the concept of ESP, which is
crucially concerned with enabling learners to perform certain language tasks. Proficiency tests
for specific purposes should, therefore, be able to give a reliable in indication on whether a
candidate is proficient enough to carry out the tasks that will be required. Yet although specific
language proficiency tests seem to be a logical extension of the ESP principle, they remain
problematic. Why is this so?
a) As already noted, proficiency tests are primarily criterion-referenced, and therein lies the
problem: what should the criteria be? Should they vary with different sub¢ areas? What skill
and know ledge enable someone to perform particular tasks? How specific are those skills and
that knowledge to any particular task? This argument might be countered by having real-life
tasks as tests. However, this is rarely possible to set up in practice and so still leaves us with the
problem of determining what features are crucial to the real life performance.
b) How specific is specific? Can a test in Engineering, for example, be a valid indicator for all
branches of Engineering - marine, electronic, civil, electrical, mechanical, aeronautical etc?
Alderson and Hughesc (1981) make this point in considering the British Council's ELTS test:
'which EL TS modular test, for example, out of the six presently available (Physical, Life,
Social and Medical Sciences, Technology and General Academic) should be taken by a student
of Urban and Regional Studies, whose course will include Law and Economics courses as well
as courses in Technology? ... What about the (frequent) cases of students who have a
background in Physical Sciences, who are coming to the UK to do a (to them) novel course in
Technology? Do they take the Physical Science test or the Technology test?'
Until we know more about what enables a language user to perform particular
communicative tasks, we must view proficiency tests as only approximate guides.
COURSE EVALUATION
ESP course is like any courses that its existention is justified as a particular educational
need. Through evaluation, ESP course is actually fulfilling the needs. The sponsor also needs
clear information to be supplied the suitable course and as well further investment and support
on the result.
Establishing the meeting of aims, evaluating the ESP course is the solution. From many
revisions of course, the information is gotten and also help to guide the design of similar
courses. Showing the various parties involved, course evaluation is as a social role. Teacher,
learners, sponsors etc are the parties involved that their opinions are important.
There are four aspect of ESP based on Alderson and Waters (1983):
a. What should be evaluated?
b. How can ESP courses be evaluated?
c. Who should be involved in the evaluation?
d. When (and how often) should evaluation take a place?
1. What should be evaluated?
Everything that is significant must be evaluated. However, there are two important things
that must be constraints:
a. The ability of the teacher collect the information
b. The ability of the teacher to use the information for evaluating. Remember, it can be annoy
everyone if you collect the information in detail. It is possibly at inconvenience, time and
expense, and then doing a little or nothing with it. Then the colleagues may not cooperative.
ESP course is to meet two main needs of the learner. The first one is the need as language
learners and second is the need as language users. Through the aims of ESP course, its
evaluation is concerned with assessing the extent of their needs.
There are two basic questions before assessing the students.
a. Is the course fulfilling the language learners’ needs?
b. Has the course fulfilled or is the course fulfilling language users’ needs?
Then, you must to find the answer “yes”. The next question that must be asked is
What areas of need are not being/have not been fulfilled?
After getting the answer of the areas of the course, you are focusing attention to
identifying the sources of these problems.
a. Were the unfulfilled needs identified during the course design process? If not, why not?
b. How can the course be changed to take these needs into account? If these needs were identified
during the course design process, why are they not being / have they not been fulfilled? Is the
fault in the syllabus, the materials, the teaching and learning techniques, the testing procedures,
logical/administrative arrangements and the course system?
2. How can ESP courses be evaluated?
In theory, there are many ways to evaluate ESP course, from simulation to suggestion boxes.
However, in practice the most techniques to evaluate ESP course are test results, questionnaires,
discussion, interview, and informal means (comment, casual chats, etc.)
The suitable techniques are used base on the teaching situation best. For gathering the
evaluation information is in the first stage process. Then, collected and summarized it if it is
extensive. It needs to be discussed with all interested parties and conclusion drawn. Finally, it is
included in evaluation report as a basis further discussion and decision making.
3. Who should be involved in the evaluation?
In every colleague will vary about who involved in ESP course evaluation. It is likely in
practice there are ESP teaching institution, the ESP teachers, the learners, and the course
sponsors. Evaluation is concern with people evaluation of value and the opinions. It is related
with their interest and concern. Getting learners’ views are more difficult because they do not
give real view. Learners thought that the views will influence their assessment. Therefore, the
views of learners could support the ESP course and useful for feedback. The way of getting
information by asking who you ask and how you ask is important.
4. When (and how often) should evaluation take a place?
According to Hutchinson and Water in 2008, the most appropriate time to evaluate the
course are:
a. In the first week of the course
b. At regular intervals throughout the course.
c. At the end of course.
d. At the end of the course in each meeting, because it’s effective to know how
well the course achieved the goal.
It’s difficult to prescribe how often course evaluation should be done. It’s need sensitivity
and respond in evaluating the course. Besides, it’s also depend on the characteristics of the
individual teaching situation.
Conclusion
In this section we have considered some of the mechanics of evaluating learner
performance, and in particular we have outlined some of the problems associated with specific
purpose testing. The value of tests depends primarily on how they are used. Teachers and
learners need, first of all, a positive attitude to tests. Since test are here to stay (at least until we
can think of a better alternative), it is no good closing our eyes to them and hoping they will go
away. Rather, tests should be recognized for the important role they play in the teaching-
learning process, and every effort made - by teachers and learners - to get the most out of them.
We need to see test results less as an end in themselves and more as the starting point for
genuine negotiation and interaction between the teacher and the learners, and among the
learners themselves.
Evaluation can fulfill two functions – assessment and feedback. Assessment is a matter
of measuring what the learners already know. But any assessment should also provide positive
feedback to inform teachers and learners about what is still not known, thus providing important
input to the content and methods of future work. The fact that most ESP learners are adults,
there is no reason why the educational use of tests should not be taken a stage further, with
learners becoming increasingly involved in providing feedback to themselves and their peers,
and sharing in decisions about the most appropriate procedures for evaluating their progress.
