Tools and Techniques of assessment

TOOLS AND TECHNIQUES OF ASSESSMENT OF LEARNING

 Characteristics of Good measuring Instrument:

Essential Qualities:

    Like a good tool or a machine, a good measuring instrument must meet certain minimum requirements before it is deemed to be reliable. What are the essential characteristics and requirements of a good measuring instrument? These are planning, validity, reliability, objectivity, discriminating power, adequacy, practicality, comparability and utility. All of these qualities are criteria that must be given serious thought when a teacher selects a measuring instrument.  

     Any good measuring too should possess certain characteristics. So, evaluation technique is judged for its adequacy efficiency and consistency as measuring device on the basis of commonly accepted qualities. These are Validity, Reliability, Objectivity and Usability or Practicability.

1.      VALIDITY: 

         The test should be Valid. It should fulfill the objectives for which it is meant. If our aim is to measure the ability to understand and application of mathematical facts then language difficulty and speed should not come in way. Thus, it should be free from defects that are likely to affect the valid results.

1.         The Validity of a test is defined as “the accuracy, with which a test measures whatever it is supposed to measure.”

2.       Validity is the accuracy with which a test reliably measures what is relevant.

3.       “The efficiency with which a test measures what it attempts to measure”.

The validity of a test is affected by the following factors.

1.       Unclear Directions: When the directions don’t clearly indicate to the pupils the answer will tend to reduce the validity. For example,

Unclear: State whether the following statements are true or false. Give reasons.

Clear: State whether each of the following statements is true or false. If false, give reasons.

2.       Reading Vocabulary: If the reading vocabulary is poor the students fail to reply to the tests item, even if they know the answer. It rather becomes a reading comprehension test for them, and the validity decreases.

3.       Difficult sentence construction: If a sentence is so constructed as to be difficult to understand, students would be unnecessarily confused, which will affect the validity of the test, for example,

          “Carefully select the material to be interpreted so that the interpretation to be made will not be insignificant and non-representative of the course content and non-behavioural objectives.”

4.       Poorly constructed test Items:   Poorly constructed test items reduce the validity of a test .

For example:

Poor: The famous rivers in India are__________.

There are many correct answers to this test items ; e.g., (I) in the north part of India (ii) south Indian rivers?....

5.       Use of inappropriate items: With the help of objective type items, the pupils’ power of organizing matter cannot be judged. By written test items the pupils’ correct pronunciation cannot be judged. The use of such inappropriate items lowers validity.

6.       Medium of expression: English, as the medium of instruction and response for non-English medium students, creates many serious problems. Such tests through the English medium, instead of measuring learning outcomes in a subject, measure primarily the knowledge of English, which ultimately affects the validity of a test.

7.       Difficulty level of items: Too easy or too difficult test items would not discriminate among pupils; thereby the validity of a test will be lowered.

8.       Influence of extraneous factors: Extraneous factors, like the style of expression, legibility, mechanics of grammar (spelling, punctuation, etc.) handwriting, length of the answer, method of organising the matter, etc., influence the validity of a test. If a student gets a higher score because of his ability to draw artistic diagrams in science, the score because of his ability to draw artistic diagrams in science, the score does not tell the whole truth about his achievement in science; hence the tests is not valid. Lack of neatness, while answering attest item in algebraic equations, should not influence the score.

9.       Inappropriate time limit:  In a speed test, if no time limit is given, the results will be invalidated. In a power test, an inappropriate time limit will lower its validity. Our tests have become power tests as well speed tests. Hence, much care should be taken in fixing the time limit.

10.   Inadequate Coverage of Content: If a test covers only 5 to 6 topics out of 20 topics then it fails to measure adequately what it is supposed to measure. Essay type items generally fail to cover a vast portion of the content. Inadequate sampling lowers validity.

Generally in easy type tests the examiner is greatly influenced by the factors such as style of expression, length of the measure, handwriting the method of organization etc. All these factors lower the validity of the test.

11.   Inadequate weightage:  Inadequate weightage to sub-topics or objectives or various forms of questions would call into question the validity of a test.

     Validity is the most important consideration on the selection and use of any testing procedure.

It should always be remembered that------

(a)    Validity is an inclusive term, and includes and is affected by all the other characteristics of the measuring instrument.

(b)   Validity is a matter of degree. It may be high, moderate or low. It does not exist on an all-or-none basis. For example, the result of a Geometry test may have a high validity for indicating logical deductive reasoning, a moderate validity for indicating skill in drawing figures and low validity for indicating computational skill.

(c)    Validity is specific rather than general. A test may be valid for one specific purpose but not for another, valid for one specific group of students but not for another. For example, an ordinary foot-rule may be----

(i)                  Valid for measuring the length of a segment in millimeters but not valid for measuring the same in fractions of a millimeters;

(ii)                Valid for measuring the length of a room, but not for measuring the distance between Bombay and Surut;

(iii)               Valid for drawing a segment but not for drawing a circle.

An achievement test in mathematics may be--

(i)                  Valid for Std. VIII and not for Std. X;

(ii)                Valid for std. VIII in Karnataka State but not valid for the same standard in Gujarat State;

(iii)               Valid for Std. VIII in an urban area but not valid for the same standard in any rural area.

Types of Validity

    The following four types of validity have been identified and are commonly used in educational and psychological measurements:

(i)                  Content validity (how well our test sample represents the universe of criterion behaviour);

(ii)                Concurrent validity (how closely test scores are correlated with present criterion behaviour);

(iii)               Predictive validity (how well test scores predict future criterion behaviour);

(iv)              Constructive validity (how well our test seems to measure a hypothesized trait.)

2. RELIABILITY:

The reliability of a test may be defined as,

1.       “the degree of consistency among test scores”;

2.       “the degree of consistency with which the test measures what it does measure.”

      If a watch regularly goes 2 minutes ahead of the Indian Standard Time, it may be said that it is not valid, for it does not measure, what it is supposed to measure; but it is reliable in the sense that it consistently goes 5 minutes ahead

    A test score is called reliable when we have reasons for believing it to be stable and trustworthy. Any measuring device provides only very limited data. What we hope for is the fact that a student’s score should be similar under slightly different conditions. If we measure a student’s level of achievement, we hope that his score would be similar under different administrators, using different scores, with similar but not identical items, or during a different time of the day.

     In physical measurement, ordinarily very reliable measures can be obtained. This is true primarily for three basic reasons:

(a)    Physical characteristics can be usually measured directly rather than indirectly;

(b)   The instrument used to obtain the measure is quite precise;

(c)    The traits or characteristics that are measured are relatively stable.

       Psychological and educational measurements are typically much less reliable than physical measurements. A psychological or educational measurement is indirect and is connected with less precise instruments or traits that are not always stable or well-defined. There are many reasons why a pupil’s test score may vary. Some of them are:

(a)    Trait instability:The characteristics we measure may change over a period of time;

(b)   Sampling error: Any particular questions we ask in order  to infer a person’s knowledge may affect his score;

(c)    Administrative error: Any change in direction, timing or amount of rapport with the test administrator may cause score variability;

(d)   Scoring error: Inaccuracies in scoring a test paper will affect the scores.

(e)   Other factors:  Such things as health, motivation, degree of fatigue of the pupil and good or bad luck in guessing may cause score variability.

Methods of Estimating Reliability

     Reliability is usually expressed by a coefficient of correlation which is called the reliability coefficient. The methods used to estimate reliability differ in that they allow different sources of error to show up. Many approaches can be used to estimate reliability; but the more common ones are:

(a)    Test Retest Method;

(b)   The Alternate or Parallel Forms Method;

(c)    The Split Half Method;

(d)   The Rational Equivalence Method.

All these methods are highly statistical; they are; therefore, not very useful to the ordinary classroom teacher.

Factors Influencing Reliability

      The factors that affect the reliability estimates are:

 (i)Method: The method used in obtaining data on reliability affects the reliability coefficient. The    

     method that takes into account both stability and equivalence will tend to give a lower coefficient

    than the other methods.

(ii) Interval:With any method involving two testing occasions, the longer the interval of time  

        between two test administrations, the lower the coefficient will tend to be.

(iii) Test Length:Adding equivalent items makes a test more reliable, while deleting them makes it 

less reliable. For example, a test of 40 items is more reliable than a test of l10 items, and less   reliable than a test of 100items,

   (iv) Speed:A test is considered to be a pure speed test if everyone who reaches an item gets it right,      

             but no one has time to finish all the items. Here, score differences depend upon the number of     

          items   attempted. The opposite of a speed test is a power test. A pure power test is one in which    

         everyone has time to try all items but, because of the difficulty level, ordinarily no one obtains a

         perfect score. Few tests are either pure speed or pure power tests. However, to the extent that a

          test is speeded, it is inappropriate to estimate reliability through the split-Half Method. In that case

        the Test-Retest method or parallel Forms Method should be used.

(v)Group Homogeneity: Other things being equal the more heterogeneous the group, the higher the reliability. In other words, the test is more reliable when applied to a group of students with a wide range of ability than one with a narrow range of ability.

(vi) Difficulty of the Items: Tests in which there is little variability among the scores gives lower reliability estimates than tests in which the variability is high. Too easy or too difficult tests for a group will tend to be less reliable because the differences among the students in such tests are narrow.

(vii) Objectivity of Scoring: The more subjectively a measure is scored, the lower its reliability. Ordinarily objective type tests are more reliable than subjective or essay type test. However, objectivity should not    

     be confused with reliability. A test having a high scoring objectivity, such as True-false test, may be

      quite unreliable as a measuring instrument.

(viii) Ambiguous Wording of Items: When questions are interpreted in different ways at different times by the same students, the test becomes less reliable.

(iX) Inconsistency in Test Administration: Inconsistency in test administration such as deviations in timings, procedure, instruction, etc., fluctuations in interest and attention of the pupils. Shifts in emotional attitude, etc. make a test less reliable.

(X) Optional Questions: If optional questions are given, the same students may not attempt the same items on a second administration; thereby the reliability of the test is reduced.

Relation between Validity and Reliability

(i)                  Validity is sometimes defined as truthfulness while reliability is sometimes defined as trustworthiness. Truthfulness and trustworthiness are the two aspects of what is essentially the same thing, namely, test efficiency.

(ii)                In order that a test should be valid, it must first of all be reliable. If we cannot get a scale to give us a consistent weight measure, we certainly cannot expect it to be accurate. Note, however, that a measure might be very consistent (reliable) but not accurate (valid). A scale may record weights which are 10 gm.  too heavy each time .In other words, reliability is a necessary but not sufficient condition for validity. A reliable test may not necessarily be valid; but a test cannot be valid unless it is reliable.

(iii)               Neither validity nor reliability is an either. They are degrees of each.

(iv)              Since a single test may be used for many different purposes, there is no single validity index for a test. A test that has some validity for one purpose may not at all be valid for another. But for a test a single reliability coefficient may be estimated by statistical procedures.

(v)                Validity includes reliability. A classroom test should be both consistent and relevant; this combination of characteristics is called validity. Thus, validity has two major aspects-reliability and relevance.

3. OBJECTIVITY:

A test is objective when the scorer’s personal judgement does not affect the scoring. Objectivity in a test makes for the elimination of biased opinion or judgement of the person who scores it. A measuring instrument is said to be highly objective if the score assigned by different but equally competent scorers is not affected by the judgement, personal opinion or bias of the scorers.

        In an objective test, items can readily be scored as right or wrong. The true-false type, the alternate response type, the matching type, the multiple-choice type test items are highly objective, while essay type items are highly subjective.

    Objectivity is a pre-requisite of reliability and therefore of validity. Objective judgements are accurate and hence tend to be reliable. But this does not mean that only objective type test items should be used and short-answer or essay type test items should not be used at all. The most important thing is to make the evaluation tool as objective as possible.

     The objectivity of a test can be increased by-----

(a)    Using more objective type test items;

(b)   Making essay type test items more unambiguous, well-constructed, giving specific directions which would establish a framework within which students can operate;

(c)    Preparing a marking scheme or a scoring key;

(d)   Setting realistic standards;

(e)   Asking two independent examiners to evaluate the test and using the average score of the two as the final score.

      The objectivity as well as the reliability and validity of a test may be expressed with the correlation coefficient. The coefficient of correlation between scores assigned to a group of papers by the same examiner on two occasions is called the coefficient of objectivity.

A Test is objective when the examiners judgment does not affect the scoring.  Objectivity in a measuring instrument means the degree to which equally competent users get the same result. This is largely responsible for the development of standardized and objective type tests. Objectivity is a prerequisite to reliability of a test. Judgments on this line are accurate and hence tend to be reliable. In a highly objective instrument the scores assigned are affected by the judgment personal opinion or the scores. 

4. Usability or Practicability:

Practicability is an important criterion for assessing the value of a test; and it depends upon a number of factors.

(A)   Ease of Administrability:

(i)                  Definite provision should be made for the preparation, distribution and collection of test materials.

(ii)                Instruction to the pupils should be simple, clear and concise.

(iii)               Sample items should be illustrated by practice exercises.

(iv)              The test format should be such that pupils will have no difficulty in reading the items, in recording their answers, in moving from one page to the next, etc.

(v)                Illustrations should be clear cut and easily tied up with the appropriate test items.

(vi)              Mechanical features of a test, such as page size, length of line and kind of type, should be such as to facilitate the administration of a test.

(B)   Ease of Scoring:

(i)                  The results of a test possessing scorability should be obtainable in as simple, rapid and routine a manner as in proportion to their importance.

(ii)                The test should be subjected to accurate scoring even by persons not conversant with their content.

(iii)               No algebraic manipulations should be required to get the scores.

(C)   Ease of Interpretation:

(i)                  The raw scores of a test should be easily converted into meaningful derived scores.

(ii)                It should be feasible to interpret the results with the competence of classroom teachers. No specially trained personnel should be required in order that the results may be interpreted validity.

(D)   Economy:

(i)                  The economy of a testing programme should be computed in terms of the validity of the tests per unit or cost.

(ii)                Economy refers to the cost as well as the time required for administering and scoring a test. Any test of a duration which does not exceed a period of 45 minutes is preferred by teachers.

(iii)               A test is called practicable if it can be easily administered and is acceptable to average    examinees.

      A good test is useful in many ways for example, a test result may be used for improvement of teaching may be useful to measure some desired quality ability in the student useful in finding out deficiencies in pupils so that remedial measures by taken up for their removal act.

     While preparing a test, the time and cost of administration must be taken into consideration.

     The test should be usable and should serve a definite need in the situation in which is used.

ACHIEVEMENT TEST:

   Achievement tests are the most frequently used evaluation tools in the educational setting belonging to the paper and pencil category.

    The term achievement refers to the acquisition of all the behavioural changes belonging to the cognitive, affective and psychomotor domains.

  *   It is an instrument designed to measure relative accomplishment in the specified areas of learning.

*   Ebel defines” Achievement test is one designed to measure students group of somebody of knowledge or is proficiency in certain skills.

 *  Freeeman defines test of Educational achievement as “designed to measure knowledge, understanding, and skills in a specified subject or group of subject”.

Characteristics of a good test

Test preparation activities which promote quality, long-term learning are appropriate, even essential. Good test-taking skills and appropriate content learning can reduce the likelihood that extraneous factors will influence students’ test scores. The various characteristics of a good test are:

§  It can be tried out and selected on the basis of its difficulty level and discriminating power.

§  Directly related to the educational objectives.

§  It should possess description of measure behavior in realistic and practical terms.

§  Contains a sufficient number of test items for each measured behavior; concerned with important and useful matter; comprehensive, brief, precise and clear.

§  It should be divided into different knowledge and skills according to behavior to be measured.

§  Standardized the items and made instructions clear so that different users can utilize it.

§  Rules and norms have to be developed so that various age groups can use at various levels.

§  It provides equivalent and comparable forms of the test.

§  A test manual has to be prepared, which can act as a guide for administering and scoring.

Need for Achievement test:

1.       To best the understanding of the student about the concept, principles, laws and theories of a subject achievement tests are needed.

2.       It is necessary to know the level of knowledge, understanding, application and skill of the student.

3.       To test the basic knowledge of the student regarding the subject.

4.       To identify students of maximum and minimum level achievements to provide proper guidance to them.

           Functions of Achievement tests:

1)  Achievement tests are frequently employed to check and attainment of minimum performance of                        students.

 2) They are important tools in counseling. They are helpful in assigning grades.

 3) They also constitute an important feature of remedial training programme.

 4) For all types of learners, the periodic Performance of students.

 5) It helps in the selection of the student they are helpful in the placement and Classification.

  6) For all types of learners, the periodic administration of well-constructed and properly

     Chosen achievement test serves to create love in learning.

 7) Achievement tests are employed as aids in the improvement of instructional techniques and     

      the revision of curriculum content.

STRENGTH OF ACHIEVEMENT TESTS:

1.       They are valid measure of achievement.

2.       They are reliable measuring instruments because of their objectivity.

3.       They are objectively and accurately scored.

4.       They are easy to administer.

5.       Because of frequent requisitions, they are improved at intervals.

6.       They are provided with norms, which enable the teacher to compare individual or groups of individuals or even schools.

WEAKNESSES OF ACHIEVEMENT TESTS:

1.       They lead to undesirable study habits because students try to memorize bits of information with no regard to logical sequence and reasoning of facts.

2.       They encourage uncritical thinking because they provide no score for elaboration in meaning.

3.       Miss-interpretation of results may lead to erroneous conclusions about individuals.

4.       Many important and significant aims of education can’t be measured, if items are restricted to factual information.

ESTABLISHING CLASSROOM ACHIEVEMENT TEST VALIDITY:

         For establishing the validity of teacher-made achievement test in any subject, the following criteria’s may be adopted:-

1.       Use objective as a basis for question development.

2.       Use accepted question development principles.

3.       If multiple behaviours are being sampled, they should be in proper proportion.

4.       Extraneous sources of influence on test responses need to be eliminated or controlled.

5.       Extraneous sources of distraction during test administration must be controlled.

6.       Verify correctness, appropriateness, and accuracy of answers to items and scoring of tasks.

7.       Assessment materials must be appropriate to the level of functioning and experience.

8.       Check the reliability of the measures used.

9.       Where possible, check test scores against external criteria.

10.   Test items that are related to change during instruction should be retained for future use.

11.   Use the most appropriate format.

TYPES OF ACHIEVEMENT TESTS:

1) Teacher made tests

 2) Standardized Achievement tests.

     Teacher made tests are classified into oral, written and performance test.

ORAL TEST:

The process of conducting the test orally ie., asking the questions orally and receiving the answer orally from the students is called oral test.

 The oral examiner can pose a question and modify the same it is not understood. Because of this flexibility an examiner can probe the depth of the students understanding better through oral test.

Uses of oral test (Merits):

1) It helps in testing the previous knowledge of the students. Based on the previous knowledge, the teacher may easily impart the new knowledge.

2) Oral questions put time to time, help the teacher in judging whether the students are following the lesson or not.

3) The teacher may easily pay individual attention by distributing oral questions all over the class.

4) Oral questions help much in carrying out recapitulation and revision work through proper practice or drill.

5) Oral questions helpful in fixing the learning of new principles and formulae.

6) It encourages healthy competition among the students.

7) Oral questions help in developing the habit of exactness and precision.

8) Oral questions help in creating as well as maintain interests of the students in the study.

9) Oral questions develop the habit of quick learning and thinking and therefore proves a very economical device.

10) It is used at the lower stages where it is difficult for the students to give written response.

DEMERITS:

1) It is a time consuming one.

2) It does not extensively or efficiently measure all the abilities of the learner.

3) It permits inferences and favoritism of teachers.

4) It leaves no permanent objective record of pupils performance.

5) Its use on a mass scale is highly limited.

6) It is highly subjective.

7) It is not equally fair and just to all pupils.

WRITTEN TEST:

The test in which the students respond to the teacher in writing is called a written test.

Advantages:

1) It gives us a most stable, reliable and valid type of evaluation.

2) It can be played carefully, keeping in view objectives, content and learning experiences.

3) The answer can be assessed with perfect objectivity and more validly and reliably with the help of a marking scheme and a scoring key.

4) Answer books can be preserved for a long time, making measurement possible.

5) It helps teachers to plan and guide and instructional programme.

6) Many students can be evaluated at a time and at various places.

Disadvantages:

1) Understanding may not be judged.

2) Written tests preparation takes a long time. So, it cannot be taken as a continuous part of education.

3) Certain objectives such as skill in handling apparatus, recording of the reading of an instrument etc., cannot be evaluated.

4) Written tests require a considerable amount of expenditure.

PERFORMANCE TEST:

      The test, which measures the performance of students in psycho-motor tasks, is known as performance test.

 Example-1: Students learn how to conduct experiments in laboratory smoothly and systematically. The performance test measures to what extent students have mastered the conduct of experiment with required speed, accuracy and strength.

Example-2: Teacher provides learning experience in the art playing musical instruments with a specific objectives. Teacher conducts performance tests to assess to what extent students have mastered theart of playing musical instrument.

     In a performance test two aspects are involved. One is assess the process of performance, and the second one is to assess the product of performance.

I. The process of performance of any psycho-motor act involves:

1. Doing with correct speed.
2. Doing with required accuracy.

 3 .Doing with required strength.

 4. Doing with proper coordination between various acts.

 5. Doing by handling the equipment properly.

II. Test of product performance involves:

1.       Quantity of the product with required specifications i.e. size colour combinational getup.

2.       Properties of the product desired.

3.       Raga, Tala, Laya, and Swara in music etc.

Tools to be used in performance test:

1. Observing the performance by using observational schedule.
2. Rating scale.
3. Check list.
4. Verification of records.
5. Interview the students with interview schedule.
6. Video recording technique.

Importance of Performance test:

1.       Performance test is essential to assess to what extent the objectives related to psychomotor task are realized.

2.       Performance test helps the teacher to know the weakness of students in performing the given task.

3.       It helps to provide practice to get mastery in performance.

4.       It helps the teacher to correct the mistakes in performance.

5.       It helps to judge the quality of performance of the students.

6.       It helps the teacher in giving feedback to the learner regarding his performance.

7.       It helps the students to know their physiological limit in performing the psychomotor activity.

8.       It helps the teacher to provide practice session individually to master the skill in performance.