Paper I – Experimental Designs and Psychometrics

Introduction to Experimental Design; Types of Experimental Designs: Within Subjects, Between subjects, Factorial; Basic Terminology in Experimental Designs; Basic Terminology of Statistical Analysis.  

The assessment of individuals: Measurement in the social sciences; Historical highlights of measurement; statistical background – scales of measurement, the normal distribution, score meaning; Identification of construct – links between constructs, construct cleanliness, single vs multiple construct

Designing and writing items: Empirical, theoretical and rational approaches to item construction, assessing behaviour – critical incident technique, plot testing, summary and next step.

Designing and scoring response: open-ended responses; close ended questions; Dichotomous responses; multiple choice tests- distracters, guessing, speed and power tests, omitted and partial credits; continuous responses – summated rating scales, likert-scale, visual analogue scales, pictorial, adjective rating scales.   

Item analysis : Meaning steps; classical test theory, modern test theory

Reliability of test scores and test items : Test –retest reliability, alternative forms reliability, measures of internal consistency -  split – half, cronbach’s alpha, coefficient theta, Kuder Richardson.

Assessing validity using content and criterion method : Asking the test takers and subject matter experts, assessments using correlations and regression; multiple criteria, group differences and test bias.