Tools for Data Collection Measurement & Scaling

Commonly asked questions

Tools/instruments/inventories are they synonymous? Questionnaire and scale are they the same? Scale and index the difference? Factor and dimension the same? Test/measure What is it? What is cultural transportability? What is a standardized questionnaire/tool? How is a tool developed? When to use what type of tool? When is pre-testing done? Before or after validation?

Tools for Data Collection

Observation Interview method Questionnaire Projective techniques Socio-metric technique

Observation
We are almost constantly engaged in observation of things, objects, processes and even thought-ways of people It is our basic method of obtaining information about the world around us Observation becomes a scientific tool for the researcher to the extent that it serves a formulated research purpose, is planned systematically, is related to more general theoretical proposition, is recorded systematically and is subjected to check and controls on validity and reliability Many types of data sought by the researcher can be obtained thro direct observation though direct observation is not the only method

Advantages of Observation
One great asset of the observational technique is that it is possible to record behavior as it occurs Most items of our behavior are so much a part of our habit that they escape an aware detection and resist translation into words Studies may deal with subjects who are not capable of giving verbal reports of their behavior or feelings for the plain reason that they cannot speak, e.g., infants. Such studies necessarily depend on observation Observation is independent of peoples willingness to report

Types of Observation
Controlled/Uncontrolled observation Structured/Unstructured/Partially structured observation Participant/Non-participant/Disguised observation

Interview Method
Why interview method in research?
To understand the experience and perceptions of the world from the subjects point of view To provide the opportunity for professional conversations with a directed structure and purpose To focus on collecting the kind of in-depth information that can only be acquired through a personal exchange

Steps in Interviewing
Opening the interview Negotiating the purpose Searching for the answer Communicating information to the respondent Closing the interview

Steps in Interviewing Opening the Interview

Obtaining the first supportive response is most important. Importance of non-verbal cues Gaining approachability Obtaining responses as Can I help you? Building trust

Steps in Interviewing Negotiating the Purpose of the Interview

Consolidating approachability Generating respondents interest Careful listening and inquiry Searching for start ups

Steps in Interviewing Searching for Answers

Start with a smile Listen carefully Be interested Take counter questions seriously Ask questions till you get the complete answers Never think you know the answer Get the consent to return if more help is required

Steps in Interviewing Communicating Information to the Respondent

Avoid jargons Maintain appropriate intellectual levels Tell or show the details

Steps in Interviewing Closing the Interview

Withdraw slowly from the environment Thank the respondent Inform him for data usage Inform him of accessibility

Interviewing - Advantages
Together with questionnaire, interviews make up the survey method, which is one of the most popular design approaches in social/management research. Flexibility interviews can be adjusted to meet many diverse situations High response rate ensures a relatively high response rate Easiness in administration Opportunity to observe non-verbal behavior Opportunity for clarification and probing Opportunity to record spontaneous response Control over identity of the respondent Completion of tool can ensure that all items in the tool are adequately covered

Interviewing - Limitations
Interviews are more costly and time consuming Likely to be affected by the interviewer's biases Can be at times inconvenient to respondents Less anonymity of the respondent Less effective in the case of sensitive issues

Projective Techniques
The Projective Hypothesis The projective hypothesis (Lawrence Frank, 1939) asserts: when people try to understand vague or ambiguous unstructured stimuli; the interpretation they produce reflect their needs, feelings, experience, prior conditioning, and thought processes. The Problem: The difficulty is in ascertaining which particular needs, feeling, experience, prior conditioning, thought processes are reflected. Projective techniques can draw equally upon the imagined and real, the conscious and unconscious, the recent and the old. Much room for interpretation is left given to the tester making test validation impossible.

Projective Techniques cntd

PTs are indirect means of elicitation that enables the respondent to project beliefs and feeling onto a third party, to an inanimate object, or into a task situation. Respondents are encouraged to describe or react to stimuli in their own words, by little prompting from the researcher. Respondents are expected to interpret the stimuli or the situation.

Types of Projective Techniques

The most common projective techniques used in social research are: Word association Sentence completion Third-person technique TAT

Word Association
The subject is presented with a test of words, one at a time, and asked to respond with the first word that comes to mind. e.g., Kent-Rosanoff Test Dark Deep Mountain Fruit Smooth Slow Rough Hard Stem Cottage Bitter Thirsty

Sentence Completion
This method is also based on the assumption of free association. The respondents are asked to complete a number of incomplete/partial sentences or phrases. e.g., People who work late are. A female manager is liked by. .. Other people usually I worry over. When I was a child

Third-Person Technique
You found this grocery list in a supermarket. Characterize the local woman who brought the groceries, write briefly about her personality and values Mutton Bread Hamburgers Cornflakes Pizza Coke

Thematic Apperception Test (TAT)

TAT consists of series of pictures in which the research topic is the center of attention. The researcher ask the respondent to tell what is shown to be happening in the picture and what the character in the picture might do next. Picture Frustration Test (Rosenzweig Test) Inkblot Test Herman Rorschach

Questionnaire
A questionnaire consists of a number of questions printed in a definite order on a form (or set of forms) The forms are usually mailed to the respondent who are expected to read, understand and reply to them in writing in the relevant spaces provided for the purpose on the form/s. A respondent has to answer the questions on his own, ideally. A schedule also has a reference to proforma containing a set of questions, handed over to the respondent and filled in the presence of the interviewer.

Advantages of Questionnaire
It does not call for any special skills or training on the part of the investigator It covers a large number of people spread over a large territory making it economical in terms of money, time and energy It is an impersonal technique. Uniformity from one measurement situation to another is provided by way of standardization It ensures anonymity The respondent gets ample time to respond

Limitations of Questionnaire
It can be administered only on subjects with a considerable level of education In a mailed questionnaire, the proportion of returns is very low The respondent can misinterpret questions The respondent need not have clear views on the issues probed All respondents need not show a sense of responsibility The researcher is not in a position to vary the stimuli or social atmosphere impinging upon the subjects according to his designs

Data Gathering Methods Important Guidelines

Pre-testing of the Tool: This should be done with a group of people similar to the intended population and study subjects This is a trial run to test the data collection tool and will allow a prediction of how respondents will interpret and react to the questions Gains of Pre-testing: Locates problems before starting with data collection Provides overall importance of respondents and interviews Facilitates correction and modification in the tool and may suggest even altering wordings of questionnaire items

The Concept of Measurement

Measurement is the process of assigning numbers or labels to objects, persons, states, or events in accordance with specific rules to represent quantities or qualities of attributes.

Measurement
Measurement Assigning numbers or other symbols1 to characteristics of objects 2 according to certain prespecified rules3 this definition contains 3 parts We measure certain characteristics of the objects and not the objects (such as preferences, attitudes etc) There is one to one relationship between the numbers and characteristics measured Creating a continuum, on which measured objects are located/placed (scaling)

Measurement of Variables
Conceptual variables Measurement The ideas that form the basis of Actual empirical referents a research hypothesis. Usually that represent the stated in an abstract manner conceptual variables Example: Study Time ------- Hours of study (1 hr, 2hrs, 3hrs etc.) Aggression ------- No. of presses of a button that administers shock to another person Self-monitoring skill ------- Scores of R. D. Lennoxs selfmonitoring skill scale

Development of a Measurement Scale

Scale A scale is a set of symbols or numbers so constructed that the symbols or numbers can be assigned by a rule to the individuals (or their behaviors or attitudes) to whom the scale is applied

The Measurement Process

Utilizes the scale
Which leads to

Research findings

Identify the concept of interest

Use the conce pt to

If the evaluation is satisfactory, then the researcher

Evaluate the reliab.& validity of the scale

That require the researcher to Which enables a Measurem researcher to create ent scales

Develop a construct
Which is used to create

Operational definition

Which enables a researcher to develop an

A conceptual definition

Measurement & Scaling

Social and management research, irrespective of type and nature, involves measurement of variables Variables can be: Qualitative Quantifiable Quantitative Variables can be measured at four levels: Nominal Ordinal Interval Ratio

Measurement & Scaling

The level of measurement of a variable is decided by the properties possessed by the scales used for the purpose. The properties being: Ability to distinguish between variables attributes Ability to capture the degree variation Equality of difference between anchor points Possession of an absolute zero point If the scale is having property 1 Nominal If the scale is having properties 1 and 2 Ordinal If the scale is having properties 1,2 and 3 Interval If the scale is having properties 1,2,3 and 4 - Ratio

1. 2. 3. 4.

Variables & Measurement

Based on the nature of operational definition Qualitative Identifiable categories (A) Nominal (e.g., Male and Female) - A Quantifiable Ranking/Ordering possible (B) Ordinal (e.g., Height of male and female) A and B Quantitative Equal distance between points in-built (C) - Interval A, B and C - True zero (D)? If yes, Ratio A, B, C and D

1. 2. 3. 4. 5.

A pure categorical variable, in which the numbers serve only as labels or tags to identify and classify the objects Assign numbers to categories that have no special order to one another 1 5 3 2 4 (Range) * * * * * (Phenomenon) e.g., What is your nationality? Indian British American Japanese Others e.g., Sex, Marital status, Social status, Education, Days of week/month Job type: Executive, Technical, Clerical

Nominal Measurement

Ordinal Measurement
Measures phenomenon that can be rank-ordered or sequenced in terms of more or less of the phenomenon being measured. But the anchor points are not equidistant. 1 2 3 4 5 (Range) * * * * * (Phenomenon) e.g., How often you make plant visits? Never A few times a year Once a month Once a week More than once a week e.g., Size of the dress: Short, Medium, Large, Extra large

1. 2. 3. 4. 5.

Ordinal Measurement
A scaling method, in which the numbers are assigned to the objects to indicate the relative extend, to which some characteristic is possessed Here we try to determine whether an object has more or less of a characteristic than some other objects. We cannot determine the magnitude of the characteristic.

The zero position is not defined.

Interval Measurement
Measures phenomenon on the basis of equal underlying intervals between the anchor points on the measurement scale. 1 2 3 4 5 (Range) * * * * * (Phenomenon) e.g., Indicate your overall approval of the decision: 1 2 3 4 5 Do not approve --------------Approve e.g., Temperature scales

Interval Measurement
A scale, in which the numbers are used to rate object/characteristics of the object such that, numerically equal distances on the scale, represent equal distances in the characteristics being measured. Here an arbitrary zero is fixed. The distance between 4-5 is equal to distance between 5-6 etc., Most of the researcher treats this scale as ratio scales and compute all the relevant measures. Measures of central tendencies and dispersions are permissible measures

Ratio Measurement
Measures phenomenon on the basis of equal underlying intervals on the measurement scale with an absolute zero point. 0 1 2 3 4 (Range) * * * * * (Phenomenon) Height Length Time Income Market share How many subordinates you have?----no. of subordinates

Ratio Measurement
It is a combination of Nominal, Ordinal and Interval scales with an absolute zero position. All the arithmetic calculations are permitted. Measures such as Weight, Height, Age (Years/Months) are examples of ratio scales. All the measures such as regression/correlation/measures of central tendencies/measures of dispersion/ are permitted. But very rarely used in consumer researches.

Levels of Measurement - Summary

Criterion Nominal Ordinal Interval Ratio

Properties

Naming

Naming Ranking

Naming Ranking Equal intervals

Scoring

Naming Ranking Equal intervals Zero point

Scoring

Measurement Categorical

Ranking

Mathematical function

None

None

Addition & Subtraction

All the four functions

Measurement Error
Interviewer error Ambiguity scale/questions

Interviewer

Respondent

Inaccuracy of response

Attitude and Attitude Scales

Attitude Scales

Scale A coherent set of items related to factors that are regarded as indications of a more general concept Item A single component in the scale Congeneric items When all items lead to one and the same factor they are known as congeneric (uni-dimensional) items Index A set of items (say, 4 or 5 items) that are reflective of a variable. The items will have equal weightage.

Attitude and Attitude Scales

ABC Affective, Behavioral, Cognitive components How do we measure attitudes? 1. Observation and qualitative methods 2. Self reports e.g., Head movement, eye contact Measurement of attitudes The Lost Letter Technique Galvanic Skin Response (GSR) Facial Electromyography Recording (EMG)

Scaling Techniques
Scaling Techniques No. of Dimensions Format Types

One dimensional Multi-dimensional Comparative scales Non-comparative

Rank Paired Social Constant order comparison distance sum

Continuous rating

Itemized rating

Likert type Semantic Stapel summated differential scale

Dimensional Scales
One-dimensional scales Procedures designed to measure only one attribute of a respondent or object Multi-dimensional scales Procedures designed to measure several dimensions of a respondent or object

Comparative & Non-comparative Scales

Comparative Scales In this category of rating scales, the positions on the scale are expressly defined in terms of a given population, a group or in terms of people with known characteristics. The respondent for example, may be called upon to indicate whether an individuals problemsolving skill or some other attribute most closely resembles that of others.

Comparative Scales & Non Comparative Scales

Comparative Scales: It involves a direct comparison of stimulus objects. (Prefer Pepsi/ Coke or Cadburys/Nestle) Since they possess the characteristics of rank order/ ordinal scales, they are often referred as Non-Metric scales. Non-comparative Scales Also known as Monadic/Metric scales in which each object is scaled independently of the others in the set. Often they belong to the categories of Interval or Ratio Scales.

Multi-Dimensional Scales Rank Order

Rank Order Respondents are given a list of items and asked to compare the items with a view to assign individual rank. This method is difficult if there are more than five. Therefore, the paired comparison method is used. For example, A, B, C, D are there: A and B, A and C, A and D, B and C, B and D, C and D are compared. No. of comparisons = N(N-1)

Multi Dimensional Scales Rank Order Scales

Several objects or characteristics of the objects are given with the respondents The respondents are asked to rank them according to the given criteria. For example, the respondents are asked to rank the following brands of two wheelers based on mileage of the vehicle Brands Yamaha Hero Honda Bajaj Suzuki TVS Rank

Multi Dimensional Scales Rank Order Scales

Since the respondents classify the objects based on some criteria and rank them, the measurements are ordinal scaled one. The method forces the respondents to differentiate among the objects taken for study purpose. Very simple method in the sense that the respondent has to take (r-1) decisions if r objects are given to him/her.

Measures such as Spearman Rank correlations are frequently used to summarize the results.

Multi Dimensional Scales Paired Comparisons

The respondents are given a pair of objects at a time and asked to choose one from among them based on the given criterion For example, based on maintenance cost choose the best brands from the given set of motor cycles
Yamaha Hero Honda Yamaha Hero Honda Bajaj Suzuki

Bajaj

Suzuki

Multi Dimensional scales Paired Comparisons

Based on the criteria given (maintenance cost) if the brand is superior (lesser cost of maintenance) the respondents can assign the value 1 and otherwise 0 The sum of the score across the row/column can reveal how many times the brand preferred over other brands. Normally the researcher has to include provision for neutral answers.

Multi-Dimensional Scales Bogardus Social Distance Scale

Attitude towards members of social or ethnic groups that is based on the assumption that ones liking for a group is reflected in the social distance that one finds acceptable in relationships with the member of the social or ethnic group compared.

Multi-Dimensional Scales Constant Sum Scales

Requires the respondents to divide a given number of points, typically 100, among two or more attributes based on the importance of the person. Number of points allocated to each alternate indicate the rank. The more points an attribute receives, the more important the attribute is. If an attribute is not at all important, assign it zero points. If an attribute is twice as important as some other attribute, it should receive twice as many points. Hours of service.(points), Friendliness.(points), Distance from bank.(points)

 The respondents are asked to assign/allocate a constant sum (say 100 pts. or 100%) to the set of objects/characteristics of the objects If some of the attributes are not very important or not at all important to the respondents, can assign zero also The basic question is whether he/she is exhausting all the 100 points/% Then the researcher has to find out the average for all the characteristics across the respondents. Even though the scale slightly appeared to be metric, it is considered as ordinal since the points are assigned on comparison of objects/stimulus.

Constant Sum Scaling

Sl. No
1. 2. 3. 4. 5. 6.

Characteristics

Points

Drum Type Color Scratch Resistance body Water Consumption Power Consumption Spares Availability

Non Comparative Scales

In general, the respondents do not compare the objects being rated with any other objects in the stimulus set or with some pre-specified standard (such as their preferred brand/ideal brand).

Non Comparative scales

Continuous rating scales

Itemized rating scales 1. Likert scales 2. Semantic Differential 3. Stapel

They rate one object at a time according to the criterion given/considered

Non Comparative Scales Continuous Rating / Graphic Scaling

The respondents rate the objects by placing a mark on an appropriate position on a line, that runs from one extreme of the criterion variable to other extreme. In the scales, the line may contain scale points in the form of numbers or brief description. How would you rate the training program? The worst 0 10 20 30 40 50 60 70 The best Form 1: 0____________________100%

Form 2: The worst 0_________________100 The best Form 3: Very bad 0_______NB/NG________100 Very good

Non Comparative Scales Itemized Scales

A scale, having numbers or brief description associated with each category The categories are ordered in terms of scale positions The respondents are required to select the specified category that best describes the object

Non-Comparative Scales Likerts Method of Summated Rating

Developed by Rensis Likert Respondents require to show his or her degree of agreement or disagreement with each of a series of statements about the stimulus objective Analyzed by assigning a numerical value such as -2 to +2 or 1 to 5 with an analysis for item by item basis (profile analysis) Or total sum for series of statements for each respondents (summated approach) e.g., Strongly disagree Strongly agree

Non Comparative Scales Semantic Differential Scales

It includes various subscales that measure the different dimensions of the attitude It is a 7 point rating scale, with end points associated with bipolar labels that have semantic meaning (opposite) Example: Rate the SMS brand on the following: Reliable +3 +2 + 1 0 -1 -2 -3 Unreliable Modern +3 +2 +1 0 -1 -2 -3 Old fashioned

Non Comparative Scales Stapel Scale

Developed by Jan Stapel A uni-polar rating scale with 10 categories ranging from -5 to +5 without zero Stapel scale is used when bipolar statements cannot be recreated It is used to measure both the duration and intensity of an attitude simultaneously +5 + 4 +3 +2 +1 (e.g.supportive) -1 -2 -3 -4 -5

Stapel Scale cntd

Respondents are asked to indicate by selecting an appropriate numerical response category, how accurately/inaccurately each characteristics describe the object The advantage over semantic scales is that it does not require any pre-testing of the adjectives in use The scales used least of all Also the Thurston scale, which is very difficult to construct is used less frequently by the researchers

Developing a Tool/Scale
Option 1: Adopting a standardized, validated culture-free tool/scale developed by experts in the field for the use in a similar setting or population - Measures may be accepted Option 2: Scale requiring minor modifications (like semantic issues, jargons, different population etc) - Re-establish validity (sometimes) and reliability Option 3: A all-new scale to be constructed/developed - Involves construction of the scale followed by the establishment of validity and reliability measures

Issue of Cultural Transportability

Measuring life styles of women employees: How often do you smoke in a day- at home and outside? Measuring diversity awareness of managers: Are you in favor of the provisions of the EEO and the underlying principles of the affirmative policy? Measuring attitude towards HR practices: Do support the recent hike in dating allowance for the staff? Measuring attitude towards road traffic offences: Indicate your preference regarding the number of points to be cut for lane violations

Developing a New Scale

Survey of all available literature and measurements, if any, to collect information regarding all dimensions, aspects, facets, explanations and features of the variable under study Develop a set of all inclusive items/descriptions/characteristics Convert them into statements/questions/checklist Administer among experts make a short list Make a coherent set of questions, administer and collect data Do data reduction (Factor analysis) Establish validity and reliability measures Refine it by subsequent use

Defective Instrument

Reasons: Insufficient operational definition Poor selection of content items It causes distortions: Can be too confusing and ambiguous Use of complex words and syntax beyond comprehension Leading questions Ambiguous meanings Mechanical defects Multiple questions

Questionnaire Filling Biases

Central tendency - Giving the central/neutral answer Agreement to the questions - Agreeing with the statements given Social desirability - The need to answer positively Scores obtained = True value + Random error + Non random error

Questionnaire Design Major Decisions

What should be asked? How should each question be phrased? In what sequence should the questions be arranged? What questionnaire lay out will best serve the research objectives? How should the questionnaire be pretested? Does the questionnaire need to be revised? Basic criteria a questionnaire must meet: Relevance (avoiding unnecessary/irrelevant questions) Accuracy (ensuring reliable and valid information)

Phrasing Questions
Open-ended response vs. Fixed-alternative questions Fixed-alternative questions Simple --dichotomy questions Determinant-choice question Frequency-determination question Attitude rating scale Checklist question

The Art of Asking Questions

Focus (on the variable) Be specific avoid ambiguity Have a general idea of respondents profile Be crisp and simple Avoid leading questions Avoid loaded questions Avoid double negative Avoid double barreled