The current issue and full text archive of this journal is available at www.emeraldinsight.com/0263-5577.

htm

IMDS 106,5

A proposed white-collar workforce performance measurement framework

644

Udomsak Suwansaranyu
Mitsubishi Fuso Truck, Pathumthani, Thailand, and

Kongkiti Phusavat
Department of Industrial Engineering, Kasetsart University, Bangkok, Thailand
Abstract
Purpose To propose a structured framework for measuring white-collar workforce. Design/methodology/approach The proposed framework, designated as the multi-dimension measurement process or the MDMP, is based on the nature of white-collar work as well as on the strengths of current measurement techniques such as Zigons. The experiment on comparing the MDMP with several techniques was conducted. The analytic hierarchical process (AHP) has been adopted to determine the usefulness and applicability of the MDMP. The follow-up discussions with the participants and the surveys to external experts have also been made. Findings The research results imply potential usefulness and applications for the MDMP. Relatively to others, the MDMP has performed very highly with the set of prioritized criteria (from the AHP) that is used for this comparison, e.g. reliability, strategic congruence, measurement coverage, and user acceptance. Based on this experiment, the MDMP appears to ensure the alignment between strategies and measurement, and also to gain the acceptance by both workers and supervisors. Research limitations/implications Given the small size (16 participants) and the limited scope (participants mainly from the accounting and nance areas), it is not possible to provide a denite conclusion on the effectiveness of the MDMP. More experiments and tests will be needed to determine the level of the MDMP generalization. Practical implications A basis or a starting point to help develop a tool that can be used to measure white-collar workforce. Originality/value This research incorporates several aspects relating to the work performed by white-collar workforce from outputs/outcomes, skills, behavior, and organizational goals. The needs to develop the framework that measures and captures the performance of white-collar workforce have been cited by several sources for many years. The focus on measuring the workforce level is mainly for appraisal/administrative purposes. As a result, the information may not reect all aspects relating to the white-collar work. Keywords Performance measures, White-collar workers Paper type Research paper
Industrial Management & Data Systems Vol. 106 No. 5, 2006 pp. 644-662 q Emerald Group Publishing Limited 0263-5577 DOI 10.1108/02635570610666421

1. Introduction The term performance measurement has had signicant roles in business and operational management during the past few decades. Many organizations have used performance measurement as a primary tool for communicating direction, establishing

accountability, dening roles, allocating resources, monitoring and evaluating activities, linking organizational processes, establishing goals and benchmarks, and initiating changes to ensure continuous improvement (Viken, 1995). Simply put, it is a critical element of a management process. This is supported by the statement of you cannot manage what you cannot measure by Deming (1986). Rummler and Brache (1995) reinforce such a statement by concluding that the selection of measures and related goals is the greatest single determiner of an organizations effectiveness as a system. Owing to its importance, performance measurement has received lots of attentions from a number of researchers in various elds of study including psychology, human resource management, economics, accounting, and industrial engineering. When focusing on the industrial, national, and international levels, the approaches have been designed and developed by the eld of economics such as the total factor productivity (TFP) technique. At the organizational, functional, program, and project level, there are several disciplines that have examined and developed the performance measurement approach. The balanced scorecard concept (Kaplan and Norton, 2004) has been used to measure organizational performance levels. At the same time, the approach developed by Sink and Tuttle (1989) have been cited by several sources in both public and private sectors when measuring functional and organizational performance. At the group and individual levels, there are various concepts and methods, ranging from motivational approach (by industrial psychologists) and appraisals for salary structure/job assignments/workload analysis extended (by human resource specialists), to piece-rate and standard times (by industrial engineers). Nevertheless, for the group and individual levels, there is a lack of clarity and comprehensive taxonomy to determine the compatibility between the specic circumstance and the particular approach (Ramirez and Nembhard, 2004). The US Bureau of Labor Statistics has shown that the proportion of white-collar workers has increased dramatically during the past few decades. By 1980, white-collar workers represent approximately more than 60 percent of the entire workforce. This trend is consistent with the progress and movement in the economy among the developed countries. Their economies have shifted from the agricultural- and industrial-based to service- and knowledge-based. As a result, the emergence of white-collar workforce represents both potential opportunities and problems in businesses and management. The opportunity to explore new ideas and approaches on measuring the work created by white-collar workforce is challenging. At the same time, the difculty level in measuring the performance level of white-collar workforce is perceived to be greater relatively that of blue-collar workers. Blue-collar workers who are on the production line have received a lot of attention because their performance is simple to measure and analyze. Applying the time and motion studies (developed by Frederick Taylor in 1911) can ensure successful blue-collar workforce measurement. The following example helps demonstrate the key differences, based on Table I. The nature of white-collar and knowledge work is quite different from that of the shop oor. In fact, extension of the performance (including the productivity aspect) measurement from the successful shop oor application to the ofce and laboratory stops abruptly when the output of the off-line process is no longer clear and when higher productivity becomes only one of several desired outcomes. Measuring the performance of white-collar workers is common in back rooms of service institutions.

Performance measurement

645

IMDS 106,5

Gregerman (1981)

Beruvides and Koelling (1992) Job task pursuit can be best described as perplexity, problem, program, or project on the uncertainty scale Decisions involved in the task are mainly semi-structured to unstructured Endeavors are tactical and strategic (i.e. tactical wide efforts are directed toward a tangible result using quantitative standards, strategic global efforts are aimed at a general direction using qualitative measures) Stage of maturity can be described in the control to optimization level (i.e. control steady state achieved by eliminating variations, optimization modication of the plan and control to improved performance)

1. Knowledge work is usually non-repetitive 2. Output of knowledge work is usually impacted by a number of external factors 3. Output of knowledge work is usually difcult to quantify 646 4. There are many approaches to knowledge work 5. The effectiveness of the work depends mainly on the approach selected by knowledge worker Knowledge worker usually requires good problem solving skills 6. Task of knowledge worker usually takes longer than that of blue-collar worker 7. Knowledge workers usually work in group and require group recognition and participation. It is a rare knowledge worker who functions best alone. In fact, observations lead to conclusion that most knowledge workers perform best in concert with others. These collaborations range from interaction with peer groups to membership internal company service organizations to cooperative relationships with Table I. external resources such as suppliers, Denitions of white-collar customers, and consultants work

However, as the work becomes less repetitive and more creative in its requirements, measuring the apparent workow, outputs, and inputs becomes less and less meaningful. Both outputs and inputs are mainly intangible and complex, and the work from one white-collar is part of a larger work process. Sumanth (1998) and Thor (1987) raise an interesting aspect of measuring white-collar work. A lawyer handling utility right-of-way issues might be measured on right-of-way obtained per month, but on multi-year between-the-company liability cases, what does productive legal work really mean? Furthermore, it is much more difcult to measure quantitatively the work performed by an engineer or a researcher than that of a shop oor worker. Presently, current measurement effort focuses on various dimensions the can be used as performance surrogates that reect the tasks and work performed by white-collar workers. These dimensions include quality, satisfaction, cost, etc. Nevertheless, there is a general agreement that it is not an effective way to capture, assess, measure, and evaluate the performance of white-collar workers. As a result, there is a perception that white-collar work is immeasurable and its performance information is of little usefulness (Thomas and Baron, 1994). This attitude is shortsighted because measuring white-collar workforce may provide opportunities for more creative ways to perform work and to add value into the work processes. 2. Problem statement There are three key problems or obstacle when attempting to establish performance measurement for white-collar workforce. They are:

(1) what to measure; (2) how to measure; and (3) cultural issues. The rst obstacle is on what to measure. A number of challenges arise when determining what to measure for white-collar workers. First of all, no matter the job description for white-collar workers essentially is a series of statements and a set of responsibility. However, the more complex of the task and the more exible structure that an employee has, the less concrete job statement will t what the employee does. Thus, the higher a person moves up in an organization and the more varied and subtle the work, the more difcult it is to identify quantitatively and qualitatively the results that represent more than a fraction of his or her effort (Levinson, 2003). As clearly stated in the previous section, the quality of service, advice, or innovation work is more difcult to measure than the quality of widgets. Trying to measure performance of white-collar using all numeric measures likely produces some irrelevant measures and may lead to unwanted behaviors. The second problem is how to measure. There are a couple of items to be considered when deciding how to obtain performance results from white-collar workforce. Using nancial measures with a team with very little control over it does not make much sense (Zigon, 1998). People tend to come up with nancial measures since they believe that they are connected with organization goals. That, however, does not make much sense with functions like human resources or accounting, which their roles do not directly and immediately impact on organization nancial performance. Worse yet, the budgets or headcounts are allocated and rationed with the companys operating output. Sometimes, a research budgets are reduced because of the decline in sale revenue. However, sales and R&D activities deal completely in a different time frame (Thor, 1987). In addition, a complicated measurement system may not be better or more effective than the simple one. Attempts to develop the ultimate system for measuring white-collar workers performance levels have often led to the development of impressive, but massive and complicated, systems with a lack of condence from related parties. Scientists or engineers whose performance is measured on 20-30 variables typically ignore most or all of them. Finally, for the second consideration, the subjects of measurement objectivity and frequency are important. Since, applying non-numeric measure to measure the performance of white-collar workers is common, it is crucial to ensure that information from these measures are obtained from the right persons with low subjectivity. In addition, the performance information should be provided often enough to avoid miscommunication. According to Zigon (1998), a new staff likely wants to be given performance information more often. The third roadblock is cultural issues. Sanchez (2000) suggests the measurement effort should be aware of cultural differences. It is observed that all approaches and techniques to performance measurement tend to incorporate cultural assumptions that may or may not reect usability. For example, the queuing concept assumes people remain orderly in line while waiting for a service. Some demonstrations of these differences are as follows. In some cultures, the hierarchal rank may be of greater concern. As a result, employees feel neither the right nor the duty to determine their own performance measures and goals. For many techniques, the performance feedback should focus primarily on the performance level. In Asian and some Latin cultures, staffs expect to be given feedback on their integrity, loyalty, and cooperative spirit as

Performance measurement

647

IMDS 106,5

648

much as their ability to perform or achieve a goal and performance levels. At the same time, many techniques concentrate on measuring individual performance (similar to the effort for blue-collar workforce). Nevertheless, for several Japanese organizations, measuring group performance is preferred. Given the above problems and circumstances, it is important to clearly identify the research scope and objectives (Figure 1). The bold blocks represent the focus of this research. The primary objective of this research is to explore and to propose a structured approach to measure the performance levels of white-collar workforce. Then, the next task is to develop a process to help fulll the research objective. This research process involves several steps. (Figure 2) The rst step is to examine current techniques in measuring white-collar workforce. There are ve techniques selected in this step. They are as follows:
Performance Measurement

Administrative Purposes Recruitment, retention, termination (e.g. a career path, etc.) Promotion (e.g. as skill requirements) Motivation -

Improvement Purposes Analysis on problems and root-cause for improvement interventions Target setting -

Strategic Purposes Seeking desired outputs, which contribute to the overall goals of the organization Communicating organization goals

Organizational Level

Functional Level

Individual Level

Figure 1. Background of performance measurement and research focus

Blue Collar Workers

White Collar/Knowledge Workers

Examine Current Measurement Techniques Initial Evaluation of the Current Measurement Techniques

Identify Criteria for Technique Evaluation

Propose the Structured Approach, the MDMP

Test and Experiment the MDMP with Current Measurement Techniques

Discuss Research Findings

Figure 2. Research process

Conclude and Recommend Future Research Opportunities

(1) (2) (3) (4) (5)

Graphic rating scale (GRS). Behavioral anchored rating scale (BARS). Management by objectives (MBO). Zigons measurement technique (Zigons). Total measurement development (TMD) by Tarkenton productivity group.

Performance measurement

The second step is to identify a set of criteria to be used for assessment and evaluation on the current techniques. The third step is to design, modify, and develop a structured framework for measuring white-collar workforce performance. This approach is referred to as the MDMP. The fourth step is to experiment and test the MDMP. Within this step, the discussion and training sessions have to be made to a group of the participants. This is to ensure their familiarity with the purposes of the experiment as well as the implementation of the six techniques and their results. There are 16 participants. Their work is in the areas of accounting and nance. The six techniques for testing are the MDMP, the GRS, the BARS, the MBO, Zigons, and the TMD. The AHP is used to prioritize the set of criteria from the second step and to determine whether the MDMP approach and its results strongly correspond to this set. The fth step is to discuss the ndings, including the feedback from external experts and comments from the participants. The last step is to provide conclusions, and shortcomings, to elaborate on academic contributions, and to recommend potential future research. 3. Research process There are ve white-collar workforces performance measurement techniques selected for this research. Appendix 1 briey describes each technique. The primary reason for their selection is due to their recognition. Secondly, the time taken to be familiarized and then to implement these techniques is relatively short. This will be important when the experiment is conducted. The next step is to summarize the ongoing problems and the comments in several literatures so that a set of criteria can be identied. This set is important in two ways. The rst one is to help determine the strengths and weaknesses of the ve current measurement techniques. The second implication is to help compare the usefulness and applicability of the ve current measurement techniques and the proposed one, the MDMP. The set of criteria consists of measurement coverage or comprehensiveness, reliability, acceptability, measurement standards, measurement complexity, measurement frequency, and strategic congruence. Some criteria may contain more details. Appendix 2 describes each of the ve criteria used for this second step in the research process. To complete this second step, an initial evaluation is conducted. This initial evaluation helped verify the need to develop a new framework (or the MDMP) from the practical point of view. In addition, the initial evaluation helps identify some of the strengths and weaknesses embedded in each of the ve current measurement techniques. In the initial evaluation, two senior budgeting (accounting and nance) ofcers of one of Thailands largest nancial institutions were asked to implement the ve measurement techniques, including the task of developing performance measures for white-collar workers. The two senior ofcers were then requested to compare the results from these ve techniques with the set of criteria (stated earlier). Table II shows the results of this initial-evaluation study. These results conrmed the need to develop a new framework from the practical perspective, and

649

650

IMDS 106,5

Criteria

Measurement coverage

Reliability

Acceptability Measurement standards Measurement complexity and prioritization Simplicity Prioritization Low Low Low Low Low Low

Measurement frequency Strategic congruence

Table II. Results of the initial evaluation on the ve current measurement techniques Sub criteria Process-based measurement Behavior- and skills-based measurement Result-based measurement Inter-rater reliability Test-retest reliability No Yes No Low Low Low Low Simple No Yes No High Medium High Medium Moderately simple GRS BARS MBO No No Yes Very high Very high High High Simple Medium Medium High Zigons No No Yes High High Medium Medium Complex High Medium High TMD No No Yes High High Medium Low Very complex Medium Low Medium

also helped identify the strong and weak characteristics of each of the ve current measurement techniques. It should be noted that before and during the initial evaluation, numerous discussion sessions have been held to ensure the understanding of their roles/expectation and of performance measurement background. The next step in the research process is to develop a new framework for measuring the performance level of white-collar workforce. The information from the previous step helped this development tremendously. The MDMP combines the practical advantages from the ve current measurement techniques. The development of the MDMP has also taken into the consideration the nature and characteristics of current work being experienced by white-collar workforce such as ad hoc projects, roles of technology used for task or work completion, and uncertainty created by external factors. The MDMP consists of four aspects. They are: (1) developing the measures; (2) conducting measurement; (3) performing analysis; and (4) carrying out recommendations. The emphasis of the MDMP is on other relevant aspects apart from the development of the measures such as the measurement process, the analysis and the deployment of recommendations. Figure 3 shows the overall concept of the MDMP. The steps required for implementing the MDMP are as follows: Issue I: develop the long- term perspective measures: (1) identify external drivers and their directions; and (2) develop strategy, measures and targets that are in line with the external directions (people, system and organizational procedures). Issue II: develop the measures under operational and accountability perspective: (1) draw a customer diagram and list down all the results produced; and (2) classify works into strategic and routine work.
Overall Company Objectives Operational perspective Results Results Process Skills & Behavior Psychological Aspect Self Esteem Compatibility Accountability perspective

Performance measurement

651

Long term perspective People System Procedure

Figure 3. Framework and steps for the MDMP implementation

IMDS 106,5

652

For strategic work, measures have to be carried out in four aspects: (1) results; (2) process; (3) behavior; and (4) physiological aspect. For routine work, only measures on the results are to be carried out. Measures need to be logically linked to the long-term perspective. The measures and the information from these measures derived by the MDMP cover several areas, namely, long terms, operational and accountability. This is to ensure individuals can balance both short- and long-term focus. Basically, the measures in the long-term perspectives would provide the infrastructure for individuals in order to allow them to achieve the breakthrough-performance level. At the same time, the measures in the accountability perspective would allow the balance of the focus on both routine and strategic tasks. See Appendix 3 for the results from having implemented the MDMP to measure the performance levels of accountants and nancial analysts. The next step is to test the usefulness and applicability of the MDMP. The experiment was conducted with a group of 16 staffs who have worked in the areas of accounting and nance. Figure 4 shows the overview of the tasks completed in this step. Before the experiment, lengthy discussion sessions were made to ensure proper background, clarity of their expectation, and their roles during and after the test. The AHP was used for two purposes: (1) to determine the priority and relationships among the criteria in the set; and (2) to determine the level of usefulness and applicability of the 6 measurement techniques with respect to the set of prioritized criteria.
Task 1: Selection of Participants

Task 2: Familiarization of the Experiment with Participants

Task 3: Identification of Factors on Performance Measurement

Figure 4. Tasks completed during the experimental/testing step in the research process

Task 4: Verification on Compatibility of the Factors with the Set of 7 Criteria and Prioritization of these 7 Criteria (thru the AHP)

Task 5: Evaluation of the Goodness of the 5 Current Measurement Techniques and the MDMP to respond to how well each technique corresponds to the prioritized set of criteria (thru the AHP)

The AHP is a multi-attribute decision tool. It aims to integrate the results (in this case, the opinions from the participants) with pair-wise comparisons. It is a method useful for breaking down a complex and unstructured situation into smaller parts; arranging these parts into a hierarchic order, assigning numerical values to subjective judgements on the relative importance of each part, and nally synthesizing the judgment to determine which part has the highest priority (Saaty, 2001). Essentially, there are three principles underlying the AHP. These are the principle of: (1) constructing hierarchy; (2) establishing priorities; and (3) logical consistency. The AHP depends mainly on the experience of the participants. Given this feature, the experiment may be conducted without requiring a large number of participants. Table III shows the participants proles. On the average, they have a nine-year working experience and their performance levels have been extensively evaluated at least once. Coupled with the discussion sessions, the quality of participants should no longer represent a major concern. During the experiment, the participants were rst asked to freely and independently identify the critical factors of a performance measurement approach. This activity was important so that the issues of subject familiarity can be resolved. At the same time, these factors initiated from the participants would be compared with the set of criteria, earlier identied from the second step. In other words, the next task was to match these factors with the seven criteria. It was found that these factors are consistent with the set of seven criteria. This compatibility task was necessary to ensure that the 16 participants had basic understanding on the subject of performance measurement and were qualied to provide the judgment that later would be used to evaluate the usefulness and applicability of the ve current measurement techniques and the MDMP. Then, the participants provided the opinions (judgment) on the factors they earlier had identied by using the AHP. The feedback from the participants implied that white-collar workers place the highest value on the measurement reliability. This is likely due to the fact that they have long been suffered from the non-reliable measurement approach. The strategic congruence, measurement coverage, an ability to support different types of work, and user acceptance were considered to be the critical criteria that should be embedded in a measurement task. The better the technique can relate to these criteria implies that it is more useful and better applicable for measuring the performance level of white-collar workforce. To complete the
Type of work Senior Operational level levela 3 6 9 2 5 7 Education Bachelor . Bachelor degree degree 2 6 8 3 5 8

Performance measurement

653

Total Male Female Total 5 11 16

Number of years of experience 11.4 8.5 9.4 Table III. Proles of participants

Note: aSenior staffs are those with more than ve years of work experience with at least one person under supervision

IMDS 106,5

654

experiment, the discussion sessions were conducted separately between the two groups of participants: senior and operational staffs. The observation results from this classication were quite intriguing. Senior staffs appeared to value the strategic congruence aspect about the same level relatively to that of reliability. They believed that the acceptance of their subordinates is crucial to the success of a performance measurement approach. Operational staffs, on the other hand, paid most of their attention to reliability. The study also discovered that the measurement simplicity aspect was not considered to be important. The analogy of a sand cone model, in Figure 5, is used to summarize these prioritized ndings in terms of nice to have, should have, and must have. Tables IV-VII demonstrate the overall results from the rst four tasks. In regard to the consistency of the experiment, these priorities (or prioritized criteria) are obtained with the 0.8 percent consistency ratio that is quite excellent compared to the standard of 20 percent for the size of 10-30 participants (Takala, 2002). Saaty (2001) states that, with larger sample sizes, the 10 percent or less is acceptable. In addition, the implementation index (IMPL), to be derived by dividing standard deviation by the priority of that aspect (also known as a variability coefcient), was used. The lower the index represents to the higher the reliability of the priority of that characteristic (Leskinen and Takala, 2005). All IMPLs were lower than the acceptable level of 1.0 with the lowest indexes placed on reliability and strategic congruence as stated in table. At this moment, the task of prioritizing the criteria to be used for comparing and evaluating the usefulness and applicability of the ve current measurement techniques and the MDMP has been completed. Finally, the next task was to test the ve current measurement techniques and the MDMP. This task also involved the same group of 16 participants. They provided their opinions on the results after the implementations of these ve techniques and the MDMP with respect to the set of prioritized criteria. Tables VIII and IX demonstrate the experimental results, given that the participants were well-informed and -experienced with each technique, including the MDMP. 4. Analysis and conclusion The results from Table VIII can be analyzed and concluded as follows. The MDMP appears to correspond well with a prioritized set of seven criteria. It is perceived to be the most useful technique during the experiment and to be readily applied relatively to

Measurement Simplicity

Nice to Have or Option

Figure 5. Prioritized criteria for evaluating the usefulness and applicability of the ve current measurement techniques and the MDMP the sand cone model analogy

Measuree Acceptance Measurement Coverage

Measurer Acceptance Should Have or Support Ability to support different types of work Must Have or Foundation

Reliability

Strategic Congruence

Critical factors identied by participants Yes Yes Yes Yes No Yes No No No No No 22a 9 4 4 3 1 1 1 1 1 1 48

Compatibility to a set of seven criteria

Number of appearances

Percentage of appearances 45.8 18.8 8.3 8.3 6.3 2.1 2.1 2.1 2.1 2.1 2.1 100

1. Reliability 2. Ability to point out strength and weakness 3. Easy to measure and understand 4. Ability to support different type of work 5. Ability to compare performance with other colleagues 6. Acceptance of management 7. Transparent 8. Ability to link to compensation 9. Ability to measure both productivity and efciency 10. Ability to measure only relevant subjects 11. Effective data collection process Total

Notes: a The initial answers from participants were evaluated and followed up. They were rephrased (this is completed with the participant consent). Therefore, it is possible that the results shown in the number of appearances column may be more than the number of participants. This is because the same response may be stated more than once by a participant. For example, a participant may rate both reliability and objectivity in the top three priority factors. Then, the objectivity was analyzed and eventually was rephrased as reliability

Performance measurement

655

Table IV. Number of appearance of top three critical factors

IMDS 106,5

656

the other ve current measurement techniques. Based on the experimental results, the MDMP provides the reliable outputs (i.e. measures) that are aligned with organizational strategies. This nding is important since these two criteria represent the must have or foundation group that reects usefulness and applicability of a performance measurement technique. This alignment also ensures that the attention will be made to nd ways to improve the performance level of white-collar workers (instead of controlling and administering purposes). Based on the feedback from the participants, the MDMP appears to capture and address the primary nature of white-collar work. The MDMP focuses on outputs and outcomes. At the same time, it attempts to incorporate and capture the activities performed for task completion. Given this appearance, the acceptance from both workers (measuree) and supervisors

Theoretical criteria 1. Reliability 2. Strategic congruence 3. Measurement coverage 4. Ability to support different type of work 5. User acceptance-measurer 6. User acceptance-measuree 7. Measurement simplicity Total Consistency ratio

Overall (percent) 24 16 14 13 13 13 7 100 0.8

Senior staff (percent) 18 18 14 13 13 15 8 100 1.1

Operational staff (percent) 35 13 13 13 11 10 5 100 3.4

Table V. Group prioritization on the set of seven criteria using the geometric mean method

Characteristics 1. 2. 3. 4. 5. 6. 7. Reliability Strategic congruence Measurement coverage Ability to support different type of work User acceptance-measurer User acceptance-measuree Measurement simplicity

Overall 0.43 0.30 0.72 0.50 0.80 0.58 1.59

Senior staff 0.29 0.27 0.81 0.56 0.72 0.54 1.69

Operational staff 0.30 0.38 0.67 0.43 1.06 0.63 0.92

Table VI. IMPL results

Weight (percent) 1. 2. 3. 4. 5. 6. 7. Reliability Strategic Congruence Measurement coverage Ability to support different type of work User acceptance (measurer) User acceptance (measuree) Measurement simplicity 24 16 14 13 13 13 7

Table VII. Weight assigned on the set of criteria

(measurer) is relative high with respect to other ve measurement techniques. Nevertheless, the MDMP is not perceived to be simple and user-friendly. This can be attributed to the measurement coverage or comprehensiveness. At the same time, the MDMP may be more complex that because it has combined several strong features from many techniques. From the follow-up interviews with the participants, there may be negative impacts on measurement simplicity from excellent measurement coverage as well as the criteria on strategic congruence and ability to support different type of work. Owing to the trade-off issues derived from the conicting effects among a prioritized set of seven criteria, the questions on measurement simplicity require further examinations and studies so that the MDMP can become more practical and easier to use. In regard to the feedback from the external experts, they generally support the ndings on the prioritized set of criteria when evaluating the usefulness and applicability of the ve current measurement techniques and the MDMP. They also provide positive opinions on the framework and its components (such as the steps) of the MDMP. Their concern is more on the utilization and sustainability of the MDMP. There were several questions that had been raised, such as how to consistently records and keep the results from the MDMP, the cost of implementing and using the MDMP, the ability to integrate the MDMP and its results to the management process, whether

Performance measurement

657

GRS (percent) Reliability Strategic congruence Measurement coverage Ability to support different type of work 5. User acceptance (measurer) 6. User acceptance (measuree) 7. Measurement simplicity 1. 2. 3. 4. 7 6 9 8 9 10 28

BARS (percent) 6 4 6 6 7 7 22

MBO (percent) 11 13 6 5 10 9 24

Zigons (percent) 15 13 12 13 19 14 15

TMD (percent) 23 22 22 25 24 24 7

MDMP (percent) 37 41 45 43 31 36 5 Table VIII. Results from prioritized criteria

Consistant ratio GRS BARS MBO Zigons TMD MDMP (percent) (percent) (percent) (percent) (percent) (percent) (percent) Reliability Strategic Congruence Measurement coverage Ability to support different type of work 5. User acceptance (measurer) 6. User acceptance (measuree) 7. Measurement simplicity Overall Rank 1. 2. 3. 4. 0.9 1.5 2.0 2.5 1.2 1.8 2.6 1.6 1.71 0.98 1.23 1.09 1.16 1.35 1.85 9.37 5 1.50 0.62 0.86 0.86 0.85 0.97 1.45 7.10 6 2.71 2.11 0.81 0.67 1.28 1.13 1.59 10.31 4 3.69 2.09 1.67 1.69 2.34 1.82 1.03 14.33 3 5.73 3.51 3.13 3.39 2.98 3.09 0.48 22.31 2 9.09 6.59 6.20 5.72 3.89 4.77 0.33 36.59 1 Table IX. Consistency ratios and weighted results on prioritized criteria

IMDS 106,5

658

the sustainability question was part of the development process, etc. It should be noted that the external expert did not participate in the experiment. They are from various countries such as the US and Australia. The surveys were randomly distributed to them after the ndings had been nalized. Their comments were necessary to help verify the research results and the conclusion. With respect to the research limitations, the MDMP still needs to be further examined. Owing to the small size of the participants, the ndings may have to be re-examined. More participants in the experiment can help conrm the original results on two areas: (1) the set of prioritized criteria (from Figure 5 and Table VII); and (2) the usefulness/applicability level of the MDMP (Table VII). In addition to the accounting and nancial areas, the MDMP should be tested by other white-collar professions such as research, engineering, planning, marketing, etc. On the other hand, it is important that the characteristics of white-collar workers (such as motivation, self-managed work, language prociency, etc.) are to be recognized and should become more explicit in the performance measurement technique. To further analyze the usefulness and applicability of the MDMP, several issues will have to be addressed. In addition to the questions raised by the external experts, the areas such as what circumstance (e.g. long experience, highly motivated, self-managed tasks, good command in language, strong organizational culture, etc.) are compatible with the implementation and sustainability of the MDMP. Owing to the fact that the experiment involved only the group of 16 accountants and nancial analysts, the generalization on the MDMP usefulness and applicability is not possible at this time. In addition, the MDMP should explicitly consider the roles of emerging technology and constant changes in work environment so that the impacts on the performance levels of white-collar workforce can be better understood. The trade-off among seven prioritized criteria used to compare and evaluate different measurement techniques represent another area in which more studies can be made. For example, the main effort should be between the measurement simplicity and the remaining criteria. Furthermore, the MDMP should be observed within the context of a performance-based contract. Can the MDMP strengthen the application of this contract? This performance-based contract has become more important as well as more common in the workplace where white-collar work predominates such as law and consultant rms. Finally, instead of focusing on comparing with different measurement techniques from the set of criteria (as shown in Appendix 2), the future research effort on extending the work on the MDMP should concentrate on evaluating and appraising its conceptual framework and measurement dimensions with other models/methodologies (as clearly summarized by Ramirez and Nembhard, 2004). In conclusion, there are several interesting and useful ndings from the research. Although the proposed framework needs to be further investigated and examine, a few contributions can be specied within the context of the research in the area of performance measurement for white-collar workforce. The rst one is the issue of what should be used to help evaluate a new framework/technique/approach to be developed for measuring the performance levels of white-collar workforce. The prioritized set of criteria (shown by the sand cone model Figure 5) helps identify the level of importance on design/ development consideration items. In addition, the framework for

white-collar workforce performance measurement is proposed and tested (Figure 3). This framework can potentially be a reasonable starting point (since it has combined strong features from other techniques) for continuous research. Finally, the research process itself can be further discussed. The effort may focus on what should be the viable research process to help verify the work on developing a new framework, approach, or technique in measuring white-collar workforce performance.

Performance measurement

659

References Beruvides, M.G. and Koelling, C.P. (1992), Management tools: a technological issue in white-collar/knowledge work, Journal of Management of Technology III, Institute of Industrial Engineers, pp. 1357-63. Brown, M.G. and Svenson, R.A. (1998), Measuring R&D productivity, from the Zigon Performance Group web site, available at: www.zigonperf.com Cherrington, D.J. (1991), The Management of Human Resource, Allyn & Bacon, New York, NY. Deming, W.E. (1986), Out of the Crisis, MIT Center for Advanced Engineering Study, Cambridge, MA. Gregerman, I.B. (1981), Knowledge Worker Productivity, AMACOM, New York, NY. Jones, S.D. and Schilling, D.J. (2000), Measuring Team Performance: A Step-By-Step, Customizable Approach for Managers, Facilitators, and Team Leaders, Jossey-Bass, New York, NY. Kaplan, R.S. and Norton, D.P. (2004), The Strategy Maps, Harvard Business School Press, Cambridge, MA. Kaydos, W. (1991), Measuring, Managing, and Maximizing Performance, Productivity Press, Atlanta, GA. Leskinen, J. and Takala, J. (2005), How to develop holistic satisfaction in Finnish ice hockey business environment, International Journal of Management & Enterprise Development, Vol. 2 No. 1, pp. 38-45. Levinson, H. (2003), Management by Whose Objectives, Harvard Business School Press, Boston, MA. Noe, R.A., Hollenbeck, J.R., Gerhart, B. and Wright, P.M. (1996), Human Resource Management, McGraw-Hill, New York, NY. Ramirez, Y.W. and Nembhard, D.A. (2004), Measuring knowledge worker productivity: a taxonomy, Journal of Intellectual Capital, Vol. 5 No. 4, pp. 602-28. Rummler, G.A. and Brache, A.P. (1995), Improving Performance: How to Manage the White Space on the Organization Chart, Jossey-Bass, San Francisco, CA. Saaty, T.L. (2001), Decision Making for Leaders, RWS Publication, New York, NY. Sanchez, C.M. (2000), Performance improvement in international environments: designing individual performance interventions to t national cultures, Performance Improvement Quarterly, Vol. 13 No. 2, pp. 56-70. Schutz, W. (1994), The Human Element: Productivity, Self-Esteem, and the Bottom Line, Jossey-Bass, San Francisco, CA. Sink, D.S. and Tuttle, T.C. (1989), Planning and Measurement in Your Organization of the Future, IIE Press, Norcross, GA. Sumanth, D.J. (1998), Total Productivity Management, St Lucie Press, Delray Beach, FL.

IMDS 106,5

660

Takala, J. (2002), Analyzing and synthesizing multi-focused manufacturing strategies by analytical hierarchy process, International Journal of Manufacturing Technology and Management, Vol. 4 No. 5, pp. 345-50. Thomas, B.E. and Baron, J.P. (1994), Evaluating knowledge worker productivity: literature review, USACERL Interim Report. Thor, C.G. (1987), 10 rules for building a measurement system, White paper for the American Productivity and Quality Center. Viken, K. (1995), Team Measurement: Some Whys, Whats and Hows, Center for the Study of Work Teams, University of North Texas, Austin, TX. Zigon, J. (1998), Measuring the hard stuff: teams and other hard-to-measure work, Zigon Performance Group, available at: www.zigonperf.com

Further reading Dixon, J.R., Nanni, A.J. and Vollmann, T.E. (1990), The new performance challenges: measuring operations of world-class competitions, Down Jones-Irwin, New York, NY. Green, K.W., Medlin, B. and Whitten, D. (2004), Developing optimism to improve performance: an approach for the manufacturing sector, Industrial Management & Data Systems, Vol. 104 No. 5, pp. 106-14. Helo, P. (2005), Managing agility and productivity in the electronics industry, Industrial Management & Data Systems, Vol. 104 No. 7, pp. 567-77. Helo, P. and Szekely, B. (2004), Logistics information systems, Industrial Management & Data Systems, Vol. 105 No. 2, pp. 5-18. Laurent, A. (1998), Premium on performance, available at: www.govexec.com Minor, J.B. (1992), Industrial-Organizational Psychology, McGraw-Hill, New York, NY. Phusavat, K., Suwansaranyu, U. and Takala, J. (2003), Evaluating productivity measurement approaches for white-collar workers, Proceedings of the 4th Annual International Symposium in Industrial Engineering, Kasetsart University, Thailand, pp. 40-9. Saaty, T.L. (1980), The Analytic Hierarchy Process, McGraw-Hill, New York, NY. Schainbatt, A.H. (1982), How companies measure productivity of engineers and scientists, Journal of Research Management, Vol. XXV No. 3. Sink, D.S. (1985), Productivity Management: Planning, Measurement and Evaluation, Control and Improvement, Wiley, New York, NY. Yasmin, M.M., Wafa, M.A. and Zimmerer, T.W. (1995), A prole of successful U.S. manufacturers: a starting point for evaluating the effectiveness of manufacturing strategies and practices, Industrial Management & Data Systems, Vol. 95 No. 10, pp. 8-18.

Appendix 1. Description of the ve current white-collar workforce performance measurement techniques The details of the approaches being evaluated are as follows. GRS is the most common form of the attribute approach to performance measurement. Using this method, the manager considers one employee at a time, circling the number that signies how much of that trait the individual has. GRS can provide the rater with a number of different points (a discrete scale) or with a continuum along which the rater simply places a check mark (a continuous scale) (Noe et al., 1996).

BARS builds on the critical incidents approach, the approach that requires managers to keep record of specic examples of effective and ineffective performance on the part of each employee. The BARS is designed to specially dene performance dimensions by developing behavioral anchors associated with different levels of performance (Noe et al., 1996). MBO is popular in both private and public organizations. In an MBO system, the top management team rst denes the companys strategic goal for the coming year. These goals are passed on to the next layer of management, and these managers dene the goals they must achieve for the company to reach its goals. This goal-setting process cascades down the organization so that all managers are setting goals that help the company achieve its goals. These goals are used as the standards by which an individuals performance is evaluated (Noe et al., 1996). The Zigons third-generation process for measuring performance has been developed by Jack Zigon based on many years of his experience. It is basically a six-step approach with a focus on the results. The outstanding benets of this approach include the fact that it ensures the strategic alignment of the measures with the organization goals. Additionally, the approach provides an easy to follow roadmap to develop measures for hard to measure work. The steps included are review the organizational measures, dene measurement starting points, weight the results, and develop performance measures and standards and decide how to track them (Zigon, 1998). The TMD is a result-oriented measurement approach developed by the Tarkenton Productivity Group in the 1980s. The approach was developed specically for measuring performance of white-collar and knowledge workers. The approach ensures that the measures obtained reect the views of all related parties. These include strategic self-view, customer view, accountability view, and information system view (Jones and Schilling, 2000).

Performance measurement

661

Appendix 2. Details on a set of criteria used to assess the strengths and weaknesses of performance measurement techniques The set of criteria includes: . Measurement simplicity. The scientists or engineers whose performance is measured against 20-30 variables typically ignore most or all of them (Brown and Svenson, 1998). . Compatibility to the nature of white-collar workers. Nature of white-collar work must be considered as well as uncertainty relating to task completion. . Reliability. Whenever it is possible, it would be better to use quantiable units. However, that is not always the case for work performed by white-collar workers. It is also possible to utilize subjective gures or judgment. . Measurement coverage or comprehensiveness. An effective white-collar performance measurement approach should address various aspects, including the focus on process, behavior/skill, result, and psychology. This would basically allow us to point out where the problems and opportunities are (Kaydos, 1991), (Gregerman, 1981), (Viken, 1995), (Cherrington, 1991) and (Schutz, 1994). . Strategic congruence. This implies the extent to which the measurement approach brings out the performance level that is congruent with the organizations strategy, goals, and culture. . Acceptability. acceptability refers to whether the people who utilize the performance measurement approach accept its results. . Measurement standards. This has to do with the target setting process. . Measurement frequency. The frequency that the measurement has been carried out has an inuence on the effective of the performance measurement.

IMDS 106,5

Appendix 3. Example of measures developed by implementing the MDMP

Operational Perspective

662

Result oriented measure Number of days before the deadline which we can submit the report

Process oriented measures Amt of time spent on consolidating data, review and make adjustment if any

Amt of time spent on preparing the input files Parts Sales

Amt of time spent on obtaining COS & price

Amt of time spent on obtaining Volume

Amt of time spent on obtaining the input files Opex

Amt of time spent on obtaining the input files Sales Allow

Number of days before the deadline which we receive the FX Rate

Amt of time spent on preparing the input files Opex

Amt of time spent on preparing the input files Sales Allow.

Behavior and skills oriented measure Using assumptions obtained from depts to develop forecast and at the same time provide feedback to the depts for managing the business Psychological aspect measure Self-esteem and Atmosphere Compatibility

Long- term perspective People Number of days in training Business process Procedure % of procedures developed to total processes System % of manual work

Number of days in training Technical

Corresponding author Kongkiti Phusavat can be contacted at: fengkkp@ku.ac.th To purchase reprints of this article please e-mail: reprints@emeraldinsight.com Or visit our web site for further details: www.emeraldinsight.com/reprints