International Journal of Engineering Research & Technology (IJERT)

ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

Cobit 4.1: A Maturity Level Framework For

Measurement of Information System
Performance (Case Study: Academic Bureau at
Universitas Respati Yogyakarta)
Herison Surbakti
Universitas Respati Yogyakarta
Indonesia

Abstract - Universitas Respati Yogyakarta requires methods thorough as IT audit framework as developed based on the
and structured approach in its evaluation, especially in rules/procedures of internal company/institution where COBIT is
academic departments. This should to be done to assess the fit used, so the time will be measured in accordance with the
between institutional objectives with management that has conditions, rules, procedures and norms that work in the
been applied. COBIT is a framework used to assess, measure company. COBIT has also been developed on an ongoing basis by
and control the performance of institutions in the professional NGOs auditors spread throughout much of the
management of IS/IT. COBIT is also accepted and country, where in each state builts a relationship that can manage
harmonized by its users, because the framework is built from these professionals.
the goal, rules and institutional policy where all processes are
analyzed by looking at the alignment between the objectives to Performance measurement in academic information
RT
be achieved by the procedures/policies implemented by the system at the Universitas Respati Yogyakarta that utilizes IT as a
institution. In this study the author uses COBIT (Control means of supporting the process is expected to support the
Objectives for Information and Related Technology) version management of the education process. For example at the
4.1 on the domain Planning Organization (PO). The results of beginning of the selection for new students, the learning process
IJE

the study conducted performance measurements are made in which is done, lectures supporting components such as the
the form of academic information system analysis, mapping method used, the curriculum and other provisions such as faculty,
the level of maturity and recommendation for Universitas students, facilities, other facilities until the graduation of students
Respati Yogyakarta, which is expected to be a management who need to be evaluated in order to produce quality and good
model for other institutions. service and competitive education.
Keywords 1.1. Problems
COBIT, measurement of performance, audit, CSF, KGI, KPI
1. There has been no measurement of the performance model of
academic information system at the Universitas Respati
Yogyakarta.
1. INTRODUCTION 2. COBIT as a framework used as a reference for measuring the
Some companies do not hesitate to invest their share in performance of academic information system at the
the field of Information Technology (IT), although the Universitas Respati Yogyakarta.
investments make enormous drain on the budget. This is done as 1.2. Limitations
an effort to get the convenience and benefits of the use of IT, Limitation of the problems in this study is:
which is expected to help the performance of the company to 1. This study refers to the standard of COBIT 4.1 (Control
conduct a competitive business strategy. However it turns out, the Objectives for informatian and related technology) made by
investment made by the outcome often is not generated. the IT Governance Institute (ITGI) an institution that manage
and organize IT governance.
Assessment and evaluation of investments that have 2. COBIT domain used is on Planning Organization (PO).
been issued for the implementation of IT is proper to be 1.4. Research Objectives
considered. Based on some research explained that the company
The purpose of this study is:
has begun to realize and start doing performance measurement
1. Make a model of the academic information system refers to the
and evaluation. To make the use of performance measurement
COBIT framework according to the characteristics of
and management of IT, then The IT Governance Institute (ITGI)
Universitas Respati Yogyakarta.
as an institution conducting the IT governance arrangements that
2. Gain the insight about the performance of academic
have standardized tools or frameworks is widely used in the world
including COBIT, ITIL, COSO, ISO, and so on [2],[4],[5],[9]. information system at the Universitas Respati Yogyakarta.
3. Perform analysis and assess the fit between policies on
COBIT is a reference method/framework for academic standards at the Universitas Respati Yogyakarta
measurement and control of information technology. COBIT implementation, it is used as a reference for the evaluation of the
framework is a standard that is considered the most complete and existing academic system at the Universitas Respati Yogyakarta.

IJERTV3IS080897 www.ijert.org 999

(This work is licensed under a Creative Commons Attribution 4.0 International License.)
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

2. LITERATURE REVIEW

2.1. Information Technology Architecture

The Information Technology Architecture (ITA) is
described through a set of views, each from the perspective of a
different stakeholder. An individual view captures items
meaningful to the stakeholders as elements and their
interrelationships expressed in a standard form, the structure of
the view, and the views correlation with other views. Typically,
groups within the organization possess a mixture of Commercial
Off-The-Shelf (COTS), frameworks and custom-developed
legacy applications as well as data stored in databases and
directories. In such an environment, functionality and data
embedded in one application cannot be easily merged with
functionality and data provided by another. Such an endeavor
usually requires the creation of a yet another separate application.
Further, each of these applications relies on a proprietary client
for user interaction [10].

Figure 2.2 Relationship of data, information, and knowledge

RT
IJE

Figure 2.3 Description of the data, information and knowledge

Figure 2.1 Conceptual View of the architecture (EOHHS CTO
Organization Information Technology Architecture, Version 2.0, 2007) 2.3. Academic Information System
To achieve the goal of the learning process that is done,
2.2. Information System it needs a system to regulate the course of the process of the
It conceptually describes the data objects, events, learning system, from start of new admissions, teaching and
activities, and transactions, which have no meaning or effect learning activities, faculty, students, lecturer's method used, the
directly to the user. Information is data that has been processed in curriculum, the process of filling up the study card, and the
such a way that increases user knowledge [10],[1],[3],[4]. While graduation evaluation process. This is what is known as the
knowledge is a combination of instincts, ideas, rules, and academic system [4],[9]. Academic System is made with the
procedures, that drives the actions or decisions. The relationship purpose of regulating the learning process to fit the expected
between data, information and knowledge can be mapped in goals, in accordance with the vision and mission of educational
Figure 2.2, while the data, information, and knowledge can be institutions that shelter, and also in accordance with the rules
described in the form of a pyramid, as shown in Figure 2.3. instituted university education.
2.4. COBIT Framework
COBIT Framework is an IT governance framework
aimed at management, IT service staff, department control, and
audit function more critical to the business process owner. COBIT
Framework as a framework for the management needs for
measurement and control of information technology provides the
tools to measure the ability of information technology that will
continue to be developed. COBIT framework is created to serve
as a reference to exercise control over information technology and

IJERTV3IS080897 www.ijert.org 1000

(This work is licensed under a Creative Commons Attribution 4.0 International License.)
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

can ensure confidenciality, integrity and availability of the data From the data obtained, and then the researcher
[5],[6],[7]. performed the steps in the performance measurement system of
The results obtained from the IT Governance is used as academic information such as explained figure 3.1.
a reference or management guidelines, one of which is COBIT
which has a measuring tool such as maturity models, CSF, KGI
and KPI. COBIT has a measurement indicator management of
information technology in business processes. The relationship
between the measuring tool set is described in Figure 2.4.

Figure 2.4 Model CSF, KPI dan KGI

3. RESEARCH METHOD
The research method used in this study:
1. Studying the source literature of COBIT Framework, audit
trails, and academic information system.
2. Data collection from internal documents regarding
institutions such as vision, mission, goals, IT architecture,
organizational structure, master plan development, including
RT

IT management policies.
3. General identification of the learning process that is carried
out at the Universitas Respati Yogyakarta.
IJE

4. Perform an analysis of the weaknesses, strengths,

opportunities and challenges of the institution (SWOT
Analysis).
5. Direct observation of the academic activities to gain
referring to the direct point of the matter.
6. Perform analysis of the academic information system that
refers to the COBIT framework using a questionnaire.
7. Conducting interviews to related parties such as bureau of
academic chief, chairman of the study program, IT staff and Figure 3.1 Stages Performance and Measurement of Academic
administrative staff, and faculty. Information System
8. Perform data analysis.

3.1 Stages of Performance Measurement of Academic 4. IMPLEMENTATION

Information System 4.1 Performance Measurement of Academic Information
The research method used is performance System Framework Based on COBIT
measurement system with reference to the academic information To perform the management and measurement of
system and considering some points as follows: performance against the academic information system, the
1. Internal Institute Environment framework which is used as a reference or guideline is COBIT
1. Vision, Mission, and Institution Goals Framework which includes the maturity model, CSF, KPI and
2. Academic Institutions Activity Cycle KGI. In general, IT arrangement in Universitas Respati
3. Strategy and Scope of Business (Business Scope) Yogyakarta aims to drive, ensure, and control the institutions in
4. Distinctive Competence order to achieve goal, namely to reduce the risk factors and make
5. Development Master Plan improvements in all aspects so as to support the institution's
6. Planning factors performance [4],[8].
7. Strategic Planning Figure 4.1 describes the framework for the
8. Architectural Institute of Information Technology establishment of IT management adopted from COBIT domains
9. Integrated Academic Information Management directly with Planning Organization (PO) as a focus of study.
System of Institutions
10. Integrated Systems Students

IJERTV3IS080897 www.ijert.org 1001

(This work is licensed under a Creative Commons Attribution 4.0 International License.)
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

Figure 4.1 IT Institutional Management Framework Based on COBIT's

PO domain (ITGI, Management Guidelines, 2008)
RT
4.2 Performance Measurement Model of Academic
Information Systems
Figure 4.2 Managing and Organizing the design of academic
information system
IJE

Academic information system procedures and policies

in its practice have clear rules, standardized and should be
4.3 Performance measurement process on the domain of
Planning and Organizing
informed. The determination of the performance achievement
The measurement process is expected to manage IT
indicators (strategic objectives) using CSF, KPI and KGI are used resources and achieve strategic goals and objectives such as
to achieve this. Figure 4.2 is a planning model for the effectiveness, efficiency, confidentiality, integrity, availability,
management of academic information system at the Universitas compliance and reliability of the academic management
Respati Yogyakarta with reference to the COBIT Framework. information systems, so this stage is the translation of the vision
and policy institutions. To make the process of performance
measurement at the PO domain, the used of indicator known as
KGI (Key Goal Indicators) and KPI (Key Performance
Indicators), this both indicators related to the CSF (Critical
Success Factors) because CSF was raised from the need for
business done by management and if not done then it will hamper
the pace of organization.
To map the institutional objectives and business
requirements, the process can be done by translating and
identifying indicators associated with the domain Planning
Organization, such as: Establishing the performance of each
process PO, Assign KGI, KPI Assign, Assign CSF, and Maturity
models. The measurement process is shown in Figure 4.3.

IJERTV3IS080897 www.ijert.org 1002

(This work is licensed under a Creative Commons Attribution 4.0 International License.)
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

to, but they may not address business needs.

4 Managed Service levels are increasingly defined in the

system requirements definition phase and
incorporated into the design of the
application and operational environments.
Customer satisfaction is routinely measured
and assessed. Performance measures reflect
customer needs, rather than IT goals. The
measures for assessing service levels are
becoming standardised and reflect industry
norms. The criteria for defining service
levels are based on business criticality and
include availability, reliability, performance,
growth capacity, user support, continuity
planning and security considerations. Root
cause analysis is routinely performed when
service levels are not met.

5 Optimised Service levels are continuously re-evaluated

Figure 4.3 Performance Measurement Process of Academic Information to ensure alignment of IT and business
System Management objectives, whilst taking advantage of
technology, including the cost-benefit ratio.
4. Descriptive Maturity Model and Measurement All service level management processes are
Techniques subject to continuous improvement.
Descriptive measurement techniques are made by the
nominal size to sort objects from the lowest to the highest, these
measurements only give the order by rank. Measurements were
carried out directly from values that refers to the value of the
existing sorting in maturity models as show in table 4.1. This simplifies the process of calculating and mapping the
maturity model. These measurements were made to map the
Table 4.1 Maturity Model (ITGI, Management Guidelines, 2008) position of IS governance maturity model into the existing, so that
from the model that will be scalable of the IT management in
RT
Level Name When planning and organizing domain occupies a level determined in
accordance with criteria that are owned. From its placement then
0 Non-Existent Management has not recognised the need for it can be used to evaluate candidates for the management for
a process for defining service levels. further improving and enhancing the performance of the
IJE

Accountabilities and responsibilities for management of the existing IS governance in the institution.
monitoring them are not assigned.

1 Initial/Ad-Hoc There is awareness of the need to manage

service levels, but the process is informal
4.1 Summary Results of Audit Implementation Design
and reactive. The responsibility and Model
accountability for defining and managing Recapitulation of this implementation shows all the
services are not defined. If performance results of data from questionnaires filled out by respondents from
measurements exist, they are qualitative different levels of management. The questionnaires recapitulation
only with imprecisely defined goals. use descriptive measurement techniques. This is a standard
Reporting is informal, infrequent and
indicator outcome of determination recap associated with
inconsistent.
quantitative data expressed in simple calculations such as the total
2 Repeatable There are agreed-upon service levels, but
value of the whole, the index, average, and percentage. The
they are informal and not reviewed. Service answers given by the respondents will have scores that is
level reporting is incomplete and may be equivalent to each of its maturity level. The total respondent is 25
irrelevant or misleading for customers. and the recapitulation of PO results in table 4.3.
Service level reporting is dependent on the The respondents calculation summary per PO (Planning
skills and initiative of individual managers. Organization) domain by using descriptive calculation formula
A service level co-ordinator is appointed obtained results as shown in Table 4.3 and 4.4.
with defined responsibilities, but limited
authority. If a process for compliance to
SLAs exists, it is voluntary and not enforced.

3 Defined Responsibilities are well defined, but with

discretionary authority. The SLA
development process is in place with
checkpoints for reassessing service levels
and customer satisfaction. Services and
service levels are defined, documented and
agreed-upon using a standard process.
Service level shortfalls are identified, but
procedures on how to resolve shortfalls are
informal. There is a clear linkage between
expected service level achievement and the
funding provided. Service levels are agreed

IJERTV3IS080897 www.ijert.org 1003

(This work is licensed under a Creative Commons Attribution 4.0 International License.)
 International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181
Vol. 3 Issue 8, August - 2014

Table 4.3 PO Domain Questionnaire recapitulation

5. CONCLUSION
PO TOTAL INDEX % LEVEL
From the results obtained it appears that the institution
PO-1 56 2.24 44.80% 2 has a different level for each PO domain. In general, institution
occupies level 3, which means that the institution has procedures
PO-2 46 2.19 61.33% 3 in management, has been communicated and documented for each
element in the institution. However, the implementation is still
PO-3 49 2.33 65.33% 3 highly dependent on existing human resources willing to perform
the procedure or not. That existing procedures are still limited to
PO-4 56 2.67 74.67% 4 the formalization of implementation available, and management
needs to improve planning and organizing.
PO-5 44 2.10 58.67% 3 The discrepancy between the technical implementation
of management policies with existing will be reduced by the
PO-6 57 2.71 76.00% 4 control and measurement refers to guidelines such as COBIT
Framework KPI, KGI, CSR and maturity models in COBIT
PO-7 66 3.14 88.00% 5 Framework for monitoring ranging from policy-setting;
monitoring is being conducted at the time, until the goal is
PO-8 59 2.81 78.67% 4 achieved.

PO-9 61 2.90 81.33% 4 6. REFERENCES

PO-10 56 2.67 74.67% 4

[1] Alter, Steven, 1992, Information System A Management

TOTAL 713 33.53 920.80% 36
Perspective, Fourth Edition, Prentice Hall Inc.
[2] Garca, Victoriano Valencia., Vicente, Eugenio J. Fernndez, Dr.,
MEAN 71.3 3.353 92% 3.6 and Aragons, Luis Usero, Dr., 2013, Maturity Model for IT Service
Outsourcing in Higher Education Institutions, (IJACSA)
International Journal of Advanced Computer Science and
Table 4.4 Percentage scale at the level of maturity Applications, Vol. 4, No. 10, 2013.
[3] Gottschalk, Petter, 2012, Knowledge driven service innovation and
RT
Scale Level Maturity Level management : IT strategies for business alignment and value
creation, ISBN: 1-4666-2512-0, 978-1-4666-2512-9, IGI Global.
0 17% 0 Non-Existent [4] Henderi, 2010, Good IT Governance: Framework and Prototype for
Higher Eduation, Creative Communication and Inovative
IJE

18 33% 1 Initial/Ad-Hoc Technology Journal vol 3, no.2 ISSN: 1978-8282.

[5] ISACA, 2006, Integrating COBIT into the IT Audit Process
34 50% 2 Repeatable (Planning, Scope Development, Practises), IT Governance Institute.
[6] IT Governance Institute, 2007, COBIT 4.1, Framework Control
51 66% 3 Defined Objectives Management Guidelines Maturity Models, IT
Governance Institute, ISBN 1-933284-72-2.
67 83% 4 Managed [7] ISACA, 2006, IT Governance Global Status Report.
[8] Law, M. Averill, dan Kelton, David W., Simulation Modelling and
84 100% 5 Optimised Analysis, Second edition, McGraw-Hill., Singapore, 1991.
[9] Maria, Evi, 2012, The Measurement Of Information Technology
Performance In Indonesian Higher Education Institutions In The
Context Of Achieving Institution Business Goals Using COBIT
Based on the results from Table 4.3, for each process in Framework Version 4.1 (Case Study : Satya Wacana Christian
the PO domain, it obtained graphs as in the figure 4.4 below: University, Salatiga), Journal of Arts, Science & Commerce, E-ISSN
2229-4686, ISSN 2231-4172.
[10] Skoczylas, Robert., Sevier, Raoul., Garden, Martin., Snyder, Jason.,
2007, Commonwealth of Massachusetts, 2007, EOHHS CTO
Organization Information Technology Architecture, Version 2.0.

Figure 4.4 Measurement graphs in maturity model

IJERTV3IS080897 www.ijert.org 1004

(This work is licensed under a Creative Commons Attribution 4.0 International License.)