Lirneasia Tremanual v21

Download as pdf or txt
Download as pdf or txt
You are on page 1of 51

Manual of Instructions

for conducting the


Telecom Regulatory Environment (TRE)
Assessment

12 Balcombe Place, Colombo 8, Sri Lanka

asia@lirne.net

Last Edited 24 March 2008 by LIRNEasia

Version 2.1
Table of Contents

 
TABLE OF CONTENTS .............................................................................................................................................. II
1. INTRODUCTION AND ACKNOWLEDGEMENTS................................................................................................... 1

2. WHAT IS THE TRE? ..................................................................................................................................................... 2


2.1 WHAT THE TRE MEASURES............................................................................................................................... 2
2.2 A BRIEF HISTORY ................................................................................................................................................. 3
2.3 HOW IT’S DONE .................................................................................................................................................. 3
2.4 THE THREE SECTORS ......................................................................................................................................... 4
2.5 THE SEVEN DIMENSIONS .................................................................................................................................... 5
2.6 THE LIKERT SCALE............................................................................................................................................ 6
2.7 THE RESPONDENTS: CATEGORIES, WEIGHTS, MINIMUM NUMBERS ............................................................... 6
2.7.1 Categories....................................................................................................................................................... 6
2.7.2 Weights........................................................................................................................................................... 7
2.7.3 Minimum numbers ........................................................................................................................................ 7

3. BEFORE YOU BEGIN: SOME FAQS ........................................................................................................................ 9


3.1 WHO OWNS THE TRE METHODOLOGY? ........................................................................................................... 9
3.2 WHAT QUALIFICATIONS ARE REQUIRED FOR COUNTRY RESEARCHERS? ........................................................ 9
3.3 CAN IT BE A SINGLE COUNTRY SURVEY? ......................................................................................................... 10
4. GETTING STARTED .................................................................................................................................................. 11
4.1 ENSURING CONFIDENTIALITY ......................................................................................................................... 11
4.1.1 Why does Confidentiality matter? ............................................................................................................... 11
4.1.2 How can Quality be ensured? ..................................................................................................................... 11
4.1.3 The role of the Data Guardian .................................................................................................................... 12
4.1.4 What the Researcher should do to ensure confidentiality.......................................................................... 12
4.2 THE TRE TOOLKIT .......................................................................................................................................... 14
4.2.1 The TRE Manual......................................................................................................................................... 14
4.2.2 The TRE Cover Letter ................................................................................................................................. 15
4.2.3 TRE Questionnaire...................................................................................................................................... 15
4.2.4 The Summary of Regulatory Events ........................................................................................................... 16
4.2.5 TRE Excel Data Files.................................................................................................................................. 16
4.2.6 The TRE Country Report ............................................................................................................................ 17

5. CONDUCTING THE TRE STUDY ............................................................................................................................. 19


5.1.1 Respondent Categories ................................................................................................................................ 19
5.1.2 How many in respondents – the sample size............................................................................................... 19
5.1.3 How to compile the list of respondents........................................................................................................ 20
5.1.4 What information is needed? ...................................................................................................................... 21
5.3.1 Different Survey Channels .......................................................................................................................... 22
5.3.2 Web Survey .................................................................................................................................................. 24
5.4.1 Use of Industry Events................................................................................................................................. 25

ii
5.4.2 Optimizing pre-contact ................................................................................................................................ 26
5.4.3 Follow-up ..................................................................................................................................................... 26
5.4.4 Use of colored paper .................................................................................................................................... 27
5.4.5 Incorporating new respondents................................................................................................................... 27
6. COLLECTION AND ANALYSIS OF RESULTS........................................................................................................ 28
7. REFERENCES ........................................................................................................................................................ 34
8. ANNEX A1: TRE COVER LETTER............................................................................................................................ 35

9. ANNEX A2: TRE QUESTIONNAIRE ....................................................................................................................... 36

10. ANNEX A3: TRE SUMMARY OF REGULATORY EVENTS .............................................................................. 40

11. ANNEX A4: THANK YOU NOTE ....................................................................................................................... 45

12. ANNEX A5: RESPONDENT LIST......................................................................................................................... 46

13. ANNEX A6: TRE DATA FILE ............................................................................................................................... 46

14. ANNEX A7: CONFIDENTIALITY AGREEMENT................................................................................................ 47

iii
1. Introduction and Acknowledgements
The aim of this manual is to:

1. to explain the Telecom Regulatory Environment (TRE) Assessment

2. to provide detailed instructions on how to conduct a TRE study

It aims to provide step by step guidance on how to conduct a TRE study, and provides
templates for the key tools that are required for the study.

The manual builds on the latest experience of having implemented the TRE survey in
emerging Asia as well as Latin America.

LIRNEasia is thankful to the researchers who provided their experiences from the past
surveys, enabling us to document best practices and practical tips that help future
researchers. We also acknowledge the input provided by the researchers of our sister
research network in Latin America that used an early version of this manual to conduct the
TRE surveys there and provided feedback based on their experience.

We appreciate if users of this manual report any errors to asia@lirne.net so that we may
continue to improve its quality.

1
2. What is the TRE?
2.1 What the TRE measures

The desired objective of telecom policy reform and regulation is improved sector
performance, measured in four dimensions: connectivity, price, quality of service and
choice.

Investment is a necessary condition for sector performance.

Risk is the primary determinant in making the investment decision – higher the risk, higher
the expected rate of return. At the point of investment, investors consider risks associated
with three environments:

• Macro-level or country
ƒ Market or commercial, and
ƒ Regulatory

The macro-level or country risk is defined as factors that may affect the entire economy,
such as inflation and foreign exchange risk, as well as overall political stability. Commercial
risk is comprised of factors such as demand, effect of substitutable products and services,
and performance of competitors. Regulatory risk is a term of art, defined by Spiller and
Levy (1994) to refer to risk emanating from government action, including but not limited to
the actions of the actual sector-specific regulatory agency with authority over the industry
in question.

Risk is partially a matter of objective analysis - an investor can calculate an expected rate
of return on a new investment based on factors within his control and assumptions based
on factors outside his control. However, risk is, to a great extent, also matter of perception.
Macro-level/Country Risk and as Regulator Risk are both difficult to measure objectively.
But at a minimum a subjective measure of both Country Risk and Regulatory Risk is a
necessity in making the investment.

The scope here is the regulatory environment within which telecom operators and potential
new entrant’s function, that is, a subset of the overall Regulatory Risk environment here
described as the “Telecom Regulatory Environment” (TRE) that includes only the telecom-
specific aspects. This manual presents a tool to measure the TRE in a country.

The TRE tool presented here is a measure of perception that is affected by a number of
different factors. For example, the context of the investment (new vs. incremental) and
nature of the telecom sub-sector (mobile vs. fixed) will affect the perception of the TRE.

2
The TRE has many uses: it is a diagnostic instrument for assessing the performance of the
laws affecting the telecom sector and the various government entities responsible for
implementation. If the scores are low in one aspect against another, it may be that the
regulatory performance needs to be improved. If the performance considered
satisfactory, it may also be possible that the problem is the communication of the
regulatory actions. If the latter conclusion is reached, the appropriate action would be to
improve the way the agency communicates its actions. The TRE can also be used as a
tool for investors to assess regulatory risk in a country. Particularly for investors facing
investment opportunities in the telecom sectors of more than one country, the TRE can
provide a ranking of the countries in terms of telecom-specific regulatory risk.

2.2 A brief history

The original TRE instrument was designed to assess regulatory effects on investment
(Samarajiva & Dokeniya, 2005). It asked stakeholders to assess the telecom regulatory
environment across five dimensions (market entry, allocation of scarce resources,
interconnection, regulation of anti-competitive practices and universal service obligation)
for the fixed and mobile sectors. The dimensions were adapted from the Reference Paper
of the Fourth Protocol of the General Agreement on Trade in Services1. A dimension for
Tariff Regulation was added. The Reference Paper also refers to the Independence of the
Regulator, but this was left out of the dimensions selected because it is seen as a process
variable different from the other outcome variables. The initial TRE surveys (the pilots as
well as the 2006 implementation across 6 countries) therefore had a total number of 12
items (6 in each sector) that required a response. Learning from the 2006 survey, and in
keeping with the changing nature of the telecom sector, new dimensions and new sectors
have been added, as we discuss below.

2.3 How it’s done

The TRE asks senior level stakeholders to assess the Telecom Regulatory Environment in a
country across a number of dimensions.

It makes considerable effort to be parsimonious in the questions because the ideal


respondents are senior managers, including CEOs of operators. A lengthy questionnaire
runs the risk of it being passed down to others to complete.

The respondents are asked to rate the quality of the regulatory environment for each
dimension on a Likert scale ranging from 1 (highly ineffective) to 5 (highly effective). So the

1
http://www.wto.int/english/tratop_e/serv_e/telecom_e/tel23_e.htm
3
respondent has to select a score (1, 2, 3, 4 or 5) and simply circle it (or click, in the case of a
web-based survey). Posing questions in this format ensures that responses can be easily
analyzed without losing any qualitative information as often occurs when using open
ended questions.

The TRE instrument is administered at the same time in six countries by a team of
researchers.

Each instrument is accompanied by a short narrative statement describing each of the


dimensions, using language from the Reference Paper as much as possible, and a bland
summary of significant telecom policy and regulatory actions taken within the previous 12
months. The survey is accompanied by a cover letter stating that participation is voluntary
and that respondent confidentiality is guaranteed.

Questionnaires are sent to large number respondents from agreed-upon categories.


Follow-up emails and phone calls are made to ensure a high response rate.

While the TRE Scores themselves are the most direct output of a TRE study, more meaningful
analysis is done by analyzing the TRE scores in light of actual sector performance indicators
for a particular country.

2.4 The Three sectors

Starting in 2008, the TRE in any country will be done for 3 sectors - fixed, mobile, and
broadband.

It has been argued that with increased fixed-mobile convergence, the separation
between the Fixed and Mobile sectors is irrelevant. LIRNEasia however cites the 2006 (most
recent) TRE survey data, where there is significant difference between the scores received
for the two sectors, in each dimension. This reveals that at present, at least in developing
Asia, the regulatory environments faced by the two sectors are different. Indeed, the
convergence of fixed and mobile TRE scores will be one of the best pieces of evidence on
actual (as opposed to wishful) fixed-mobile convergence. On the other hand, India has
now (actual) converged regulation and therefore the two sectors are treated as one.
Therefore in the case of India, it may be appropriate to send out a joint “Fixed+Mobile”
questionnaire instead of two separate questionnaires.

In addition to the Fixed and Mobile sectors, the third sector of Broadband is being added in
the 2008 survey. In the developing world, Broadband access is emerging as a new ground
for policy making and private sector service provision. But unlike the developed world,
Broadband may not mean the tradition fat pipe reaching homes. Instead most access
may come from mobile broadband. Therefore, the term Broadband refers to multiple

4
modes of accessing the higher speed internet – be it through mobile phones, other mobile
devices, internet kiosks, home PCs. Given that the survey is currently dominantly
conducted in emerging economies we take the lowest (slowest) commonly accepted
definition of Broadband, which defines it as “and ‘always on' internet connection with a
minimum download/ upload speed of 256kbps (OECD Communication Outlook 2007).

As noted above, all 3 sectors may not be applicable to all countries. Or other minor
definitional changes may also need to be made. These country-specific customizations to
the methodology should be discussed with LIRNEasia and agreed upon prior to conducting
the survey.

2.5 The Seven dimensions

For each of the 3 sectors mentioned above, the respondent will assess (i.e. provide a score
on a scale of 1 – 5 for) each of the following dimensions:

1. Market entry

2. Allocation of scarce resources

3. Interconnection

4. Regulation of anti-competitive practices

5. Universal service obligation

6. Tariff regulation

7. Quality of Service

Of these, the first 5 dimensions are based on the Reference Paper of the Fourth Protocol of
the General Agreement on Trade in Services (WTO, 2007) and reflect the broadest
international consensus of the most important aspects of telecom regulation2.

Tariff Regulation was added following pilot studies and input from researchers.

Based on early 2008 discussions it is agreed that Quality of Service (QoS) is increasingly
important. While the traditional (incumbent led) fixed sector always had basic QoS
measurements, increasingly QoS for mobile is becoming important – for example,
completing a financial transaction via the various m-payment methods only feasible if the
mobile signal doesn’t drop half-way through the transaction. Similarly, depending on the

2
The Reference Paper also contains a dimension on the independence of the regulator. However
because it is seen as a process variable different from the other outcome variables, it is left out of
the TRE dimensions.
5
application being used Broadband QoS becomes increasingly important (e.g. simply
browsing can tolerate higher levels of latency that VOIP applications simply cannot). At
the moment, objective measures for measuring Broadband and Mobile QoS are neither
common nor standardized. However meaningful perception measures about regulation
related to QoS can go a long way in putting QoS on the regulatory/policy agenda.

2.6 The Likert Scale

Each of the seven dimensions is scored on a scale of 1 – 5, where 1 is Highly Unsatisfactory


and 5 is Highly Satisfactory.

The raw data collected for the survey using the Likert scale are ordinal data and therefore
the distinction between neighboring points on the scale is not necessarily always the same.
For instance, the difference in effectiveness expressed by giving a score of 4 rather than 3
might be much less than the difference in effectiveness expressed by giving a score of 5
rather than 4.

Since the ordinal data collected are summarized in to final TRE scores, it is possible to
transform the 1 to 5 scale to a different scale like -2 to 2 which makes it easier to interpret.
However, there is a possibility that a transformed scale portrays a different level of
perception to the respondent. For example giving a score of 3 in a 1 to 5 scale and giving
a score of 0 in a -2 to 2 scale might not appear the same for some respondents.

Therefore it is recommended that the 1 – 5 scale be kept constant in doing the TRE survey

Obviously the Likert scale may be subject to distortion from couple of causes. Respondents
may avoid extreme scores (central tendency bias); or try to portray themselves or their
organization in a more favorable light (social desirability bias).

2.7 The Respondents: Categories, Weights, Minimum Numbers

2.7.1 Categories

The different stakeholders that are involved in the TRE have been grouped into three
categories according to their common interests.

• Category 1: Stakeholders directly affected by telecom sector regulation

o E.g. Operators, Industry associations, Equipment suppliers, Investors

• Category 2: Stakeholders who analyze the sector with broader interest

6
o E.g. Financial institutions, Telecom consultants, Law firms

• Category 3: Stakeholders with an interest in improving the sector to help the public

o E.g. Academics, Research organizations, Journalists, Telecom user groups,


Civil society, Former members of regulatory and other government agencies,
Donors

Note that certain Financial Institutions (listed in Category 2) may take an equity stake in an
operator and thereby become an Investor (Category 1). It could also be that within the
same financial institution, respondents from one unit (say, the Equity Research division) falls
into Category 2 (since they analyze the sector as a whole) while another unit (say, the
Investment/Asset Management division that owns shares of the operator) falls into
Category 1. In such instances the researcher must deal with the categorization of
respondents on a case-by-case basis.

2.7.2 Weights

Each respondent category’s contribution is of identical importance to the final TRE scores.
Therefore, if there are an equal number of respondents for each category, the TRE
Assessment will reflect the views of the respondents of each category in an equal manner.

However, the sample selection procedure used in this study does not, by itself, produce
equal numbers of respondents from each category, because response rates will differ.
Different number of respondents in each category will result in over-representation of some
categories and under-representation of others. This will cause problems in comparison.

Ideally each category should make the same contribution to the final result. In order to
achieve this balanced representation, over-represented categories are given a weight of
less than one and under-represented categories are given a weight of greater than one, in
such a way that all three categories equally contribute to the final score.

2.7.3 Minimum numbers

Even though weighting deals with numerical imbalance, it is important to have sufficient
number of respondents for all four categories because the sample size determines the
precision (a measure of how consistently the result is determined by repeated
determinations without reference to any "true" value) with which population values can be
estimated; i.e., larger the sample, the more precise the estimate. As a practical matter,
sample size is often the dominant factor in determining the precision. Very few respondents
from a particular category will make the final TRE score highly sensitive to each
respondent’s input in that category.

7
Having taken the above factors as well as the practical constraints associated with the
survey in to account, it is recommended (required) to have a minimum of 15 respondents
from each category. In micro-states (e.g. Bhutan, Maldives), a minimum of 5 respondents
from each category is required.

Having a larger number of respondents per category also allows more sophisticated
analysis without compromising the respondent’s anonymity. For example, Category 1
represents players who are already in the market. Certain regulatory regimes may unduly
favor those who have already passed (by whatever means) the barrier to entry. For these
respondents, overly cumbersome market entry procedures may indeed serve as a positive
and keep new entrants out. However to analyze at the level of respondent categories, a
larger data set is required.

8
3. Before you begin: some FAQs

3.1 Who owns the TRE methodology?

The TRE methodology/tool was developed by Rohan Samarajiva of LIRNEasia, a think-tank


that works across the Asia Pacific. As a public interest research organization, and in
keeping with LIRNEasia’s mission of bringing about evidence-based policy reform, we are
interested in having the TRE implemented (and analyzed) in as many countries as possible.
In addition to the 6 country TRE study conducted by LIRNEasia in 2006, there is a 9 country
survey planned for 2008 for emerging Asia. LIRNEasia’s research partners (in the larger
LIRNE.NET network) in Africa and Latin America have already conducted TRE surveys in a
number of countries in those regions, and the results are available publically.

Other researchers interested in conducting a TRE study in a set of comparable group of


countries are required to contact LIRNEasia to discuss specifics of the conducting the TRE
survey. In particular, as a form of quality control it is important for information about
respondents to be maintained in a manner that ensures confidentiality. LIRNEasia will
facilitate this role and act as the Data Guardian of personally identifiable data, so that any
queries about the legitimacy of the survey data can be answered/verified (by revealing
only the necessary information). LIRNEasia will also make available access to the existing
web-based survey to such researchers.

3.2 What qualifications are required for country researchers?

Simplistically, anyone with basic research capabilities who follows the steps listed here can
conduct a TRE survey.

But conducting the survey is simply the first step in a much large analysis. The results need
to be analyzed in the historical context of the telecom regulatory developments, as well as
the current performance of the sector. Therefore the analysis requires deeper insight and
knowledge that is sector specific.

But even more important than telecommunication sector knowledge is the name-
recognition/reputation of the researcher and/or his/her institution. The TRE survey targets
CEO or equivalent senior personnel at telecom companies, government and civil society.
They are unlikely to agree to complete a survey (however short) unless the researcher or
the researcher’s organization is recognized as one that is engaged in this type of work,
having carried out such work in the past.

9
Furthermore, in small markets individual respondents can be easily identified (i.e. in some
countries there is only one mobile service provider). The researcher’s credibility and track
record go a long way in ensuring the respondents that his/her responses will be kept
confidential.

The caliber of the researcher (and his/her organization) is therefore one of the key success
factors in conducting a TRE.

As the developer of the methodology, and as the organization has conducted several TRE
surveys in the past (results of which have been presented in international forums), LIRNEasia
will also provide credibility to the study.

3.3 Can it be a single country survey?

The TRE quantifies the perception of the regulatory environment in a country. It is not
possible to pick/define some arbitrary absolute score that must be achieved/passed in
order for a country to be considered a high performer. What can be said instead is that if
country A which has higher TRE scores than Country B, that Country A has a better
regulatory environment. Or if Country A improved its scores from time T to time T+1, then its
regulatory environment has improved.

Therefore the TRE study almost always lends itself to be conducted in more than 1 country,
simultaneously and being repeated over time.

If a researcher insists on conducting a TRE in only one country, then the results become
meaningful after a 2nd TRE study is conducted after a certain time period has elapsed (e.g.
after 1 or 2 years). In the extreme case, one could argue it is possible to do the TRE for a
country just once, and use the scores for the 3 sectors to compare the regulatory
environments that impact each of the sectors separately – but this too would suffer from
lack of historical or cross-country context.

10
4. Getting Started
4.1 Ensuring Confidentiality

4.1.1 Why does Confidentiality matter?

The value of the TRE is that it obtains the perceptions of the senior most stakeholders in a
country’s telecommunications sector. This is why the TRE is kept extremely short, and
designed in a manner that requires quick responses - 3 sectors, but the same 7 questions
repeated for each sector, requiring a simple click (for the online survey) or simple circling
of a number (for the paper survey). However respondents will not provide honest responses
if they feel that the specific responses can be tracked-back to (or identified with) them.
This is particularly true if the respondent has negative scores to assign. This reluctance is
wholly justified, especially since there is a history of regulators exerting significant power
over stakeholders. In most circumstances, a new entrants cannot be seen to “criticize” (by
giving low TRE scores to) the regulator which also governs it. This becomes particularly
relevant in markets where players are identified easily – e.g. markets where there is only 1
operator in each sector.

Therefore it is vital that confidentiality of the respondent is ensured in order to

• To maintain and protect the respondents’ business interests


• To encourage people to take part in the study
• To obtain truthful and accurate information from respondents

One simple solution therefore is to ensure that no personally identifiable information is


collected or stored beyond the completion of the TRE survey (i.e. insisting that researchers
destroy all such records immediately after data collection).

4.1.2 How can Quality be ensured?

However, access to such (personally identifiable) information may be important in ensuring


the quality and thereby credibility of the TRE scores. What if, for example, there is a
situation where the results (TRE scores) of a particular country are being disputed several
months/years after a survey is completed? Having access to the specific response sheets
may be vital in such situations – enabling researchers to go back and verify that the correct
numbers are used. Furthermore, as the TRE is widely adopted, the ability to do such
verifications can be important in quality control. Therefore a record of respondent
information becomes a necessity.

11
4.1.3 The role of the Data Guardian

The challenge therefore is to strike the right balance between ensuring quality at the same
time as ensuring confidentiality. The requirement is to ensure that this information is kept
confidential, accessed only when necessary, for a specific purpose and by a minimum
number of persons.

In order to do this, LIRNEasia will appoint one person, not directly involved in conducting,
analyzing or writing about the TRE studies in any country to maintain the TRE scores
associated to each respondent. Should the need arise this person will be empowered to
access this information and verify information without revealing any information that will
enable the association of results with individual respondents.

The Data Guardian and the process recommended below are meant to ensure that:

a) Personally identifiable information is never co-located (i.e. in the same place, such
as the same file) as the TRE scores

b) Should the need arise there is a way to associate individuals with the TRE scores
assigned by them.

4.1.4 What the Researcher should do to ensure confidentiality

As researchers engaged in the TRE study, the following minimum steps are to be followed:

1. All personnel (the researcher, other staff such as research assistants, subcontractors
and other third parties) who have access to direct data from a respondent should
sign the Confidentiality Agreement. By signing the agreement you provide security
and limit the use of and disclosure of information/results. See Appendix A7 for the
Confidentiality Agreement

2. During the survey, any completed paper questionnaires should be physically


secured and kept under lock and key. All digital files should be password protected.

3. If at any point of conducting the survey a respondent wishes to stop participating, all
records related to that individual’s name/contact information should be deleted
from all physical and digital media associated with the TRE study.

4. Upon completion of the survey, all records must be digitized, to enable analysis.
However during digitization, digital data is separated into 2 different files in order to
maintain confidentiality:

12
a. The first file (the Respondent List, explained in detail later) should contain the
Respondent Name, Designation, Organization, Contact Information,
Questionnaire ID/URL and a unique Respondent Number. Questionnaire
ID/URL corresponds to the number written on the paper survey, or the unique
ID generated for the web-survey. The unique Respondent Number is
allocated generated by the researcher (simple 1, 2, 3, … series of numbers for
each person who completed a questionnaire), and enables each
respondent to be identified uniquely with a number. Have the Respondent
Number therefore enables the researcher to find the Name and other
information for a respondent.

b. The second file (the TRE Data File, again explained in detail later) contains the
actual the Respondent Number and TRE Scores for each number. This file
does NOT contain names, organizations or contact information of any
respondent. Only the Respondent Number differentiates one respondent
from the other. But without the Respondent List file, associations between
respondent and scores cannot be made.

5. Immediately upon creating the Respondent List, it should be sent to the Data
Guardian appointed by LIRNEasia.

a. The file must be password protected. The password must be emailed to the
Data Guardian in a separate email. If a CD/DVD based copy is to be sent, a
reputable courier service must be used and the data should be password
protected. The password must be sent to the Data Guardian separately
(different courier package or via email).

b. After (e)mailing the file and making sure it has been received by the Data
Guardian (and that it can be opened and read without technical problems),
ALL copies of the file should be removed from the researchers
computer/back-up storage devices.

c. If paper surveys were completed by respondents, these should be sent to the


Data Guardian, via a reliable courier service. Any photo-copies of
completed questionnaires in possession of the researcher must be destroyed
after verifying receipt of originals at LIRNEasia.

6. The TRE Data File can now be used by the researcher and anyone else who has
signed the Confidentiality Agreement. However as soon as data entry is completed,
a copy of this file should also be sent to LIRNEasia/research coordinator.

a. As always the digital file must be password protected and the password must
be sent separately. If CD/DVD based copies are being sent, a reputable
courier service must be used and passwords sent separately.
13
7. Depending on the country, the contents of TRE Data File may still be sensitive.
Though it contains no personally identifiable information, it does list the Category (1,
2 or 3) to which each respondent belongs to. In countries where there is only 1
mobile operator (for instance), listing a response from a particular respondent
number belonging to Category 1 may give sufficient information to identify the
operator. Therefore researchers should exercise care in sharing the TRE Data File
with anyone outside of the research team. Even within the research team, only
those who have signed the Confidentiality Agreement should be allowed access to
the raw data.

8. Equal care must be taken in displaying results. For example, whether in a public
forum or in private/small groups, results should ALWAYS be shown for the whole
sample or by respondent category as whole, and NEVER for individuals (even if
individuals are only identified by the Respondent Number). As noted earlier,
displaying information by Category should be done with care and not done if there
is any risk of exposing (or even hinting at) the names of companies/respondents.

4.2 The TRE Toolkit

The TRE Toolkit provides a set of tools that enable researchers to carry out a TRE study across
countries. The Toolkit consists of the following documents:

(i) TRE Manual


(ii) TRE Cover Letter
(iii) TRE Questionnaire
(iv) The TRE Summary of Regulatory Events
(v) TRE Respondent List and TRE Data File
(vi) TRE Report Template
(vii) TRE Thank You Note
(viii) Confidentiality Agreement

All researchers must be familiar with these tools. It is important that researchers engaged in
the TRE study use the documents (ii through vii) with minimal edits in order to ensure
comparability of data, standardization of process and quality of output. Any significant
variations/edits to these documents must first be discussed with LIRNEasia.

4.2.1 The TRE Manual

The TRE Manual (this document) provides researchers with a guide to conduct the study.
This includes the relevant background and methodology of the TRE Assessment as well as

14
descriptions on documents and templates required to carry out the study. It should be
familiar to all researchers engaged in a TRE study. It will be reviewed and updated at
regular intervals.

4.2.2 The TRE Cover Letter

The TRE Cover Letter (see Annex A1) provides an introduction to the study. It should be sent
to each of the respondents you intend to survey. The letter provides respondents with
information about the study, and reassures them of the confidentiality of their responses. It
is issued by LIRNEasia, and identifies the researchers has being associated with LIRNEasia for
the particular TRE study. The TRE Cover Letter includes:

• An introduction to the person who will be deploying the questionnaire


• A brief description of the TRE Questionnaire itself
• Why the respondent was chosen for this study
• Terms of participation and confidentiality
• Contact details of the lead organization
The researchers should edit the Cover Letter provided in Annex 1 (customize it to include
the Researcher name, organization and so on) and email it to LIRNEasia’s TRE Research
Coordinator who will in turn sign it, put it on LIRNEasia letter head and send it back to the
researcher. The researcher then sends out a copy of the Cover Letter to each potential
respondent.

4.2.3 TRE Questionnaire

The TRE Questionnaire is the (electronic or physical) form that is used to record responses.
Respondent who are targeted in the study are sent the TRE Questionnaire (see Annex A2)
along with the Summary of Regulatory Events (see below).

The TRE Questionnaire contains:

• An introductory section, which describes what is covered under each dimension being
measured (for all 3 sectors).
• Three Likert scales upon which respondents mark their perceptions on the effectiveness
of each dimension of the TRE for each of the 3 sectors

In each questionnaire that is sent out vial mail or email, an identifying number must be
written so that when it is mailed back the researcher knows which respondent it is from. This
Questionnaire ID can be as simple as 1, 2, 3…, n.

The TRE Questionnaire for the 2008 study is available in Annex A2.

15
4.2.4 The Summary of Regulatory Events

Each TRE Questionnaire is accompanied by the Summary of Key Regulatory/Policy Events.

This is a fact sheet/description of regulatory events over the past 12 months. All important
regulatory and policy events must be listed, irrespective of whether they are positive or
negative developments.

The language used to describe these developments must be as neutral and bland as
possible, in order to not influence the respondents.

Each time a TRE is done in a country, the researcher conducting the study must prepare a
new Summary.

See Annex A3 for an example of the Regulatory Events sheet complied for India in the 2006
survey. Please note however that this document will vary significantly across countries and
across time.

4.2.5 TRE Excel Data Files

The TRE Tool kit comes with 2 separate Excel templates:

a) The Respondent List File: This is a file that contains the respondent name, designation,
organization (or Category), contact information and the unique Respondent
Number assigned to him/her. This file is created during the data collection stage.
The Respondent Number can be a simple series of 1, 2, 3,..n numbers assigned to
each respondent that has completed a survey. If web-based surveys were
completed, a unique number generated for each respondent (and emailed as part
of the URL – see details below on the web survey) must also be included in the
Questionnaire ID/URLcolumn, against the relevant names. If paper (printed)
questionnaires were completed, the number you assigned at the time of sending
out the Questionnaire should be included in this column. As noted earlier, the
Respondent List should be emailed to the Data Guardian, and all copies in
possession of the researcher destroyed immediately. The file must be password
protected, and the password must be emailed to the Data Guardian in a separate
email. Same should be done if a CD/DVD based copy is couriered

b) The TRE Data File: This file contains worksheets that have been pre-formatted with the
necessary formulas embedded to aid the researchers in data collection and data
analysis. This file contains no personally identifiable information on respondents, and
instead contains only the Respondent numbers that were generated in the
Respondent List file above. The Data Template consists of multiple work-sheets that
enable data entry, automatic generation of weighted scores, automatic generation
16
of graphs (charts) for each dimension, automatic generation of the total TRE scores
for each sector. A final copy of this file must be provided to LIRNEasia. The file must
be password protected, accessible only to the researchers that have signed the
Confidentiality Agreement, and not be shared with third parties. It’s safer to always
display information for the whole sample. Information per category can be
displayed if there is no risk of individual organizations/persons being identified.

4.2.6 The TRE Country Report

The Report Template provides the researcher with the key elements that are required in a
final report that presents the results of a TRE assessment in a country. The report contains
the following sections.

• Title Page: with author(s), contact information


• Acknowledgments: Apart from acknowledging persons or organizations that have
aided the researcher(s) during the study, all reports must explicitly mention the
following sentence: “This research was carried out with the aid of a grant from the
International Development Research Center, Ottawa, Canada.”
• Summary: An executive summary that covers the key aspects of the TRE study and
results in the country
• Table of Content: automatically generated
• Introduction
• Methodology: referring to the LIRNEasia TRE methodology, how/when the study was
done, how samples were selected, any previous studies, changes in method from
previous studies (if applicable), deviations from published and accepted TRE
methodology.
• Findings / Results: This is the most detailed section of a TRE Report. Subsections (a
total of seven) should be used to present the TRE scores for each dimension and to
analyze the results in the context of actual sector indicators. For example, the
subsection on the Market Entry dimension should present the TRE scores for the
Market Entry dimension, and analyze them in the context of recent licenses that may
have been issued (or explain the lack of new entrants), the transparency (or lack
thereof) of processes to be followed by operators who wish to enter the market, the
licenses that are required, evidence of ease of market entry and level of
competition as shown by HHI calculations, market share pie charts and so on.
Additional subsections can be added as needed. If a previous TRE study has been
conducted in the country, a comparative analysis of previous vs. current results must
also be done, and sector performance data used to explain (where possible) the
changes in scores obtained In particular, relationship between the changes in TRE
scores and changes in sector investment must be presented.
• Conclusions: A section on what can be concluded from the data (both TRE and
sector indicators)
• Recommendations / Steps Forward: measures that can be taken by
• Appendices: containing detailed tables showing scores (for the whole sample, or by
category, if not too revealing), a copy of the actual Summary of Regulatory Events
17
used during the survey, copies of other material used during the survey and any
other supporting information must be placed in a series of Appendices.

4.2.7 TRE Thank You Note

The TRE Thank You Note (see Appendix A5) in addition to thanking respondents for
contributing to the study also offers an opportunity for future collaborations with the
respondent and his/her organization The contact details of the lead organization, and the
country researcher are also included in the event that the respondent may have any
further questions.

The TRE Thank You Note should be sent to the respondents at the end of the study, along
with the TRE Report for that country. If the final TRE Report is being delayed for any reason,
a Thank You Note should be sent immediately upon completion of the survey with a
promise to send the TRE Report upon completion.

The Thank You Note is on LIRNEasia letter head and signed by the research coordinator.
The individual researcher may add a more personalized note on top of this (or a
personalized email).

18
5. Conducting the TRE Study
This section provides details on the steps to be followed in conducting a TRE study in a
country.

5.1 Selection of the Expert Panel of Respondents

The legitimacy and success of a TRE study depends on how successful a researcher is in
targeting the right senior level stakeholders and getting them to complete questionnaires.
Therefore compilation of the respondent panel is one of the first and most important
activities of any TRE study.

5.1.1 Respondent Categories

As mentioned there are 3 categories of respondents:

• Category 1: Stakeholders directly affected by telecom sector regulation, such as


operators, Industry associations, equipment suppliers and re-sellers

• Category 2: Stakeholders who analyze the sector with broader interest, such as those
working for financial institutions, Telecom consultants, Law firms

• Category 3: Stakeholders with an interest in improving the sector to help the public
such as academics, research organizations, journalists, telecom user groups, civil
society, former members of regulatory and other government agencies, donors

5.1.2 How many in respondents – the sample size

For each category, a sufficient number of target respondents need to be identified. Past
experience shows that even in the best case less than 45% of those approached agree to
complete the questionnaire. Often the response rate is less. Therefore, even for the most
credible and well-connected researcher, it is necessary to have at least 33+ respondents
that can be approached, in order to ensure that at least 15 completed questionnaires are
received at the end. In the case of micro-states, where a minimum of 5 responses per
category is required, at least 11+ respondents need to be approached.

Even though the assignment of weights (explained later) deals with numerical imbalances,
it is important to have sufficient number of respondents in all 3 categories because the
sample size determines the precision with which population values can be estimated. Very
few respondents from a particular category will make the final TRE score highly sensitive to
each respondent’s input in that category, and may provide a distorted view of the
regulatory environment.

19
5.1.3 How to compile the list of respondents

Some TRE country researchers are well connected within the telecom sector of a country
that they will easily be able to compile the list and the contact information. However in
most instances it is necessary to compile a new list or to update older ones – the persons
holding key positions within companies may have changed within a short period of time,
especially in a dynamic sector such as telecom. Therefore the following steps are
recommended:

1. Search the Internet, scan the trade press, scan popular media, talk to persons who
are knowledgeable and connected in order to identify:

a. the names of operators, industry associations and equipment suppliers that


are active in the telecom industry (the trade press may indicate companies
active in the sector)

b. the names of law firms, financial analysts and consultants who follow the
telecom sector (consultants for example may publish reports; financial
analysts will publish equity research reports on the telecom sector, and so on).

c. the names of civil society organizations, ex government officials and other


persons in Category 3 (academics, for example may reside within the
relevant technical/engineering/communications/ law departments within
universities)

2. Identify personnel within each organization identified above. The key is to identify
the personnel within each organization that have a direct
expertise/experience/interest in the telecom sector.

a. In the case of operators and equipment manufacturers the CEO is an obvious


target. However in addition, the senior-most persons who handle regulatory
affairs are as important. There might be a regulatory affairs division or a legal
department within the operator’s organization which may yield suitable
potential respondents.

b. In the case of retired government personnel, ideally persons who have been
involved in telecom regulation in the recent past should be selected. Persons
who recently vacated (transferred from/retired from) the posts of a regulator
are ideal.

c. In identifying respondents from academia, a search engine such as Google


Scholar or specialized academic databases will yield the most appropriate
candidates. All searches should target the specific country – i.e. academics
who research the specific country, not necessarily based in the country.

20
d. Journalists are somewhat easier to identify by following the trade and popular
media over a period of time. For example, start with the popular business
newspapers and identify journalists who have written relevant articles.
Contact the newspaper to obtain the contact information for the journalist.
Similarly for electronic media (TV, internet, radio).

e. In some cases the targeted person may be obvious or well known. If not, the
researcher may need to contact the organization via phone or email and
obtain the information for specific personnel. Being courteous, explaining the
reasons for the request and providing information about the researcher and
his/her organization will help in obtaining information. Private sector firms may
be particularly reluctant to give direct contact information, but direct
contact information is exactly what is needed. Many surveys can end up in
the inboxes of the assistants of senior managers, and never move beyond.
Past experience shows that phone calls are almost always more productive
than emails in finding information.

5.1.4 What information is needed?

For each respondent, the name, designation, organization and the different modes of
contacting the respondent (email, fax, and mailing address) are the most important
information. Apart from this, the researcher can also maintain the following information in
order to track progress during the survey: the date on which the TRE Cover Letter (and any
other pre-contact information) was sent, the date on which the questionnaire was sent, the
date on which the response was received, the date on which follow up was done, method
of communication used, date thank you note was sent.

5.2 Compiling the Summary of Regulatory Events

The purpose of this document is to refresh the respondent’s memory on key events that
occurred in the previous 12 months. It is given to respondents with the TRE Questionnaire.
The document needs to be objective and accurate, and be written in bland language so
as to not (positively or negatively) influence the scores given by respondents.

In order to prepare a list of regulatory events that occurred in the past 12 months, the
researcher may need to:

• Consult a senior official at the telecom regulatory agency of the country


• Use the Internet
• Read relevant newspapers and magazines to ensure the details are accurate
• Consult experts in the telecom sector for information
21
• Circulate a draft among a small number of experts to obtain agreement

Format: Events should be listed in reverse chronological order, with the most recent event
listed first. The date (or timeframe) for each event should be accompanied by a brief
description. An example (from a previous study, for a particular country) is provided in
Annex A3. However the Summary of Regulatory Events is country and time specific. The
researchers need to exercise his/her judgment in deciding which events to include or
exclude. A short document is likely to be read (or at least scanned) by busy respondents,
while a longer document is clearly more comprehensive.

5.3 Applying the TRE Questionnaire (conducting the survey)

5.3.1 Different Survey Channels

Theoretically it is possible for the survey to be administered via a number of different


channels: a web based survey, email, mail, fax, telephone/in person interview. Each has
pros and cons, and the appropriateness and effectiveness of each channel is often
culturally dependent. A quick summary of each channel is provided next.

Keep in mind that the TRE Cover Letter, the TRE Questionnaire (with introductory section
containing definitions of each sector/dimension) and the TRE Summary of Significant Events
MUST be given/sent to the all respondents irrespective of the channel used.

Channel Advantages Disadvantages

Web Survey • Substantially reduces costs • Respondents need some


(using the • Web survey infrastructure already computer literacy and access
web site provided (by LIRNEasia) and doesn’t to internet
have to be developed each time • Access to the web page
provided by
• Faster speed (for respondent and needs to be controlled
LIRNEasia) researcher)
• Increased respondent flexibility (when
compared to phone/in-person
interviews) since they can access the
web site and respond at their leisure.
Email • Cheaper than the some of the other • This method is limited to those
methods as there are no postage or with email facilities.
travel costs • Due to the nature of online
• Absence of delays that are attributed networks it is harder to ensure
with tradition mailing systems. confidentiality during the
• Invitations and information regarding process.
the study can be sent easily prior to • Respondents need to be
the deployment of the questionnaire. provided with detailed
This enables the researcher to rapidly instructions.

22
determine the level of participation. • Response rates can be slow.
• Some evidence from the US (research Although research shows that
undertaken by the Colorado State email surveying methods have
University) indicates that the response higher response rates, this is
rate is higher in email surveys than in limited only to the first few
interview or mail surveys days. After this it has similar
(http://writing.colostate.edu/guides/re rates to mail surveys
search/survey/). However this may not (Oppermann, 1995)
be
• Research undertaken by the Colorado
State University indicates that people
tend to answer more honestly to
electronic surveys.
• Due to the high speed associated with
email services, respondents usually
reply promptly. You can also reach
overseas respondents relatively easily.
Postal • Cheaper than some of the other • Generally gives low response
Service methods such as face-to-face rate. Research undertaken by
interviews. the Colorado State University
Note: A self- • Respondents have greater flexibility in has indicated that mail-in
addressed, finding an appropriate time to fill out surveys have a response rate
postage the questionnaire. of about 20%.
paid • Reduced bias in the responses • Findings of the DIRSI research
envelope (researchers are unable to team in Latin America are that
should unconsciously influence responses or this is the least effective means
ALWAYS be develop a personal impression on the for achieving responses.
enclosed respondent since there is little or no
along with direct contact).
the
Questionnair
e so that
responses
can be
returned
easily

Fax • One of the fastest methods to • Could be more expensive than


communicate with respondents, with other methods due to telecom
the exception of email costs.
• Both researcher and
respondent need access to a
fax machine. Previous studies
show that responses are
delayed due to the
respondents not having direct
access to a fax.
• Reduced confidentiality, as
company fax machines tend
to be placed in open / public
locations.

23
Telephone • Responses are immediate. If the • The cost of conducting the
respondent is not willing to participate survey is high due to telephone
in the survey he/she will inform the charges – especially if
researcher immediately. This eliminates respondents are overseas.
the time lags that occur when • The timing of the phone call is
questionnaires are sent and responses important. If the call is made
are received. during an inconvenient time
• If a respondent requires further the respondent may
information regarding the immediately reject the survey,
questionnaire they will be able to even if he/she is likely to
clarify the issue immediately. accept it if sent via email..
• It may be inappropriate, for
example, for junior researchers
to contact senior personnel by
telephone (depending on the
country).
Personal • This is the most favoured channel • The cost of conducting the
Interview when conducting the survey. It survey is high, as you have to
enables data to be collected in a factor in the larger number of
timely manner, and also enables the people required to undertake
creation of a network of personal the interview, the cost of travel
relationships for future TRE studies. to meet respondents, cost of
• Issues can be clarified easily as you scheduling these meetings
can communicate directly with the (telephone calls, emails) and
respondent. extra effort required.
• As with telephone interviews, • Although responses are
responses can be collected obtained immediately, the
immediately. You will also be able to entire interview process may
determine immediately if a take a significant amount of
respondent wishes to participate in the time.
study. • Face-to-face interviews may
involve greater levels of bias
both on your part and that of
the respondent.
• The attitude of the researcher
during the
interactions/interviewing may
have a significant bearing on
the outcome of the study. If
the respondent feels offended
or insulted s/he could refuse to
participate in the survey or
could provide misleading
information.

5.3.2 Web Survey

24
LIRNEasia has already created a user friendly web-application that enables respondents to
easily implement a web-based survey. As this is a new tool implemented in the 2008 TRE
research cycle, it’s worth mentioning in some detail.

1. The researcher logs into the system at http://www.tre.lirneasia.org/admin using the


allocated username and password (Researchers interested in using the web for the
survey should contact LIRNEasia via email and request a username and password).
When the researcher logs in he/she can:

a. Enter the Summary of Significant Events by clicking on the “Edit Fact Sheet”
tab

b. Enter the list of respondents who wish to do the survey online by clicking on
the “Respondents” tab. This page also indicates whether each respondent
has participated in the survey.

c. View responses and download responses in Excel format by clicking on the


“Data” tab

2. When the new respondents are added to the system will then assign a unique ID for
them, and create a URL of the form
http://www.tre.lirneasia.org/tre.php?dXNlcl9pZF9f=XzEwMDAwMDBh that can be
emailed to them (the unique ID is encrypted to maintain confidentiality of
information).

3. The researcher then email the respondent directly and send them the generated
URL, making sure the email contains the Cover Letter (Annex A1) with instructions for
the respondents to click on the link to complete the survey.

4. When a respondent clicks on the link to start the survey, the encrypted username will
be used to validate the respondent. He/she can then complete the survey. The
Summary of Significant Events and a description about each dimension are also
available online to the respondent. Respondents may access the page many times,
but only one submission will be allowed. It is not possible to change their responses
subsequently.

5.4 Increasing the number of responses

5.4.1 Use of Industry Events

Industry events are excellent opportunities to approach a number of respondents within a


short timeframe. People attending such events are already in an “out of office” mindset
and may be more predisposed to interviews. During the 2006 cycle, a LIRNEasia researcher
25
was able to get all the CEOs of the telecom operators in a particular country to complete
the questionnaire by targeting them (in one room) at an industry event. Of course the pre-
planning that enabled the researcher to attend this meeting took significant effort and
networking.

Industry events also facilitate the building of future connections which will be useful when
conducting questionnaires in the future.

5.4.2 Optimizing pre-contact

The pre-contact package is a simple method by which you invoke a respondent’s interest
in both the questionnaire and the study.

Initial contact should be made via email as it is an effective way to make an introduction.
Email is fast and enables a great deal of information to be transferred relatively easily. If
the respondent does not use email, then mail or fax services can be used (making
allowances for postal delivery in the case of mail).

The information should be presented clearly and in simple language, and should contain t
a minimum, the TRE Cover Letter introducing the researcher, introducing the study and
assuring the respondent that his/her responses will be treated confidentially. In certain
instances researcher may need to provide more detailed information such as previously
published TRE results, description of LIRNEasia or the researcher’s organization and so on.
These need to be handled on a case-by case basis and LIRNEasia TRE Research
Coordinate will assist with the compilation of such information.

Often, respondents are interested in the final results of the TRE study. Therefore a promise to
share the final report(s) should be made during the pre-contact stage (and followed
through at the end of the study).

Experience shows that it’s ideal if pre-contact is made approximately one week prior to the
deployment of the Questionnaire. Often the timing is crucial. If pre-contact is made too
late then it would not make the required impression. If it is made too early there is a danger
that the respondent could forget about the study.

The researcher should be prepared to answer questions from the respondents on the study
and the documents that have been sent – this can happen via email exchanges or when
the researcher follows up with phone contact

5.4.3 Follow-up

Following up after pre-contact is perhaps the most important activity undertaken during
the data collection process. Without follow up phone calls, emails or faxes, the response
26
rate is likely to be very low. Even to reach the 45% average response rate achieved in past
surveys, follow up is needed.

Follow up should start approximately a week after the pre-contact information is sent out to
the respondents in the case of mail, and within 2 days in the case of email or fax.

Assuming the researcher has made verbal or email contact with the respondent after the
pre-contact has been made and the respondent has agreed to participate in the survey,
the Questionnaires will be mailed to the respondent. Follow up phone calls/emails should
be made within 1 week of the Questionnaire being sent, because if the questionnaire has
not been filled by this time, the respondent has most likely forgotten about it.

If at least 2-3 follow-up calls have been made (or emails being sent) and there is no
response or no plausible explanation is given for non-response (at the pre-contact stage or
after sending the questionnaire), it is generally safe to assume that the particular
respondent is not interested in participating in the survey. Over-enthusiastic following up
may result in the busy respondent simply passing on the survey to a junior person to
complete in order to avoid being harassed.

5.4.4 Use of colored paper

The TRE Cover Letter and TRE Questionnaires can be printed on colored paper, if mail is
used. This has several advantages:

• It is more eye catching and could result in the respondent showing a greater initial
interest in the study.
• It attracts attention when lying amongst a stack of other documents.
• During follow up the document can be easily identified.

5.4.5 Incorporating new respondents

If the number of responses received is not enough even after significant follow-up, then the
researcher has to bring in additional respondents. Potential new respondents in the
required categories need to identified, and the processes above have to be repeated but
with greater intensity, as the process cannot be undertaken for a third time.

Note however that expanding the number of respondents should be a secondary choice.
First priority should be given to obtaining responses from the initial group of respondents.
Adding new respondents is second best since these 2nd round choices are likely to be less
directly relevant to the study (assuming all key stakeholders were targeted in the first
place).

27
6. Collection and analysis of results
6.1 Calculation of Weights

Although the Excel template included in the TRE Toolkit automatically generates weighted
scores and figures, it is important the researcher has a thorough understanding as to how
they were generated. For this reason, the process is briefly described in 3 simple steps.

Step 1: Calculation of weights

To calculate the weight for a category, divide the total number of respondents in all the
categories by 3 and divide the answer by the number of respondents in that category.

Example:

Suppose the total sample size is 63 and its composition is as follows.

Category 1 : 32 respondents

Category 2 : 11 respondents

Category 3 : 20 respondents

Therefore the weights for each category will be as follows

63
Category 1 : 3 = 21 = 0.656
32 32

63
Category 2 : 3 = 21 = 1.909
11 11

63
Category 3 : 3 = 21 = 1.050
20 20

Step 2: Weighting the responses

Next multiply the scores by the corresponding weight value calculated in Step 1. You
should multiply all the responses of category 1 respondents by the weight of category 1
and responses of category 2 respondents by the weight of category 2 and so on.

Example:
If the following are the scores given by a category 1 respondent, then multiply
each score by 0.656 to get the weighted score.

28
Original Responses

3 2 4 2 3 4 5 2 3 4 2 3

Weighted Responses

1.97 1.31 2.62 1.31 1.97 2.62 3.28 1.31 1.97 2.62 1.31 1.97

Step 3: Calculation of Final Score

The simple averages of the weighted responses for each dimension are taken as the final
score for each dimension.

6.2 Using the two Excel Files

Two Excel files, Respondent List (TRE_Respondent_List.xls) and TRE Data File
(TRE_Data_File.xls), are included in the CD in the TRE Toolkit (or available as separate
attachments, if this Manual is emailed).

6.2.1 TRE Respondent List

The Respondent List is used to store the details of respondents. Each respondent should be
given a unique Respondent Number. That same number should be used as a reference
when storing data in the TRE Data File.

6.2.2 TRE Data File

The TRE Data File is used to perform calculations of the TRE results for the study. The
template provides a simple interface in which data from the TRE Questionnaires can be
entered. The required weighted results will then be calculated automatically.

This file has four worksheets.

The first worksheet titled Data (Screenshot 1) is used to manually enter the responses. You
should enter the following information in the worksheet:

• The respondent number

• The score for each dimension

• The respondent category

29
Enter zero (0) for unanswered questions to make sure it is ignored from the final score. Enter
all the other respondent data in the “Respondent List.xls” file.

Screenshot 1

After all the data has been entered into the system, the Output worksheet (Screenshot 2)
will automatically display the final weighted scores for each dimension.

Screenshot 2

30
The other two worksheets give graphical representations of the final weighted outcome.

The Individual Scores Chart worksheet (Screenshot 3) gives a comparison of the weighted
scores for each sector for each dimension.

The weighted Total Scores Chart (Screenshot 4) provides a comparison of the weighted
total score for each sector.

Screenshot 3

Screenshot 4

31
6.2.3 Creating a password

Once data entry is completed, protect the data by creating a password so that only
authorized persons can access and view the data.

Step 1 - Left-click on the "Tools" menu in Excel


Step 2 - Select "Options" from the drop-down menu
Step 3 - On the "Options" screen that appears left-click on the 'Security' tab to display the
Security window
Step 4 - On the 'Security Tab' enter a password in the 'Password to Open:' box.

Finally the researcher needs to send the TRE Data File to the Data Guardian and destroy all
the copies of the TRE Data File in researchers’ computer after receiving an
acknowledgement from the guardian.

6.3 Dealing with non-conforming data/unusual situations

32
Write the section (titled Quality Control) from previous, but more meaningfully. See AKM’s
comments about things being not clear.

6.4 Writing the TRE Country Report

The TRE Country Report is the 2nd output in a TRE study (the 1st being the TRE Data File), and
it is the most important. It contains factual information (i.e. the actual scores, from the TRE
Data File) as well as analysis and inferences.

The primary role of the report is to explain the TRE scores in light of actual (telecom) sector
performance indicators. This requires in-depth understanding of the historical context of
regulation in the country, understanding of the institutional framework, analysis of the
indicator data and ability draw linkages between perception and performance.

The secondary role of the report is to present the methodology of the study, provide a
background and summarize the results.

The important Sections of the Report have already been highlighted earlier in this manual.
New sections maybe added. For example if a TRE study is done for the very first time in a
country, a regulatory and market time-line that graphically represent key events in the
country’s telecom sector since liberalization (or since another defined dated) can be
included (instead of doing so just for the preceding 12 months).

The Report should contain references for all material cited, formatted in American
Psychological Association (APA) referencing style.

33
7. References
Levy, B. & Spiller, P. (1994). The Institutional Foundations of Regulatory Commitment: A
Comparative Analysis of Telecommunications Regulation. Journal of Law, Economics and
Organization, 10(2), 201-46.

Oppermann, M. (1995). Email- surveys- Potentials and pitfalls. Journal of Marketing


Research, 7(3), 29-33.

Samarajiva, R. & A. Dokeniya with Sabina Fernando, Shan Manikkalingam & Amal
Sanderatne (2005), Regulation and investment: Sri Lanka case study, in Stimulating
investment in network development: Roles for regulators, eds. A.K Mahan and W.H.
Melody, pp. 141-76. Monte Video: World Dialogue on Regulation.
http://www.regulateonline.org/content/view/435/31/

Samarajiva, R, Galpaya, H. & Ratnadiwakara, D. (2007). Telecom Regulatory Environment


(TRE): assessment, methodology and implementation results from five emerging economies.
Presented at the 35th Telecom Policy Research Conference, Fairfax VA, USA on September
28 - 30, 2007.
http://web.si.umich.edu/tprc/papers/2007/683/SamarajivaGalpayaRatnadiwakara_TPRC.p
df

34
8. Annex A1: TRE Cover Letter
<Date>

To Whom it may concern:

LIRNEasia is conducting a Perception Survey of the Telecom Regulatory


Environment (TRE) in <name of country> as part of <an/a> <inter-country /long term>
comparative research of the regulatory environments within countries.

Ms/Mr. <name of researcher> from <name of executing organization> will be conducting


the survey in (name of country) on behalf of <name of lead organization>.

The TRE Assessment developed by LIRNEasia, a think tank working across the Asia Pacific
region. TRE surveys have already implemented in a number of countries. Findings of the
most recent study is at
http://www.lirneasia.net/projects/current-projects/measuring-ict-sector-
regulatoryperformance/.
The TRE Assessment for <country> will be conducted annually and will provide all key
stakeholders with an overview of the TRE in the country.

You have been chosen to complete this questionnaire because you have been identified
as a telecom expert or an important stakeholder in the telecom sector. We invite you to
complete a short questionnaire that will take less than 5 minutes of your time. There are 21
short questions, and you need to indicate your scores for each a scale of 1 to 5. The
questionnaire is accompanied by a summary of significant telecom policy and regulatory
actions taken over the previous 12 months for your easy reference.

All responses will be kept confidential and anonymity is guaranteed.

Participation in this survey is voluntary and you have the right to terminate the survey at any
time. <Name of executing organization> will ensure that your responses will remain
confidential.

If you have any questions or concerns about this project or <name of lead organization>,
please do not hesitate to contact me on (email address/ telephone number). Additional
information about our organization and activities can be found on our website: (website
address).

………… (Signature)…………
<Name of Project Coordinator @ LIRNEasia>
<Designation>
<Name of Lead Organization>

35
9. Annex A2: TRE Questionnaire
Questionnaire Number : ………………

Telecom Regulatory Environment for <Country>


You are kindly requested to make your frank assessments of the telecom regulatory environment
(TRE) for the year 12 months ending <Month, Year> for the fixed, mobile and broadband telecom
sectors on a five-point scale.
The dimensions used in this questionnaire are broadly based on the WTO Regulatory Reference
Paper (GATS Protocol 4) and are briefly described below. A fact-sheet of key events in the Telecom
Regulatory Environment is also attached for your reference for the period <start month> – <end
month> <year>.
Completing the Questionnaire should take less than 5 minutes of your time. Please email the
completed questionnaire to <email address> or fax it to <fax number>. If you prefer, you can
complete the same survey online by simply going to <URL, unique>.

Dimension Aspects Covered

Market Entry Transparency of licensing. Applicants should know the terms, conditions,
criteria and length of time needed to reach a decision on their application.
License conditions. Exclusivity issues.

Scarce Resources Timely, transparent and non-discriminatory access to spectrum allocation.


Numbering and rights of way: frequency allocation, telephone number
allocation, tower location rights.

Interconnection Interconnection with a major operator should be ensured at any technically


feasible point in the network. Quality of interconnection comparable to similar
services offered by own network. Reasonable rates for interconnection.
Unbundling of interconnection. Interconnection offered without delay.
Sharing of incoming and outgoing IDD revenue. Payment for cost of
interconnection links and switch interface. Payment for cost of technical
disruption of interconnection.

Tariff Regulation Regulation of tariffs charged from consumers.

Regulation of Anti Anti-competitive cross subsidization. Using information obtained from


Competitive competitors with anti-competitive results. Not making technical information
Practices about essential facilities and commercially relevant information available to
competitors on a timely basis. Excessive prices. Price discrimination and
predatory low pricing. Refusal to deal with operators and other parties.
Vertical restraints. Technical disruption of interconnection. Sharing of towers
and facilities by parent company and subsidiaries in different segments of the
market.

36
Universal Service Administration of the universal service program/fund in a transparent, non-
Obligation (USO) discriminatory and competitively neutral manner and is not more burdensome
than necessary for the kind of universal service defined by the policymakers.

Quality of Service Description to be added after obtaining a good definition


(QoS)

Please TICK the number that best represents the quality of the regulatory environment for
each dimension. The lower number represents Highly Ineffective and the higher number
represents Highly Effective.

FIXED SECTOR Telecom Regulatory Environment <start-month, Year- <end-month, Year>

F1 Highly Highly
Market Entry
ineffective  effective
1 2 3 4
5
F2 Highly Highly
Access to Scarce
ineffective  effective
Resources
1 2 3 4
5
F3 Highly Highly
Interconnection
ineffective  effective
1 2 3 4
5
F4 Highly Highly
Tariff Regulation
ineffective  effective
1 2 3 4
5
F5 Highly Highly
Regulation of
ineffective  effective
Anti-competitive
Practices 1 2 3 4
5
F6 Highly Highly
Universal Service
ineffective  effective
Obligation (USO)
1 2 3 4
5

37
F7 Highly Highly
Quality of Service
ineffective  effective
(QoS)
1 2 3 4
5

Comments:

Please TICK the number that best represents the quality of the regulatory environment for
each dimension. The lower number represents Highly Ineffective and the higher number
represents Highly Effective.

MOBILE SECTOR Telecom Regulatory Environment, for <start-month, Year- <end-month,


Year>

M1 Highly Highly
Market Entry
ineffective  effective
1 2 3 4
5
M2 Highly Highly
Access to Scarce
ineffective  effective
Resources
1 2 3 4
5
M3 Highly Highly
Interconnection
ineffective  effective
1 2 3 4
5
M4 Highly Highly
Tariff Regulation
ineffective  effective
1 2 3 4
5
M5 Highly Highly
Regulation of
ineffective  effective
Anti-competitive
Practices 1 2 3 4
5
M6 Highly Highly
Universal Service
ineffective  effective
Obligation (USO)
1 2 3 4
5
38
M7 Highly Highly
Quality of Service
ineffective  effective
(QoS)
1 2 3 4
5

Comments:

Please TICK the number that best represents the quality of the regulatory environment for
each dimension. The lower number represents Highly Ineffective and the higher number
represents Highly Effective.

MOBILE SECTOR Telecom Regulatory Environment, for <start-month, Year- <end-month,


Year>

B1 Highly Highly
Market Entry
ineffective  effective
1 2 3 4
5
B2 Highly Highly
Access to Scarce
ineffective  effective
Resources
1 2 3 4
5
B3 Highly Highly
Interconnection
ineffective  effective
1 2 3 4
5
B4 Highly Highly
Tariff Regulation
ineffective  effective
1 2 3 4
5
B5 Highly Highly
Regulation of
ineffective  effective
Anti-competitive
Practices 1 2 3 4
5
B6 Highly Highly
Universal Service
ineffective  effective
Obligation (USO)
1 2 3 4
5
39
B7 Highly Highly
Quality of Service
ineffective  effective
(QoS)
1 2 3 4
5

Comments:

10. Annex A3: TRE Summary of Regulatory Events


This is ONLY an example from the 2006 survey for India, covering the 12 months preceding
the survey. Please note a new one for each country must be completed by each
researcher.

Key Regulatory Events for India; June 2005-June 2006

Date Subject

2006

27 June Study Paper on Financial analysis of Telecom Industry of China and India.

16 June Consultation Paper On Admissibility of Revenue Share between Visiting Network


and Terminating Network for Roaming Calls. The key issue in this paper is that in
case of roaming, whether the terminating network service provider should get
only the prescribed termination charges or in view of higher roaming charges,
should there be any revenue share arrangement between the visiting network
service provider and the terminating network service provider.

13 June Consultation Paper on Interconnect Usage Charges (IUC) for Short Message
Service (SMS). This consultation paper mainly discusses the need for regulatory
intervention for Interconnect usage charges, specifically for SMS carriage and
termination charges.

12 June Consultation paper on Allocation and pricing of spectrum for 3G services and
Broadband Wireless Access. This paper discusses 3G spectrum allocation and
pricing related issues, issues related to spectrum for Broadband Wireless Access.
These technologies hold great potential for the rapid and comparatively
inexpensive deployment of broadband services especially in rural India.

6 June Proposed amendments in the Cable Television Networks (Regulation) Act, 1995

40
and the existing Telecom Licenses for facilitation of growth of IPTV services

24 May Consultation Paper on Fixing the Benchmarks pertaining to Quality of Service for
Broadband. This paper discusses the various issues relating to Broadband Quality
of Service parameters, the international practices, various broadband access
technologies and also suggests various Quality of Service parameters for
Broadband and their benchmarks.

21 April Consultation Paper on Issues relating to Commercial Tariff.

21 March The Telecommunication Tariff (Forty third Amendment) Order 2006 (3 of 2006).

21 March Regulation on Code of Practice for Metering and Billing Accuracy.


http://www.trai.gov.in/trai/upload/Regulations/44/regu21mar06.pdf

20 March TRAI releases Recommendations on Next Generation Networks (NGN)

10 March The Telecommunication Interconnection Usage Charges (Seventh Amendment)


Regulation (2 of 2006)

In Schedule III of The Telecommunication Interconnection Usage Charges


Regulation, 2003 (4 of 2003), the following entries shall substitute the existing entries
relating to paragraph 3.2.2:-

3.2.2 For calculating ADC , Adjusted Gross Revenue shall have the same meaning
as given in the respective licenses;

PROVIDED that in calculating the ADC as a percentage of Adjusted Gross


Revenue (AGR) of a Universal Access Service Licensee/Basic Service Operator, the
revenue from Rural Fixed Wireline subscribers shall be excluded.”

8 March TRAI provides its recommendations on mobile number portability: Mobile Number
Portability implementation process should be initiated in our country. A time frame
of 12 months between the acceptance of recommendation by the Government
and launch of this facility is recommended. It is recommended that this facility
should be available to mobile subscribers tentatively by 1st April 2007.
7 March The Telecommunication (Broadcasting and Cable) Services (Second) Tariff (fourth
Amendment) Order 2006 (1 of 2006). To give effect to this a Tariff Amendment
Order has been issued in which the words Ordinary Cable Subscriber, Commercial
Cable Subscriber has been defined and the definition of ‘charges’ has been
amended and a new clause to give effect to the relevant date for determining
the ceiling in respect of commercial cable subscriber has been introduced. The
proposed amendment is intended to be a short-term measure and would be
reviewed on the basis of detailed examination as indicated in para 3.

27 Feb. Direction to Mobile Service providers in the States of Maharashtra, Tamil Nadu,
West Bengal and Uttar Pradesh not to charge differential tariffs for calls
terminating in BSNL network and other service providers networks

41
23 Feb The Telecommunication Interconnection Usage Charges (Sixth Amendment)
Regulation 2006 (1 of 2006)

Salient features

• The total amount of ADC shall be reduced to Rs.3335 crore and estimated
ADC for BSNL would be Rs. 3,200 crore. Substantial reduction (about 33%)
in the amount of ADC
• There will not be any ADC on per minute basis on domestic calls.
• ADC on International Long Distance traffic shall continue to be on per
minute basis but at a reduced rate of Rs 1.60/minute (more than 50%
reduction) for Incoming International calls, this in turn will reduce arbitrage
and hence grey market. ADC on outgoing international calls have been
reduced to Rs.0.80/minute (reduction more than 65%).
• All licensees of Unified Access Service, Cellular Mobile Telephone Service,
National Long Distance Service and International Long Distance Service
shall pay 1.5% of their AGR as ADC to the BSNL. BSNL will retain ADC
chargeable as percentage of its AGR. Unified Access Service
Licensee/BSOs retain ADC as percentage of AGR of wireline subscribers
and the balance shall be paid to the BSNL.
• For estimation of ADC as a percentage of AGR, of access providers, the
revenue from the rural subscribers shall be subtracted.
• The UASLs/BSOs other than BSNL would retain ADC in terms of percentage
of AGR and also on outgoing international calls from their wireline
subscribers.
• No change in mobile and fixed termination charges from the existing level
of Rs.0.30 per minute.
• Death of distance acknowledged by moving over to a ceiling carriage of
Rs. 0.65/minute irrespective of distance.
• No ADC charge on rural revenue of operators to incentivize penetration
of telecom services in rural areas.
• Strengthening of monitoring mechanism of payment & receipt of ADC by
operators.

16 Jan TRAI issues Consultation Paper on “Tariff Plans with Life Time Validity”

12 Jan Consultation Paper on Issues pertaining to Next Generation Works (NGN)

Issues : -

• Awareness and relevance: Is NGN relevant for India? When should the
industry migrate? For which category of stakeholders is NGN relevant?
• Regulatory approaches: Is there need for regulatory initiatives on NGN?
Should there be ‘light touch’ regulation or are there areas needing more
detailed regulation? What regulatory incentives could help boost benefit
from NGN and reduce risks? Will a move to NGN in rural areas reduce the
gap between urban and rural tele-densities? If yes, how to push NGN to
rural India? What interconnection regime needs to be developed in the
NGN context?
42
• Migration issues: Is there a role for Regulator to ensure smooth migration?
2 Jan Consultation Paper on Issues relating to Convergence and Competition in
Broadcasting and Telecommunications.

2005

2 Dec The Register of Interconnect Agreements (Broadcasting and Cable Services)


(Second Amendment) Regulation 2005 (12 of 2005). TRAI has decided to amend
the existing clause 6 and make consequential amendments in clause 5 of the
above regulation so as to enable the Authority to specify a particular procedure
in regard to the manner of filing of data or information; to the form or formats of
filing; to the number of copies to be filed; and, to such other procedural aspects
connected and incidental to the filing of details of interconnect agreements
through a simplified process instead of the need to amend the regulations every
time whenever a change in procedure is necessitated.

2 Dec Draft Regulation on Intelligent Network Services in Multi Operator, Multi Network
Scenario Regulation 2005

Salient Features :

• All telecom consumers in the country shall have access to Multi-Operator


Multi-Service Intelligent Network (IN) Platform of their choice and no
Operator should be allowed to block his consumers from accessing IN
platforms of his choice.
• It shall be the Access Providers’ prerogative to deploy their Intelligent
Network (IN).
30 Nov TRAI issues Direction to Cellular Mobile Service Providers for ensuring Quality of
Service that the Quality of Service parameters, including the level of POI
congestion, in its network should be strictly within the benchmark laid down by the
Authority.

3 Nov TRAI reiterates its Recommendations pertaining to Local Loop Unbundling and
Fiscal Incentives for Broadband.

3 Oct TRAI provides its recommendations on growth of telecom services in rural India

16 Sept The Telecommunication Tariff (fortieth Amendment) Order 2005, (7 of 2005) In


exercise of the powers conferred upon it under sub-section (2) of the section 11
read with section 11(1)(b)(i) of the Telecom Regulatory Authority of India Act,
1997, the Telecom Regulatory Authority of India.

8 Sept The Telecommunication Tariff (thirty ninth Amendment) Order 2005, (6 of 2005). re-
fixed IPLC tariffs. The new ceiling tariffs for three most commonly used capacities
i.e. E-1 (Speed of 2 Mega Bits Per Seconds), DS-3 (Speed of 45 Mega Bits Per
Seconds) and STM-1 (Speed of 155 bits per seconds)

1 July Regulation on Quality of Service of Basic and Cellular Mobile Telephone Services,
2005 (11 of 2005) modify some parameters, some deleted and also introduce
43
some new parameters.

24 June Consultation Paper on Issues related to Entry Fee & Annual License Fee for ISP
License with Virtual Private network (VPN). This consultation paper presented a
scenario in the country as well as international scenario of some other countries
and different types of VPN’s & background about principles of license fee for
various telecom services.

6 June Consultation paper on Measures to promote competition in International Private


Leased Circuits segment in India; This paper presented a scenario in the country as
well as international scenario of some other countries, and the technical issues &
regulatory concerns arising out of the current situation.

44
11. Annex A4: Thank You Note

<Date>

<Name of Respondent>
<Designation>
<Organization>
<Address>

Dear< Sir/ Madam>:

LIRNEasia would like to thank <name of respondent’s organization/you> for contributing to


the perception survey of the Telecom Regulatory Environment carried out in <Year survey
was done> Ms/Mr. (name of researcher) from (name of executing organization).

Please find attached a copy of the <Title of report> for your reference.

We appreciate your input to our project and we look forwarding to working with you again
in the future.

If you have any further questions or concerns about the TRE study or the attached report,
please do not hesitate to contact me. Additional information about our organization and
other activities can be found on our website: (website address).

………… (Signature)…………
(Name of LIRNEasia TRE Research Coordinator)
(Designation)
(Name of Lead Organization)

45
12. Annex A5: Respondent List
See Microsoft Excel file/template, attached separately

13. Annex A6: TRE Data File


See Microsoft Excel file/template, attached separately

46
14. Annex A7: Confidentiality Agreement
The following agreement is signed by the TRE researcher. It should be done at the very
beginning of a TRE survey, before any contact with potential respondents is made.

In all matters related to the 2008 TRE research, I hereby agree to uphold the highest ethical
standards and to follow the steps specified in the TRE Manual published by LIRNEasia.

Specifically, I agree to:

1. Ensure that all research assistants, subcontractors or other third parties who may
come into contact with the TRE data will sign, and be subject to the same
confidentiality agreement
2. To ensure that information about respondents (or anything that enables
identification of individual respondents) is kept separately from the TRE scores they
have expressed by:
a. following the steps outlined in the TRE Manual to create two separate files
(the Respondent File and the TRE Data file)
b. ensuring that physical and digital data is kept under lock & key or password
protected at all times during data collection
c. ensuring that the Respondent file and any completed paper questionnaires
are provided to the Data Guardian, password protected (password to be
sent separately) immediately after data collection
d. ensuring any and all other copies of the Respondent File are
deleted/destroyed from digital or physical storage after it has been received
by the Data Guardian appointed by LIRNEasia
e. ensuring that the TRE Data file information is not shared or displayed with
anyone outside of LIRNEasia and your research team (who are subject to this
terms of confidentiality) in any manner that will enable association between
persons and the scores (see TRE Manual for details)

Name in Full: ______________________________________________________________________

Signature: ___________________________________ Date:_____________________

47
Please complete the above information, sign and date it, and return to LIRNEasia

48

You might also like