Bates V Post Office: Steve Parker Witness Statement 1

Filed on behalf of the: Defendant
Witness: Steven Paul Parker

Statement No.: First
Exhibits: SPP1
Date Made: 16 November 2018
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
IN THE HIGH COURT OF JUSTICE

QUEEN'S BENCH DIVISION
ROYAL COURTS OF JUSTICE
B E T W E E N:
ALAN BATES & OTHERS

Claimant
AND
POST OFFICE LIMITED

Defendant
WITNESS STATEMENT OF STEPHEN PAUL PARKER
I, STEPHEN PAUL PARKER of Lovelace Road, Bracknell, Berkshire RG12 8SN WILL
SAY as follows:
1. I am employed by Fujitsu Services Limited (Fujitsu) as Head of Post Office

Application Support.
2. I am authorised to make this statement on behalf of Post Office Limited (Post

Office), the Defendant in these proceedings, in relation to the Horizon Issues trial
listed for March 2019.
3. The facts set out in this statement are within my own knowledge, or if they are
outside my knowledge, I have explained the source of my information or belief.
4. In this statement I refer to copy documents attached and marked Exhibit SPP1.
BACKGROUND
5. I started working on the Royal Mail Group Account, later called the Post Office
Account, in July 1997, before the introduction of Horizon. I have continued to
provide support to the Post Office Account in my various roles at Fujitsu
throughout the whole of Horizon’s life.
6. Prior to my work on the Post Office account, I held a number of roles within the IT
industry, including an in-house operations and support role for critical online
systems, support consultancy services and design and development role in an
AC_152871086_1 1
E2/11/1
End User Technology group in relation to Microsoft tools for internal and external
customers.
7. I began working in July 1997 as a support consultant and deputy manager within
the Software Support Centre (SSC) for the Royal Mail Group Account, providing
third line application support for the Horizon application. Within this role I was the
lead designer and part of development team for the internal website providing
support knowledge (KEL) database, technical documentation management and
operational change control. I also assisted the SSC manager in the provision of
the support service and operational management. Although I did not have the
formal title, I acted as the deputy manager to the SSC as a whole.
7.1 Between December 2009 and March 2010 I was a full time Problem Manager /
Operational Manager of the SSC, responsible for the management of incidents
through the whole support process.
7.2 In March 2010 I became the Manager of the SSC and was responsible for the
provision of third line application support to the Post Office Account, including the
management of the staff working on the account. The SSC subsequently
expanded into a shared service, providing support services to a number of Fujitsu
customers, the largest of which is still the Post Office Horizon system. As head
of this unit I am currently responsible for strategic support, managing 25 – 40
staff.
RICHARD ROLL'S STATEMENT
8. I have been asked to comment on the witness statement of Mr Roll dated 11 July
2016 put forward by the Claimants in relation to the Horizon Issues trial in March
2019. I worked with Mr Roll while he was employed by Fujitsu in the SSC (I
understand from Fujitsu's HR department that this was from 5 March 2001 until 17
September 2004). As explained above, although I did not have the formal title, I
acted as deputy manager while Mr Roll was there.
9. The Horizon system was rolled out across the Post Office network between 1999
and 2000. In 2010 there was a migration from the system commonly referred to
as "Legacy Horizon" to an online version ("HNG-X" or "Horizon Online").
Therefore, when Mr Roll refers to the Horizon system he is referring to Legacy
Horizon and where I respond to an assertion made by Mr Roll I am also referring
to Legacy Horizon unless otherwise stated. Much of my statement also applies to
Horizon Online, however.
10. I found Mr Roll to be a conscientious worker and provided him with a personal
reference for a position that he applied for after leaving Fujitsu (it was a personal
AC_152871086_1 2
E2/11/2
reference because Fujitsu does not allow staff to provide references on behalf of
the company).
11. In his statement Mr Roll suggests that there were frequent instances of software
problems in Horizon that had an impact on branch transaction data and which
Fujitsu resolved "remotely" (i.e. not in a branch), not merely by changing the
software but also by frequently changing branch transaction data (by injecting
new transaction data and by editing or deleting existing transaction data), without
informing branches that such actions were being taken. As I explain below, those
suggestions are incorrect and Mr Roll's account of Fujitsu's actions and powers is
inaccurate and misleading.
12. In order to explain why, it is necessary to have a proper understanding of what

Fujitsu was able to do, what the effects of its actions were and the extent to which
such actions had any effect on transaction data in branches. This is described
further below, but I would like to make some general responses immediately.
13. It is correct that the SSC had (and has) the ability to view data in branches and
other sources such as data centres remotely (in read-only mode): that is to be
expected given their support role which is described in paragraph 26 below.
14. Post Office did (and does) not have the same ability – what it could (and can) do
is view copies of transaction data as replicated from Horizon to a variety of Post
Office systems. In Horizon Online, facilities have been added which allow Post
Office to remotely examine branch data held in the Branch Database (BRDB) in
read-only mode for a variety of business reasons, such as monitoring levels of
cash that are in branches based on the data entered onto Horizon by branch staff.
15. It is also correct that issues sometimes arose which necessitated changes to the
Horizon software. However, this was not something that Mr Roll played any
significant part in, as I describe in paragraphs 34 and 35 below.
16. It was (and is) theoretically possible for there to be a software problem which
could cause a financial impact in branch accounts, but this was (and is) extremely
rare and Horizon's countermeasures were (and are) very likely to pick such
matters up. In my experience, these problems have always represented a very
small proportion of the issues which led to software changes and a very small
proportion of the overall issues dealt with by the SSC.
17. On the very rare occasion that a software problem which could cause a financial
impact in branch accounts arose, it would be investigated and resolved and
Fujitsu would determine its impact on the Horizon estate and inform Post Office of
any financial impact on branches so that they could be resolved.
AC_152871086_1 3
E2/11/3
18. In Legacy Horizon it was possible for the data in a particular counter in a branch
to become inconsistent with replicated copies, and Mr Roll appears to be
describing this in paragraph 17 of his statement. In that situation there could be
remote management by Fujitsu to correct the problem, but branch transaction
data was not changed in any way. As explained in paragraph 55 below, the work-
round involved replicating the correct data from another counter in the affected
branch or from the data centre copy.
19. The suggestion that Fujitsu edited or deleted transaction data is not correct. In
Legacy Horizon it was not possible to delete or edit messages that had been
committed to the message store. See the explanation of my colleague Torstein
Godeseth in paragraph 37 of his witness statement dated 27 September 2018
which reflects my understanding and experience of Horizon’s functionality.
20. In para 18 of his statement, Mr Roll also suggests that in Legacy Horizon he and
others in the SSC could "insert transactions and transfer money without the sub-
postmaster knowing". However:
20.1 No Fujitsu personnel have ever had the ability to "transfer money" out of Horizon
into, for example, an individual's account.
20.2 Some members of the SSC were (and some remain) able to insert transaction
data. SSC access privilege gave the ability to inject transactions, but appropriate
change controls were in place and no such insertion would have happened
without complying with those controls.
20.3 I should make it clear that, in this witness statement, I am concentrating on what
the support process is designed and able to do and not any malicious misuse of
these facilities. Malicious misuse makes most things possible, as with any other
IT system, however, Horizon had a number of checks and security settings in
place that made it very difficult to carry out malicious misuse. In any event such
misuse would have been discovered by consistency checks or colleagues (all
access was controlled and audited) and would have resulted in instant dismissal.
But even a malicious user would not have been able to “transfer money”.
21. As there is no way in which money could be taken from a branch and moved
anywhere else (for example into the employee’s bank account), it follows that
there was no motive for anyone to do this. It is not clear to me why anyone at
Fujitsu would insert transaction data into a branch's accounts without there being
a legitimate reason for doing so. Furthermore:
21.1 Any transaction that was inserted would immediately cause a discrepancy to
arise in the branch's accounts. For example, if a transaction were to be inserted
AC_152871086_1 4
E2/11/4
which stated that £1,000 of stamps had been bought by a customer who paid
cash, that would immediately cause a reduction in stock levels of stamps in that
branch and the branch would have £1,000 less in cash than Horizon expected it
to have.
21.2 In other words, although a transaction could be inserted, it would immediately

become apparent that this had been done and ultimately it would not benefit any
member of staff to behave in this way.
22. It is correct that the "remote access" described above could have been carried out
without the permission of a Subpostmaster. However, any additional transactions
inserted remotely would be identifiable as such from the transaction logs that are
available to Subpostmasters from Horizon.
23. I will now describe the structure of the Horizon support teams during the period
when Mr Roll was employed by Fujitsu before responding to the specific
suggestions that Mr Roll makes.
The Horizon Support Teams
24. There were four lines of support for Horizon while Mr Roll was employed by
Fujitsu and they are described in paragraph 26 below. There are still four lines of
support for Horizon today, albeit that some names have changed and some
responsibilities have moved around teams.
25. It is common within the industry to have a multi-level support model. Generally,
as you move up through the levels of support the cost of the staff providing the
service increases because they are more qualified. Having said that, there is
often overlap of skills between adjacent lines of support and while a team may be
responsible for a particular level of support, staff within that team can have skills
which allow them to perform a role that is more usually performed by the next
level of support.
26. The four lines of support for Horizon while Mr Roll was employed were as
follows:-
st st
26.1 1 line: The 1 line involved several different elements:
1
26.1.1 the Horizon Service Desk (HSD) was a helpdesk operated by Fujitsu
that branches could contact with issues relating to the Horizon
application or the hardware provided in branch by Fujitsu to run the
1
The HSD has also been known as the Horizon System Helpdesk and the Horizon
Incident Team.
AC_152871086_1 5
E2/11/5
Horizon application. I estimate that there were around 80 members of

the HSD while Mr Roll was employed, who:-
(a) dealt with straightforward enquiries such as password issues and

scheduling engineers to attend branches to investigate reports of
hardware issues;
(b) monitored the live estate and took corrective actions defined in
knowledge documents (this role was fulfilled by two units: the
System Management Centre (SMC) for data centre events; and
the Counter Eventing Team for Post Office counter events. The
corrective actions did not involve any changes to any branch
transaction data); and
nd
(c) referred other issues to the 2 line support function.
st
26.1.2 there was also a 1 line Communications Management Team operated
by Fujitsu which specifically focused on communication incidents; and
st
26.1.3 Post Office also operated a 1 line helpdesk for operational issues
called the National Business Support Centre (NBSC).
If a branch required assistance to attempt to determine the cause of a

discrepancy they would contact NBSC in the first instance. Discrepancies are not
unusual in a retail system. They indicate a difference between the operator’s
declaration of cash and stock on hand and the systems calculation and as such
are a business operation issue. However, it was not always possible for NBSC to
identify the cause of a discrepancy. For example, a user may enter a deposit of
£100 into a customer's bank account on Horizon but rather than taking £100 from
the customer, they may make a mistake and give the customer £100 as if it had
been a withdrawal. In that scenario, NBSC would not have been be able to
identify the cause of a discrepancy. Clearly, NBSC is also unable to assist when
losses have been caused by theft.
If NBSC were unable to identify the cause of a discrepancy they would often fall
back on a default statement along the lines of “this looks like a software issue" so
that the SSC would investigate it. However, Mr Roll's statement that "[i]f an error
was referred to us then it was extremely unlikely to be due to a mistake made by
a postmaster" is not correct. The vast majority of discrepancies investigated by
the SSC as pseudo “software issues” were (and are) not caused by software
issues.
AC_152871086_1 6
E2/11/6
nd nd
26.2 2 line: 2 line support was provided by senior members of the HSD and SMC
rd
and junior members of the SSC (as explained below, the SSC also fulfilled a 3
nd
line support function). The 2 line support function mainly involved staff
searching knowledge articles based on the descriptions of issues reported by
branches, gathering evidence and applying simple, well-defined work-arounds
(often on the phone). An example of this would be resetting passwords.
rd rd rd
26.3 3 line: the SSC also provided 3 line support. The staff that provided 3 line
support had a detailed knowledge of the Horizon application based on
documentation and some inspection of source code. They:-
st nd
26.3.1 designed, tested and documented work rounds for the 1 and 2 lines
of support;
26.3.2 applied analytical skills to the symptoms and evidence gathered by the
st nd
1 and 2 line functions and undertook in-depth investigation into
incidents (incidents are the basic unit of work for the support team and
come from helpdesk calls and other Horizon support teams);
26.3.3 undertook complex configuration (configuration items can be used to

alter the behaviour of the application) and data fixes which might have
required the generation of special tooling;
26.3.4 designed, wrote and documented new support tools;
26.3.5 undertook source code examination, complex diagnosis and

documentation (including methods to recreate faults) of new application
th
problems before sending them to the 4 line support group for root
cause software fix; and
26.3.6 provided technical support to other internal Fujitsu teams working on

2
Horizon.
rd
The 3 line support function used a system called Peak (until 2003 it was called
PINICL) to log and manage incidents passed to them which were suspected to be
faults. It also maintained a Known Error Log (KEL) which describes the
symptoms of problems with some analysis of causes, (potential) solutions to the
problems and workarounds that might be needed before a permanent solution
2
An example of this which applies to Horizon Online is support to the Management
Support Unit (MSU) who are responsible for the Reconciliation Process documented in
SVM/SDM/PRO/0020 (Reconciliation and Incident Management Joint Working SVM/SDM/PRO/0012
Document). The reconciliation process also applied to Legacy Horizon. {F/1051}
AC_152871086_1 7
E2/11/7
can be implemented. The Peak and KEL systems are still in use today and are
described further in paragraph 62 onwards below.
th th
26.4 4 line: 4 line support staff had an intimate knowledge of narrow areas of the
system and were (and are) ultimately responsible for the production of permanent
fixes to repair the root cause of an incident or problem in the live application.
They had knowledge of computer languages which they used to amend source
code to fix problem in the live application code. There was often overlap between
th
4 line and developers, who added new features into the application.
27. The structure of the support for Horizon is broadly the same today. One of the
main changes is that the HSD function is no longer operated by Fujitsu (it has
been operated by Atos since June 2014).
Further context – call volumes and transfers
28. Between 1 January 2001 and 31 December 2004 (the years Mr Roll worked for
Fujitsu), the SSC received a total of 27,005 calls, meaning that on average 563
calls per month were dealt with over this 4 year period. This is shown by a
spreadsheet prepared by a team in the SSC which appears at Exhibit SPP1 (Tab {F/1839}
"RRP_Live_Peaks_Into_SSC"). Where (as here) I analyse data in this statement,
the analysis is mine.
29. Transferred calls (i.e. those not resolved by the SSC) are of interest. A very small
th
proportion of calls transferred to 4 line support would have concerned software
errors requiring resolution, so it would be interesting to know the number of calls
transferred to them. However, while the SSC have records of the volume of
transferred calls, we do not retain records of where they are transferred to and it
th
is not the case that all of these would have been transferred to 4 line support.
For example, incidents would often arrive at SSC from internal teams for routing
back to helpdesks.
30. As evidenced by the data in Tab "RRP Live Peaks Out of SSC" of Exhibit SPP1, {F/1839}
an average of 78 calls per month (14%) were transferred to teams outside SSC,
nd th
for example, to 2 and 4 line support. This indicates that only a small volume of
calls received and escalated to the SSC related to software errors requiring
resolution.
31. In terms of calls to the HSD, Fujitsu only holds records of the volume of calls
made to HSD from December 2008 onwards. I understand from my colleague
Sandie Bothick (Fujitsu’s Service Delivery Manager) that the HSD received
13,225 calls in December 2008 and 13,005 in January 2009. The witness
statement of Angele Van Den Bogerd provides that Post Office’s NBSC call
AC_152871086_1 8
E2/11/8
volume data shows an average of 1,096 calls per day were received in the period
2015 to 2018. While these figures relate to different periods, they indicate that
st
only a tiny proportion of issues reported to 1 line support helplines were
escalated to SSC.
th
32. From the SSC, only a tiny proportion of incidents were escalated to the 4 line
support team. It follows that only a tiny fraction of incidents raised actually needed
to be looked at by the only team who might potentially effect changes in software.
The SSC and Mr Roll's role within it
33. I agree with Mr Roll's recollection that there were around 30 individuals working
th
on the 6 floor in Bracknell (i.e. in the SSC) at any one time while he was
employed.
nd rd
34. As noted above, the SSC team provided both 2 and 3 line support. As with
any mix of people, there are (and were) various levels of talent within SSC. Mr
Roll was primarily used in Operational Business Change (OBC), which involved
supporting the engineers who were opening and closing branches and increasing
and decreasing the number of counters in branches. Mr Roll would also have
been regularly correcting the application environment after engineers had
replaced failed counter hardware and clearing temporary files to increase disk
nd
space. This could fairly be described as 2 line work and it was done by the SSC
because it required a higher level of access to the system than other support
teams had.
rd
35. Some members of the 3 line support group identified the need for software fixes
th
via source code examination and would pass this on to the 4 line team for a
code fix to be written. Mr Roll did not play any significant part in this and was not
involved in any extensive source code examination. An application code fix would
rd
not be written by anyone in the 3 line team and he was not involved in the
th
provision of 4 line support
36. I disagree with Mr Roll's suggestion that much of the work being carried out by the
SSC while he was employed could be described "as "fire fighting" coding
problems in the Horizon system." There were times when the SSC was very
busy, for example, networking problems causing application issues across the
whole estate and data centre outages. But there were only rare circumstances
where a coding issue had an estate wide impact and, in those instances, Mr Roll
would have been involved executing avoidance actions to mitigate impact to the
estate (i.e. following established work-arounds) rather than working on the root
cause.
AC_152871086_1 9
E2/11/9
37. The SSC had (and has) access to view, but not amend, source code. Senior
members of the team would have looked at it from time to time to confirm exactly
how the Horizon application would process a given input and what the outputs
would be when investigating specific issues or general education on how the
system works. However, the access was rarely used. Moreover, Mr Roll was not
working at a level where he would be required to review code. His role in the
SSC was predominantly following work around processes designed by other
people and making configuration changes.
38. In support of the statements above relating to SSC workload I have undertaken
some analysis of the work carried out by SSC between 1 January 2001 and 31
December 2004 (as stated above Mr Roll was employed from 5 March 2001 until
17 September 2004). My colleague John Simpkins provided the summary data
from the Peak system, which I analysed.
39. When an incident is resolved, the SSC team member (or technician as they are
sometimes called) types a summary of the incident (known as a Final Response)
and allocates a response code to the incident in order to classify it. While
guidance is provided on when to use each response code (see paragraph 64.5
below), allocation is the subjective view of the technician closing the incident and
there is no re-examination of the response codes later to ensure consistency.
40. With that in mind, the final response codes that were allocated to incidents (i.e.
Peaks) reported to SSC between 1 January 2010 and 31 December 2004 were
as follows:-
40.1 known issue / work around - 35.3%;
40.2 admin - 27.5%
40.3 reconciliation - 15.7%;
40.4 potential user error - 10.9%;
40.5 potential software error - 8.3%; and
40.6 hardware error - 1.2%.
41. It should be noted that:-
st
41.1 a major part of 1 line’s raison d'être is to deal with user error and therefore the
st
percentage of issues attributable to user error would be much higher at 1 line;
AC_152871086_1 10
E2/11/10
41.2 very few hardware incidents reached the SSC because they were the preserve of
st
the HSD (i.e. they were relatively easy to spot and therefore filtered out by 1 line
support);
41.3 8.3% of calls to the SSC (2252) are attributed to potential software errors over
these four years. This includes duplicates and does not provide any clarity on the
significance of the error corrected. Many software errors, particularly in a new
product, were insignificant, such as correcting capitalisation in printed output. I
cannot be more precise without examining each Peak and even then it might not
be possible to determine how a Peak should have been properly classified
(Peaks are essentially notes made by Fujitsu personnel to chart the progress
made in resolving an issue and these notes can vary in fullness and clarity); and
41.4 Classifying an incident as a "potential" software error does not necessarily mean
that there was a software error and, even if there was, it does not mean that the
error was one that could have caused a financial impact in a branch's accounts (a
large proportion of these would be errors in numerous data centre resident
systems that the Subpostmaster never sees – errors were often as trivial as the
use of “Kg” instead of “kg” on receipts). As stated in paragraph 16 above, such
errors were extremely rare. They were all resolved (resolutions include a source
code fix, a configuration fix or a workaround).
42. Mr Roll's suggestion in paragraph 10 that software issues in Horizon "routinely"

caused discrepancies in branch accounts is misleading. In the vast majority of
cases such an occurrence would cause a receipts and payments (R&P) mismatch
that would be flagged by the branch system as part of the balancing process (the
Horizon system carries out self-consistency checks which generate alerts in the
event of a receipts and payments mismatch that are picked up by SMC and
incidents raised for the SSC) and appear on MSU reporting. These would then
be investigated and resolved by the SSC.
42.1 Since the introduction of Horizon in 1999 there have been 735 live incidents
which refer to “Payments and Receipt mismatch” (i.e. incidents recorded against
components of the system providing Horizon service to Post office rather than
incidents raised against test systems). This figure has been obtained using a
textual search across all incidents where the title or one of the incident updates
contains all of the words “receipts”, “payments”, “mismatch”. It should be noted
that this is not 735 unique incidents; there will be a lot of duplicates with the same
root cause. The only way to determine how many unique incidents there were
would be to manually review all of the incidents. All of them were resolved.
AC_152871086_1 11
E2/11/11
42.2 These incidents are reported as a result of self-consistency checks carried out by
Horizon. It should be noted that a R&P mismatch is not only caused by a
software error. It can also be caused by incorrect product reference
(configuration) data
42.3 Receipts and payments mismatches happened more often during the early life of
{F/1839}
Horizon (see Tab “All RnP by RCode and Date” of Exhibit SPP1). My analysis of
that data shows that there were around 8.6 such incidents per month on average
between 1 January 2001 and 31 December 2004 (417 out of a total of 27,005
incidents into SSC or 1.5% of SSC incidents during that period).
43. Mr Roll refers to a "perception…that the Service Level Agreements between Post
Office Ltd and Fujitsu involved financial penalties payable by Fujitsu to Post
Office" (paragraph 12). I am aware that there were Service Level Agreements for
issues such as stuck transactions (Fujitsu had 10 days to retrieve transactions
that had not replicated from a counter). It is quite normal for contracts such as
the one between Fujitsu and Post Office relating to Horizon to have such
agreements. The same level of diligence was (and is) applied to all incidents,
whether an SLA was relevant of not. The possibility of financial penalties was
never a factor for the SSC.
44. I do not understand what Mr Roll means when he says that "any discrepancy in
the post office accounts had to be resolved speedily" (paragraph 12). There was
(and is) a process run by the Management Support Unit (MSU) which involves
examination of various system reporting and may result in Business Incident
Management Service (BIMS) entries going to Post Office. An incident may also
be raised by MSU with the SSC to provide support to the MSU in resolution of the
BIMS. These are subject to Service Level Agreements and Mr Roll may be
referring to this process. However, if Mr Roll is suggesting that Fujitsu routinely
rushed out fixes or workarounds due to SLA time pressure, that is not the case.
Fixes would be expedited based on service impact. It would be quite wrong to
suggest that they were not done properly because of any SLAs.
45. It is correct that there are a limited number of opportunities to release software
updates and that these releases could take 6 weeks or more to be released to the
live service (Mr Roll’s statement, paragraph 13). These longer timescales would
be employed for non-urgent updates wrapped up into a consistent set for
deployment. On those rare occasions when a problem has an impact on the
financial integrity of the system a "hot fix" would be deployed which involved
much shorter timescales. I would expect a timescale measured in days not weeks
to deploy a hot-fix. Mr Roll also states that a bug could reappear several weeks
AC_152871086_1 12
E2/11/12
after a coding fix had been released due to software issues. I am aware of only
one or two cases where a fix regressed in my time at Fujitsu.
46. In paragraph 14 Mr Roll states, "I would reiterate that the main recurring issues
were software issues." It is a symptom of working within a software support team
that the majority of issues that come in have been attributed to a software issue
by, for example, a lower line of support. This can lead to a mind set of “look at all
these Horizon errors”, but what this indicates to me is that the previous levels of
support are functioning correctly, removing the majority of other causes (user /
hardware problems). It does not indicate that the majority of Horizon errors could
be attributed to software.
47. Mr Roll states (paragraph 7) that "[s]oftware programs were written by us to strip-
out irrelevant data, to enable us to more easily locate the error." The support
tools are used to filter information and present information to technicians in ways
that make the support process easier. I am aware of two support tools (also
known as software programs) that were written while Mr Roll was employed by
Fujitsu:-
47.1 the Smiley support tool written by my colleague John Simpkins, which
amalgamates information from various sources (e.g. databases) into a single
view pertinent to a particular support task and provides a unified interface to run
various tools to achieve a single support outcome; and
47.2 another tool (I cannot remember its name) written by my colleague Richard
Coleman whose function was to retrieve messages (i.e. data) from the
correspondence server to local text files for examination and which was
eventually subsumed into the Smiley support tool.
47.3 Neither of these tools changed the underlying data in a branch's accounts.
The work carried out by Mr Roll
48. Mr Roll states that he would investigate financial discrepancies that had arisen in
branches by "work[ing] sequentially through all transactions over the relevant
period, and also work[ing] through thousands of lines of computer coding"
(paragraph 7).
48.1 It is not the role of the SSC to routinely investigate discrepancies.
48.2 In very rare circumstances a discrepancy could be caused by a software issue

th
and in those circumstances it might be necessary for the 4 line support function
to work through thousands of lines of computer coding to determine the root
AC_152871086_1 13
E2/11/13
cause. However, Mr Roll would not have worked through thousands of lines of
computer coding to investigate a discrepancy in a branch: he did not work in the
th
4 line.
49. Mr Roll further states that if SSC was "unable to find the cause of the discrepancy
then this was reported up the chain and it was assumed that the postmaster was
to blame" (paragraph 10). That is not my experience: it is a simple truth of support
that the majority of issues reported in a system are attributable to user action or
user misunderstanding of system functionality. Hence, someone working in a
support environment analysing a new issue would examine the possibilities of
user error as a first hypothesis but any final conclusion is only generated based
on the evidence. Where the evidence does not support a conclusion that there is
a problem with Horizon, the SSC feeds the existent factual data back to Post
Office and might say something along the lines of "all indications are that the
branch has made a mistake", but Fujitsu neither attributes "blame" or agrees the
final conclusion with the Postmaster
50. When an incident is resolved, the SSC team member (or technician as they are
sometimes called) types a summary of the incident (known as a Final Response)
and allocates a response code to the incident in order to classify it. While
guidance provided on when to use each response code, allocation is the
subjective view of the technician closing the incident and there is no re-
examination of the response codes later to ensure consistency.
51. The Peaks that Mr Roll works on while employed by Fujitsu indicate that he dealt
{F/1839}
with 915 incidents (see Tab "Final Responses" of Exhibit SPP1). To give some
clarity on these incidents I have analysed their final response codes that were
allocated to them by Mr Roll. He classified them as follows:-
Response code % allocated

by Mr Roll
Known issue / work around 61.9
Reconciliation 14.5
Admin 9.3
Potential software error 3.2
52. This supports my recollection that Mr Roll mainly followed work-arounds devised
by other people (61.9%) and that he was rarely involved in the detailed
examination of potential software errors (3.2%). As explained in paragraph 41.4
above ‘potential’ software errors do not necessarily mean software errors, let
alone software errors resulting in discrepancies to branch accounts.
AC_152871086_1 14
E2/11/14
Remote access
53. I understand from Post Office's solicitors that the term "remote access" has been
used to describe Fujitsu and Post Office carrying out the following actions when
not physically present in a branch:-
53.1 accessing branch data in read-only mode;
53.2 inserting new transaction data; and
53.3 editing or deleting existing transaction data.
54. As noted above, it is correct that Fujitsu had (and has) the ability to view data in
branches remotely given their support role.
55. Mr Roll claims that "[d]uring the course of resolving the software issues, we would
frequently access a Post Office counter IT system remotely" (paragraph 15). Mr
Roll gives the example of ("when a binary bit would "flip", thus a "1" became a
"0""):-
55.1 This probably relates to a condition known as a CRC (Cyclic Redundancy Check)
error which would happen when a hard drive became faulty at a branch counter.
Although this is a hardware problem, remedial action was needed by the SSC to
resolve. To clarify, this process did not involve changing any transaction data.
55.2 As explained by Mr Godeseth in paragraph 35 of his statement, in Legacy

Horizon:-
3
55.2.1 all counter data was held in a bespoke message store in each branch
and the data was replicated to all counter positions in the branch and
from the branch to the data centres where it was held in
correspondence server message stores;
55.2.2 any data inserted into the message store at the data centre (for
example reference data or authorisations for banking transactions)
would be replicated back to the branch counters; and
55.2.3 selected data was extracted from the correspondence servers to

update Post Office's back end systems.
55.3 If one of the sets of data on a branch counter became corrupted it would
generate an event that would be picked up by the SMC and/or reported to HSD
3
A message means data and transaction data is a subset of the data in the message
store.
AC_152871086_1 15
E2/11/15
by the branch (an incident reporting a “CRC Error”). There were a total of 629
CRC errors over the life of Legacy Horizon (see Tab "All CRC by Date" of Exhibit {F/1839}
SPP1).
55.4 The issue would be reported to the SSC, who would delete the entire set of data
on that counter and replace it with a copy of the data from one of the other
sources that had not been corrupted. While this process involves deleting and
replacing a set of data, no new data is produced; all that happens is that the
replicated data is used to replace the data that has become corrupted from
another counter in the branch. It would have been necessary for the SSC to
inform a branch before carrying out this task because it is likely that any attempt
to use that counter would fail while the process was being carried out. In Mr Roll’s
capacity as an OBC specialist, he would have been involved in this type of
activity.
55.5 Alternatively, Mr Roll's reference to a binary code flipping may relate to a

configuration item (an item that can change the behaviour of the application)
which can become locked in the wrong binary setting (1, 0). One example of
which would be a stock unit lock which, in the wrong state, would prevent
updates to stock units within a branch. This issue was corrected by a member of
the SSC accessing Horizon remotely, but it did not involve accessing or editing
transaction data in any way or re-creating databases.
55.6 I cannot think of any other examples of incidents that Mr Roll may be referring to
in paragraph 15.
56. Mr Roll claims that "some errors were corrected remotely without the sub-
postmaster being aware" (paragraph 16).
56.1 It may be that Mr Roll is referring to issues relating to the end of day concept in
Legacy Horizon. Essentially there was a cut-off point for transactions every day
at 7:00pm and each counter had to write an end of day message to the branch's
master counter to enable the master counter to write a branch end of day
message, which would then trigger the data centre to harvest messages
(including details of transactions) to Post Office's back end systems.
Occasionally a counter in a branch would fail to write an end of day message and
there was a process for correcting this. The issue would be reported to SSC by
way of an incident (either as a result of a call to HSD or sometimes Fujitsu could
spot issues via system events).
AC_152871086_1 16
E2/11/16
56.2 In lay terms, SSC would force the counter to generate a report based on the data
already in the counter. A message injected in this way would go into the audit
trail. This would not alter the branch’s transaction data.
57. Mr Roll also states that, there were some errors where it was necessary to
"download and correct the data and prepare it for uploading back on to the post
office computer, then call the postmaster to inform him that there was problem
and that we needed two or three minutes to correct it" (paragraph 17).
57.1 It is not clear what errors Mr Roll is referring to or how he says they
were corrected. The issue referred to could be another instance
where the work round of re-creating transaction data from a
replicated copy was required as described in paragraphs 55.3 and
55.4 above.
SUMMARY
58. In summary, the suggestion that Fujitsu would manipulate a branch’s transaction
data in a way which was detrimental to a particular Postmaster and undetectable
is wrong.
58.1 All support action taken by Fujitsu is directed to ensuring that legitimate
transactions entered by Subpostmasters are correctly processed by the Horizon
application and correctly passed to other POL systems as appropriate.
58.2 Any financial corrections required are communicated by Fujitsu to Post Office for
execution or approval.
58.3 Any support intervention that requires the insertion of a transaction is identifiable
in the audit trail.
58.4 There is no financial incentive for a support technician to circumvent the controls
within the system.
58.5 There are strong controls in place to prevent intervention by support staff with
malicious intent or misguided attempts at financial gain.
59. The statement that issues with coding in the Horizon system were extensive and
impacted branch finances is incorrect for the reasons stated above.
KELS AND PEAKS
60. I have been asked by Post Office's solicitors to provide some more information on
the two toolsets used to support the Horizon system, the KEL knowledge base
AC_152871086_1 17
E2/11/17
(AKA KEL) and the incident management system (PINICL / Peak). This
information is applicable to both Legacy Horizon and Horizon Online except
where explicitly stated otherwise.
61. Fujitsu use a custom solution, developed and administrated by the SSC, which
allows us to record support knowledge into a structure known as a KEL (Known
Error Log). KELs record support knowledge which is intended to assist staff in
the support and understanding of the Horizon system. KELs do not contain the
history of an incident (see my analysis of the Peak system below). KELs are
generated for a number of reasons, for example: to explain system behaviour or
messages that originate from central and counter systems; to record symptoms
and outcomes from incidents referred by help desks or identified as a result of
Fujitsu's reconciliation processes; and to record information on issues seen in test
environments (resolved before the feature is passed on to users).
61.1 The acronym KEL is a misnomer inherited from a previous system. KEL entries
are support knowledge entries and do not have a one to one relationship with
errors on the system. There are a lot more general supporting knowledge KELs
than KELs relating to specific errors.
61.2 Guidelines for the generation and use of KELs are documented in
SVM/SDM/PRO/0875 (End to End Support Strategy) section 11 Knowledge Base {F/823}
Maintenance. Although some aspects of this document need revision due to
changes in the support structure for Horizon, Section 11 is still fundamentally
correct in relation to Horizon.
61.3 KELs reflect community sourced knowledge to assist staff involved in the support
of the Horizon solution. There are no mandated rules for when a KEL should be
created other than a desire to make resolving a problem easier for all concerned.
Guidelines exist in SVM/SDM/PRO/0875. Any KELs created or updated are {F/823}
referred to the SSC for approval to provide a basic check that the information at
the time of the change is valid.
nd
61.4 KELs can be created by the senior people in HSD (2 line) to supplement their
own knowledge base (rather than taking an active role in the KEL process) and
st
the SMC monitoring team. The 1 line helpdesk function do not create KELs
st
(HSD 1 line used their own knowledge base).
61.5 A KEL is a living document that reflects support knowledge at a given point in
time. KELs are not designed to provide a history of a particular symptom or
support process and a particular KEL cannot be considered the definitive source
containing all possible information regarding the problem it addresses. It is up to
AC_152871086_1 18
E2/11/18
the technician to check other potentially relevant KELs and information sources
(e.g. support guides) and use their analytical skills to determine the right course
of action to take in a given situation.
61.6 There have been historic requests to remove large numbers of KELs based on
date updated to reduce the number of search results that are returned to help
desks when naive search terms were used. The dates given in the KELs, while
an indicator of potentially irrelevant support information, are not precise. This led
to the concept of “deactivated” KELs (deactivated KELs are removed from the
default searches support people use although the user interface allows the user
to explicitly request a search to include deactivated KELs). At the time of writing,
there are 113 deactivated Legacy Horizon KELs and 1024 deactivated Horizon
Online KELs.
61.7 For most of its lifetime, there has been no fixed routine for the review of KELs. If
a technician recognises an inaccurate KEL as they analyse information they are
expected to update in order to improve the knowledge base. Approximately 2
years ago the KEL review forum was introduced. This forum meets weekly to
review KELs and update or deactivate as appropriate.
61.8 Before creating a KEL there is an expectation that the creator will search the
existing information to ensure that it is not duplicated. Because people express
information using different words, it is not possible for the system to perform such
a check. Human fallibility and unique expression mean that duplicated
information is present in the KEL system.
61.9 Archiving: There is no requirement to keep historic support information. Once an

item is no longer relevant to current systems it can be removed without any
implications to the support of the system. KELs are deleted when they have no
value to the support of the current systems. This can happen at any time and is
carried out by the SSC who are the final arbiters of what information is currently
relevant. That does not mean that all current KELs are still relevant; it may be
that irrelevant KELs have not been deleted yet. At the time of writing there are
1,491 deleted KELs.
61.10 KEL entries have a single field to record an incident (Peak) reference. This is not
a record of all incidents the KEL is relevant to. Generally, it is used to record the
Peak that was being investigated when the KEL was created or updated. This is
a manual entry and is not checked by the system because a KEL may not have
an associated incident. There is no requirement to update the KEL when
information in it is re-used to provide guidance on a different incident.
AC_152871086_1 19
E2/11/19
62. Peak (not an acronym) is browser-based software incident and problem

management system used by Post Office Account for all development, test and
st
support teams except the 1 line help desk. It enables details of the incident and
diagnostic progress to be captured in a searchable format and allows the tracking
of problems from detection through to resolution. Peak was developed in-house
by the SSC from the PinICL system it replaced in 2003. The system has been
customised and enhanced over its 15 years of operation and still continues to be
developed to Post Office Account requirements today.
62.1 When Peak was implemented, data from PINICL was migrated to Peak.
st
62.2 One source of Peak incidents are the 1 line support teams (including the HSD
the NBSC helplines). Peak is also used to process incidents generated by other
support units, monitoring and testing teams.
62.3 The structure of a Peak enforces a workflow (it gives a process structure leading
to a defined outcome). As a result, the Peak system has also been used to
record and progress other items loosely associated with incident management.
For example, Release management process, reference data delivery process,
Post Office Data Gateway route definition process. So the Peak system contains
incidents which do not directly impact the Horizon service provided to the Post
Office counters (AKA “Live service”). These additional types of “incident” are
differentiated by the Peak type: L = Live service, R = Release management etc
62.4 For most of its life cycle a Peak is assigned to a particular support team and a
person within that team who is responsible for the next action on the incident. As
the incident is progressed by various members of the support community, they
add textual comments and supporting evidence to the Peak to document the
progress of the incident. These updates are date / time stamped and form a
record of the diagnostic and resolution process. Progress updates cannot be
deleted / amended by users once committed to Peak. A Peak may also be
transferred between teams and people as it progresses to final resolution.
62.5 A final response code (numeric) is applied to an incident when it has reached its
conclusion along with text that supports the response code. Response codes are
the subjective opinion of the person closing the incident and are not subject to
review but they remain the best way to classify the final outcome of an incident.
Response codes and their expected usage are documented in
SVM/SDM/PRO/0875 (End to End Support Strategy) in Section 9 Incident closure {F/823}
categories. Although some aspects of this document need revision due to

changes in the support structure for Horizon, Section 9 is still fundamentally
correct and is relevant to Horizon (Legacy and Online).
AC_152871086_1 20
E2/11/20
62.6 The Peak system has no archiving policy. Effective incident management
requires that you can track the current issues and those from the recent past.
Data retention on Peak has been impacted by a lack of disk space, the primary
cause being large evidence files attached to Peaks. I am told by my colleague
who maintains the Peak system that:-
62.6.1 encrypted evidence files are removed one month after the incident is
closed; and
62.6.2 due to disk space issues in the past it has been necessary to remove
evidence attached to older Peaks.
62.7 KEL references can be added to a Peak entry. There is no system check to
ensure a KEL has been added since KELs are not relevant to all incidents being
processed.
62.8 Incident priorities and appropriate usage are described in SVM/SDM/PRO/0875 {F/823}
(End to End Support Strategy) in Section 7 and SVM/SDM/PRO/0018 (Incident {F/1691}
Management Process). Incidents with a financial impact on branches are treated
as high priority.
62.9 Target times to resolve software incidents are described in SVM/SDM/PRO/0875 {F/823}
(End to End Support Strategy) in Section 8.
KEL ANALYSIS
MR COYNE'S KELS
63. I have been shown paragraph 5.114 of Mr Coyne’s report, in which he says that
he has analysed 5,114 KELs to determine the scope and impact of potential
Peaks. Mr Coyne explains that out of these 5,114 KELs, he believes he has found
163 that could be of "significant interest" and that he has referred to 76 of these in
his report.
64. Post Office's solicitors have reviewed Mr Coyne's report and have provided me
with a list of KELs that they have identified as being referred to in the report
(Coyne KELs). I do not know why there appears to be a difference between this
number and the number of 76 quoted by Mr Coyne.
65. It is not clear what Mr Coyne means by “significant interest”. It may be that he
means that a KEL presents evidence of a bug, error or defect in Horizon that has
AC_152871086_1 21
E2/11/21
E2/11/22
Filed on behalf of the: Defendant
Witness: Steven Paul Parker
Statement No.: First
Exhibits: SPP1
Date Made: 16 November 2018
Appendix 1
KEL Mr Coyne's Report Fujitsu's Comments

ID Short-name Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
1. wrightm Payments 5.6 - 5.11 This issue is reported as causing This issue, referred to as the 'Payment Mismatch' by Mr Coyne, is dealt The issue caused a receipts and
33145J Mismatch discrepancies showing at the with substantively in the second witness statement of Torstein Olav payments mismatch in the accounts
Horizon counter which disappeared Godeseth. of affected branches, which was
{F/1450}
when branches followed certain detected by the monitoring of system
process steps. While a workaround events by Fujitsu. Post Office were
was established by KEL wright informed and corrected the relevant
33145J, it is not clear how many branch accounts.
corrections were required to fix all
instances of this or event that all
instances were fixed.
E2/11/23
AC_152871086_1 1

2. acha1233J Unexplained 5.21 – 5.23 Discrepancies between the branch The cash planning system calculates the branch's cash levels figures from No impact.
Discrepancies cash declarations and the amount one day to the next. This was a timing issue due to figures from a previous
{F/951}
(cash received by SAP (the cash day being used in association with other figures from the current day.
declarations) management system). The KEL
states that this is not a user error or This issue affected the figures being used by the back end cash planning
anything that can be corrected at system and did not affect any branch accounts. The KEL exists to deal
branch level. This is therefore with further complaints.
consistent with the problem being
due to the existence of a software In the event that there is a discrepancy between the amount received by
bug. the Post Office cash centre in a cash pouch returned by a branch and the
amount the branch entered on Horizon as having been put in, this would
be identified by Post Office and a Transaction Correction for the
appropriate amount would be issued.
E2/11/24
AC_152871086_1 2

3. acha1717T Unexplained 5.22 There is an acknowledgment that By way of context, this is a generic KEL relating to the handling of calls None.
{F/1303} Discrepancies cash declaration discrepancies concerning discrepancies. There are a number of possible causes for
(cash could be due to an "Unknown unexplained discrepancies between the cash declaration and figures on
declarations) System Problem". SAP, including:
 Subpostmaster has made an incorrect declaration
 Transactions as recorded on the system do not match what actually
happened at the branch
 Outstanding recovery
 Withdrawn products
 Known system problem (these should be monitored for or be easy to
spot from events etc.)
 Unknown system problem
This term "unknown system problem" is a term that the KEL creator would
have used to indicate that the issue may have been caused by a new
(previously undetected) defect, that would follow the normal diagnosis
process and will then become a known system problem once fixed.
The relevant Peak (PC0202239) relates to an investigation surrounding {F/688}

the possible explanation of a £240 discrepancy. The investigation
concluded that the discrepancy was likely to have been caused by human
error and it appears that the Postmaster accepted this conclusion.
E2/11/25
AC_152871086_1 3

4. acha621P Unexplained 5.23 Evidence of cash declaration This issue, also referred to as the 'Dalmellington' issue is dealt with This issue caused a discrepancy in
Discrepancies discrepancies arising from clerks substantively in the second witness statement of Torstein Olav Godeseth. the Subpostmaster's outreach branch
{F/1426}
(duplicate Rem duplicating Rem In transactions as Fujitsu identified the branches affected by this issue and gave the which was easy to identify from the
In) a result of wrong messages being information to Post Office to resolve the issue with affected branches. transaction logs available through
presented on the Horizon counter Horizon and the fact that separate
screen. receipts were printed for each
transaction. Post Office issued
Transaction Corrections or advised
Subpostmasters how to take
corrective action to remove the
discrepancies.
E2/11/26
AC_152871086_1 4

5. LKiang3014S Unexplained 5.24 Issue following multiple cash This KEL relates to the next KEL (MScardifield2219S) as follows: This issue may have had a
Discrepancies declarations and trial balance report {F/428} temporary financial impact, but it
{F/405}
(multiple cash being inaccurate. On this occasion  LKiang3014S describes the symptoms of the issue; and would have been resolved when the
declarations) the support department was unable  MScardifield describes the cause of the issue. branch next declared the cash held in
to identify the root cause of the the branch accurately.
discrepancy although it was This was a software/ environmental issue involving the Horizon system
reported that a correction could be struggling to find cash declarations, which would tend to happen if a
made at the Post Office counter Subpostmaster was undertaking multiple declarations in a stock unit.
level by redoing the cash
declaration using the same amount This particular instance involved Riposte failing to find one of the Cash
already declared. Within KEL Declarations and thus generating a temporary discrepancy. If left
MScardifield 2219S Fujitsu identify unresolved this would result in a loss to the Postmaster for this period.
the underlying software bug as However, when the Subpostmaster subsequently declared the cash held
being caused by 'cached data' not in branch accurately, this would generate an equal and opposite
being updated via Riposte. discrepancy that would cancel out the earlier discrepancy.
This issue was not resolved, but further diagnostics were added to enable
further investigation should the problem happen again.
E2/11/27
AC_152871086_1 5

6. MScardifield2 Cached Data 5.24 Delay in 'cached data' being This KEL relates to the previous KEL (LKiang3014S) as follows: As above.
219S Delay updated via Riposte. This resulted {F/405}
{F/428} in incorrect data being presented in  LKiang3014S describes the symptoms of the issue; and
any discrepancy, variance and  MScardifield describes the cause of the issue.
balance reports.
This is not a 'Live' issue, but something found on a Test rig. Experiments
Mr Coyne comments that this is did find that this could occur around once in 100,000 balance calls when
likely to be confusing for the the rig was heavily loaded.
Postmaster and could lead them to
making unnecessary modifications if 13 Live Peaks reference this KEL, each Peak was treated separately and
they are unaware that the problem the issue was discussed with the Subpostmasters. The work around of
should clear itself overnight. subsequently correctly declaring cash declarations was identified and
communicated to Subpostmasters.
E2/11/28
AC_152871086_1 6

7. DSeddon542 Cash Pouch 5.25 A failure in pouch delivery resulted The cause of this was that the system was in “Price shopping mode” This had a financial impact, but when
6P Delivery in a cash gain when the branch which can be used to buy, for example, £15 of 1st class stamps. Price the branch investigated the
{F/356} declaration was carried out. shopping is not supported for products like cash, so when the branch discrepancy the failed remittance
attempted to remit the cash deliver in while in Price shopping mode the would have been identifiable in the
cash remittance failed. logs that are available to users of
Horizon.
In this particular KEL, the remittance was repeated later and was
successful. A fix to ignore “Price shopping mode” for Rems was applied to In addition:-
Live in April 2007.  a critical event was written
that will have been picked
up by Fujitsu's support
teams and Fujitsu will have
advised Post Office to
contact the branch; and
 Post Office's own
reconciliation procedures
would have identified that
the remittance in was not
completed successfully
and contacted the branch
and resolved the matter
with a Transaction
Correction.
E2/11/29
AC_152871086_1 7

8. acha194L Automated 5.26 Problem affecting around 15% of Automated cash declarations from self service kiosks failed at random. No impact on branch accounts.
Cash kiosk branches that prevented these Analysis shows that this was due to rounding errors on the arithmetic
{F/1314}
Declarations branches from being able to carried out due to use of incorrect data types. The business impact of the error is
automatically make cash on cash management and the
declarations. The lack of cash declarations have no impact on Branch accounts, but will delivery of cash meaning the cash
result in cash planning potentially having incorrect or missing figures and needed at a branch may be
thus failing to calculate accurate amounts of cash to send to the branch. incorrectly estimated, leading to late
or insufficient cash deliveries.
This was a coding error and was fixed in December 2015.
The vast majority of self service
kiosks are located in Crown branches
and the others are located in large
mains branches.
9. DSeddon314 Reference Data 5.30 Examples have been found in KEL/ The particular KEL related to the incorrect maximum values being set up No impact - it was not possible to
Q Peak records that indicate in Reference Data and the counter not handling this error correctly. This rem in the coin and therefore it could
Reference Data has an impact upon was caused by a human error by Post Office in the Reference Data. not be sold by branches.
{F/335}
daily counter activities. Specifically, it was not possible to rem in a particular commemorative coin
because it did not have the requisite Reference Data.
The Reference Data was fixed on 14/3/06 and a code fix to handle the
scenario better was applied in June 2006. This did affect a number of
branches during the day that the problem was live, but a message was
sent out to all branches advising them of the issue and how to correct it.
E2/11/30
AC_152871086_1 8

10. johnbascoG5 5.31 An Automated Payment transaction For some unknown reason, the counter was associating an Automated No impact.
222H was reported as having failed due to Payment transaction with an invalid client code. There was no fault in the
“Unknown Agent Code 3046”. Client Reference Data. The fault was impossible to reproduce, and so not
{F/1661}
account code 3046 was found to not understood.
exist in Reference Data and the
fault was not reproducible when the Failure to complete a transaction would not produce an error in accounts -
problem was analysed and tested. It a double entry transaction would either all succeed or all fail.
was acknowledged that, due to the
business impact, a fix would be
provided to check and validate the
client account code exists in
Reference Data before the
transaction is committed.
11. acha10L Reference Data 5.32 KEL documents how branches were Incorrect Reference Data was issued, which had the effect that payments No impact.
Errors unable to accept cards for rent and of Council Tax to Vale of Glamorgan Council were rejected. The
{F/497} council tax payments due to Reference Data was fixed overnight and it all worked the next day. Failure
incorrect Reference Data. to complete a transaction would not produce an error in accounts - a
double entry transaction would either all succeed or all fail.
E2/11/31
AC_152871086_1 9

12. MWright1458 Withdrawn 5.33 Issue involving withdrawn products When products are withdrawn, there is a withdrawal process which Temporary financial impact caused
Q Products impacting a branch accounting requires users to have Remmed Out the relevant products (which involved by the Subpostmaster's failure to
{F/419} position because the Postmaster the stock being returned to the stock centres) or the stock would need to follow the correct procedure in
will have products that cannot be be adjusted. The grace period for the Subpostmaster either Remming Out relation to the Remming Out of
accounted for as there is no or adjusting the stock was 6 weeks. After the Reference Data was obsolete stock.
remaining Reference Data for them withdrawn, any unused stock would result in a discrepancy (a change was
to later declare that stock item in the introduced by Impact in 205 which meant that this could no longer occur). Fujitsu was able to resolve the issue
accounts. The assumption was that the branch had actually sold the missing stock by adding a compensating
and failed to record it on Horizon so the stock was removed from the transaction.
system and a cash discrepancy would be generated for its value.
In this particular case, the branch failed to Rem Out its unused stock of
obsolete products (stamps) within the grace period despite being asked to
do so on multiple occasions. This left the Subpostmaster with a problem
she could not solve herself, but needed Post Office support to do so. To
resolve the issue, corrective transactions were added to the Message
store in consultation with Post Office at the data centre to reflect the value
of the obsolete stock which enabled the branch to roll over correctly.
E2/11/32
AC_152871086_1 10

13. wbra5353J Reference Data 5.34 The customer was charged twice for This was a fault involving a self-service kiosk, which resulted in a No impact.
Errors the same transaction which was customer being debited three times, after which the session was
{F/1205}
reported to be a side effect of errors cancelled.
within Reference Data.
This was an issue with how the NCR kiosk interfaced to Horizon, which
attempted multiple Debit Card transactions with the same ID. NCR
diagnosed this as being due to some invalid Reference Data being sent to
the kiosk.
The fault appears to result from two causes:
i. faulty Reference Data (the result of human error), which was

easily corrected
ii. a fault in the kiosk software, which came from an external
supplier
E2/11/33
AC_152871086_1 11

14. GMaxwell365 Duplicate 5.38 Failures or interruptions in service This was an issue where approximately 835 transactions were sent to No impact.
1K Payment with the harvesting process can Streamline (Post Office's previous payment services provider) on 2
Transactions cause duplicate payment successive days due to an operational issue in Horizon back end
{F/342}
transactions to be processed. processing. These were identified by Streamline as duplicates and not
processed so customers were not charged twice. This also resulted in
errors picked up by Fujitsu's Reconciliation service so it was trapped by
both ends of the interface. The DRS (Data Reconciliation service) takes
inputs from multiple sources (counters, TPS database and FI reports) and
compares them and produces reports detailing transactions where they
disagree.
15. surs357P 5.38 Failures or interruptions in service Same type of incident as previous KEL at row 14 - same analysis. No impact.
with the harvesting process can
{F/500}
cause duplicate payment This KEL additionally states: as no customer accounts have been debited
transactions to be processed. twice no further reconciliation is needed.
By way of further explanation, the reason for there being 2 KELs is that
this issue occurred in both December 2004 and again in March 2009.
However, in both cases both Streamline and Fujitsu picked up the
problem.
E2/11/34
AC_152871086_1 12

16. jharr832S Recoverable 5.41 Acknowledgment that the recovery This KEL relates to a case where the system was behaving as designed, There would be no impact if the user
Transactions process (in relation to recoverable but the user in branch failed to answer the questions correctly and did not followed the recovery process
{F/408} transactions) is a complex area. follow the recovery process properly. presented by Horizon. If the user
failed to do this there could be an
impact due to the user's error.
17. cardc464Q Failed 5.42 Particular difficulty in processing a This KEL does not indicate any software error. The specific scenario As Mr Coyne acknowledges, in this
Recoveries recoverable transactions. In this described by this KEL would always appear as a failed transaction at a case there was no impact on the
{F/761} instance the settlement of the Branch, so it is highly unlikely that any cash was exchanged and the branch account.
transaction had not been written branch or customer was out of pocket.
into the Branch database so the
recovery failure would have had no All failed recoveries are monitored by Fujitsu and result in the exact
impact on branch or customer circumstances being checked out and a transaction correction issued in
accounts. the event of a discrepancy.
All failed recoveries are automatically identified in a daily report that the
security operations team receives (formerly MSU). They review this and
raise Peaks for any that require further investigation by the SSC team.
E2/11/35
AC_152871086_1 13

18. seng2037L {F/1040} 5.43 These KELs describe how various These are both generic KELs describing how support should process There would be no impact if the user
and transaction states may also indicate specific scenarios found on reconciliation reports. These reports are followed the recovery process
acha959T {F/1700} a failed recovery. generated daily for banking/debit card/e top up transactions where the presented by Horizon. If the user
counters, agent and financial information (FI) feeds differ in some way (for failed to do this there could be an
example, the counter timed out a transaction but the FI authorised it and impact due to the user's error.
ring fenced the funds). They are sent daily to the security operations
team. Each transaction is checked and sometimes further investigation is
required from the SSC team via a Peak.
These KELs describe legitimate states that can occur during reconciliation
following a failed transaction which is usually recovered from correctly.
There are 23 legitimate (normal process) states and 39 error states that a
transaction can enter. Each state has a specific meaning depending upon
the responses from the Counter, TPS and FI. A legitimate state would be
that the TPS and Counter both know about a transaction but the FI may
never send information about it (state 6); this is a legitimate state but it
would warrant investigation.
E2/11/36
AC_152871086_1 14

19. dsed4733R 5.44 Example where transaction This KEL refers to a failed recovery report (report of failed recoveries) and This was a zero value transaction so
(no 20) recovery has failed due to a wrongly in particular some unexpected items in it. there was no impact on branch
named recovery script. Jason accounts.
{F/1106}
Coyne refers to a Horizon system The existence of the failed recovery report is evidence of routine
error arising because of incorrect robustness countermeasures in place to deal with failed recoveries. In If the transactions had a financial
Reference Data. this case, the unexpected behaviour seems to have arisen from faulty value then the issue would have had
Reference Data, which was corrected within a couple of days. an effect on the branch. However, it
would have been raised on Fujitsu's
This KEL refers to problems specifically with an ADCScript-HPBB_REC1 failed recovery reports, investigated
recovery script. HPBB stands for Home Phone Broadband and it appears and reported to the POL via a BIMs
that the transaction would have been a customer application for a Post report.
Office Home Phone Broadband service where information is collected
from the customer but no payment is made.
E2/11/37
AC_152871086_1 15

20. PSteed2847N Incorrect 5.51 Software issues resulting in Horizon A user had Remmed In some cash to the wrong stock unit. When they Temporary financial impact which
Mathematical applying the wrong mathematical realised this they carried out a reversal of the Rem In but, due to a was obvious to the Postmaster (who
{F/152}
Sign sign when reversing transactions software error the value of the Rem In was doubled instead. The reversal reported the issue) and corrected by
(i.e. a plus (+) instead of a minus (- receipt would have informed the user that something had gone wrong and Post Office issuing an error notice.
)) in this case they stated that their Rem In amount had doubled when
reporting the issue. Presumably they looked at a remittance report or
balance report to see this.
A Rem In reversal is not a particularly common transaction and it was

prohibited as part of Impact changes in 2004. The user has attempted it
for a specific reason so if after performing the reversal it hasn't had the
desired effect you would expect the user to clearly notice this and raise a
call with the helpdesk to query it. Upon confirming the error the NBSC
could then issue an 'error notice' to correct any anomaly.
It took just 13 days from reporting to active a fast track fix.

E2/11/38
AC_152871086_1 16

21. cardc5756N Pouch Rem 5.52 This is an example where the This concerned the reversal of a Rem Out of a pouch, a rarely used Temporary financial impact which
Out Reversal system failed to reverse all items in process, by which when a banch has prepared a pouch returning cash to would be picked up by Post Office's
{F/441}
a multi-line pouch, meaning only the the cash centre and then realised that they either made a mistake (as cash centre reconciliation process
first item was reversed. happened in this case) or need the money after all. It was not possible to and corrected by a transaction
rule out the possibility that this was caused by a software issue, but it was correction.
not possible to replicate it so this could not be investigated further.
The Rem Out reversal appears to have gone wrong in this case and only
part of the pouch contents was reversed, thus leaving some of the value
still in suspense.
22. GCSimpson1 Foreign 5.54 All currencies in branch doubled up The discrepancy (between the Horizon record and physical cash) was Temporary financial impact which
049L Currency following successful balancing eight picked up in branch and a call raised. It appears that this particular was resolved when the branch
Discrepancies days previously. incident was resolved by the branch upon monthly balancing. carried out the next monthly balance
{F/207}
(when it declared the actual value of
Due to the delay in providing the information to the development team, the currencies held in branch).
lack of any record of this incident having happened previously and there
being no further reports of similar problems, we are unable to confirm the
root cause.
E2/11/39
AC_152871086_1 17

23. MHarvey3527 Insufficient 5.55 Evidences that in certain The KEL says: No impact.
I Diagnostic Data investigations there was insufficient
{F/246} diagnostic data to be able to fully 'As this is, at the moment a one off event and clearly no further progress
diagnose an issue. can be made at this stage, I have therefore closed PC113202 as
"insufficient evidence". However, any further occurrences should be sent
to APS Counter Dev for investigation.'
This is an issue in the transferring of copies of transactional data from

Horizon to Post Office's back-end systems. The specific data mentioned
here is not financial. The underlying issue was that there was
unnecessary validation of copies of Reference data being passed back to
Post Office and this validation was removed as part of Impact (a joint
working body to introduce improvements to the system and processes) in
2005.
24. CObeng1123 Stock Gains 5.56 Unexplained discrepancies (gains) This was a complicated memory loss issue in branch. Extensive searches Not known due to the age of the
Q for different stock unit types (Cash were made for memory loss issues in test at the time and only one was matter.
{F/185} and Stamps) was reported. The found and explained (which did not relate to this issue). It therefore
incident remained unexplained and appears that this was a one off incident. Due to the passing of time, we're
no record of advice having been unable to identify the cause, but if the counter software was running short
provided to Postmasters. of memory we would expect the counter to display a warning to the user
which would have been seen.
25. DRowe1625K 5.57 This KEL is not available.
E2/11/40
AC_152871086_1 18

26. dsed525Q PIN pad 5.68 There was a failure in the This was a faulty PIN pad. It prevented the Postmaster from carrying out No impact.
Failures Postmasters being able to transact some transactions but this would not affect branch accounts.
{F/512} various types of transactions
including payment transactions
using a PIN pad. An error message
and code were generated, and a
new PIN pad was the
recommended solution.
27. surs3941P PIN pad 5.68 There was a failure in the This appears to have been an issue with a corrupt customer card. It was No impact.
Failures Postmasters being able to transact set up with no CVM (CVM is something on a card that indicates whether a
{F/616}
various types of transactions PIN or a Signature is to be used to authorise a transaction). SSC suggest
including payment transactions that this was an attempt to do a Balance Enquiry on a Credit Card (which
using a PIN pad. An error message isn’t allowed). Reference Data should have prevented that being
and code were generated, and a attempted and it's not clear why it didn’t. It could have been a corrupt card
new PIN pad was the that the customer had.
E2/11/41
AC_152871086_1 19

28. BrailsfordS22 PIN pad 5.68 There was a failure in the This was a faulty PIN pad. It prevented the Postmaster from carrying out No impact.
39K Failures Postmasters being able to transact some transactions but this would not affect branch accounts.
various types of transactions
{F/674}
including payment transactions
using a PIN pad. An error message
and code were generated, and a
new PIN pad was the
29. cardc219R PIN pad 5.69, 5.135 This KEL indicated that PIN pad The KEL relates to a specific (one-off) hardware issue with the old No impact.
{F/1152} Failures related issues would usually result Hypercom PIN pad where the transaction was authorised but not
in the recommendation of a new confirmed due to the hardware fault. The process is request, authorisation
PIN pad, regardless of the error. In and confirmation, however, the final stage of confirmation did not take
this case a transaction had been place. As per the KEL, this was identified as part of the reconciliation
declined by the PIN pad but did not process (i.e. the same DRS reconciliation process referred to above) and
get reversed passed back to Post Office and a Transaction Correction issued
automatically. There was no impact on branch accounts, however, the
customer was charged twice and they should have been reimbursed by
Post Office's back end processes.
30. dsed4733R 5.92 Identifies multiple failed recoveries See analysis at line 19 above. See analysis at line 19 above.
occurring because of a wrongly
{F/1106}
named recovery script.
E2/11/42
AC_152871086_1 20

31. obengc5933K 5.93 There was a loss of This is further evidence of the failed recovery report doing its job and This may have caused a temporary
communications following network alerting Fujitsu to failed recoveries to enable them to investigate them and financial discrepancy
{F/757}
banking transactions and the make any necessary corrections to accounts by sending a BIMs to Post
printing of the customer receipts Office. If this had not happened, a transaction correction would have
resulting in a message to the data been issued as a result of Post Office's own reconciliation processes.
centre timing out. Consequently, the
Postmaster was asked to follow The incident was caused by a complex 'grey' communications failure (i.e.
recovery but the transaction was the network kept switching between good and bad; not solid good or solid
only able to recover partially. bad), which the development team could not reproduce.
The KEL gives no reason to suppose that, even if this condition had
persisted, the backstop of reconciliation and Transaction Corrections
would not have corrected any resulting errors in accounts.
As per KEL, the failed recovery will be centrally reported and investigated
via the DRS reconciliation process.
32. wrightm33145 Payments 5.116, 5.118 Non-zero trading position. This is the same issue as line 1 above. See analysis at item 1 above. See analysis at item 1 above.
J Mismatch
{F/1450}
E2/11/43
AC_152871086_1 21

33. ArnoldA2153 Withdrawn 5.117 This generates the same Receipts This issue was found during internal Horizon Online testing and fixed There may have been an issue if the
P Stock and Payments Mismatch error before Horizon Online went live. branch had continued to rollover
message (to KELs in lines 1 and 31) before the products were reinstated,
{F/1448}
but in fact relates to a mismatch In 2016, some products were withdrawn during a trading period when they however, in the 2 cases recorded,
during the balancing of a stock unit were still being traded. This is contrary to Reference Data procedures the branches were advised to delay
that contains withdrawn product. and caused an issue in branches that had traded those products. the rollover until the products were
re-instated so no impact on branch
A fix was put in place involving the products being re-instated and the accounts.
branches affected rolled over successfully.
34. ballantj1759Q Payment 5.118 Details 3 conditions that may cause This is a generic KEL ensuring that the event monitoring team raise a call N/A (this is a generic KEL).
Mismatch a receipts/ payments mismatch that every time a Receipts and Payment mismatch event is seen. It references
{F/798}
may impact on branch accounts. other KELs that are known issues for specific cases. This is an example
of how our monitoring is made to work effectively. All of these calls get
investigated and may need manual correction to avoid errors in branch
accounts. From a Horizon perspective, none of the calls raised should
therefore be left without investigation / resolution.
E2/11/44
AC_152871086_1 22

35. acha1357Q Incorrect 5.120 It is possible for discrepancies to The issue related to declarations from the same trading period a year ago There may have been some financial
Declarations have been accepted by the becoming visible again and thus causing confusion. It only affected impact, but there were reports
{F/1655} Postmaster based upon incorrect branches that had done a Stock Declaration the previous year but available in branch that could have
declarations. The problem could normally didn’t do them. The fix was to change the archiving strategy so been used to identify incorrect
arise due to old stock declarations that all declarations that had not been updated for 6 months were declared amounts.
not being automatically removed removed. A check was made at the time for any branches that had old
from the system. These discrepancies that might become current again in the next 2 months (to Further, a corresponding gain / loss
could only be removed by making allow for the archiving fix to be made) and these were removed manually would occur in a subsequent trading
zero-value declarations or deleting by MSC. A period so that would resolve the
the stock unit then waiting overnight issue.
before balancing. The initial call was raised on 11th Feb 2011. A central workaround to
avoid further issues was implemented under MSC and the official fix was
released in June 2011.
E2/11/45
AC_152871086_1 23

36. acha3145Q 5.121 Provision of a full support solution This KEL is cited in the previous KEL dealt with at line 35 and relates to a There may have been some financial
for incorrect stock declarations and stock balancing problem caused by the user doing some uncommon impact is is branch process at this
{F/1383}
discrepancies. sequence – i.e. not caused by withdrawn products. stage, it is effectively as if the SPM
had wrongly declared the cash or
stock and the system will warn them
that this does not match the
calculated value. In this case it was
an old declaration that got included.
Redeclaring the cash/stock will fix the
issue. If they do not they will roll with
a loss this BP/TP but have a gain
next TP/BP.
37. allend1645p Horizon 5.129 Provides an example of Horizon’s Horizon allowed the clerk to select 'Debit card' as a method of payment This is a case of the Postmaster
Interface weak interface controls and lack of and later switch to 'fast cash' at the end of the customer session. This was being responsible for errors made by
{F/813}
Controls data entry validation. In a single a subsequent user error which involved the user at the branch failing to staff. This would have shown as a
sales transaction the user was able take payment of £500 for the 540 euros. discrepancy caused by user error in
to select and enter different not taking the payment that was due
methods of payment (Debit Card from the customer and the
and Fast Cash). Horizon allowed Postmaster would be liable for this.
the transaction to be settled via Fast
Cash when the Debit Card payment
method had already been selected.
E2/11/46
AC_152871086_1 24

38. acha621P 5.130 The correct screen to successfully This issue, also referred to as the 'Dalmellington' issue is dealt with Temporary financial impact which
process a cash pouch did not substantively in the second witness statement of Torstein Olav Godeseth. was rectified by a Transaction
{F/1426}
appear resulting in the clerk in an Correction being issued.
outreach branch inadvertently
doubling up the amount of cash
recorded. The issue appears to
have been caused because of an
earlier system logout or inactivity
which in turn resulted in incomplete
checks being conducted by Horizon
post logon.
E2/11/47
AC_152871086_1 25

39. EJohnson393 Horizon 5.132 This enabled Postmasters to carry Whilst Remming In currency it is possible to create a transaction with a No impact.
7R Interface out “Rem In” transactions without a positive quantity and a zero value .
Controls value being entered.
{F/248}
This issue was caused by the Reference Data test team (not following the
correct process). The functionality was changed the following year so that
amounts did not need to be entered.
Auto Rems (introduced around 2004) meant that the content of cash and
currency pouches was sent to the branch electronically so when a pouch
was delivered, the system automatically told the Subpostmaster what the
content was and used that value for the Rem in rather than asking the
Subpostmaster to key it in.
E2/11/48
AC_152871086_1 26

40. PSteed145J Phantom Sales 5.133 Involved 'phantom' sales appearing These "phantom" sales were caused by hardware problems and fixed by No impact If the transaction related
on the Horizon counter screen but replacing hardware. to stock, when the branch declared
{F/174}
which had not been selected by the their stock and cash the discrepancy
user. would cancel out (e.g. a sale of
stamps would reduce the stock of
stamps and increase the cash figure
by a corresponding amount; when
balancing the correct number of
stamps should be declared and this
will cancel out the effect of the
phantom sale of stamps).
41. pcarroll1235R Screen Freezes 5.133 Instructions on how to deal with These instructions (which relate to issues that do not concern bugs or No impact on branch accounts. .
environmental issues and hardware errors in Horizon) were distributed to those who called for help via this
{F/353} are contained in this KEL. Jason KEL.
Coyne says "it is not known how
widely these were distributed to
SPMs", implying that they should
have been.
E2/11/49
AC_152871086_1 27

42. jharr1323L 5.136 Example of a successfully recorded This relates to a fishing rod licence request not sent to the Environment No impact.
transaction initiated in a Post Office Agency and was only detected at a later date. In this particular instance,
{F/1542}
branch (where a customer receipt the transaction had been reversed by a user; this is not a software issue.
was generated) which failed to Once a transaction is reversed, all relevant data is discarded and not sent
appear in the Post Office Data to the AP Client as its effectively as though the transaction never took
Gateway. place (Fujitsu only keep Post Office Data Gateway records readily
available for 30 days) but it is committed to the audit store and therefore
any additional investigation would have needed to be undertaken by
audit).
It is not known why the reversal took place. This could be due to
fraudulent activity or could be that the customer sought a refund and the
money was refunded.
E2/11/50
AC_152871086_1 28

43. MArris3433I 5.137 The ability of Horizon to erroneously This was a software bug which allowed a transaction to be recorded twice No impact (fixed in testing).
(no 46) record the same transaction twice after a session transfer and was a fairly rare circumstance. This issue was
after a session transfer to a different picked up during internal testing and despite numerous attempts
{F/168}
counter. This happened with both development were unable to recreate the scenario. There are no Peaks
NS&I (National Savings & which refer to this KEL.
Investments) and Network Banking
(NWB) transactions. The KEL was
passed to a development team to
provide a bug fix as part of the S60
rollout but it is unknown if this was
ever resolved.
E2/11/51
AC_152871086_1 29

44. CharltonJ275 Previous Key 5.139 Counter level corrections made via A software error in a PDL file meant that when a user used the "Previous" No impact.
2T Software Error the “Previous” key led to both the key for a transaction that used an ADC Script, old and amended values .
old value and amended value being were stored and used. This resulted in an incorrect transaction for the sum
{F/1060}
stored and used in error in the amount of both the old and amended values being added to the sales
transaction. A fix was released in basket.
the Live environment eight days
after the issue was first raised. As the user has used the "Previous" key to go back and amend a value, it
should have been obvious to the user if then a completely different value
item is added to the sales basket. If, for whatever reason, this was not
noticed then the the customer will end up being overcharged as the
system will ask the user to take a larger payment. Assuming the user does
what the system says there will be no impact to the branch accounts.
This issue affected 3 products and only occurred when "Previous" key
was used to correct the amount entered. It was fixed 8 days after it was
spotted. A search was made for all branches that had used those
products twice within a session and the results were sent to Post Office.
E2/11/52
AC_152871086_1 30

45. SSur343P 5.141 Example of a declined network An error in network banking caused the customer's account to be debited No impact.
banking although the transaction failed at the branch and the Postmaster was told
{F/211} transaction that resulted in money at the counter that the transaction failed. Banking transactions with a
being taken from the customer’s response code of 26 (FI Unavailable, Try again later) would be recorded
account. as zero value transactions at the branch, a DECLINED receipt would have
been produced and so no money should have been handed over to the
customer. Therefore no impact on branch accounts. On rare occasions
the financial institution may have debited or credited the customer bank
account despite us not receiving the authorisation. In these instances, if
the automatic reversals fail to resolve matters then the issue would be
picked up as part of the DRS reconciliation process meaning Fujitsu would
inform Post Office of what has happened at the counter so that they can
liaise with the financial institution to ensure their systems match so
customer is not out of pocket. The root cause was between the NBE and
the FI, which is outside of Horizon.
E2/11/53
AC_152871086_1 31

46. LKiang3526R 5.142 Examples of E-Pay transactions This was caused by two authorisation agents being active at once, when No impact.
crediting the customer account only one should have been, resulting in the phone being credited with £10
{F/260}
twice although only one payment twice and ePay charging Post Office or the phone provider (depending on
has been taken. the arrangement between the two) being charged twice, meaning the
customer would have got two sets of top up for the price of one.
This is a back end system problem which would be picked up by counter

measures, causing a BIMs to be raised.
47. SSur5310P 5.142 Examples of E-Pay transactions Similar to the KEL above at line 46. No impact.
crediting the customer account This is a back end system problem which would have had no impact on
{F/355} twice although only one payment the branch accounts. Despite a phone being topped up twice at the
has been taken. branch only a single top up would have been recorded in branch along
with the required payment for it.
Again, this is a back end system problem and would need to be resolved
by Post Office centrally and would not impact branch accounts.
E2/11/54
AC_152871086_1 32

48. pothapragada 5.165 Jason Coyne says "it is The issue was the ability to declare stock that could not then be This may have had a financial impact
c4359R acknowledged that simple fixes transacted (due to Reference Data rules). but if so it would be due to human
{F/836} ought and were implemented to error (i.e. declaring that it held an
either fix bugs or provide additional The only impact on a branch's account were if the branch were to actually item of stock that it couldn't transact).
data validation checks". declare that it held an item of such stock. This is unlikely as the item had This discrepancy would be removed
been withdrawn and should be returned to the stock centre. It should also if the branch accurately declared that
be noted that most branches do not undertake stock declarations as stock it had no such stock.
is normally manged using stock adjustments (which didn’t have this
issue). Should the stock be declared by mistake by user-error, then a
further declaration of the correct (i.e. zero) holdings would resolve the
issue.
49. Marris4123N 5.165 Jason Coyne says "it is This was a problem observed during test in Disaster Recovery for DVLA No impact.
acknowledged that simple fixes transactions - a very rare circumstance, which should be handled
{F/165} ought and were implemented to correctly, but would have no impact on branch accounts in the unlikely
either fix bugs or provide additional case that a branch encountered the issue
data validation checks".
When recovering a failed counter, the user is asked to input data from a
receipt. To handle poor typing there is a check sum which should ensure
that it has not been altered. This bug relates to the fact that not all the
data entered is controlled by the check sum. Therefore, when a tester
deliberately input incorrect data, the system did not detect it. NB this did
not include financial data.
E2/11/55
AC_152871086_1 33

50. acha2230K Additional 5.186 Jason Coyne highlights a problem This issue, also referred to as the 'Local Suspense Account' issue is dealt This issue had a financial impact
Checks with additional checks which were with substantively in the second witness statement of Torstein Olav which was resolved by Post Office
{F/1150}
implemented to identify system Godeseth. writing off discrepancies.
errors/ inconsistencies when
balancing
during branch and references a
note within the KEL as follows:
"This should never happen -

something has gone horribly wrong.
Or possibly the checks haven’t been
implemented as intended.”
E2/11/56
AC_152871086_1 34

51. dsed2049S 5.187 Jason Coyne suggests that this KEL This KEL relates to withdrawn products that were converted to cash on No impact if the correct process is
highlights the lack of system rollover but the losses are carried forward into the next period instead of followed for returning the withdrawn
{F/1137}
communication and/or support being dealt with there and then. products to Post Office. In the event
communication in respect of certain that the process is not properly
system features which could Withdrawn products should be sent back by the Postmaster to Post Office followed, a Transaction Correction
subsequently result in errors. so the branch is not holding stock that cannot be sold. can be issued to correct any impact
on branch accounts.
If this process is not followed, the branch will be left with a loss at the next
trading period and could be corrected by a Transaction Correction. The
stock will also be converted to cash if the Postmaster has purchased it
personally, for example. The fix was made to make this clear to the
Postmasters.
Jason Coyne implies this was a bug which took 6 months to fix. There
was a minor bug in that the cash value of the withdrawn stock was not
included in the current rollover, but delayed until the next rollover.
However there was no specific loss to the branch (other than the value of
the withdrawn stock which they were responsible for as described).
E2/11/57
AC_152871086_1 35

52. acha3250R 5.189 The reconciliation process used by This was an issue with Post Office's back end reports, caused by timing No impact.
Post Office to assist with identifying issues (APS transactions arriving a day late) which were outside of Post
{F/1304}
any accounting differences is not Office or Fujitsu's control.
able to easily identify genuine
differences and/or differences This caused discrepancies in certain back-end reports, which could
resulting from external APS however be understood by cross checking other reports. This is a “false
transactions from old trading dates. error” being reported by reconciliation relating to transactions occurring
around the end of day, being 7pm.
There is nothing wrong in the actual transactions – just an error in the way
that reconciliation totals are calculated.
53. acha1941L 7.6 During the recovery process (when The KEL says: As there is no reconciliation needed,
some transactions recover but there is no impact on branch
{F/960}
others fail to recover) it is only the "This is not really a problem, it is just confusing when investigating a state accounts
recovered transactions printed on 4 call. The Disconnected Session receipts will show all the transactions in
the receipt. The disconnected the session. The successfully recovered transaction needs no
session receipt should also identify reconciliation."
those transactions not recovered.
These are printed for the This shows that any incomplete information went not to the branch, but to
Postmaster to retain. someone in Post Office or Fujitsu investigating a state 4 call.
E2/11/58
AC_152871086_1 36

54. surs1147Q Failed 7.7 This relates to the solution to a While the advice to the user in the KEL is somewhat counter-intuitive and No impact.
Recovery failed recovery requesting the user indicates that nothing should be done and the user should wait for the
{F/1433} to log onto the relevant counter and counter to time out, leaving the system error, there is nothing to suggest
start the recovery process, but the recovery was not resolved. The root cause of this issue was an error
leave the counter displaying the in Post Office's script relating to the Dangerous Goods products, that
system error message. resulted in recovery failing. We're unable to tell whether or not the script
was corrected.
This would have no direct impact on branch accounts, but clearly would
be very inconvenient as the counter is out of action.
55. wrightm33145 Payment 7.9, 7.41 Cited by Coyne. This issue, also referred to as the 'Payment Mismatch' issue is dealt with See item 1 above.
j Mismatch {F/1450} substantively in the second witness statement of Torstein Olav Godeseth.
56. boismaisons1 Disc Space 9.12 This KEL describes the running This is an issue with hard disks. Disc space sizes have nothing to do with No impact.
328M Sizes commands on counters to assess branch transaction data which in any case at the time (2012) was not
{F/1052} disk space sizes. stored in the branch.
E2/11/59
AC_152871086_1 37

57. SeemungalG Transaction 9.49 Records an instance where This was an issue when posting transactions from old Horizon to BRDB No impact.
519Q Amendments transaction prior to a branch migrating from old horizon to Horizon Online. In some
amendments carried out are cases, such transactions fell foul of validation in TPS and needed to be
{F/1210}
causing exceptions. amended before being sent to Post Office's backend systems. Such
amended transactions were also posted to BRDB. Any amendments were
related to the trading period they related to and not any financial values.
They were only in BRDB to be used if the branch migrated during the
current trading period from Old Horizon to Horizon Online.
Tip Repair is a back end process to make sure all required transactions
get sent to the relevant external systems. There is no effect on the branch
accounts.
58. MHarvey2255 Corrective 9.50 Records the manual addition of This is a back-end balancing issue caused by missing or invalid No impact.
P (no 64) Balancing corrective Reference Data. The insertion of correction records is done into the TPS
Transactions balancing transactions inserted by database to allow the branch data to be forwarded to POLSAP. There is
{F/345}
SSC affecting the TPS system. no effect on the Branch Database therefore no effect on the Branch
Accounts. It is not understood why this is marked as remote access as the
work is done within the Data Centre.
E2/11/60
AC_152871086_1 38
APPENDIX 2
KEL Fujitsu's analysis

ID Nickname Summary of the KEL Analysis Impact on branch accounts
1. acha423K Cash Button The Cash button on the settlement This is not a bug, rather it is a feature of how the system No impact unless the system is misused by
screen can be used for either a receipt in operates. branch staff.
{F/379} or a payment out of cash. Horizon
decides on the context depending on
whether the stack total button says TAKE
or PAY.
2. acha488S Large This involved both transactions being Although the clerk took the money from the customer, the No impact.
Transactions completed in a single session. session wasn't settled because £1m is too large for FastCash.
{F/481} The settlement should instead be entered as two £500,000 cash
payments. This is an issue with hitting system limits with very
large transactions over £1M. This would be very noticeable and
there is an avoidance action to take two smaller payments.
Looking at the Peak, the Transactions were actually recovered

automatically by the system (AP Txns were recoverable in old
Horizon) so no impact on branch accounts once recovery was
carried out.
3. acha508S Postmaster reports problems with This was a bug in the handling of multiple bags of coins when Temporary impact that would have been
remming out, in particular differences being Remmed Out. It was fixed by a code fix issued in April identifiable from the reports available to
{F/403}
between the two receipts which are 2007 and fully rolled out by June 2007. Subpostmasters. The KEL description states
E2/11/61
AC_152871086_1 39
printed after the pouch barcode is that "differences between the two receipts
th
scanned. The original problem was found on 12 Feb 2007 and was which are printed after the pouch barcode is
presumably due to a software update rolled out at that time. scanned… The second receipt (Office Copy)
Investigations were carried out and a list of affected branches only shows one bag of each".
was generated (this is no longer available) and provided to Post
Office. NBSC was informed about a workaround to the issue. It would have been rectified by a transaction
correction.
4. acha522T Cash This involved two separate cash This is not a bug, rather the incident appears to be caused by Impact caused by human error.
Withdrawal transactions most likely by two separate human error of the user not reading the screen carefully when
{F/697} customers as follows: doing a withdraw limit CAPO transaction after failing to settle an
earlier customer basket.
 customer 1: given £200 in cash
 customer 2: given £320.90 in cash
(being the total value of the stack)
The loss was caused by the user not

settling stack in between banking
transactions.
5. acha2140S £1m Cheques If the branch holds cheques to a value of This will be rare and if encountered it will be highly visible by the No impact.
more than £1M, then the value of Postmaster.
{F/1374}
cheques cannot be adjusted using the
normal Stock Adjustment mechanisms. The best thing to do would be to rem out the Cheque for £1M
and then any further adjustments can be handled as normal.
6. acha3347Q If a stock unit carried forward from This was an issue for a branch following migration from old No impact.
Horizon is deleted before the first trading Horizon to Horizon Online and only affected a branch if it had
{F/702}
period rollover on Horizon Online, the deleted a Stock Unit after migration and before the first rollover.
E2/11/62
AC_152871086_1 40
check for 'last stock unit' may not be The issue would be clearly visible as the Branch would be
applied properly, and all the stock units unable to rollover without following a complex work around.
can roll over without Local Suspense
being cleared.
Local Suspense for a particular trading

period has to be zero before the branch
can be rolled over. Once a stock unit is in
the new trading period, it can put
gains/losses into Local Suspense and
clear them, but this has no effect on
Local Suspense for the old trading period
- think of them as two separate
containers.
7. acha3610P Advantageous There are advantageous exchange rates This appears to be a misunderstanding of what exchange rates No impact.
{F/679} Exchange for transactions over certain limits to use. The “over 500” rate applies to when the sterling value is
Rates (shown on Foreign Currency report as over £500 (not the Euro value).
DDE for Euros, DDU for US dollars). In
this case the limit was £500.
The user pressed 'Buy Euros' and

entered 600. The exchange rate shown
was the standard exchange rate, not the
rate for transactions over the 500 limit,
the reason being the £500 limit had not
been reached.
8. acha4221Q The clerk went into the 'Delivery' menu This was a bug in the early days of Horizon Online following an Financial impact would have been clear to the
{F/794}
th
and scanned two pouches (one of unusual (but valid) sequence of events. It was fixed on 19 branch because a duplicate receipt was
E2/11/63
AC_152871086_1 41
currency and one of coins). The second April 2010. printed. It would have also been identifiable
pouch was recorded twice on the from the reports available from Horizon.
system, resulting in a loss of £80.
It was resolved by a transaction correction
Two Remittance In slips relating to the being issued.
second pouch were output, both
identical, as well as one for the first
barcode.
9. acha4353P The counter froze while cashing a Postal This was due to an invalid Postal Order with a negative value No impact.
{F/828} Order and now recovery won't complete being set up by an external system which resulted in a counter
so the counter could not be used The being frozen.
counter was rebooted, but when they A fix was applied to check the value of Postal Orders coming
logged back in they got a Postal Order from external systems as being positive and this was applied on
encashment transaction recovery 15/8/2011. A system fix was made to allow recovery to be
message for a negative amount, followed bypassed on this counter to make it useable again.
by the Invalid Value message again.
10. acha5226J When a branch puts through a bureau This issue was fixed in October 2010 No impact.
transaction in excess of £5,000 a
{F/734} message should appear on the screen to
remind the user to ask the customer to
take two forms of ID from the customer
to conform with Anti Money Laundering
regulations. This reminder prompt is not
appearing on the Horizon Online
counters.
11. acha5259Q The Postmaster wrote a discrepancy of It appeared to only affect branches balancing in April 2010 and Temporary financial impact which would have
£167.17 to Local Suspense, then this 33 branches were identified as being impacted. Details of these been cancelled out in the following period by a
{F/637}
was cleared from Local Suspense as branches were passed to Post Office. corresponding discrepancy.
E2/11/64
AC_152871086_1 42
normal and the Postmaster selected to

make good the losses. At this point the This was an issue found in the early days of Horizon Online and
system printed out a final balance report was resolved in July 2010.
for the trading period with the cash
figure amended and nil discrepancy.
Normally the system would then come up
with a message to confirm rollover but
instead went back to the screen asking
how the discrepancy was going to be
made good.
12. AChambers183 Report Issues A transaction for £6.67 was done at A timing issue to do with printing reports at a counter. There No impact.
3 17:23 and was settled at 17:25. The daily was no impact on the branch accounts – just confusion due to a
APS transaction report was done on a transaction being missed from a report and the report not being
{F/186}
different counter at 17:25 and did not re-printed when it appeared.
include the APS transaction.
13. AChambers225 Postmaster sold some foreign currency This was not a bug, rather an issue in how a Currency Impact, but guidance on how to correctly
2R (1000 euros, sale value: £750). The transaction was incorrectly reversed on old Horizon. perform an existing reversal was all that was
Postmaster realised the transaction had needed to rectify it.
{F/316}
been settled to the wrong product in this Foreign currency transactions consist of two parts: the currency
case being cash instead of debit card. itself, which has a value based on the exchange rate, and
Existing Reversal was used to reverse margin, which is added to cover the cost of the transaction.
the transaction, and then re-run correctly. When the transaction was reversed, the Postmaster entered the
When it came to balance at the end of transaction for the cash settlement part of the transaction. While
the trading period, the currency stock the Postmaster believed the whole transaction had been
holding on the system was too high by reversed, it had not as the margin had not been reversed. When
1000 Euros. When corrected, this gave a the stock unit was balanced, the wrong number of Euros
gain on currency of £720 and a cash loss became apparent. The stock holding was corrected by the
E2/11/65
AC_152871086_1 43
of £750, being a net loss to the branch of declaration of the actual number held. Again, this did not correct
£30 the margin, which is generated as part of the currency sale and
is not directly linked to the stock.
The way Reversals are handled on Horizon Online means that

this sort of issue can no longer happen.
14. AChambers413 Multiple quantity for stamps/postage When a quantity greater than 1 is entered for a Smartpost This may have had an impact but a user should
4R label affects cash settlement or transaction, the Quantity is not reset to 1 when the user moves have been able to spot it and the sums involved
subsequent transactions. on to the settlement screen. are likely to be small due to the issue affecting
{F/319}
mails products.
If the transaction is settled to Fast Cash / Fast Cheque or Debit
Card, this doesn't matter, but some users habitually use the
Cash (F2) button to enter the cash presented by the customer,
then give the customer change as indicated by the new stack
total. If this is done, the cash amount entered is multiplied by
the Quantity and hence the new stack total is wrong.
This was fixed in December 2005 (Reference Data fixed).

15. AChambers441 Receipts and This appears to have been an issue with This was picked up by Reconciliation Reports(looking for Temporary financial impact which would have
3Q Payments doing a Transfer Out which was not receipts and payments mismatches) and investigated. been cancelled out in the following period by a
{F/266} Mismatch picked up correctly when balancing. corresponding discrepancy
16. AChambers571 Postmaster balancing on counter 1, then This occurred due to a counter failure around the time EOD No impact.
1K completing on counter 2 due to counter 1 activities were happening and thus resulted in a mismatch in
{F/188} timing out, causing a discrepancy Reconciliation reports.
between reports.
17. Agnihotriv245L Horizon The last digit of the exchange rate is No Financial impact. It was fixed in September 2010. No impact.
{F/667} Online occasionally being displayed incorrectly
E2/11/66
AC_152871086_1 44
Exchange on the rates board.

Rate
The system does use the correct values
to calculate the rates -the issue here is
with the display only. However, because
the display is on the Customer facing
rates board there is potential for
annoying Customers as they may get a
slightly different rate to that advertised on
the board.
18. AOConnor158I Transaction Failed to Harvest a quantity of 11743997 This is another problem with limits. The branch tried to declare a No impact.
{F/81} Limits pennies as being too large. cash holding of over £100,000 in 1p and this hit a limit in
Reference Data. The fix was to police the system limit on
declarations.
19. AOConnor5257I The Postmaster remmed in a cheque for This is a user error in how Post Office cheques were handled. Temporary impact caused by user error.
{F/80} £3,200, however, it did not show up on Following advice and Guidance the problem was resolved.
his balance snapshot or adjust stock, but
it is showing in his Suspense Account.
20. arnolda229R An Open Value Encashment for £5.00 There was a typographical error in the script causing the issue. No impact (resolved before Horizon Online
was performed, the transaction was This issue was found during testing of Horizon Online and fixed went live).
{F/530}
authorised and added to the Basket but before the first counter went live.
the counter crashed before the Basket
could be settled. On Login Recovery was
invoked and a 'Recovery Failure' receipt
was printed for £0.00.
21. ArnoldA2341L Currency All currency codes, in terms of "IMoney" This was an issue identified by developers regarding the way No impact.
{F/533}
E2/11/67
AC_152871086_1 45
Code objects should be ISO-compliant, or currency was handled within the counter code. It had no impact
Validation verified by £-sign However, we do not in the real world it was closed.
validate this at all anymore, in order to
support future currency codes. This
means that junk codes are accepted.
22. ballantj020J Postmaster states that she has sold a There were 2 issues here: This may have resulted in a small impact due to
{F/698} Lyca top up for £10, the message appear a failure to follow the correct procedure.
unable to connect to the data centre then 1.There was an issue with how Reference Data was generated
logs Postmasters out. which resulted in some counter scripts failing. This was fixed on
20/08/2010 (2 days after this problem occurred);
The transaction request has been 2. The Ref data issue caused an e-pay transaction to fail and
authorised and the reversal may not be the Postmaster didn’t handle the recovery correctly
effective which will cause an 'E21'
reconciliation error. The failure to handle recovery correctly may have resulted in a
loss of £10.
23. ballantj2547K The Transaction Processing System This is a problem with Smart Post which seemed to write slightly No impact.
Total and Counter total values for the corrupt transactions in that there were missing attributes
{F/633}
Number and Absolute Quantity columns required by back end systems and reconciliation systems, but
are the same but the Absolute Value for are complete as far as the branch accounting is concerned.
Counter Total is greater than the
corresponding Transaction Processing This therefore has no impact on branch accounts, but does
System Total by £14.80. result in reconciliation errors which are fixed by amending the
transaction copies in the backend systems. The fixes are
If the session nets to zero (add up all the always to dates and not to values.
SaleValues for the same SessionId) no
reconciliation is needed. If it doesn't, a This KEL refers to amount mismatches, but the amounts used
correction must be made to send the by the reconciliation are different from those used by
data to POLFS (see <a accounting. They should be identical, but in this case are not.
E2/11/68
AC_152871086_1 46
href=kel_view_kel.jsp?KELRef=Maxwell The accounting values are the correct amounts.

G460L>MaxwellG460L</a>) and the PM
may need to be told about a possible
receipts and payments mismatch, or at
least watch out in case one is raised.
24. ballantj3342L Reconciliation picked up a scenario As the transaction failed there was no impact on the branch No impact.
associated with a failed banking accounts.
{F/600}
transaction. Specifically, the request
from the counter never reached the This did identify some issues in the way that failed transactions
Branch Access Layer. were handled and why they resulted in reconciliation errors and
The application event log shows these were fixed in April 2010
AdminCfg receiving data and causing a
VPNKeyChange
In this case the request never reached

the Authorising Agent and therefore no
money was requested, the C0 did reach
the Authorising Agent but was
unexpected and has caused this
reconciliation incident.
Instruct MSU that no reconciliation

required.
25. bambers3553L Currency A MoneyGram Send transaction was This was an issue found during testing of Horizon Online and No impact.
Specification initiated. There are 3 options to define relates to a “£” sign being displayed when the user is being
symbol the amount being sent: asked to input an amount in another currency to a MoneyGram
{F/488} i. £ including fee transaction.
ii. £ not including fee It was agreed not to fix this until a later time (not clear if it ever
E2/11/69
AC_152871086_1 47
iii. Receive Amount excluding was fixed, however, the MoneyGram product was re-engineered
fee. in 2015 so it would have behaved differently after that time
The 3rd option allows a customer to anyway).
specify an amount to be sent in local
currency, such as $300, in which case This would have no impact on branch accounts, but may have
the receiver will get $300 and the send caused some mild confusion to the Subpostmaster.
amount is calculated in sterling.
However, at the input of the receive
amount, the prompt appears (correctly)
as 'Amount in USD' but the input box
(which is a currency datatype) has a '£'
symbol present. This leads the user to
think that they have selected the wrong
option and would lead to incorrect
amounts being entered here. Please see
the attached screen shot for evidence.
26. bambers4236K Electronic Top ETU E-voucher for £10.00 is erroneously This is an issue in the Counter Training service in that it doesn’t No impact.
Ups ("ETU") declined as a New Reversal. support reversals of E Top ups.
{F/524}
.
The basic problem is that, to support It was discovered during testing and was agreed that this would
ETU Reversals, we rely on the be a restriction on the functionality supported for Counter
Authorising Agent remembering details Training.
of the original ETU transaction. In a CTO
we only have an Agent Simulator and it As this only impacted counter training then there is no impact on
is not configured to handle ETU any Live Branch accounts.
Reversals. Given the simplicity of the
Simulator it would be very difficult to
support ETU Reversals.
E2/11/70
AC_152871086_1 48
This as a restriction for CTOs but is not

documented in DES/GEN/REP/0006 nor
REQ/CUS/STG/0004
27. BluerP5546R Upside Down For PING transactions (Transaction This was a cosmetic issue when processing Transaction No impact.
Pound Sign Acknowledgments) £ signs will appear Acknowledgments on a Training Counter in that the “£” sign was
{F/700} as upside down question marks in the displayed incorrectly. This has no Financial Impact as it was
training counters. only affecting training counters.
This issue was spotted in internal testing and we had no reports

of this issue from Live Counter Training Offices. A fix went live
in January 2011.
28. cardc235Q Drop and Go The user initiated a Drop and Go This was a problem in handling errors correctly in a Drop and This would have caused a loss in the branch
transaction for £100 which failed due to Go script provided by Post Office. accounts, although the issue was identified by
{F/1660}
timeouts. Following the failure, a success the Subpostmaster and it would have been
message was displayed. The user This was passed to Post Office to fix the scripts. resolved by a transaction correction.
settled the transaction and the customer
handed over £100. The customer
checked the balance and stated that the
top up had not gone through so the clerk
performed another Drop and Go
transaction which was successful. The
customer has paid in £100 but the
branch account has been debited by
£200. Accenture verified that only the
second Drop and Go top up was
successful.
29. cardc339P Receipt A Transfer Out of £511 cash was done The system is behaving correctly and there would be no No impact - the Transfer Reports and the
{F/377}
E2/11/71
AC_152871086_1 49
Printing on counter 14, session id 14-2966714-1. financial impact on the branch account, but the Postmaster may Transaction log will show exactly what has
A corresponding Transfer In was done be confused as to what exactly has happened. actually happened.
immediately afterwards on counter 2,
session id 2-2046304-1. Before the
Transfer In had completed, a receipt with
the wrong session id (14-2966714-1)
was printed. After the Transfer In had
completed, the correct receipt was
printed.
The extra receipt, which is almost

identical to the actual Transfer In receipt,
is quite confusing. It may lead the
Postmaster to suspect that the transfer
has been carried out twice, when in fact
it has not. Advise the Postmaster to use
the 'Preview' rather than the Print button
if they wish to view the individual
products being transferred.
30. cardc427T The transaction appeared as a state E26 This was an issue with failed banking transactions and then No impact. Following a failed banking
on the NB102 Section 2 Link report. recovery from a subsequent counter failure using the same transaction the next banking transaction may
{F/1362} identifiers. Some of these appeared in reconciliation reports reuse the same unique reference. This will
This issue has also resulted in a when they shouldn’t be and so causing additional work to cause that transaction to fail also as the
transaction being reported as both a Support teams. authorisation software knows the transaction
State 4 and State 6 - see PCPC0244934 previously failed and will not pass it on to the
However as these are all failed transactions they are all for zero FI."
PC0197368 was fixed and released as amounts and so have no impact on branch accounts.
part of release
E2/11/72
AC_152871086_1 50
CTR01_22_01_00_RELEASE (June
2010).
Unfortunately the change was regressed

as part of a BAL change in February
2015 - see peaks PCPC0243030 and
PCPC0241771. Another fix is in
progress.
31. cardc2326R The user will usually count out the cash The system is operating as designed and no change was If the correct procedures were followed there
to be paid before pressing Fast Cash, requested by Post Office. would be no impact.
End-of-session because once they have pressed Fast
sales prompts - Cash then the amount to be paid out is There may be confusion in relation to the way Sales Prompts
usability issues reset to zero and the stack disappears. are handled at the end of a session.
{F/739} However, if the transaction results in a

sales prompt then the stack is not
cleared and the amount payable remains
on the screen. If the user selects the
sales prompt and transacts another
product, this is added to the stack and
the 'Total Due To Customer' is updated
by the relevant amount.
If the user is distracted or busy then they

potentially pay out the new amount in
addition to the original amount. The
system is working as designed and
Postmasters should be referred to the
E2/11/73
AC_152871086_1 51
NBSC.
32. cardc3335R A call was raised with e-pay to determine Not a bug. Certain E Top Up products had been withdrawn but No impact.
why requests were being declined. This the Reference Data had not been updated to remove them from
Vodafone Text is e-pay's response: The reason for the the counter. This meant that e-pay declined the requests.
Pack Vouchers Vodafone £10, £15 and £20 Text Pack Following the investigation, then the Reference Data was
being declined vouchers being declined by e-pay is updated to remove the products from the counter the following
because they were deactivated on weekend.
{F/516} request by Vodafone. We sent out a
Product Configuration document in May
detailing this change. This document was
sent to Dave Cooke at Fujitsu as well as
Clare Tetley and Iain Gilbert at Royal
Mail. The deactivation was rolled out on
June 1st.
33. cardc3415N When a branch migrated from Horizon to This was a problem in the migration of a branch from old No impact - the figures post migration were
Horizon Online, differences were Horizon to Horizon Online. correct and the issue down to an inaccurate
{F/666}
reported between the 'Pre Quantity pre-migration report.
Move' and 'Post Quantity Move' figures. The report produced pre-migration on the old Horizon didn’t
take into account a Transaction Correction carried out in the last
trading period so adjust stock levels. The report produced post
migration was correct.
.
34. cardc5946P Halifax / Bank of Scotland bank cards Branches with an “&” in their name were resulting in Banking No financial impact on branch accounts.
declined with response 05 - 'do not transactions being declined by Halifax / Bank of Scotland.
{F/808}
honour'. No information was given by Halifax / Bank of Scotland
regarding why they are declining the cards. This was fixed by
May 2011.
E2/11/74
AC_152871086_1 52
35. CCard1223Q {F/340} Counter hangs when attempting to clear It would appear that an Invalid option was presented on the No impact.
local suspense during stock unit rollover. menu of options available when settling local suspense. This
seems to have been fixed shortly afterwards.
36. CCard4658N The stock unit balance report only Buttons were introduced to record when cash was added and No impact.
{F/336}
includes figures for Add or Remove Cash removed due to variances being spotted. However the
transactions done in the current behaviour of these was not carried forward from one balancing
balancing period. It should however period to the next and so caused confusion.
show the cumulative total since the start
of the trading period. When the problem was identified, the buttons were “padlocked”
and a fix was issued in March 2006.
37. CharltonJ222L Log on event timestamp can be after the The log on event appears to be recorded at the time the log on No impact.
{F/1155}
log off and other associated events when is processed by the HBS server, but the other rep events are
looking at rep events for the HBS Kiosks recorded against the "DateTime" field in the incoming message.
38. CharltonJ2752T {F/1060} See item 24 of Appendix 1. See item 24 of Appendix 1. See item 24 of Appendix 1.
39. CObeng1123Q {F/185} See item 44 of Appendix 1. See item 44 of Appendix 1. See item 44 of Appendix 1.
40. acha4349K Reconciliation reports relating to declined This has no impact on the branch but caused unnecessary work Zero value transactions have no financial
{F/715} e-pay transactions are not clearing down for the Fujitsu and Post Office's reconciliation teams. Issue was impact on branch accounts.
correctly. The affected transactions are fixed on 11/10/2010.
zero value, and have been declined by e-
pay.
41. acha4745R This was an issue relating to back end KEL suggests issue was with reconciliation reports not being Not known.
{F/1643}
reconciliation where there was a £20 (manually) processed correctly. Peak PC0219762 applies. This
difference between 2 totals relating to is due to a recovery performed at the branch on the following
millions of pound worth of LINK day which reversed the transaction. IT caused confusion in the
transactions. reports.
This was raised by POL's reporting systems.

E2/11/75
AC_152871086_1 53
42. acha5650L A user logged into a counter where there Bug in the recovery process that could post transactions to the No impact.
{F/1025} was an unsettled banking transaction wrong TP / BP. This would result in 2 discrepancies in 2
requiring recovery, which had been done separate periods which would cancel out, so no long term
in stock unit AA. impact on Branches.
Stock unit AA was still in TP 12 BP 4. Fix issued in June 2010, but in theory could impact counter that
Stock unit BB was already in TP1 BP 1. migrated and hit this problem on the day of migration to Horizon
Online until September 2010 (when migration completed)
Recovery completed successfully,
correctly writing the transaction and its
settlement into stock unit AA, but for TP
1 BP 1.
The stock unit was short by the amount

of the banking transaction in TP 12 BP 4,
but then had a matching gain in the
following period.
If the TP/BP is incorrect, but the stock

unit will roll into that period in the future,
then this problem will cause a
discrepancy in the current period, but it
will be balanced by an equal and
opposite discrepancy in the future.
Advise the PM of this.
43. acha633R The Settle Gain/Loss Centrally products This is an issue identified by Post Office rather than a branch in No impact.
{F/763} have a minimum transaction value of that branches were being allowed to Settle Centrally small
£150 and should not be available if the losses (limit should be £150 or more). This was identified on
E2/11/76
AC_152871086_1 54
discrepancy to be cleared from Local 15th May 2010 and the fix rolled out to all branches by 5th July
Suspense is less than £150. Horizon 2010
Online does not appear to check the
minimum value when building the pick
list and so branches can choose settle
Centrally for any value.
44. AChambers253 This was an issue picked up by the Unclear what branches would have done at the time, however Any impact would have been very minimal,
L reconciliation checks due to smart Post the impact is likely to have been very small (pence rather than pence rather than pounds.
not correctly checking that the pre-paid pounds). This was fixed in 2005.
{F/449} amount for postage was less than the
actual amount and then attempting to
generate a postage label for a negative
amount
45. AllenD2519J POLSAP report that a particular TC or Back end problem, involving data sent to POL from a PO client. No impact.
TA is missing from the expected BLE file. This is an issue caused by incorrect Reference Data with
{F/1644}
The TC / TA entry is actually present in passing data to POL’s Back end systems (POL SAP) and may
the BLE file, but it lacks the TC / TA delay the processing of a TA / TC.
Reference value which allows POLSAP
to identify the item.
46. AllenD429U One of the central systems failed during Looks like a back end problem, with no impact on branches No impact.
the evening and when it restarted it This is a problem in the data Centre which delays passing data
{F/1668}
picked up the wrong time, which meant to Post Office's back end systems. However, it has no impact
that when it was trying to decide whether on Branch Accounts.
a Txn happened before or after 7pm (the
EOD cut off) if got the answer wrong and
this meant that some data was
associated with the wrong day in back
end systems.
E2/11/77
AC_152871086_1 55
47. cardc4027Q This was a problem in incorrectly The issue first occurred in 2011 and another similar issue No impact.
handling a transaction which had been occurred in 2013. It appears to be related to the old Hypercom
{F/1149}
rejected by the PIN PAD resulting in a PIN Pads which were replaced after the second occurrence of
spurious reconciliation error on a report. this issue so no further action was taken
As the transaction had been clearly declined, then there would

be no financial impact at the Branch.
48. cardc5444K The Postmaster received a Planned In a Shared SU, then it is possible to make multiple Cash Temporary impact due to human error.
Order as follows: Declarations from different tills which are added together.
{F/740} Based on the last declared Cash on
Hand figure of In this case a Cash Declaration had been made accidently for
£97,875.00 notes + £9,156.29 coins Till 16 (instead of Till 1) which resulted in the value of cash
on 13.07.2010 you will need to remit to being doubled.
the Cash Centre
£85,000.00 in notes on your next This should have been spotted when the branch balanced.
scheduled collection day. However, before then the Cash Planning identified that the
The PM had declared cash for all his Branch had too much cash and asked for some to be returned.
stock units on that day, however his
actual Cash Declaration value was much This is a case of branch user error and there was no actual
lower than the Planned Order cash impact on the accounts, once the spurious Cash Declaration
declaration value. The Planned Order was identified and removed.
asked him to remit too much cash.
The PM normally used Till Id 1 for his
shared stock unit, but had accidentally
done a declaration with Till Id 16 during
the previous week. He was unaware that
the Till 16 declaration was being added
E2/11/78
AC_152871086_1 56
to every overnight cash declaration sent

to the cash centre.
E2/11/79
AC_152871086_1 57

Bates V Post Office: Steve Parker Witness Statement 1

Uploaded by

Copyright:

Bates V Post Office: Steve Parker Witness Statement 1

Uploaded by

Document Information

Original Title

Copyright

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Bates V Post Office: Steve Parker Witness Statement 1

Uploaded by

Copyright:

Filed on behalf of the: Defendant

Witness: Steven Paul Parker

Claim No: HQ16X01238, HQ17X02637 & HQ17X04248

IN THE HIGH COURT OF JUSTICE

ALAN BATES & OTHERS

POST OFFICE LIMITED

WITNESS STATEMENT OF STEPHEN PAUL PARKER

1. I am employed by Fujitsu Services Limited (Fujitsu) as Head of Post Office

2. I am authorised to make this statement on behalf of Post Office Limited (Post

RICHARD ROLL'S STATEMENT

12. In order to explain why, it is necessary to have a proper understanding of what

21.2 In other words, although a transaction could be inserted, it would immediately

The Horizon Support Teams

Horizon application. I estimate that there were around 80 members of

(a) dealt with straightforward enquiries such as password issues and

If a branch required assistance to attempt to determine the cause of a

26.3.3 undertook complex configuration (configuration items can be used to

26.3.4 designed, wrote and documented new support tools;

26.3.5 undertook source code examination, complex diagnosis and

26.3.6 provided technical support to other internal Fujitsu teams working on

Further context – call volumes and transfers

The SSC and Mr Roll's role within it

40.1 known issue / work around - 35.3%;

40.2 admin - 27.5%

40.3 reconciliation - 15.7%;

40.4 potential user error - 10.9%;

40.5 potential software error - 8.3%; and

40.6 hardware error - 1.2%.

41. It should be noted that:-

42. Mr Roll's suggestion in paragraph 10 that software issues in Horizon "routinely"

The work carried out by Mr Roll

48.1 It is not the role of the SSC to routinely investigate discrepancies.

48.2 In very rare circumstances a discrepancy could be caused by a software issue

Response code % allocated

53.1 accessing branch data in read-only mode;

53.2 inserting new transaction data; and

53.3 editing or deleting existing transaction data.

55.2 As explained by Mr Godeseth in paragraph 35 of his statement, in Legacy

55.2.3 selected data was extracted from the correspondence servers to

55.5 Alternatively, Mr Roll's reference to a binary code flipping may relate to a

KELS AND PEAKS

61.9 Archiving: There is no requirement to keep historic support information. Once an

62. Peak (not an acronym) is browser-based software incident and problem

categories. Although some aspects of this document need revision due to

Claim No: HQ16X01238, HQ17X02637 & HQ17X04248

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

The relevant Peak (PC0202239) relates to an investigation surrounding {F/688}

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

The fault appears to result from two causes:

i. faulty Reference Data (the result of human error), which was

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments

KEL Mr Coyne's Report Fujitsu's Comments