App Development: An NHS Guide For Developing Mobile Healthcare Applications

NHS Innovations South East
App Development:
An NHS Guide for Developing Mobile
Healthcare Applications
May 2014
Developed with funding from the Intellectual

Property Office Fast Forward 2013 Competition.
CONTENTS
EXECUTIVE SUMMARY ............................................................................................................................................ 5
MOBILE APP FIELD GUIDE STAGES .......................................................................................................................... 6
STAGE 1 – PRE-DEVELOPMENT ............................................................................................................................... 7
Needs Assessment .............................................................................................................................................. 8
Why Develop an App? ..................................................................................................................................... 8
Compeve Analysis ........................................................................................................................................... 9
Stakeholder / Target Audience Analysis ............................................................................................................. 9
Choice of Operang System Plaorm ........................................................................................................... 10
Internal Development or Outsourced? ............................................................................................................. 11
Business Jusficaon Case ................................................................................................................................ 11
Value Proposion .......................................................................................................................................... 11
Paent /Organisaonal Benefit .................................................................................................................... 11
Resources & Costs ......................................................................................................................................... 11
Risks .............................................................................................................................................................. 12
Commercialisaon Strategy .......................................................................................................................... 12
Intellectual Property Consideraons ............................................................................................................ 12
Reference Costs ............................................................................................................................................ 12
Summary of Key Points ..................................................................................................................................... 14
STAGE 2 – DESIGN & DEVELOPMENT .................................................................................................................... 15
User & System Requirements ........................................................................................................................... 16
Hardware Consideraons.............................................................................................................................. 16
Use OF API Components ............................................................................................................................... 17
Security and Communicaon Consideraons ............................................................................................... 17
Wireframe Development .................................................................................................................................. 18
Screenshots ....................................................................................................................................................... 18
Code Generaon ............................................................................................................................................... 18
System Integraon & Tesng ............................................................................................................................ 18
STAGE 3 – USER TESTING ...................................................................................................................................... 19
STAGE 4 - STAKEHOLDER REVIEW ......................................................................................................................... 20
© 2014 NHS Innovations South East NISE_App_guide_May14_v1

Stakeholder Review and Clinical Validation ...................................................................................................... 20
Relevant Technology Standards .................................................................................................................... 20
Clinical Risk Safety Standards ........................................................................................................................ 21
Systems & Security review ............................................................................................................................ 23
Quality of Apps and Validation ..................................................................................................................... 23
Internal Deployment Within Organisation ........................................................................................................ 24
STAGE 5 - APPS AS MEDICAL DEVICES ................................................................................................................... 25
When is an App a Medical Device Based on MHRA Guidance? ........................................................................ 25
Outline of MDD Process ................................................................................................................................ 26
FDA Guidance .................................................................................................................................................... 28
STAGE 6 – APP DEPLOYMENT CONSIDERATIONS .................................................................................................. 31
App Stores ..................................................................................................................................................... 31
NHS Choices App Store ................................................................................................................................. 32
NHS Trust Dedicated App Stores ................................................................................................................... 32
Other Routes to Market ................................................................................................................................ 32
STAGE 7 - MONETARISATION OF APPS ................................................................................................................. 33
“No Brainer” App .............................................................................................................................................. 33
App Sustainability.............................................................................................................................................. 33
Different Approaches to The Monetarisation of Apps ...................................................................................... 34
Free Apps ...................................................................................................................................................... 34
Freemium and In-app Purchasing ................................................................................................................. 37
Paid for App................................................................................................................................................... 39
The App Package ........................................................................................................................................... 40
Code Trading ................................................................................................................................................. 41
What Does All Of This Mean For The “NHS App”? ........................................................................................ 41
STAGE 8 - DIGITAL MARKETING AND ADOPTION CONSIDERATIONS .................................................................... 43
REFERENCES / RESOURCES.................................................................................................................................... 44
1st Edition © 2014 NHS Innovations South East 3

3
References ........................................................................................................................................................ 44
Websites with Additional Information and Medical App Review Sites ............................................................. 50
APPENDIX 1: Schematic - Mobile Health App ..................................................................................................... 51

4
EXECUTIVE SUMMARY
App development can be daunng for the uniniated, and a challenge for NHS organisaons to respond to the
increasing demand to develop and support mobile apps within their environment. NHS staff are increasingly
seeking to tap into the potenal informacs benefits associated with mobile devices. They join a steady stream
of individuals, organisaons or professional bodies developing apps to target local or naonal unmet needs.
This mobile app was made possible by a 2013 Intellectual Property Office Fast Forward Compeon award. This
award has enabled NHS Innovaon South East (NISE) to develop a praccal understanding of the mul-factorial
approach required for the successful development and commercialisaon of ”Healthy apps” targeng both
health conscious consumers and healthcare praconers. NISE has published this guide to share some basic
guidance on app development and the monesaon of apps to offer a sustainable plaorm for use in a healthcare
environment. The guide is unashamedly biased toward app development within an NHS environment but may be
equally relevant for organisaons or companies seeking to work with NHS organisaons. This field of innovaon
is constantly changing, so comments and viewpoints expressed reflect current pracce as of April 2014.
Key consideraons for app development include an overview of factors that need to be considered before
comming to any app development and the app development process. As with all projects, good preparaon
will help manage the app development and increase the likelihood of a more successful outcome. The
development process is facing a number of changes as many of the emerging clinical apps may require
approval from a regulatory body such as the UK Medicines and Healthcare products Regulatory Agency
(MHRA). The latest guidance (April 2014) from the MHRA regarding apps as medical devices has been
incorporated into this first edion.
The increasing use of healthcare apps has created increasing concern amongst clinical commentators and
professional bodies with regard to the quality of many published apps. NHS organisaons seeking to develop
healthcare apps should ensure that appropriate processes are put in place to check for clinical risk, security,
informaon governance, integraon with exisng infrastructure and that the app is fit for purpose.
Business jusficaon remains important, and in mes of austerity the sustainable provision of an app to deliver
beer care or perhaps to support a redesigned service is an important consideraon. Regardless of internal or
external development the funding and resourcing of apps will remain a challenge – therefore there is a need to
fully understand the opportunies that an app may provide for a given organisaon, its staff and ulmately its
paents.
There is no single monesaon model that is relevant for apps created in conjuncon with the NHS to meet an
unmet need, and a number of factors will influence the approach. A combinaon of monesaon models (or
hybrid model) may be appropriate in certain circumstances. Different approaches may be required depending
on whether the healthcare app is targeng an NHS user or health conscious consumer.
Ulmately the “health app” value should be determined by its impact on outcomes. If there is clear value in the
outcome, then it should be paid for in some form. What we have come to expect from apps as a consumer is
and may be quite different to what we expect from a medical app in the future.

5
MOBILE APP GUIDE STAGES
STAGE 1 - PRE DEVELOPMENT
STAGE 2 - DESIGN & DEVELOPMENT
STAGE 3 - USER TESTING
STAGE 4 - STAKEHOLDER REVIEW
STAGE 5 - MEDICAL DEVICE PROCESS*
STAGE 6 - EXTERNAL DEPLOYMENT
STAGE 7 - MONETARISATION OF APPS
STAGE 8 - DIGITAL MARKETING AND

ADOPTION CONSIDERATIONS
* Subject to nature of mobile app

6
What are your goals?

Needs Assessment
Why develop a mobile app?
Competitive Analysis
Am I “reinventing the Wheel”?
(Validated Apps)
STOP DECISION GO ON
Stakeholder /Target Project stakeholders?

Audience Analysis User Profiles?
Public Patient Involvement
needed at this stage?
hardware/platform preference?
Internal Development Identify Suppliers / collaborators?

or Outsourced?
Procurement process?
Value Proposition?
Business Justification Case Patient / organisational benefit?
Resources? Costs? Risks?
Commercialisation strategy?
Intellectual Property in app? -
copyright in code, database rights,
trademark branding
Sustainability?
Ongoing support and costs?
STOP DECISION GO ON
Proceed to STAGE 2

7
The widespread adopon and use of mobile technologies have the potenal to provide new and innovave
ways to improve health care delivery and the health of individuals.
Clinicians have increasingly embraced the use of both smartphones and tablets in their professional capacity
over the past three years. A survey conducted by the GMC demonstrated that in 2011 30 % of doctors used a
smartphone app. (Visser,BJ and Bouman, J. 2012) Another study concluded that this number would rise in 2012
and predicted that 83% of medical doctors would use a smartphone in their work (Baumgart DC, 2011).
The use of healthcare apps is not limited to clinicians using such tools at work. Apps also provide an
empowerment mechanism for paents so that they can take greater responsibility for their own diagnosis and
treatment. Apps for paents are being developed to support healthier living, help manage a long-term
condion and to provide inial advice on an emerging medical problem. Employers are also experimenng with
how smart phone technology can improve paent safety and outcomes, while simultaneously driving higher
efficiency.
In their third Global Mobile Health Market Report published in March 2013, Research2Guidance predicted
that the market for mobile health (mHealth) app services will reach $26 billion by 2017. Evidence was found
that the long-expected mobile revoluon in healthcare is set to happen with both healthcare providers and
consumers embracing smartphones as a means to improve health and healthcare. Top mHealth publishers
have generated more than 3 million free and 300,000 paid downloads in the USA on the iOS plaorm. The
reach on other plaorms and in other countries differs but the trends are similar. They found more than
97,000 mHealth applicaons listed on 62 full catalogue app stores. The majority of these applicaons were
general health and fitness apps which facilitated the tracking of health parameters by individuals and provided
users with basic health and fitness related informaon as well as guidance.
NEEDS ASSESSMENT
All needs assessment starts with an understanding of a current or baseline posion and what an organisaon is
trying to achieve. Put it another way - what is the goal? There are a variety of tools and techniques which are
typically used in value management or change management processes to idenfy goals. The simplest tool to
support a needs assessment is a gap analysis which idenfies what need to be done in the gap between the
current posion and a future end point or goal. If applied fairly, does the needs assessment support the
development and creaon of a mobile health app to achieve the goal, or facilitate a chance in service or process
towards that goal?
WHY DEVELOP AN APP?
Smartphone and tablets are hybrid devices with components and funconality more closely resembling a
computer. The range of funconality enables efficient communicaon in a clinical se£ng independent of the
paent’s locaon and the potenal to support improved decision-making at the point of care resulng in fewer
errors and be¥er outcomes. This is parcularly important for professionals who are constantly on the move
and rarely have the opportunity to sit down in front of a desk with a networked computer.
Apps are so¦ware applicaons which can be downloaded onto smartphones, tablets and e-readers to provide
soluons for an individual problem or to sasfy a niche requirement as opposed to huge enterprise IT projects
which are very expensive and take a long me to commission. Apps can also be accessories that a¥ach to a
smartphone or other mobile communicaon devices, or a combinaon of accessories and so¦war e.
Medical apps have an enormous potenal for improving clinical pracce by providing a quick, comprehensive,
and up to date overview of current clinical guidelines, making a differenal diagnosis, performing useful

8
calculaons and looking up a paent’s invesgaons, which could change the way healthcare is delivered in
the future. (Boulos et al., 2011).
There are two general types of mobile apps, nave and non-nave. Nave apps reside on a mobile device and
have been developed specifically for the device so ware plaorm. Nave apps can use specific coding to
ulise mobile device features such as a built in camera or GPS capability. Non-nave apps ulize browser
interfaces or other gateways to provide the end user with informaon through a mobile device. A nave
mobile app typically requires download through a market place associated with the operang system (OS),
such as iTunes for Apple OS devices, Android’s marketplace for Android OS-powered phones, the Microso
windows App store and Research in Moon’s App Store for Blackberry devices. Non-nave apps simply require
that the mobile device has the appropriate web browser or other interface enabled.
The increasing capabilies associated with mobile device hardware means that mobile apps are more capable
of delivering ever greater funconality and can respond in a quicker fashion than earlier devices. Newer web
technologies also mean that non-nave apps can increasingly compete with nave apps in terms of funcon.
Alternave approaches to app development
A number of app developers offer packaged so ware services to avoid tradional app development and
management costs. Typically based on a monthly subscripon per user model, these services (so ware as a
service (SaaS), and hybrid models) can also include mobile device management and mobile applicaon
management infrastructure as part of the overall service.
Developing an app can be a daunng process. There are lots of decisions to be made and each one can have a
dramac impact on the overall cost of the finished app and its chances of success, BUT DON’T BE PUT OFF!
COMPETITIVE ANALYSIS
There are over 97,000 apps (and growing) listed on several app stores ranging from healthy lifestyle apps to
apps targeng clinical staff. Many of these apps may be of unknown origin and without any validaon or
cerficaon from a credible body. Before comming resources to the development of an app, a search should
be undertaken to determine similar apps that may:
• Fully address the requirement of the organisaon or individual and which could be an alternave to
new app development or view as a competor app
• Address the requirements of the organisaon or individual but lack validaon by a healthcare
organisaon or cerfication by a recognised body.
• Parally address the requirements of the organisaon or individual but lack validaon by a healthcare
organisaon or cerficaon by a recognised body.
Any alternave app currently on the market and available in your chosen plaorm or plaorms could be
considered a competor app in the market. In reality only those medical apps that are cerfied or validated by
appropriate organisaons for relevant and safe content are likely to be the true competors longer term, as
market forces will favour higher quality medical apps.
STAKEHOLDER / TARGET AUDIENCE ANALYSIS
Who are the stakeholders (individuals or organisaons) that need to be involved in the app design and
development or kept informed? This includes stakeholders at departmental, divisional, management or
execuve level. The stakeholders may also be external individuals, organisaons or network of organisaons
who may have an influence on the app development or who will be impacted on the development and
deployment of the app (or app supported service). Does this app need to integrate into an exisng care
pathway and how? What do you need to do or communicate to address any idenfied risks or issues?

9
Is the target audience fully understood in terms of needs, preferences or expecta ons? For instance is the
target audience plaorm agnos c or does the app need to run on a specific hardware device or opera ng
system(see below)? Will the target audience pay for the app, want a free app, or expect it to be bundled into
an exis ng service? Does the profile of the target audience lend itself to a par cular commercialisa on
approach?
CHOICE OF OPERATING SYSTEM PLATFORM
There are numerous mobile plaorms but the two systems that dominate the market are Apple’s iOS and
Google Android. Other technology companies which also distribute apps are Microso Windows, and
Blackberry but they only retain a rela vely smaller market share. Windows Phone device s have gained
significant and increasing market share with Nokia devices in certain countries but this reflects new sales rather
than reflect the exis ng installed base (exis ng app consumers).
Mobile devices are upgraded on a regular basis with their opera ng systems correspondingly being refreshed
to take advantage of advances in the hardware technology. Apple and Google currently release newer versions
of their iOS and Android mobile opera ng systems annually with iOS 8 due on 2nd June 2014 and Android 4.5 or
5.0 due in summer 2014. This should be taken into considera on when selec ng the plaorm, as there may be
a need to address backward compa bility or release new updates to reflect newer plaorm versions.
Microso has launched Windows phone 8 with update version 8.1 due later in April or May 2014.
Apple distributes major updates to the iOS opera ng system for iPhone, iPad and iPod touch devices via iTunes.
iOS7 runs on the iPhone 4 and later, second genera on iPad and later and all models of the iPad Mini. Apple
provides third-party developers with a set of tools and Applica on Programming Interfaces (APIs) to create
apps which take advantage of the technology inside every iOS device.
Apps can only be installed onto an Apple device running Apple’s iOS opera ng system if the soware has been
‘digitally ‘signed’ by Apple. This means that for most Apple iOS users the only way to add apps to their device is
through the Apple App Store, although a private cer fica on route is available for organisa ons (See iOS
Enterprise licence).
In contrast to Apple, which only licences its iOS on its own devices, Android is open source and Google releases
the source code under the Apache License. This open-source code and permissive licensing allows the soware
to be freely modified and distributed by device manufacturers, wireless carriers and enthusiast developers.
Manufacturers that use Android include Samsung, Nexus, Motorola and HTC but the process for rolling out
OS updates across the range of manufacturers is not as efficient as Apple’s. This can also present challenges to
the app developer in terms of the way in which the app func ons from one device to another eg screen
resolu on.
Windows Mobile is an opera ng system that was developed by Microso. This OS is used in smartphones and
other mobile devices. Windows Phone 7 and the latest Windows Phone 8 are currently the more popular OS
versions. Microso may take an approach that's similar to Apple's App Store, where tablet specific apps don't
run on the phone, but phone apps scale to run on a tablet. Microso has expressed commitment to a common
app plaorm between the two opera ng systems so in due course all of the Microso apps may become
available on all Microso supported devices. Windows phone 8 is a major step towards convergence with
Windows 8. A comparison between Windows 8 and Windows phone 8 can be found at
h¢p://msdn.microso.com/en-us/library/windowsphone/develop/jj681690(v=vs.105).aspx
A number of sites have offered comparisons between the various mobile opera ng plaorms including:
h¢p://www.pcmag.com/ar cle2/0,2817,2416521,00.asp

10
hp://www.macworld.co.uk/news/apple/ios-7-vs-windows-phone-8-which-mobile-plaorm-right-you-
3471624/
An increasing number of developers are developing or redesigning apps based on a mul -device web app
format which can also make use of newer HTML5 scripts and more capable mobile devices. Open source tools
such as PhoneGap (phonegap.com) and Appcelerator (appcelerator.com) permit easy crea on of apps (“web
wrapping”) using web technologies to build apps for both Android and iOS plaorms.
hp://simpleprogrammer.com/2013/07/01/cross-plaorm-mobile-development/
INTERNAL DEVELOPMENT OR OUTSOURCED?
Do you have access to internal resources within your organisa on to develop the mobile app? In many cases a
supplier or collaborator will be required to support the development and deployment of the required mobile
app. Is there a requirement to go out to compe ve tender or single tender ac on?
BUSINESS JUSTIFICATION CASE
In many instances a business jus fica on case will be required to secure access to funding and resources
associated with the development of the mobile app. The informa on required will vary between organisa ons,
the nature of the mobile app and its alignment with organisa onal objec ves.
The business jusficaon case or equivalent document should be used to decide whether or not to proceed
with app development. Informaon gathered during this STAGE 1 can also be used to guide requirements
captured as an early part of STAGE 2.
Factors to consider in the business jus fica on case include:
VALUE PROPOSITION
In a brief and clear statement can you ar culate what the mobile app is, what need it is addressing and what it
means in terms of value to the organisa on or to the pa ent / consumer? This is selling the idea internally and
may differ from a value proposi on used to sell the app to an external audience. Eg For paents with advanced
rheumatoid arthris the iDocstar is a mobile app used by clinicians to radically simplify the complex process of
paent assessment for biologic therapies, guide paents onto appropriate treatments, and save the
organisaon per annum.
PATIENT /ORGANISATIONAL BENEFIT
Expand on the pa ent / consumer / organisa onal benefits with suppor ng evidence where possible. If
appropriate, it should be linked to organisa onal priori es, na onal or regional drivers eg. Supports Clinical
Guidelines 50 (CG50) or project is aligned withthe NHS (1.4) and Public Health (4.6) Outcomes Framework.
Other factors may be per nent such as pa ent experience, pa ent safety, organisa onal efficiency, support s
clinical governance or beer clinical audit etc
RESOURCES & COSTS
Have you fully iden fied internal and external resources needed to support the development and deployment?
How will training be addressed? What about maintenance or updates. What are the risks? Does the
organisa on have the relevant infrastructure to deploy the solu on such as wifi connec vity, mobile device
and applica on management capability? Will the app need a regular refresh of clinical content? How will you
make longer term use of the app sustainable within the organisa on?

11
RISKS
How are you managing overall project risk? Do you have appropriate terms with a contracted developer? How
are paent safety risks being addressed. Are there any communicaon or other technical risks? How are you
addressing any data security needs? Does this app need to interact with exisng enterprise systems and how
will the app impact on this? Who is checking overall quality of the app?
COMMERCIALISATION STRATEGY
Is this being developed for internal use or is there a wider need in the market? Do you understand the market
need and size? What is the route to market? Can you protect your app? Who owns your app? Academic
instuons and NHS organisaons all have access to IP and innovaon specialists that can offer support with
the commercialisaon process.
INTELLECTUAL PROPERT Y CONSIDERATIONS
The development of mHealth apps should be treated in the same manner as any other soware developed or
commissioned by an NHS organisaon. In the UK, the intellectual property rights usually associated with
soware are copyright and database rights (in the code and database) and design rights (user interface and
images, icons etc). This could also include trademarks in instances where a brand or identy is being
established. Patenng of apps is more expensive and can be more challenging unless there is a clear “technical
character” associated with the app or app-based service.
Intellectual property (IP) created with an NHS organisaon by its staff will usually be owned by that
organisaon. Terms relang to this can be found within the organisaon’s policy for the management of
intellectual property.
NHS pares seeking to explore IP associated with the development or commissioning of apps are encouraged
to seek professional assistance at the earliest possible stage, and prior to commissioning of any development
work. A list of organizaons that can offer assistance are shown within the references secon.
REFERENCE COSTS
The cost of a mobile app varies depending on the complexity, funconality, mobile plaorm and the app
developer used. Total app costs need to take into account product development costs, any regulatory costs,
developer licenses with mobile plaorm operators, and any markeng costs. Some of these may be viewed as
one off costs. Ongoing costs may include maintenance and support, upgrades and other annual license fees.
Ballpark costs
Apps can be broken into four major groups, depending on the amount of work involved in developing them.
• Simple Apps - A simple app with perhaps up to 4 screens and serves one basic funcon can range from
£1,000, to £7,000. Simple apps don't store any data about the user or about the previous uses of the
app.
• API or Database Apps - app storing informaon on the mobile device or interacng with a remote
server increases complexity. Saving data on the device, device authencaon, or syncing data
between mulple devices would expect to cost between £5,000 and £30,000.
• Enterprise or mul-feature Apps – Typical business app. Users can access informaon using the app
via any device or web-browser. The app will possess several key features and the user interface will be

12
heavily customised and branded to the organisation or service for an immersive service. These apps
can cost £30,000 and upwards.
 Games – Not surprisingly the wide range of game apps can range in price from £7,000 to £200,000.
Android versus Apple versus Microsoft
Whilst there are several platform operators in the market including Microsoft and RIM, the majority of apps
being developed are focused on the Apple iOS and Google Android platforms. Unlike the iPhone and iPad,
Android runs on a multitude of third party devices each possessing a range of associated hardware components
and different versions of Android API. This allows developers to create apps for a varied audience but can
create challenges for app functionality due to different coding needs, depending on Android version and
device. The uptake of an app can also be maximised by developing across different platforms.
OpenSignalMaps (May 2012) estimated that there were almost 4,000 different Android devices on the market
with many different screen resolutions that apps needed to accommodate to provide the best possible user
display. They also noted that the two most popular Android versions made up 75% of the devices surveyed –
leaving a significant 25% of devices running out-of-date software to support apps. Depending on the nature of
the app and its dependency on the device hardware, reliability issues can be overcome by testing the app on
different phones. This can impact on both time and cost and probably reflects that the Android apps are
typically 2.5 times more expensive to purchase than iPhone and iPad apps (Canalys, Feb 2012).
Example Costs For Common Features
 Social Media Integration - This would include allowing users to Tweet and post to Facebook from
within the app, or sign in to your app using their Facebook account - £300 to £1,000
 In App Purchases - Apple lets you charge users for additional downloads and services from within the
app using In App Purchases- £1,000 to £3,000
Additional Costs To Consider
 iOS Developer account allows publishing to the Apple iTunes store via its validation process. Cost is an
annual fee -$99
 iOS Developer Enterprise account allows self publishing to in-house app store and distribution to
employees. Does not permit distribution to other parties and is an annual fee. $299
 Google Play Developer Console account supports development and distribution via Google play and is
a one off cost - $25
 Access to Microsoft App developer resources and publish via Microsoft app sites. Cost is an annual
fee. £19 for individuals, £65 for companies.
 Servers and back-end support. £100/month - If your app relies on a web server to store your user's
data then you should expect to pay around £100/month for server support and maintenance.
 Marketing. Building a great app is a good starting point but you need to market the app or service.
Marketing is essential before during and after launching an app and should be factored into any
business case. A number of NHS or healthcare communities, forums and networks offer a cost-
effective means of communicating your app to a target audience. £1,000 to £5,000

13
SUMMARY OF KEY POINTS
• Medical and lifestyle apps provide a valuable way of sharing informaon, working more efficiently and
supporng diagnosis, treatment and paent outcomes.
• A choice of app plaorms is available with many developers currently favouring the iOS plaorm as
this is deemed to be more secure than Android, but this in turn restricts the choice of hardware, and
potenally - access to the app.
• The cost of developing an app should take into account all app features, connecvity with third party
systems, any ongoing costs and an appropriate markeng budget to support wider commercialisaon.
• A business jusficaon case is a useful way to capture necessary informaon against which an
organisaon can make a go / no go decision on any required investment (me, effort, money) and
value or impact to the organisaon or delivery of patient care.

14
STAGE 2 - DESIGN AND DEVELOPMENT
STAGE 2 - DESIGN & DEVELOPMENT

STAGE 2 – DESIGN & DEVELOPMENT
Functionality? Data flows?
User & System requirements
Public Patient Involvement?
STAGE 2 – Design & Development USE CASES.
Commercialisation route? Data
SStageSTAGE 2 – & Device Security? Medical
Device? (see stage 5)
Functionality? Data flows?
Public Patient Involvement?
User & System requirements USE CASES. Commercialisation route? Data
Early
& Device functions
Security? /user interface
Medical /
Device? (see
Wireframe Development
website framework
stage 5)
Hardware Specifications?
Early functions /user interface / website

Wireframe Development Mocked up prototype screens
Screenshots framework
permitting user interaction
feedback
Mocked up prototype screens permitting user

Screenshots interaction feedback
API Usage, App Store
Code Generation
optimisation?
Code Generation Linking

API Usage, App mobile app into existing
Store optimisation?
System Integration & testing
or new ICT infrastructure. I
nteroperability?
Linking mobile app into existing or new ICT

System Integration & testing
infrastructure. Interoperability?
User Training
User Training Proceed to STAGE 3

Proceed to STAGE 3
An example of a simple design & development process is shown above highlighting key steps in this stage.
Many app developments are typically undertaken using an agile development process. This iterative process
offers a rapid and responsive approach to the development of the app and its functionality. The approach
undertaken will vary from supplier to supplier with many developers using tried and tested processes and a
range of development tools.

15
USER & SYSTEM REQUIREMENTS
App development starts with a clear understanding of what the medical app needs to do, for whom, and the
environment it needs to operate in. In some instances, internal ICT support within the NHS or academic
organisaon can develop the user and system requirements documentaon to support internal development
of the app or inform a tender specificaon. In almost all cases where the development is outsourced, the
external supplier or collaborator will seek to establish clear and unambiguous requirements early into the
project, or even in phases depending on the development methodology.
User Requirements Specificaon or User Requirements Document (URD): This document specifies what the
app really needs to do for the customer or end user and what the developer will provide. User Cases are oen
used to walk through the user steps or interacons with the proposed app. Construcon of the document
usually requires negoaon between the customer and the developer and takes into account: the customer’s
view of what is needed, what the app really needs to do from a soware engineering perspecve, and the
technical and economic feasibility of the proposed development. The URD will usually form part of a contact
between the pares and can inform a project plan highlighng key milestones, cost and mescale.
The Systems Requirements Specificaon seeks to idenfy understand and plan for the organisaon and user
impact of the proposed app and the manner in which it will integrate with other systems and the business
environment. It will ensure that any technical “shall do” requirements are properly integrated with the needs
of the business or user. This will include consideraon for funconal requirements such as data and device
security within the proposed system environment, data rules, app store requirements, user interface,
interoperability with exisng systems, and other needs such as medical device or clinical safety requirements
etc.
HARDWARE CONSIDERATIONS
Apps can be developed for smartphones, tablets and desktop devices. Key consideraons include the purpose
of the app and the intended use of the app and device. Smartphones have grown in popularity over the years
and have revoluonised the manner in which we provide services and informaon via a truly mobile and
portable soluon. Smartphone users tend to want informaon in a hurry. Desktop and tablet users will tend to
interact with their devices in a different manner. Enterprise level apps such as paent record or outpaent
clinic soluons may lend themselves to larger portable and desk based devices.
A key consideraon between the choice of devices such as smartphone versus tablet will be determined by the
hardware specificaon of the device. Clearly a tablet or a mini tablet will have a larger screen size than a
smartphone, which means that the app can be designed based on a bigger user interface providing more
funconality on the screen, or making navigaon easier. Form based apps for instance may benefit from
larger screens on tablets compared with a smartphone which would have to be designed differently to
opmise the user experience.
Device size does affect how users hold and interact with the device and this may change in different
environments such as use on a hospital ward versus use in a home.
Tablets are not necessarily more powerful that smartphones. Some of the lower budget tablets available
(providing potenal access to a wider app audience) offer larger screen size but may lack communicaon
capability (such as 3G/4G), significant processor power, or other funcons such as GPS or cameras now
rounely found on most smartphones.
Many apps developed for smartphones with limited screen size can also be configured for work on larger
compable tablet devices eg running iPhone apps on the iPad /iPad mini unless they are a form/control-based
app. This consideraon can be factored into the design stage of the app development.

16
Another factor for consideraon is the interoperability of the mobile device with exisng systems
infrastructure or even other desktop devices (a developing area of convergence that Microso in parcular
seems to be pursuing with vigour).
Adapon of the user interface is key to developing an app for a smartphone, tablet or desktop. This will help
improve usability, performance and the overall user experience.
USE OF API COMPONENT S
An API or Applicaon Programming Interface is a process, tool or roune that supports a specific task or
interacon with a specific part of the soware. Many social network apps and adversers use API to share
informaon in a seamless way; app user authencaon is also oen supported by an API roune connecng
device to remote server. The use of APIs provides useful building blocks for developers to create powerful
mobile apps.
The opening up of APIs is part of the UK government’s overall approach to open ICT and user centred digital
services and there is an increasing call for suppliers and vendors to open their APIs to third pares.
SECURITY AND COMMUNICATION CONSIDERATIONS
Depending on the use of the mobile app, data and app security should be a major consideraon in terms of
design and development of the app. Factors to consider include deciding whether informaon actually needs
to be stored on the app, encrypon of any data stored on the app and the way that the app interacts with the
wider network. Breaches of paent confidenality, conflicts of interests and malfunconing clinical decision -
making apps could all negavely impact on paent care.
The potenal for sensive data to leave an organisaon is exacerbated by mobile devices and wireless
networks, which, by their nature, allow access to healthcare systems and databases. Securing appropriate
access rights from different devices for different users by segregang data is a complex high risk task for any
organisaon.
A survey by HP suggests that more than 86% of apps tested from the app store showed a variety of security
vulnerabilies. Arxan, a data security company argue that a similar problem is associated with Android apps.
Android OS is currently viewed by a number of app developers and ICT companies as less secure than the Apple
iOS. For many of these organisaons, and depending on the type of app, they would not choose to develop an
Android version of the app using the current Android OS.
Samsung has been trying to address this by the development of a secure app soluon called Knox (as in Fort
Knox). This is available on selected Samsung mobile devices and will allow separaon of personal and work
acvies on the phone. This is of parcular use if an NHS organisaon operates a Bring Your Own Device
(BYOD) policy within the organisaon. In Knox mode there are only certain types of applicaons that can be
used. These range from standard Samsung apps to a set of “Samsung Knox” apps that are available for
download. Local administrators can also restrict certain device funcons (such as capturing a screen shot) and
sharing of informaon.
As mobile devices and their OS are evolving so rapidly aempts to undertake systemac research studies can
be superseded by subsequent releases with addional security features. Such a study has been aempted by
Jin Han and co-workers ( Han, J et al., 2013)
New guidance for app developers was launched by the Informaon Commissioner’s office in December 2013.
The document, “Privacy in mobile apps: guidance for apps developers” sets out clear guidance to assist
compliance with the Data Protecon Act 1998 (DPA) and ensure users’ privacy. The ICO also clearly states “an
organisaon based outside of the UK that develops apps for the UK market, should consider that its users in

17
the UK will clearly expect any apps they use to respect their privacy according to the DPA”. Further details can
be found at hp://ico.org.uk.
WIREFRAME DEVELOPMENT
Wireframes are rudimentary visual schemacs that represent the framework or layout of an app to accomplish
a parcular purpose or objecve. Lacking detail, the wireframes take into account screen funconality, content
layout, behaviour, and priority or sequencing of what needs to happen with the app. Basic wireframes can be
created as simple hand drawn sketches. Wireframes can be used in mocking up different scenarios for the way
in which the app may funcon.
SCREENSHOTS
Designers can use the output from the wireframe work to mock up sample screen shots. Unlike wireframes
these will give the first indicaon of what the user interface or app screens could look like. User feedback at
this early stage is really useful, parcularly with more complex apps, as it can give an early indicaon of overall
usability issues ( layout, colours, font size, navigaon etc) prior to coding.
CODE GENERATION
Establishment of clear requirements and user interface will allow the developer to proceed with early coding of
the app or focus on coding specific funconality components of the app such as the use of APIs to permit the
app to access or interact with elements of the mobile device eg camera. Consideraon can also be given at
this stage to coding of other elements (where applicable) such as Android app indexing as part of a digital
markeng approach.
SYSTEM INTEGRATION & TESTING
The systems analysis earlier in STAGE 2 will have idenfied the requirements associated with system
integraon within the idenfied business environment. The systems integraon step will bring the app
together with other components of the business environment to assess the overarching funconality of the
mobile app. Depending on the nature and complexity of the mobile app this will usually tested in a “sandpit”
or demo environment to test and iron out any glitches.

18

STAGE 3 – USER TESTING
User Testing
STAGE 3 – User Testing
STAGE 3 – USER
User Testing Refine Prototype
DECISION DEVELOPMENT
DEVELOPMENT
COMPLETE?
Refine Prototype COMPLETE?
NO YES
Development Development
Complete? NO Complete? YES
Back to STAGE 2 Proceed to STAGE 3
Back to STAGE 2 Proceed to STAGE 4
The user testing phase may be undertaken over several iterative cycles and will usually involve your preferred
target audience (this may include job role, age, gender, device and operating system). This phase can focus on
specific components within the app such as installing the app, or interaction with specific hardware
components. Alternatively it may focus on full app functionality. Issues (or the user experience) arising from
the testing phase can be fed back to STAGE 2 to iron out design or coding problems.
A number of dedicated third party user testing companies can also be used to independently assess the human
factors associated with a new mobile app eg usertesting.com or userlytics.com.
If the mobile app is deemed to be a medical device then data from the user testing phase can contribute to the
technical file required for CE marking (see STAGE 5).

19
STAGE44- -STAKEHOLDER
STAGE STAKEHOLDER REVIEW
REVIEW

STAGE 4 – STAKEHOLDER REVIEW
STAGE 4 – STAKEHOLDER REVIEW
Clinical Safety assessment?

Stakeholder Review / Stakeholder Review /
Systems review?
Clinical Safety assessment?
Clinical
Clinical Safety
validation assessment? Systems review?
Clinical Validation
Stakeholder Review / Security review? Security review?
Systems review?
Quality check? Quality check?
Clinical Va
V lidaon
Validaon Security review?
Quality check?
PASS REVIEWS DECISION PASS REVIEWS
NO YES
Pass Reviews? Pass Reviews?

NOReviews? YESReviews?
Pass
Pass
NO YES
Back
Back toto STAGE
STAGE 2 2 Internal Deployment
WithinDeployment
Internal organisation*
Back to STAGE 2
Within organisaon*
Internal Deployment
Medical Device? organisation*
within
Medical Device?
NO Device?
Medical YES Device?
Medical
NO YES
MEDICAL
Proceed to STAGE 6DECISION
DEVICE? MEDICAL
Proceed DEVICE?
to STAGE 5
NO YES
Proceed to STA
STAGE
T GE 6 Proceed to STAGE 5
*Does not require CE marking

Proceed to STAGE 6 Proceed to STAGE 5
at this
*Does notstage
require CE marking
at this stage
*Does not require CE marking at this stage
STAKEHOLDER REVIEW AND CLINICAL VALIDATION
STAKEHOLDER REVIEW AND CLINICAL VALIDATION

The completed app needs to be subjected to a thorough review by relevant stakeholders within (and possibly
outside) the organisation. The extent of the review(s) will be dependent on the functionality of the medical
The completed app needs to be subjected to a thorough review by relevant stakeholders within (and possibly
app and its intended use.
outside) the organisaon. The extent of the review(s) will be dependent on the funconality of the medical
app and its intended use.
RELEVANT TECHNOLOGY STANDARDS
RELEVANT TECHNOLOGY
Different types of healthcare STANDARDS
apps will require a different approach to manage risk in both the development
and use of the app. Mobile medical apps can pose the same risks of failure as other medical devices, including
Different types of healthcare apps will require a different approach to manage risk in both the development
mechanical failure, faulty design, poor manufacturing quality, and user error, among other safety issues. For
and use of the app. Mobile medical apps can pose the same risks of failure as other medical devices, including
mechanical failure, faulty design, poor manufacturing quality, and user error, among other safety issues. For

20
instance, a blood-glucose monitor that gives an incorrect reading, whether through a stand-alone glucometer
or an integrated mobile medical app on a cell phone, could result in significant harm to a diabec paent who
relies on that reading's accuracy. For this reason governing organisaons such as the FDA and MHRA have
either created or are in the process of creang technology standards.
In this secon we provide a high-level overview of the criteria for qualificaon focused on IEC 62304, however
when considering app development technology standards should be effecvely analysed and implemented. It
is also important to recognise that medical standards are an evolving aspect of app development that many
app developers are sll geng to grips with. This will be very much influenced by the type of healthcare app,
but should be taken into account by anybody seeking to engage the services of an app developer, or implement
the soware within their organisaon.
Standards for medical device design
Unl recently, safety regulaons for medical device soware, at least officially, were not exceponally
rigorous. In addion, soware was not formally classified as a medical product by the Medical Devices
Direcve. This has now changed. A new regime is in force governing all medical device soware development
for all classes of device.
Previous soware safety standards were best suited to medical devices with low levels of risk, as opposed to
products where soware failure could be extremely serious and result in death. As more electronic products
have become dependent on embedded soware, the focus has shied to the reliability of soware systems
within the devices and the associated risks at all levels of usage. As a result, the new EN/IEC 62304 standard
has emerged as a global benchmark for management of the soware development lifecycle.
Risk analysis for hardware and so ware design
Medical product designers have used risk management techniques to help reduce the risks associated with
device hardware. BS/EN/ISO 14971 has tradionally been adopted as the base standard for risk management
for medical devices. The current version of this standard is considerably extended from its previous version,
and the techniques described are now intended to be applied to both soware and hardware systems.
The approach that should be taken is to consider the risks posed by the medical device as a whole, before the
soware/hardware split is decided. Hardware risk analysis can then run alongside soware risk analysis to
define the required safety systems for the device.
IEC 62304 is a harmonised standard for soware design in medical products adopted by the European Union
and the United States. Because the standard is “harmonised,” medical device manufacturers adopng it will
sasfy the essenal requirements contained in Medical Devices Direcve 93/42/EEC (MDD) with amendment
M5 (2007/47/EC) as related to soware development. This is the least onerous route to ensuring compliance
with the MDD. US FDA will also accept ANSI/AAMI/IEC 62304:2006 as evidence that medical device soware
has been designed to an acceptable standard. This standard is idencal to the EN/ISO variant in all essenal
details.
Designing to IEC 62304 ensures that quality soware is produced by means of a defined and controlled process
of soware development. This process must contain a set of requirements based on the safety class of the
soware that is being developed.
CLINICAL RISK SAFETY STANDARDS
Another important consideraon is the protecon of paent safety. With the very rapid development of apps
in the health environment it has been claimed that there is a need for be¦er safety assurance. The Health and
Social Care Informaon Centre website describes the current clinical risk safety standards. It has also expressed
paent safety issues with regard to the way in which healthcare apps may be developed in the UK.
The Health and Social Care Act 2012 states that the following must have regard to an Informaon Standard
published under the Act:

21
• the Secretary of State for Health
• NHS England
• public bodies exercising functions in connection with health services or adult social care
• anyone providing publicly funded health services or adult social care commissioned by or on behalf of
public body
Therefore all NHS organisations and their staff developing mobile applications must manage app development
in accordance with relevant information standards.
Information standards underpin national healthcare initiatives from the Department of Health, NHS England,
the Care Quality Commission and other national health organisations. They provide the mechanism for
introducing requirements to which the NHS, those with whom it commissions services and its IT system
suppliers must conform.
The following two standards, relating to patient safety, have recently been revised and approved by the
Information Standards Board (ISB). In line with current ISB practice, each standard comprises of:
 a specification, which defines the requirements and conformance criteria to be met by the user of the
standard. How these requirements are met is the responsibility of the user.
 Implementation guidance, which provides an interpretation of the requirements and, where

appropriate, defines possible approaches to achieving them.
ISB 0129 Clinical Risk Management: its Application in the Manufacture of Health IT Systems
This standard sets clinical risk management requirements for manufacturers of health IT systems. It requires a
manufacturer to establish a framework within which clinical risks associated with the design and development
of a new health IT system, or the modification of an existing system, are properly managed.
The main activities defined within the standard are the:
 identification of any potential hazards the health IT system may present to patients
 assessment of the severity and likelihood of each hazard and hence the clinical risk
 evaluation of each clinical risk to determine whether it is acceptable against defined risk acceptability
criteria
 implementation of suitable clinical risk control measures to reduce or mitigate unacceptable clinical
risks.
In undertaking these activities, it is important that their outputs are clearly documented to provide evidence of
compliance. The main documents required are:
 Clinical Risk Management Plan, which defines the organisation’s approach to clinical risk management
for a particular project
 Hazard Log, a mechanism for recording and communicating the on-going identification and resolution
of hazards associated with a Health IT System

22
• Clinical Safety Case Report, a document produced at defined stages that demonstrates the safety of
the Health IT System, in as far as it is possible at that stage
• Safety Incident Management Log, which supports the communicaon and resoluon of raised
incidents.
The requirements contained in the standard are similar to those in the standards underpinning the Medical
Devices Direcve. The standard therefore provides a suitable interpretaon of the controls needed for health
IT systems which are not currently idenfied as medical devices and not covered under the Direcve.
It is recommended that health organisaons reference this standard in any procurement or contractual
documentaon issued to manufacturers. Compliance with this standard ensures that the Manufacturer has
implemented an effecve best pracce clinical safety programme during the design and build of a health IT
system.
ISB 0160 Clinical Risk Management: its Applicaon in the Deployment and Use of Health IT Systems
This standard requires a health organisaon to establish a framework within which the clinical risks associated
with the deployment and implementaon of a new or modified health IT system are properly managed. In this
respect, many of the requirements specified in ISB 0129 are replayed in this standard to ensure consistency in
approach.
The requirements combine to establish a Clinical Safety Management System which the health organisaon can
use to manage the safety of health IT systems throughout their lifecycle.
Addional informaon on standards can be found at the Health Developer Network (see secon: Organisaons
and Companies Offering Assistance)
SYSTEMS & SECURITY R EVIEW
A proporonate check of the use of the app and its interacon with other systems should be undertaken as
part of the review process to ensure that relevant stakeholders are sasfied. Data security relang to the use
of the medical app should also be checked at this stage. A small number of data concerns emerged from within
the medical community in both the UK and US.
QUALITY OF APPS AND VALIDATION
In an overcrowded market where a large proporon of apps are never downloaded it is always advisable to
validate an app idea with a target group of potenal early adopters before expending me and money in
building a technical soluon. By involving end users during the different phases of app development, it is more
likely that something is created which users are going to need and are more likely to adopt. Feedback is
valuable as it can define which features to include, and which are unnecessary to enable the developer to build
the app in a way that these features can be easily incorporated in future iteraons as opposed to having to
recode the app from the ground up every me something new needs to be added.
The quality of published healthcare apps varies. In many instances public app stores will check apps for
compliance with their plaorm standards but not queson the clinical or health content. Discussions with the
UK regulator indicate that they are oen powerless to act as the geographical jurisdicon associated with a
published app is oen unclear (MHRA, personal communicaon). The growing use of healthcare apps has
created increasing concern amongst many clinical commentators and professional bodies with regard to the
quality of many published apps. In 2013, app developers also started claiming that Apple was restricng the
publicaon of parcular types of medical apps. Anecdotal evidence suggests that parcular concerns for Apple
are those apps that offer drug dosage informaon independently of the medicine manufacturer.

23
Recent studies (Visanathan, A et al., 2012; Hamilton, AD and Brady, RR, 2012) have addressed the lack of
evidence and clinician involvement in the design and development of medical apps raising concerns about the
reliability and accuracy of their content. This has the poten al to have a nega ve impact on pa ents. It has
been proposed that medical experts should peer review the apps although the process is not without some
deficiencies. Other scien fic valida on methods for clinical apps are available and described in detail by
(Franko, O; 2012)
Valida on methods include:
• Criterion validity – comparison against a current gold standard to demonstrate a direct

correla on between the new tool and exis ng standard using an appropriate sta s cal
test.
• Construct validity - does the new tool do what it is supposed to do. The test aims to
demonstrate an appropriate response against a real-world measure.
• Intra-observer reliability – whether there is a highly-reproducible outcome when tested

under constant condi ons by the same observer.
• Inter-observer reliability - reflects the accuracy and precision of a tool when used by various
care providers.
Content analysis - the data within an app is compared to a reliable source, such as a gold
standard textbook or guideline.
If pa ent informa on is used in the valida on tests ethical and confiden ality issues must be addressed.
Further guidance can be established via local NHS R&D offices.
INTERNAL DEPLOYMENT WITHIN ORGANISATION
Subject to comple on of relevant reviews the medical app should be deemed safe and fit for purpose for wider
deployment within the origina ng organisa on. The dis nc on between internal deployment versus external
deployment is relevant for those medical apps that are deemed to be medical devices (see STAGE 5). Apps not
deemed to be a medical device can be deployed externally and registered on relevant app stores (see STAGE 6)
SUMMARY OF KEY POINT S
• The security of many apps remains a challenge for many healthcare organisa ons. A recent security
audit by one NHS Trust uncovered hidden security breaches associated with bespoke apps developed
for that Trust. Appropriate care should be taken with regard to security and the permissions that the
app will allow with the mobile device at the me of installa on.
• Appropriate standards relevant to the type of app should be applied during development to take into
account the level of clinical and pa ent risk.

24
STAGE55- -APPS
STAGE APPSASASMEDICAL
MEDICAL DEVICES
DEVICES
STAGE 5 – MEDICAL DEVICE PROCESS

STAGE 5 - MEDICAL DEVICE PROCESS*
The regulatory environment for standalone software is constantly evolving. The regulatory framework
governing medical apps in the UK is determined at European level primarily by the Medical Device Directive
93/42/EEC(MDD)8. Individual for
The regulatory environment countries withinsoware
standalone the European Community
is constantly transpose
evolving. the directive
The regulatory into national
framework
lawgoverning
which may result in country by country variations. In Europe, the revision of the Medical Devices
medical apps in the UK is determined at European level primarily by the Medical Device Direcve Directive,
which came into effect on
93/42/EEC(MDD)8. 21 March
Individual 2010, amended
countries theEuropean
within the definitionCommunity
of a medical device tothe
transpose include the reference
direcve into naonal
to standalone
law which may result in country by country variaons. In Europe, the revision of the Medical Devicesthe
software used for diagnostic and therapeutic purposes. This amendment simply clarified fact
Direcve,
thatwhich
thesecame
products were regarded
into effect as being
on 21 March 2010,medical
amendeddevices. Medical of
the definion apps are not device
a medical specifically mentioned
to include in the
the reference
Directive at this stage but NHS organisations are advised to seek professional advice before venturing
to standalone soware used for diagnosc and therapeuc purposes. This amendment simply clarified the fact ahead
with anythese
that non-regulatory
products were app regarded
development that medical
as being may fall devices.
within the remit apps
Medical of theare
MDD. The references
not specifically sectionin the
menoned
contains a list of organisations that can provide advice in this area.
Direcve at this stage but NHS organisaons are advised to seek professional advice before venturing ahead
with any non-regulatory app development that may fall within the remit of the MDD. The references secon
The Medicines and Healthcare Products Regulatory Agency (MHRA) is an executive agency of the Department
contains a list of organisaons that can provide advice in this area.
of Health and is deemed to be the competent authority for the UK. The MHRA has responsibilities for
interpreting and enforcing
The Medicines the relevant
and Healthcare UK legislation.
Products Under the
Regulatory Agency Medical
(MHRA) is anDevices Directive,
execuve agencymanufacturers
of the Department
whoof Health and is deemed to be the competent authority for the UK. The MHRA has responsibiliesauthority
wish to place a medical device on the market must first register the device with their competent for
andinterpreting
label the device with a CEthe
and enforcing mark.
relevant UK legislaon. Under the Medical Devices Direcve, manufacturers
who wish to place a medical device on the market must first register the device with their competent authority
WHEN IS AN
and label the APP
deviceAwith
MEDICAL DEVICE BASED ON MHRA GUIDANCE?
a CE mark.
Under
WHENclause
IS2(a)
ANofAPP
the Medical
A MED Devices Directive,
ICAL DEVICE a medical
BASED ONdevice
MHRA is defined as follows:
GUIDANCE?
 any instrument, apparatus, appliance, software, material or other article, whether used alone or in
Under clause 2(a) of the Medical Devices Direcve, a medical device is defined as follows:
combination, including the software intended by its manufacturer to be used specifically for diagnostic
• and/or therapeutic apparatus,
any instrument, purposes and necessary
appliance, for its proper
soware, application,
material intended
or other arcle, by theused
whether manufacturer
alone or into be
used for human beings
combinaon, forthe
including thesoware
purpose intended
of: by its manufacturer to be used specifically for diagnosc
and/or therapeuc purposes and necessary for its proper applicaon, intended by the manufacturer to be
 diagnosis, prevention, monitoring, treatment or alleviation of disease,
used for human beings for the purpose of:
 diagnosis, monitoring, treatment, alleviation of or compensation for an injury or handicap,
• diagnosis, prevenon, monitoring, treatment or alleviaon of disease,
 investigation, replacement or modification of the anatomy or of a physiological process,
• diagnosis, monitoring, treatment, alleviaon of or compensaon for an injury or handicap,
 Control of conception;
• invesgaon, replacement or modificaon of the anatomy or of a physiological process,
and which does not achieve its principal intended action in or on the human body by pharmacological,
• Control of concepon;
immunological or metabolic means, but which may be assisted in its function by such means;
and which does not achieve its principal intended acon in or on the human body by pharmacological,
Any mobile app that fits the criteria above could be deemed to be a medical device, although depending on the
immunological or metabolic means, but which may be assisted in its funcon by such means;
type of app this remains subject to interpretation in the UK. More recently the US regulatory body, FDA, has
issued
Any newer
mobileguidance
app that on
fitsits
theinterpretation
criteria aboveincould
this area (see section
be deemed to bebelow). Further
a medical information
device, is anticipated
although depending on the
from European
type regulators
of app this remainsinsubject
due course.
to interpretaon in the UK. More recently the US regulatory body, FDA, has
issued newer guidance on its interpretaon in this area (see secon below). Further informaon is ancipated
NHS organisations are advised to err on the side of caution if there is any uncertainty with regard to the status
from European regulators in due course.
of a medical app in development.
NHS organisaons are advised to err on the side of cauon if there is any uncertainty with regard to the status
What if the app is just being used within my organisation or purely for research?
of a medical app in development.
What if the app is just being used within my organisa

on or purely for research?

25
Current guidance allows NHS organisaons to be exempt from the Medicines Device Direcve and CE marking
if the soware is being used internally or strictly for research purposes (See MHRA Bullen 18, February 2011).
Other regulaons and governance mechanisms, parcularly those associated with the conduct of clinical trials
will come into force in such circumstances.
What if I want to distribute the app elsewhere and perhaps for free?
Once an NHS organisaon seeks to distribute this further afield (outside research) then it is subject to MDD
regulaon. This is irrespecve of whether the app is being made freely available at no cost to the end user.
The MHRA refer to a term called “placing on the market” in the context of distribuon outside an originang
organisaon.
• Placing on the market means in relaon to a device "the first making available in return for payment
or free of charge of a new or fully refurbished device other than a device intended for clinical
invesgaon, with a view to distribuon, use, or both, on the Community market".
Customer or Developer: who is responsible for compliance with the MDD?
Responsibility for compliance with the MDD rests with the Manufacturer of the device. NHS organisaons that
either develop a medical device in-house or commission such a device via a third party will be viewed as the
device manufacturer. The MHRA defines the manufacturer as follows:
• Manufacturer means " the person who is responsible for the design, manufacture, packaging and
labelling of a device before it is placed on the market under his own name, regardless of whether
these operaons are carried out by that person himself or on his behalf by a third party". The
obligaons of a manufacturer under these Regulaons also apply to any other person who assembles,
packages, processes, fully refurbishes or labels one or more ready-made products or assigns to them
their intended purpose as a device with a view to their being placed on the market under his own
name, apart from a person who assembles or adapts devices already on the market to their intended
purpose for an individual paent (see Regulaon 17).
OUTLINE OF MDD PROCE SS
The Medical Devices Direcve (MDD) has three provisions for medical devices:
• Classificaon of Medical Device
• Essenal Requirements
• Conformity Assessment Route

26
Classification of Medical Device
New App?
Medical devices are categorised into
classes according to the degree of risk
inherent in the device, medical
action of the device, duration of use,
invasive nature and if the device is
powered. Excludes pharmaceuticals.
Medical Device
Class I - generally regarded as low risk.
No ISO standard is required. No notified
body involved. Can self-certify
for CE marking.
NO DECISION YES For example a standalone clinical decision
support app interpreting clinical data,
which will influence clinical
intervention or interaction with the
patient. Eg Mersey Burns App.
Placed on Market
Class IIa - generally regarded as medium
risk. Notified body has to be used to
confirm compliance with
Directive.
For example the Withings Blood Pressure
NO DECISION YES App - direct physiological measurements
using smartphone enabled
blood pressure cuff or short term
indwelling urinary catheter
MDD does not apply to App Class IIb - generally regarded as medium
risk. Notified body has to be used to
confirm compliance with
Directive. More clinical data required as
REGULATORY PROCESS part of submission.
For example: Long term indwelling
Determine device class urinary catheter
Class III - generally regarded as high risk

For example: Pacemaker device
The specific class to which a given mobile

Conformity Assessment & app is assigned is dependent on the
Essential Requirements functionality of the app and the
(Typically an ER checklist) manner in which it interacts with or
poses a potential risk to the patient.
Further information regarding classes
see: http://ec.europa.eu/health/medical-
devices/documents/guidelines/index_
en.htm
CE Marking
CE Marked Medical App
Figure: Decision tree flowchart describing the application of the

Medical Device Directive to new healthcare apps

27
Essenal Requirements
Annex 1 of the MDD (ref 1993L0042 — EN— 11.10.2007— 005.001— 25) defines 13 “Essenal requirements”
for the design and manufacture of medical devices as the minimum essenal requirements to safeguard
paents, users and third pares. These requirements spulate that the principles of safety should be integral to
the design of the product and that the product should be suitable for its intended purpose. In all cases the
manufacturer will need to meet the essenal requirements. Annex 10 (Clinical Evaluaon), Annex 11 (Criteria
To Be Met For The Designaon of Nofied Bodies) and Annex – 12(CE Marking Of Conformity) may also be
important for the design of medical apps (Annex 12 is used by default ).
Amendments to Annex 1 will expand the list to nineteen essenal requirements. The new ER 14. relates to
soware requirements and has more detail than the current Direcve. Compliance with Internaonal S tandard
IEC 62304:2006, “Medical device soware – Soware life cycle processes,” is the Standard that will be
expected by Nofied Bodies (see Conformity Assessment Route below) as a reference for ER 14.
Many soware developers meet the essenal requirements through the use of an Essenal Requirements
Checklist.
“Nofied Bodies” are naonally accredited bodies that examine the conformity of the producon process
completed on behalf of the manufacturers and whose correctness is cerfied according to uniform assessment
factors.
Conformity Assessment Route
Medical device manufacturers are given a choice of routes to meet the MDD requirements; the class aributed
to the device will dictate the path that must be followed. The conformity procedures focus on the design and
stages of manufacture.
Manufacturers must provide objecve evidence of how the design of the device meets the essenal
requirements via a technical file. A documented quality system must be in place to ensure that the device
continues to comply with the essenal manufacturing requirements and is consistent with the informaon in
the technical file.
Routes of conformity assessment are idenfied in annexes II, III, IV, V, VI, VII and VIII. Many of these MDD
annexes contain requirements that the manufacturer's quality system be assessed and be in compliance with
the applicable standard, and that those assessments be conducted by a nofied body (or through an
agreement with another organizaon) authorized to perform MDD assessments.
FDA GUIDANCE
More recently, in September 2013, the FDA issued its final recommendaons (currently non-binding) relang
to mobile medical applicaons. A number of other regulatory bodies including the MHRA are likely to follow
with similar guidance or regulaons in due course. The forthcoming FDA regulaons will be applied to “only
those mobile apps which are medical devices and whose funconality could pose a risk to a paent’s safety if
the mobile app were to not funcon as intended” and these have been termed mobile medical Apps. These
apps are defined as those that meet the FDA definion of device and either intended as:
• to be used as an accessory to a regulated medical device;
• or to transform a mobile plaorm into a regulated medical device.
“Soware applicaons that run on a desktop computer, laptop computer, remotely on a website or “cloud,” or
on a handheld computer may be subject to FDA device regulaon if they are intended for use in the diagnosis
or the cure, migaon, treatment, or prevenon of disease, or to affect the structure or any funcon of the
body of man”
All mobile medical app manufacturers should follow a quality system regulaon (which includes good
manufacturing pracces) in the design and development of their mobile medical apps and iniate prompt
correcons, when appropriate, to prevent paent and user harm.

28
For mobile medical apps, manufacturers must meet the requirements associated with the applicable device
classificaon. If the mobile medical app, on its own, falls within a medical device classificaon, its manufacturer
is subject to the requirements associated with that classificaon.
Who is a Mobile Medical App manufacturer?
A mobile medical app manufacturer may include anyone who:
• Creates, designs, develops, labels, re-labels, remanufactures, modifies, or creates a mobile medical
app soware system from mulple components. This could include a person or enty that creates a
mobile medical app by using commercial off the shelf (COTS) soware components and markets the
product to perform as a mobile medical app;
• Iniates specificaons or requirements for mobile medical apps or procures product

development/manufacturing services from other individuals or enes (second party) for subsequent
commercial distribuon. For example, when a “developer” (i.e., an enty that provides engineering,
design, and development services) creates a mobile medical app from the specificaons that were
iniated by the “author,” the “author” who iniated and developed specificaons for the mobile
medical app is considered a “manufacturer” of the mobile medical app.
The FDA notes that soware “developers” of a mobile medical app that are only responsible for performi ng
design and development acvies to transform the author’s specificaons into a mobile medical app would
not constute manufacturers, and instead the author would be considered the manufacturer
Who is not a Mobile Medical App manufacturer?
• Manufacturers or distributors of mobile plaorms who solely distribute or market their plaorm and
do not intend (by markeng claims - eg labelling claims or adversing material) the plaorm to be
used for medical device funcons.
• Third pares who solely provide market access to mobile medical apps.
• Providers of tools, services or infrastructure used in the development, distribuon, or use of a mobile
medical app.
• [Healthcare staff], who manufacture a mobile medical app or alter a mobile medical app solely for use
in their professional pracce and do not label or promote their mobile medical apps outside that
organisaon.
• Persons who manufacture mobile medical apps solely for use in research, teaching, or analysis and do
not introduce such devices into commercial distribuon (covered by other governance protocols and
regulaons).
As with all medical devices, manufacturers are legally responsible for any adverse paent outcomes associated
with the use of their device or app. NHS organisaons should seek guidance from the NHS Ligaon Authority
(NHSLA) to establish their posion with regard to product liability cover based on their compliance with any
NHSLA risk management standards and internal policies.
SUMMARY OF KEY POINT S
• The MHRA issued fresh guidance on medical device stand-alone soware (including apps) in March 2014.
Further details can be found via the references secon (other sources).
• NHS organisaons should give careful consideraon to the MDD and establish early in development
whether this is a requirement for wider adopon. No NHS organisaon wants to be subject to
unnecessary ligaon or be the first party to establish case law for the consequenal use of an
unregulated mobile medical app

29
 NHS organisations developing healthcare apps that fit the criteria of the Medical Device Directive will
be deemed to be the manufacturer even if the commissioned development work is outsourced to a
third party software developer.
 NHS organizations not familiar with the MDD process should seek advice, particularly with regard to
the appropriate conformity assessment route for their medical device or app.
 CE Mark is not the same as FDA approval. CE marking confirms safety not effectiveness of a product.
 The technical file developed for UK CE marking may contribute in part to an FDA submission but it is
likely that US-derived data will be required for any FDA submission

30
STAGE 6 - APP DEPLOYMENT CONSIDERATIONS
STAGE 6 - EXTERNAL DEPLOYMENT
APP STORES
As the number of apps created have increased exponenally it has become more difficult for customers to
locate relevant apps and for developers to bring apps to the aenon of consumers. Generic and medical app
stores have proliferated as depositories/collecons where it is possible to browse available apps, read reviews,
purchase them if a charge is payable and automacally download them on to the relevant device. The apps are
normally categorised by subject and genres. Many applicaon stores are curated and regulated by their
owners, requiring that submissions go through an approval process where applicaons are inspected for
compliance with certain guidelines (such as those for quality and content), and also require that a commission
be collected on each sale of a paid applicaon. (Apple retain 30%).
Apple launched its app store in 2008 and it allowed users to purchase and download new apps for their device
through either the App Store on the device, or through the iTunes Store on the iTunes desktop soware. To
have an app accepted in the Apple app store, app developers will require the originang organisaon to hold a
“developers license agreement” with the relevant plaorm operator. Developer licence allows development of
code which then needs to be submied to Apple for approval before publishing on iTunes.
Apple offer several types of licence and charge an annual fee, Google play and Microso charge a one off fee
and operate a simpler structure.
Apple
hps://itunes.apple.com
hps://developer.apple.com/programs/ios/enterprise/ for internal self publishing
hps://developer.apple.com/programs/ for distribuon via the apple app store
hps://developer.apple.com/library/ios/documentaon/LanguagesUlies/Conceptual/iTunesConnect_Guide
/1_Introducon/Introducon.html
The Apple app store is starng to take a tougher approach? towards healthcare apps. A number of app
developers have cited examples of new medical apps that have been rejected due to lack of supporng
evidence in relaon to the source of medical informaon contained in the app.
Google
Google Play, formerly the Android Market, is a digital distribuon plaorm for applicaons for the Android
operang system and an online electronics and digital media store, operated by Google.
Google play
hps://play.google.com/store/apps?hl=en_GB
Microso Windows (requires a windows account)
hp://msdn.microso.com/library/windows/apps/jj714071 there is a separate windows phone app site but

these are likely to be merged in Spring 2014.

31
hp://blogs.windows.com/windows/b/appbuilder/archive/2013/11/06/unifying -developer-registraon-
windows-and-windows-phone.aspx
Blackberry
hp://appworld.blackberry.com
As the number of apps has increased, so has the need for categorisaon to help users find relevant apps. On
both the Apple and Android plaorms there is a disnct category for “Medical” which can be interpreted as
aimed at health care professionals, while “Healthcare & Fitness” is oen used to describe apps for use by
paents and the general public. It is important to stress that the allocaon of apps to categories is not closely
curated – developers may submit their app categorised as they feel appropriate, and may list the same app
under mulple categories.
iMedicalApps is an app store for medical professionals, paents, and analysts interested in mobile medical
technology and health care apps. Reviews, research, and commentary of mobile medical technology are
wrien by a team of physicians, allied health professionals, medical trainees, and mHealth analysts.
NHS CHOICES APP STORE
In March 2012 NHS England launched the NHS Health Apps Library, a collecon of apps that had been reviewed
by the NHS to ensure they were clinically safe. The library is a way of offering an NHS stamp of approval to
apps so users know which ones are safe.
The apps currently included in the library are aimed at public use rather than clinicians but there are plans to
expand the library to include healthcare praconer apps in the future. Developers can submit their apps for
review and inclusion online. All apps submied to the Health Apps Library are checked to make sure that they
are relevant to people living in England, comply with data protecon laws and ulise trusted sources of
informaon, such as NHS Choices.
Once an app has met these minimum requirements, it is then checked to see whether the app could
potenally cause harm to a person’s health or condion, for example, is the app limited to providing
informaon from a trusted source – or might it go on to provide personalised medical recommendaons or
treatment opons? If so then potenally there could be harm caused by improper use of the app. The clinical
assurance team – which is made up of doctors, nurses and safety specialists, work with the developer to make
sure the app adheres to the safety standards. During this process any potenal safety concerns are idenfied
and either designed out or dealt with so that any remaining risk is at an acceptable level, thereby reducing the
risk of ligaon associated with the use of the app.
hp://apps.nhs.uk/
NHS TRUST DEDICATED APP STORES
Another recent development has been the creaon of NHS Trust dedicated app depositories, oen called app
stores. These depositories enable the NHS Trust to have complete control over apps approved for usage.
Having this structure in place benefits the Trust by safely distributing mobile apps instantly to clinicians
fostering a mobile culture with high adopon, the assurance that apps being used have gone through a
validaon process and also the support of employee-owned devices. Typically this requires mobile applicaon
management (MAM) system to manage the use of apps, security control, updates etc. Many of these are now
combined with mobile device management (MDM) soluons that allow enterprise management of mobile
devices within an NHS environment.
OTHER ROUTES TO M ARKET

32
In addion to the generic app store there are a number of credible online distribuon pla
orms for mobile
apps.
In Sometoofthe
addion these are specific
generic to athere
app store device
aremanufacturer, some such
a number of credible as Happque
online distribuonare focusedfor
pla
orms on mobile
healthcare
apps.
apps. Some of these are specific to a device manufacturer, some such as Happque are focused on healthcare
apps.
hp://www.happque.com/
In addion to the generic app store there are a number of credible online distribuon pla
orms for mobile
apps. Some of these are specific to a device manufacturer, some such as Happque are focused on healthcare
hp://apps.samsung.com
apps.
hp://www.handango.com/Home.jsp?siteId=2218
hp://www.amazon.co.uk/
The pace of app development targeng NHS services is growing at an exponenal rate. In some instances
these apps
The pace ofare
jusficaon to
appaddressing verytargeng
development
commit funding
specific paent
and
or organisaonal
NHS services
resources
needs,
is growing at and that inrate.
an exponenal itselfInis some
sufficient
instances
STAGE
these 7 are
apps - MONETARISATION
addressing very specificOF APPSor organisaonal needs, and that in itself is sufficient apps.
to
paent the acvity. NISE refers to such apps as “no-brainer”
jusficaon to commit funding and resources to the acvity. NISE refers to such apps as “no-brainer” apps.
“NO BRAINER”
The pace APP
of app development targeng NHS services is growing at an exponenal rate. In some instances
“NO BRAINER”
these APP very specific paent or organisaonal needs, and that in itself is sufficient
apps are addressing
This type of healthcare
jusficaon app canand
to commit funding be typically
resourcescharacterised by many
to the acvity. of the to
NISE refers following traits:
such apps as “no-brainer” apps.
This type of healthcare app can be typically characterised by many of the following traits:
• No immediate need to exploit outside host organisaons?
“NO BRAINER” APP
•• No clear income
No immediate generaon
need to exploit butoutside
may offer
hostcost savings
organisaons?
• App
This •typeNo needed
of healthcareto
clear income support:
appgeneraon but may
can be typically offer cost savings
characterised by many of the following traits:
• o Exisng service
App needed to support: or capability
• o
o New
No immediate service
Exisngneed toorexploit
service capability
outside host organisaons?
or capability
• Addresses
No clear organisaonal
income generaon objecves
o New service or capability but mayoroffer
metrics
cost savings
•• Improves
App needed
Addresses communicaon
to support: objecves or metrics
organisaonal
•• Improves
o delivery
Exisng of paent
service
Improves communicaon care:
or capability
• o New
Improves Paent experience
service
delivery or capability
of paent care:
• o
Addresses Paent safety
organisaonal
o Paent experience objecves or metrics
• o
o Quality
Improves communicaon
Paent safety
• App value
Improves proposion
delivery
o Quality is usually
of paent well understood by stakeholders based on a shared understanding of
care:
• need andPaent
App ovalue benefit to theis NHS,
experience
proposion local
usually health
well economy
understood by or paent carebased on a shared understanding of
stakeholders
needoandPaent
benefitsafety
to the NHS, local health economy or paent care
The “no-brainer” scenario assumes that value for money is favoured by improved and idenfiable benefits over
o Quality
any associated
The costs.
“no-brainer” scenario assumes that value for money isbyfavoured by improved and idenfiable benefits over
• App value proposion is usually well understood stakeholders based on a shared understanding of
any associated costs.
need and benefit to the NHS, local health economy or paent care
APP SUSTAINABILITY
APP SUSTAINABILITY
The “no-brainer” scenario assumes that value for money is favoured by improved and idenfiable benefits over
Depending on the
any associated type, nature and use of a healthcare app, there is oen a consideraon with regard to
costs.
funding theon
Depending creaon, implementaon
the type, nature and useand
of aongoing maintenance
healthcare app, thereofisthe app.
oen App development
a consideraon and its to
with regard
deployment come
APP SUSTAINABILITY
funding at a cost. In many cases the development of an app requires a clear
the creaon, implementaon and ongoing maintenance of the app. App development and funding strategy.
its
deployment come at a cost. In many cases the development of an app requires a clear funding strategy.
Some NHS organisaons have resorted to charity or industry sponsorship of their app. This makes sense where
Depending on the type, nature and use of a healthcare app, there is oen a consideraon with regard to
the app
Some may be targenghave
NHS a parcular
resortedhealth condion or paent group that is aligned
app. with
This the remit of awhere
funding theorganisaons
creaon, implementaon to charity
and ongoingormaintenance
industry sponsorship of their
of the app. App development makes
andsense
its
medical
the app charity
may be eg diabetes
targeng a app funded
parcular via
healtha diabetes
condion charity.
or paentPrivate
group organisaons
that is alignedsuch as
with pharmaceucal
the remit of a
deployment come at a cost. In many cases the development of an app requires a clear funding strategy.
companies may also be willing to sponsor apps aligned with parcular condions, or apps that
medical charity eg diabetes app funded via a diabetes charity. Private organisaons such as pharmaceucalwill have a clear
and
Someposive societal
NHS organisaons
companies impact. However,
have resorted
may also be willing the challenge
to apps
to sponsor charity with this
or industry
aligned approach
sponsorship
with parcular is the vast number
of their app.
condions, of
Thisthat
or apps healthcare
makes
will sense apps
have awhere
clear
the app
and may societal
posive be targeng a parcular
impact. However, health condionwith
the challenge or paent group that
this approach is aligned
is the with the
vast number remit of a apps
of healthcare
medical charity eg diabetes app funded via a diabetes charity. Private organisaons such as pharmaceucal
companies may also be willing to sponsor apps aligned with parcular condions, or apps that will have a clear
and posive societal impact. However, the challenge with this approach is the vast number of healthcare apps

33
now being generated. This will almost certainly increase the app funding requests made to charies and
industrial sponsors; which means more compeon and perhaps fewer funding opportunies.
In many other circumstances there may be a wider potenal for use of the app, and there may be a
requirement to pursue a commercial route as a means of generang income and / or offering some financial
sustainability to the longer term use of the app. Depending on the nature of the app and the manner in which
it interacts with the mobile device, there may be a requirement to update the release version of the app or
add addional funconality or content. This will be a cost consideraon above the original development and
deployment costs associated with the app. Digital markeng is another area that is frequently overlooked by
NHS organisaons pushing a healthcare app into the open market. If you are not dealing with the non-brainer
app and seeking internal funding within your organisaon to develop the app - how are you addressing app
sustainability?
Like other forms of commercially exploited soware, potenal income can be derived from:
• The app product,

• Provision of a delivery service in conjuncon with the use of the app -eg development of a telereferral
service in conjuncon with a mobile app for community based monitoring or reporng of a chronic
health problem.
• Provision of training and technical support associated with the app. (typically the domain of the
soware developer).
• A “hybrid” business model
The following secon will focus on commercialisaon models associated with the app product.
DIFFERENT APPROACHES TO THE MONETARISATION OF APPS
Consumers are increasingly reluctant to pay for apps, parcularly when so many apps are available and appear
to be free. Where payments are made, consumers are also condioned by the market to pay typical mobile
app prices for such soware (ranging from 99p to £5). This has created a challenge for developers of apps who
are seeking to cover the cost of app development and maintenance.
There is no single approach that can be used to commercially exploit this technology. A number of factors can
determine the most effecve or appropriate route to market. These include:
• Type and nature of app

• Intended audience or end user
• Risk profile
• Sustainability and income need
• Organisaonal fit / bias
Current app models largely developed for the gaming and entertainment sectors tend to be characterised
based on free, freemium and paid.
FREE APPS
Gartner (2013) noted that 91% of all apps downloaded worldwide in 2013 will be free. Current commercial
models for the majority of “free” apps are focused on app adversing or in-app purchases. Some free apps are
also associated with paid for health devices such as the Withings blood pressure monitor which can offer
alternave income streams.

34
What do we mean by app advertising?
App advertising is most commonly associated with the in-app advertising of third party products or services,
usually via banner adverts on the user screen or whole page splashscreens (typically during the initial start-up
of an app). Other forms of app advertising involve the use of push messaging to send relevant information to
existing users of the app. For instance this approach may advertise new related apps or new functionality ie
migrate the customer to some form of paid app or service.
The mobile app market is growing at an exponential rate and has paved the way for highly segmented and
target oriented advertising. Advertisers will spend a lot for customer engagement with minimal effort. At
present mobile applications are basically limited to static banner ads, mostly of dimensions 350×50. In some
instances, apps which are strategized to be initially free and later followed by download charges register less
than 2% conversion rates; most of the income comes through advertising.
Mobile app advertising revolves around advertising companies such as iAds or AdMob which monitor and drive
the ads featured. iAds monitors app ads for iOS devices while AdMob features ads for Android devices.
Registration with these companies is required if an app is to generate income through running advertisements.
Deploying a free app will bring potential earnings from the advertisements that appear at regular intervals on
the application. Depending upon the ad company’s revenue model, the charges are collected from the
advertiser and then shared between the ad company and the app owner. It’s been shown that a free
application is likely to get downloaded more often thus creating greater earning potential through
advertisements than paid apps.
Many developers track the income stream from their own apps. There is a burgeoning industry in developing
software that tracks the number of downloads and income generated from an app. NHS organisations holding
app developer licences with the platform owners such as Apple or Google can also generate reports to track
downloads and income generation.
Risks
There is however potential risks associated with the advertising model that needs to be considered and
managed. A report from researchers at North Carolina State University (March 2012) found that more than
half of apps in the Google Play store contained “ad libraries”. Some of the more aggressive ad libraries could
download and run code from remote servers with impunity, raising security and privacy concerns. These ad
libraries received their permission from when the user first installs the app with the user often unaware of
granting such permission. (see Security References: cellular-news.com). There is also some unease about the
lack of control that NHS organisation may have using one or more intermediates to target adverts toward
particular users, and more importantly ensure that certain users groups are not targeted with inappropriate
adverts eg fast food adverts associated with a weight control app.
NHS organisations are also expected to conform to existing guidelines associated with sponsorship and
advertising space such as that specified with the NHS brand guidelines (see references).
Pay-per-click advertising
The pay-per-click advertising model is very similar to Google Adsense. There are several Adsense-like
advertising companies which can be used to embed advertisements into an application:
•Admob
•Brightroll
•Google Adsense

35
•GreyStripe
•iAd
•InMobi
•Jumptap
•MdotM
•Millennial Media
•MobClix
•ZestADZ
Code from these ad companies is placed into the application during the development process. After the app
goes live and is downloaded, it requests to show an advertisement and then places that ad onto the screen.
Google finally introduced AdMob to Windows Phone 8 in October 2013 AdMob will provide a compelling
alternative to Microsoft’s own pubCenter advertising platform.
Not all of the advertising companies are the same. Some have different pay outs, different advertisements,
different conversion rates, and all have different fill rates. A fill rate is the ratio of how many times an
application requests to show an application and how many times it actually shows up. It is never 100%. This
means, sometimes ads aren’t shown when they are supposed to, which obviously affects how much revenue is
generated. (Note: this is why it’s important to design free applications so that the ads show on top of the
screen, instead of having a dedicated blacked out space for it. When a request goes out and it doesn’t fill, you
don’t want a large black rectangle to show up on the screen.) To overcome the fill rate issue, selecting one
advertising platform that has the highest pay outs but also has a high fill rate is a dilemma. This can be resolved
with a mediation service such as AdMob Mediation. Adwhirl was formerly a frequently used mediation solution
for developers but it was taken over and “retired” by Google in September 2013 although its open source code
will continue to be available on Google Code.
With a “mediation solution” an app developer can sign up for as many ad agencies as desired and the
mediation solution should ensure that there is a 100% fill rate. If an ad request goes through for one agency
and it doesn’t fill, the mediation solution will immediately go to the next agency and find an ad that is available.
There are a number of benefits of developing free apps:
 Income can be generated if the smartphone owner inadvertently hits one of the ads. Some users
however find the banner ads annoying and intrusive.
 There are more opportunities to make money from each person who downloads the app than with a
paid app.
 With a paid app, you only make money on the initial purchase, and that’s it (unless you have in-app
purchases). With free apps, those advertisements are constantly being shown every time a user loads
your application.
So what would a typical NHS view be on the use of adverts in mobile apps?
In its 2014 survey NISE asked NHS staff at a large acute NHS Trust for their views on apps. 76% of respondents
agreed that carefully positioned and targeted adverts were an acceptable compromise if it meant the mobile
app was free. Broader views in the table below show a mixed response to the use of adverts.

36
Neither
Strongly Strongly
Agree Agree or Disagree
Agree Disagree
Disagree
I am not bothered by the use of ads 4.4% 20.4% 19.9% 31.9% 23.5%
within mobile apps
I tolerate the use of ads within 6.1% 47.8% 21.5% 18.0% 6.6%
mobile apps
I do not like the use of ads within 28.5% 37.7% 21.1% 10.5% 2.2%
mobile apps
I like seeing relevant ads on my 0.4% 15.0% 25.1% 33.5% 26.0%
favourite apps
I tolerate ads when the app is 2.6% 45.2% 25.4% 18.0% 8.8%
opening up
I refuse to use or download any 7.1% 13.3% 31.9% 39.8% 8.0%
app that displays ads
I always pay for apps if it means 3.1% 12.4% 39.4% 36.3% 8.8%
that I avoid any ads
It was interes ng to note from the NISE survey that staff were asked.
Ques on: Should NHS apps carry adverts?
NHS Staff Apps vs Paent Apps
67.20% YES 48.50%

32.80% NO 51.50%
Sample comments from NHS staff included:
"Should not compromise the impar ality of NHS services"
"If you are going to use adverts then it needs to be for stuff unrelated to health ie electronics etc."
"Would be nice if adverts for items promo ng health, such as healthy food"
In summary there is a an expecta on that many healthcare apps may need to contain some form of in-app
adverts, although the majority of NHS staff as consumers would rather not have the adverts present. There is a
clear dis nc on between views rela ng to app adverts depending on whether the app is used by NHS staff or
their pa ents. In terms of a mone sa on approach the use of in-app adverts is a possibility depending on the
nature of the app, the targeted user and the ability to control the nature of adverts presented to the user
screen.
FREEMIUM AND IN-APP PURCHASING
Free to download

37
Limited funconality but unlock for more features, funconality or virtual goods
An amalgamaon of the words "free" and "premium" that refers to services, soware programs or mobile
apps that are offered to users free of charge, but typically with limited funconality, adverser support or
addional features that are only available for a premium charge. The freemium software business model
originated with shareware soware, where users are able to download soware and try it free of charge but
only for a limited me or with a restricted feature-set.
The freemium soware model has become extremely popular recently with the debut of freemium apps and
in-app purchases in the Apple App Store following the release of iOS 3.0 onwards. Freemium mobile apps are
available for download on iOS-powered devices like the iPad and iPhone without charge, but users typically
have the opon to pay for addional features.
Freemium apps now gross more than paid apps in the Apple App Store, with esmates of more than 65 percent
of all revenue generated in the App Store from in-app purchases for freemium apps.
Although we were unable to locate any specific stascs on freemium esmates on health related apps, the
common theme in arcles reviewed were discussing the games market, suggesng that this is the predominant
market. That is not to say that there are no freemium health apps just that the games market appears to offer
developers the greatest chance of a strong financial return.
The NHS survey (2014) undertaken by NISE suggests that only 27% of NHS staff have ever made an in-app
purchase with any of their downloaded apps. Respondents were asked their views on in-app purchasing as
shown below. The feedback offers a mixed view on atudes towards in -app purchasing
Neither
Strongly Strongly
Agree Agree or Disagree
Agree Disagree
Disagree
I like the idea of "paying as I go" 9.2% 25.9% 29.7% 19.7% 15.5%
I don't like in-app purchasing 29.6% 26.7% 30.8% 10.8% 2.1%
In-app purchasing is a reasonable
way to pay for addi onal material 2.1% 25.4% 34.7% 24.6% 13.1%
or informa on
If I download an app, I want to be
able to use all that the app offers 58.1% 27.8% 11.2% 2.5% 0.4%
without any further purchase.
In-app purchasing does not make it
easy for me to keep track of the 35.6% 32.6% 25.5% 5.9% 0.4%
"cost of the app"
If I buy an app I want to be able to
use all that the app offers without 59.2% 27.2% 11.8% 1.3% 0.0%
any further in-app purchase
Considera on should also be made to the differing market dynamics. For example, in the United States, where
it is widely accepted that healthcare is paid for by individuals the atude to buying a health app may be
different to that of the UK public where healthcare is free at the point of delivery.

38
PAID FOR APP
A small number of apps (par

cularly within the gaming sector) charge on a pay to download basis Some offer
full func
onality for a modest price with the business model dictated by volume of sales via an app store rather
than app price. The store will however take a percentage of the app price (Apple takes 30%).
The Apple app store encourages paid apps with top sugges
ons in their lis
ng layout and free ones at the
boom of the page. This is due to the fact that the app store directly benefits from each download made for a
paid app. Apple app store works on a revenue sharing model that allots 70% to the mobile app
developer/owner and the rest for itself.
Probably the biggest benefit of going with a paid applica

on is the income poten
al. Although it is possible to
make money with free apps, the likelihood of genera
ng huge traffic with money from adver
sement clicks is
unpredictable. Also, the app store seems to favour paid apps more than free apps. This makes sense, because
Apple doesn’t get any money from distribu
ng free applica
ons (except those that incorporate Apple’s
adver
sing plaorm, iAd).
Exposure through an app store is essen

al in order for an app to do well. This means geng onto the top 200
or top 100 or top 10 lists, geng featured on “New & Noteworthy” and “What’s Hot”, and geng exposure as
a new app in the “All [Category] iPhone Apps” space, which highlights the latest applica
ons approved in that
category, and all of this is much easier with a paid applica
on.
Many vendors combine this approach with op

onal in-app purchasing.
Some app experts suggest that it’s the confidence over the app that leads to a paid deployment. If the app is
good it will not have to depend on adver
sements to generate revenue, instead it will find its customers and
generate revenue with every download.
A small survey undertaken by NISE in 2014 asked NHS staff for their generic view on paying for medical apps.
The results are shown below.
Queson: How much have you paid for a medical app?
PRICE
3.6 2.7 3.6 0.9

Less than £1
9.1
More than £1
More than £5
27.3 60
More than £10
More than £15
More than £20
More than £50
Sixty percent of responders had spent less than £1. Twenty seven percent had spent more than £1 but less
than £5 which is representa
ve of the manner in which consumers are currently condi
oned to pay for mobile
apps. NHS staff were also asked

39
Queson: What is the most you would be willing to spend on a medical app?
The general perspecve was slightly more favourable with thirty four percent willing to pay a £1, thirty six
percent willing to pay £5 and nineteen percent willing to pay £10.
PRICE
3.6
3.6
4.5
33.6 £1
19.1 £5
£10
£15
35.5 £20
£50
This broad response does not take into account the value that a specific app may offer a user in terms of
funconality and user experience. However, NISE was encouraged that NHS staff appear to be willing in
principle to perhaps pay a bit more for a medical app. This does not necessarily mean that the pay to
download approach by itself would be enough to make the app development sustainable.
THE APP PACKAGE
App packaging provides the opportunity to embed or integrate a healthcare app into a broader service or
soluon. Typical scenarios for the app package include:
Roune regular service provision
This involves the provision of a scienfic, clinical or care service in return for a contracted fee such as an
annual soware or service licence. Examples of this are rounely found in the soware or cloud -compung
space using models such as Soware as a service (SaaS). For the end user this can have the aracon of
reduced hosng or capital costs whilst benefing from a managed service. In the context of the app, the app
becomes an enabling technology within the wider valued service.
Pay as you go service provision
Similar to the roune subscribed service provision, the pay as you go service model allows users to access an
app based service when required without being locked into a periodic or annual contract. This is also an
example of where a hybrid commercial model could potenally be used; the user accessing the service via an
in-app purchase on the phone. The pay as you go service would be equally relevant for business to business
transactions, for instance a healthcare professional purchasing laboratory analysis via a lab analysis tool.
Integraon with other technologies eg medical device
A number of companies have developed mobile apps to monitor or control other devices. This creates a
different monesaon model that leverages not just the app but also the associated device in the market. A

40
good example of this approach is the Withings blood pressure cuff which can be purchased and controlled (by
wire or wireless) via a blood pressure monitoring app on an iPhone or Android device
(hp://www.withings.com/en/wirelessbloodpressuremonitor). Such approaches potenally allow the return
on investment associated with the app to be offset by the fees associated with the medical device.
CODE TRADING
As the name implies, code trading is based on the principle that the source code or source code component
within a mobile app holds some form of intrinsic value. Like other forms of intellectual property, source code
(which can hold copyright protecon) can be sold or licenced which creates value in the app beyond its inial
use. Code trading also offers the means by which the source code or code segments from a trusted source can
be acquired without the associated development costs. Whilst common in the gaming industry, examples of
code trading associated with healthcare apps are few in number. However, as the healthcare app market
matures there is likely to be an increase in the use of code trading as a monezaon approach, parcularly
beyond the life expectancy of the original app.
A number of companies can facilitate code trading eg Udemy. One of the longest established companies is
Sellmyapplicaon.com. Some of the common licenses that are used to buy and sell apps and websites on the
sellmyapplicaon.com website include:
• Full Transfer of All Rights and Code - This involves the current owner de-lisng their app and then
allowing the new owner to relist under their own developer account.
• Distribuon License – Determine the condions under which a license holder can distribute the code.
• No Distribuon – No distribuon of code or binaries is allowed. Code can be hosted on license holder’s
servers / sites.
• •Binary restricted – Distribuon in binary / object form only is allowed.
• •Sublicensable – Code can be distributed in code or binary form.
• Derivave works – Allowing license owners to create derivave works means perming them to
modify the soware to fit their needs.
• Distribute derivave works – allows license owners to distribute a modified version of your soware
(as part of a greater work). This opon is only available if you allow distribuon of the code
(sublicensable license).
• Re-Skin App (White Label License) – License to reskin the applicaon and re-brand it using a “white
label” license. This is another common type of license sellers give to new buyers. It allows them to
purchase a fully approved app and basically remove any of the graphics, sounds, logos, etc to build a
new app that you can then market and sell as your own business.
WHAT DOES ALL OF THI S MEAN FOR THE “NHS APP”?
Let’s assume that the app has been suitably peer reviewed, is fit for purpose and has passed the necessary
quality or regulatory process. Lifestyle consumer apps are more likely to be handicapped more by the price
sensive market than an app targeng organisaon efficiency or service delivery by a healthcare praconer.
Healthcare professionals are more willing to purchase higher cost soluons that are robust, safe and assist
with the delivery of care eg mobile decision support algorithms or clinical protocols. Paents using mobile
soluons to support long term or chronic condions may also perceive greater value associated with an app,
but NHS organisaons will be keen to ensure that the financial model will not act as a barrier to such users.
An increasing number of NHS organisaons are using appropriate adversing to assist with website costs and
some are considering app adversing (anecdotal evidence), but this may be limited to apps targeng
healthcare professionals rather than paents.

41
As healthcare organisaons become beer informed about the adopon of apps as an enabli ng technology
they will be increasingly integrated into a range of services and pathways, which will lend itself to the app
package type approach. The increased value of proven source code in a slowly maturing healthcare app market
will eventually support code trading and reskinning opportunies for savvy developers.
Case Study
Benchmark against apps with accreditaon – sale price £1.99
A recent study by Cambridge University computer sciensts found that just 20% of paid apps are
downloaded more than 100 mes and only 0.2% of paid apps are downloaded more than 10,000 mes.
[Tech Crunch August 2012]
Opmisc download of 5,000 apps in first year gives income of £5,000. Realiscally this would not even
represent a break-even point.
A free App deployment with paid-for adversing may generate significant income. 20% of free apps get
10,000 or more downloads. [Tech Crunch August 2012].
Code from an ad companies, such as Google Adsense, is placed into the applicaon during the
development process. Aer the app goes live and is downloaded, it requests to show an adversement
and then places that ad onto the screen.
Depending upon the ad company’s revenue model, the charges are collected from the adverser and then
shared between the ad company and the app owner.
Let’s assume a typical one of our 5,000 users accesses this app 4 mes every month and clicks one ad.
This gives us 5,000 ad-clicks which would only bring in revenues of £2 - £3 per month. [typical pay-outs
from various internet sources]
500,000 users would bring in £2,000 - £3,600 pa
1 million users would bring in £4,000 – £7,000 pa
For an app that requires adversing and updates, 1 million users would represent less than break-even.
This process would also need some serious monitoring/management to keep it opmised.
SUMMARY OF KEY POINTS
• There is no single commercial model that is relevant for apps created within an NHS environment to
meet an unmet need.
• A number of factors including type of user, risk profile, income or sustainability need and organisaon
preference will influence the commercial model.
• Different approaches may be required depending on whether the healthcare app is targeng an NHS
user or health conscious consumer.
• People have become accustomed to free apps, but some users are also comfortable paying for apps.
• It is difficult to see that freemium will be widely used in healthcare as the business model does not
seem to be a good fit.

42
• App value should be determined by the outcome. If there is clear value in the outcome, then it should
be paid for in some form. What we have come to expect from apps as a consumer is and may be quite
different to what we expect from a medical app in the future.
STAGE 8 - DIGITAL MARKETING AND ADOPTION CONSIDERATIONS

• App value should be determined by the outcome. If there is clear value in the outcome, then it should
beare
App Stores paidcommonly
for in some form. What
deployed we have come
by customers toto
seeking expect from apps as a consumer
download/purchase is and
mobile apps and may
also be
by quite
different to STAGE 8 - DIGITAL MARKETING AND
what we expect from a medical app in the future.
developers to showcase, market and sell their applicaons. The likelihood of consumers finding useful medical
submitted - DIGITAL MARKETING

the major app AND
stores each day
ADOPTION
ADOPTION
is enormous, onlyCONSIDERA
CONSIDERATIONS
apps through browsing the overcrowded generic app stores is fairly remote. While the number of apps
STAGE 8 to TIONS
a minute fracon gets recognized by users.
Adeven esmated that 60% of the apps in the app stores have never been downloaded. Localycs (2010-2012)
App Stores
surveys have are commonlyindicated
repeatedly deployedthat
by customers seeking
22% of all apps are to download/purchase
only mobile apps and also by
used once aer download.
developers to showcase, market and sell their applicaons. The likelihood of consumers finding useful medical
apps through
If an app is tobrowsing
be widelythe overcrowded
adopted then thegeneric app stores
app needs to addisvalue,
fairly remote. While
offer a great theexperience
user number of and
apps
submitted to the major app stores each day is
promoon and markeng are essenal for wider adopon.enormous, only a minute fracon gets recognized by users.
Adeven esmated that 60% of the apps in the app stores have never been downloaded. Localycs (2010-2012)
Whether releasing
surveys have a medical
repeatedly app through
indicated that 22%the
ofapp stores
all apps areoronly
promong
used onceusage
aerwithin a healthcare
download.
organisaon, a successful markeng strategy requires planning, careful research, connuous monitoring, a
If an appaim
focused is toand
beeffecve/
widely adopted then
targeted the app so
promoon needs
thattothe
add value,
app offer aand
is noced great user experience
possibly adopted. Ifand
an app is
promoon and markeng are essenal for wider adopon.
classified as a medical device, developments and markeng must not only conform with the rules of a
parcular app storeabut
Whether releasing also to
medical the
app requirements
through the appof the regulatory
stores or promong bodies.
usage within a healthcare
organisaon, a successful markeng strategy requires planning, careful research, connuous monitoring, a
The principles for product markeng also apply to apps. Developers should inially establish whether there is a
focused aim and effecve/ targeted promoon so that the app is noced and possibly adopted. If an app is
need for the parcular app and how it differs from similar apps already on the market. Further research to gain
classified as a medical device, developments and markeng must not only conform with the rules of a
an understanding of what contributes to making an app successful should provide pointers for developers.
parcular app store but also to the requirements of the regulatory bodies.
Strategies to help an app stand out in the marketplace include presenng it from a novel perspecve and
promoon.
The principles for product markeng also apply to apps. Developers should inially establish whether there is a
need for the parcular app and how it differs from similar apps already on the market. Further research to gain
Good
an end user experience
understanding is vital and atosuccessful
of what contributes making anapp appmust be of good
successful shouldquality andpointers
provide funconforwell. The app
developers.
should be tested
Strategies to helpthoroughly
an app stand andoutperform consistently even
in the marketplace under
include the most
presenng it extreme condions.
from a novel The app
perspecve and
should
promoon.be updated regularly and accessible to the user at all mes. It should work perfectly, irrespecve of
whether the phone connecon is on or off, and ideally consume the minimum possible CPU and baery power.
Good endconstantly
If an app user experience
crashesisitvital and a successful
is unlikely that it willapp must be
be widely of good quality
adopted. and funcon
Interacon with endwell.
usersThe app
and an
should be tested thoroughly and perform consistently even under the most extreme
understanding of the customer experience could be obtained if a feedback mechanism was built into the app condions. The app
should
or a userbewas
updated
askedregularly
to rate theandappaccessible tousing
while sll the user at all mes.
it, rather It should
than during work perfectly, irrespecve of
uninstall.
whether the phone connecon is on or off, and ideally consume the minimum possible CPU and baery power.
If an
In theapp constantly
digital crashes
marketplace it is unlikely
Search Engine that it will be (SEO)
Opmisaon widelyandadopted.
App Store Interacon
Opmisaonwith(ASO)
end users and an As
are essenal.
understanding
over 90% of web traffic is driven by search engines, content creaon and social media, the benefits of the
of the customer experience could be obtained if a feedback mechanism was built into SEOapp
are
or a user was
obvious. asked
An app couldto rate
also the app while through
be promoted sll usingbuilding
it, rather
anthan
onlineduring uninstall.
presence to drive targeted traffic towards
it. Search engines also provide another route to link to the app stores -searching and ulizing keywords and
In the digital marketplace Search Engine Opmisaon (SEO) and App Store Opmisaon (ASO) are essenal. As
keyword names although they operate in a different environment.
over 90% of web traffic is driven by search engines, content creaon and social media, the benefits of SEO are
obvious. An app could also be promoted through building an online presence to drive targeted traffic towards
For effecve App Store Opmisaon an understanding of how the app store ranking and search algorithms
it. Search engines also provide another route to link to the app stores -searching and ulizing keywords and
funcon is useful. The algorithms can however be changed without noce with the subsequent effect on
keyword names although they operate in a different environment.
rankings as recently seen on iOS7 and in essence app developers have very lile control over these
algorithms
For effecveand how
App they
Store ulise keywords
Opmisaon (See reference
an understanding of secon
how thefor App
app store
store Opmisaon)
ranking and search algorithms
funcon is useful. The algorithms can however be changed without noce with the subsequent effect on
The following should aid App Store Opmisaon especially with iOS7:
rankings as recently seen on iOS7 and in essence app developers have very lile control over these
algorithms and how they ulise keywords (See reference secon for App store Opmisaon)
• Singular form words are handled beer with search algorithms and produce beer results.
• The category
The following name
should aid AppisStore
automacally saved
Opmisaon as a keyword
especially so there is no need to repeat it in the
with iOS7:
descripon.
• Singular form words are handled beer with search algorithms and produce beer results.
• The category name is automacally saved as a keyword so there is no need to repeat it in the
descripon.

43
• Keywords are as important as the app name but it is essen al to have an app tle that reads well and
succinctly describes the app.
• Preferable to use single keywords
• The App Store landing page should have an engaging appearance.
• Ra ngs and reviews are even more important in iOS7
As medical apps are a niche market, targe ng the medical community would be more produc ve rather than
relying on the saturated generic app stores. Sites that review medical apps, social media, blogs, professional
websites, publica ons and networks and integra ng the apps with mobile social networks (for example
Facebook) are probably the most produc ve. Any publicity material could include clear and detailed snapshots
and videos of the app.
If an app is compa ble with a variety of devices and plaorms then the market is poten ally larger. An app that
offers a secure online environment where passwords and sensi ve informa on are protected should be more
arac ve to clinicians.
No single method is going to boost the visibility of an app but having repor ng metrics in place which record
number of downloads, number of new users viewing the app, and how the app was located will provide
evidence as to which promo on channel has been the most successful.
Ul mately, success of the app will be determined by the driving need, the usability and value of the solu on
and accessibility. Success should also be determined by an apps’ longevity of use. In this disposable society,
par cularly with respect to apps, where oen the usage of an app may only be over several weeks, the true
value of a healthcare app will be found in the dura on of use.
REFERENCES / RESOURCES
REFERENCES
App Development
hp://www.research2guidance.com/the-market-for-mhealth-app-services-will-reach-26-billion-by-2017/
hp://www.research2guidance.com/shop/index.php/downloadable/download/sample/sample_id/262/
Baumgart DC. Smartphones in clinical prac ce, medical educa on, and research. Arch Intern Med
2011;171:1294-6
Boulos MN, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and par cipatory
healthcare: an overview, with example from eCAALYX. Biomed Eng Online 2011;10:24.
Visser,BJ and Bouman, J. There’s a medical app for that. 2012; BMJ Careers .
hp://careers.bmj.com/careers/advice/view-ar cle.html?id=20007104
Cost of apps
hp://appmuse.com/appmusing/how-much-does-it-cost-to-develop-a-mobile-app/
hp://katanacode.com/blog/posts/8-how-much-does-it-cost-to-develop-an-iphone-app
hp://appfurnace.com/2013/07/why-do-android-apps-cost-more-to-develop-than-iphone-apps/
hp://www.dailymobilereport.com/index.php/mobile-app-development-cost/

44
http://www.canalys.com/newsroom/android-apps-are-too-expensive
Regulatory considerations
http://www.mhra.gov.uk/Howweregulate/Devices/Software/index.htm
http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM26
3366.pdf
http://www.mhra.gov.uk/home/groups/comms-ic/documents/websiteresources/con114556.pdf
THE MEDICAL DEVICES REGULATIONS: IMPLICATIONS ON HEALTHCARE AND OTHER RELATED

ESTABLISHMENTS/ Bulletin 18. MHRA. Amended 11 February 2011
Technology standards
http://rdn-consulting.com/blog/2010/06/05/iso-62304-the-harmonized-standard-for-medical-device-software-
development/
http://medicaldevices.bsigroup.com/en-GB/our-services/iec-62304/
http://www.emdt.co.uk/article/developing-medical-device-software-iso-62304
http://www.emdt.co.uk/article/iec-62304-compliance
http://systems.hscic.gov.uk/clinsafety/dscn
See http://www.hscic.gov.uk/standards
Examples of validation studies of medical apps
Gregoski MJ, et al Development and validation of a smartphone heart rate acquisition application for health
promotion and wellness telehealth applications. Int J Telemed Appl 2012;2012:696324.
Kubben PL, et al An evidence-based mobile decision support system for subaxial cervical spine injury
treatment. Surg Neurol Int 2011;2:32.
Kwon S, et al Validation of heart rate extraction through an iPhone accelerometer. Conf Proc IEEE Eng Med
Biol Soc 2011;2011:5260-5263.
Mellone S, Tacconi C, and Chiari L. Validity of a Smartphone-based instrumented Timed Up and Go. Gait
Posture 2012 36(1) 163-5.
Qiao J, et al Reliability Analysis of a Smartphone-aided Measurement Method for the Cobb Angle of Scoliosis. J
Spinal Disord Tech 2012 25(4):E88-92.
Quality and Apps
http://www.imedicalapps.com/2013/06/apple-rejecting-medical-apps-drug-dosages/
Albrecht, UV et al, App-synopsis- standard reporting for medical apps. Studies in Health Technology &
Information 2013 192 ( 1154)

45
Buijink, AW; Visser, BJ and Marshall, L. Medical apps for smartphones: lack of evidence undermines quality
and safety. Evidence Based Medicine 2013 18 (3) 90-2
Ferrero, NA; Morrell, DS and Burkhart, CN Skin scan: a demonstration of the need for FDA regulation of
medical apps on iPhone. Journal of the American Academy of Dermatology 2013 68 (3) 515-6
Franko, O; Validate an App: How to Design Your Study and Get Published. Journal Mobile Technology in
Medicine, 2012 1 (2) 1-4 www.journalmtm.com
Hamilton AD, Brady RR. Medical professional involvement in smartphone ‘apps’ in dermatology. Br J Dermatol
2012;167:220
Kreiger, WH Medical apps: public and academic perspectives. Perspectives in Biology& Medicine 2013 56 (2)
259-73
McCarthy, M. FDA will not regulate most mobile apps. BMJ 2013 347 (f5841)
McCartney, M How do we know whether medical apps work? BMJ 2013 346 (f1811)
O’Neill, S and Brady, RR Clinical involvement and transparency in medical apps: not all apps are equal.
Colorectal Disease 2013 15 (1) 122
Payne, KF et al A review of current photography guidelines in relation to smartphone publishing of medical

images. Journal of Visual Communication in Medicine 2012 35 (4) 188-92
Visvanathan A, Hamilton A, Brady RR. Smartphone apps in microbiology-is better regulation required? Clin
Microbiol Infect 2012;18:E218–20
Walsworth, DT Medical apps: making your mobile device a medical device. Family Practice Management 2012
19 (3) 10-3
Security and Privacy in apps
http://ico.org.uk/news/latest_news/2013/~/media/documents/library/Data_Protection/Detailed_specialist_gu
ides/privacy-in-mobile-apps-dp-guidance.pdf
Han, J., Yan, Q., Gao, D., Zhou, J. & Deng, R. Comparing mobile privacy protection through cross-platform
applications. The 20th Annual Network & Distributed System Security Symposium, 26 February 2013.
http://www.liaiqin.com/hanjin/papers/NDSS2013Han.pdf
http://www.cellular-news.com/story/53557.php
http://mobihealthnews.com/28165/happtique-suspends-mobile-health-app-certification-program/
http://www.nytimes.com/2013/09/12/your-money/free-apps-for-nearly-every-health-problem-but-what-
about-privacy.html?_r=2&
http://www.networkworld.com/news/2013/111813-hp-ios-vulnerabilities-276063.html
http://www.theguardian.com/technology/2013/dec/11/app-hacking-commonplace-warns-security-company
http://www.samsung.com/global/business/mobile/solution/security/samsung-knox
http://www.allthingsknox.com/learn-knox/features/
http://www.samsung.com/my/business-images/resource/white-
paper/2013/11/Samsung_KNOX_whitepaper_An_Overview_of_Samsung_KNOX-0.pdf

46
http://www.parabal.com/parabal-white-papers/
App store
http://mattgemmell.com/releasing-outside-the-app-store/
http://www.zdnet.com/blog/hardware/600000-apps-in-apples-app-store-yet-i-cant-find-anything-i-
want/19549
http://www.bbc.co.uk/news/technology-23240971
App Store Optimisation
http://www.integratedchange.net/app-store-optimisation-what-you-need-to-know
http://appleinsider.com/articles/13/12/13/apple-tweaks-ios-app-store-search-algorithms-app-positions-
fluctuate-8x-more-than-usual
http://www.shellypalmer.com/2012/08/400000-apps-in-the-app-store-have-never-been-downloaded-says-
report/
App deployment (how do you get your app to market)
http://searchconsumerization.techtarget.com/feature/Mobile-device-management-vs-mobile-application-
management
http://www.softwareag.com/blog/reality_check/index.php/hot-and-not/byod-strategy-manage-the-device-or-
manage-the-data/
http://www.crn.com/news/security/240156399/mobile-device-management-market-wont-last-gartner.htm
http://blogs.forrester.com/category/mobile_application_management
(http://www.zdnet.com/blog/consumerization/10-byod-mobile-device-management-suites-you-need-to-
know/422)
Commercialisation of apps
http://www.bluecloudsolutions.com/blog/free-paid-apps-works/
http://techcrunch.com/2012/08/26/how-free-apps-can-make-more-money-than-paid-apps/
http://www.appiction.com/Free-Apps-vs-Paid-Apps
http://bgr.com/2012/09/11/mobile-app-market-analysis-free-apps-dominate/
http://www.appbrain.com/stats/free-and-paid-android-applications
http://locassa.com/free-vs-paid-apps/
http://www.thegeekafterglow.com/2012/12/whats-your-usage-of-free-vs-paid-apps.html
http://www.zdnet.com/in-app-payments-more-profitable-than-paid-apps-7000004110/
http://blog.w3i.com/2012/09/24/the-value-of-free-apps/
http://www.slideshare.net/techahead/inapp-purchases-are-in-paid-apps-are-out
http://www.gartner.com/newsroom/id/2592315

47
http://news.ncsu.edu/releases/wms-jiang-app-ads/
Code Trading
http://www.sellmyapplication.com/
https://appfresh.us/blog/5-vlad-ukis-reskins-medical-apps/
http://www.reskinningapps.com/
http://freelancedoodle.com/app-dev-4-buying-source-code-for-sale-and-reskinning-apps/
http://appdaddyblog.wordpress.com/2013/11/21/1-app-reskin-does-not-make-sense-but-many-do/
http://winningstack.com/how-to-make-complex-apps-on-a-budget/
http://pixtantblog.com/app-reskinning-101/
http://shockwavelabs.net/how-to-make-money-with-mobile-app-reskins/
http://brightnewt.com/truth-reskinning-apps/
Organisations and Companies offering guidance and consultancy
NISE (NHS Innovations South East) has been helping its NHS members identify, develop and commercialise
innovations designed to improve the quality and efficiency of healthcare services since 2004. NISE works
throughout the South East, delivering a comprehensive range of practical support services to help inventors
and their organisations bring bright ideas to reality. NISE is developing alternative business models that will
identify and address several current barriers to the commercial viability of medical apps for safe use by a wide
range of consumers including healthcare practitioners and patients alike. This will assist the NHS to leverage
app technology in the growing global market.
Further details can be found at http://www.innovationssoutheast.nhs.uk
HANDI (The Healthcare App Network for Development and Innovation) is a non-profit community interest
company that seeks to provide mutual support for developers of health and care apps and others interested in
this area. HANDI is supported by a community of clinicians, developers, health informaticians and others who
believe that lightweight healthcare apps for patients, carers and health and care professionals provide the key
to enabling IT to transform of health and care. HANDI is neutral with regard to platforms and business models
for apps.
Further details can be found at http://handihealth.org/
d4 is a non-profit organisation focused on improving patient care by placing modern technology in the hands of
doctors, nurses and other health care professionals. An immediate focus is to increase the acceptance and
affordability of mobile phones for clinicians in the UK.
Further details can be found at http://www.d4.org.uk
App / mhealth Developers
Bodymap Apps http://www.bodymapapps.com/
Oxford Web Applications http://www.oxfordwebapps.co.uk/
Froggyware GmbH http://www.froggyware.com/

48
Oxford App Studios http://www.oxfordappstudios.com/
Propagator Group http://www.propagatorgroup.com/
White October http://www.whiteoctober.co.uk/
Codeten Ltd http://codeten.com
Infomed Mobile http://www.infomedmobile.com/
Rare form New Media http://rareformnewmedia.com/home
Biff Bang Pow http://www.biffbangpow.com
Genetic Digital http://www.geneticdigital.co.uk
Electric Studio http://www.electricstudio.co.uk/
Overpass Apps http://www.overpass.co.uk/android-application-development/
The Boffin Lab http://www.theboffinlab.com
Serious Games International http://www.seriousgamesinternational.com
Black Pear Software http://www.blackpear.com
MicroStrategy http://www.microStrategy.com
Maldaba http://www.maldaba.co.uk/
Commontime http://www.commontime.com
Cranworth Medical http://www.cranworthmedical.co.uk/
Integrated Change Http://integratedchange.net
User Experience & Evaluation services
HDTI App evaluation service http://www.coventry.ac.uk/research/research-directory/allied-

health/health-design-technology-institute/business-support/healthcare-
apps/
Webcredible http://www.webcredible.co.uk/services/testing.shtml
Userfocus http://userfocus.co.uk
Digital Marketing
Integrated Change http://www.integratedchange.net/mobile-healthcare-marketing
Genetic Digital http://www.geneticdigital.co.uk
Foundry Healthcare http://www.foundryhealthcare.co.uk/

49
Other sources
MHRA hp://www.mhra.gov.uk/Howweregulate/Devices/Soware/index.htm
Health Developer Network hp://apps.nhs.uk/developers/health-developer-network/
Nofied body hp://www.bsi-global.com/CEMark/MedicalDevices/index.xalter
Medical Device Consultancy hp://c-m-s.com/
NHS Brand Guidelines hp://www.nhsidenty.nhs.uk/tools-and-resources/communicaons-

partnerships/sponsorship-and-adversing
WEBSITES WITH ADDITI ONAL INFORMATION AND MEDICAL APP REVIEW S ITES
hp://www.medicalappjournal.com/index.php
hp://www.imedicalapps.com/
hp://mobihealthnews.com

50
NHS Innovations South East
May 2014
Developed with funding from the Intellectual

Property Office Fast Forward 2013 Competition.

App Development: An NHS Guide For Developing Mobile Healthcare Applications

Uploaded by

Copyright:

Available Formats

App Development: An NHS Guide For Developing Mobile Healthcare Applications

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

App Development: An NHS Guide For Developing Mobile Healthcare Applications

Uploaded by

Copyright:

Available Formats

NHS Innovations South East

Developed with funding from the Intellectual

EXECUTIVE SUMMARY ............................................................................................................................................ 5

MOBILE APP FIELD GUIDE STAGES .......................................................................................................................... 6

STAGE 1 – PRE-DEVELOPMENT ............................................................................................................................... 7

Needs Assessment .............................................................................................................................................. 8

Why Develop an App? ..................................................................................................................................... 8

Compeve Analysis ........................................................................................................................................... 9

Stakeholder / Target Audience Analysis ............................................................................................................. 9

Choice of Operang System Plaorm ........................................................................................................... 10

Internal Development or Outsourced? ............................................................................................................. 11

Business Jusficaon Case ................................................................................................................................ 11

Value Proposion .......................................................................................................................................... 11

Paent /Organisaonal Benefit .................................................................................................................... 11

Resources & Costs ......................................................................................................................................... 11

Commercialisaon Strategy .......................................................................................................................... 12

Intellectual Property Consideraons ............................................................................................................ 12

Reference Costs ............................................................................................................................................ 12

Summary of Key Points ..................................................................................................................................... 14

STAGE 2 – DESIGN & DEVELOPMENT .................................................................................................................... 15

User & System Requirements ........................................................................................................................... 16

Use OF API Components ............................................................................................................................... 17

Security and Communicaon Consideraons ............................................................................................... 17

Wireframe Development .................................................................................................................................. 18

Code Generaon ............................................................................................................................................... 18

System Integraon & Tesng ............................................................................................................................ 18

STAGE 3 – USER TESTING ...................................................................................................................................... 19

STAGE 4 - STAKEHOLDER REVIEW ......................................................................................................................... 20

© 2014 NHS Innovations South East NISE_App_guide_May14_v1

Relevant Technology Standards .................................................................................................................... 20

Clinical Risk Safety Standards ........................................................................................................................ 21

Systems & Security review ............................................................................................................................ 23

Quality of Apps and Validation ..................................................................................................................... 23

Internal Deployment Within Organisation ........................................................................................................ 24

Summary of Key Points ..................................................................................................................................... 24

STAGE 5 - APPS AS MEDICAL DEVICES ................................................................................................................... 25

When is an App a Medical Device Based on MHRA Guidance? ........................................................................ 25

Outline of MDD Process ................................................................................................................................ 26

FDA Guidance .................................................................................................................................................... 28

Summary of Key Points ..................................................................................................................................... 29

STAGE 6 – APP DEPLOYMENT CONSIDERATIONS .................................................................................................. 31

App Stores ..................................................................................................................................................... 31

NHS Choices App Store ................................................................................................................................. 32

NHS Trust Dedicated App Stores ................................................................................................................... 32

Other Routes to Market ................................................................................................................................ 32

STAGE 7 - MONETARISATION OF APPS ................................................................................................................. 33

“No Brainer” App .............................................................................................................................................. 33

Different Approaches to The Monetarisation of Apps ...................................................................................... 34

Free Apps ...................................................................................................................................................... 34

Freemium and In-app Purchasing ................................................................................................................. 37

Paid for App................................................................................................................................................... 39

The App Package ........................................................................................................................................... 40

Code Trading ................................................................................................................................................. 41

Summary of Key Points ..................................................................................................................................... 42

STAGE 8 - DIGITAL MARKETING AND ADOPTION CONSIDERATIONS .................................................................... 43

1st Edition © 2014 NHS Innovations South East 3

© 2014 NHS Innovations South East NISE_App_guide_May14_v1

APPENDIX 1: Schematic - Mobile Health App ..................................................................................................... 51

1st Edition © 2014 NHS Innovations South East 4

Compeve Analysis ........................................................................................................................................... 9

Choice of Operang System Plaorm ........................................................................................................... 10

Business Jusficaon Case ................................................................................................................................ 11

Value Proposion .......................................................................................................................................... 11

Paent /Organisaonal Benefit .................................................................................................................... 11

Commercialisaon Strategy .......................................................................................................................... 12

Intellectual Property Consideraons ............................................................................................................ 12

Security and Communicaon Consideraons ............................................................................................... 17

Code Generaon ............................................................................................................................................... 18

System Integraon & Tesng ............................................................................................................................ 18

Alternave approaches to app development