Manufacturing Execution System (MES) : Virtual Environment
Manufacturing Execution System (MES) : Virtual Environment
Virtual Environment
Implementation Guide
Version 6.0
June, 2018
© 2018 AVEVA Group plc or its subsidiaries. All rights reserved.
The Schneider Electric industrial soft ware business and AVEVA have merged to trade as AVEVA Group
plc, a UK listed company. The Schneider Electric and Life is On trademarks are owned by Schneider
Electric and are being licensed to AVEVA by Schneider Electric.
No part of this documentation shall be reproduced, stored in a ret rieval system, or transmitted by any
means, electronic, mechanical, photocopying, rec ording, or otherwise, without the prior written
permission of AVEVA. No liability is assumed with respect to the use of the information contained herein.
Although precaution has been taken in the preparation of this docum entation, AVEVA assumes no
responsibility for errors or omissions. The information in this documentation is subject to change without
notice and does not represent a commitment on the part of AVEVA. The soft ware described in this
documentation is furnished under a license agreement. This soft ware may be used or copied only in
accordance with the terms of such license agreement.
ArchestrA, Aquis, Avantis, DYNS IM, eDNA, EYESIM, InBatch, InduSoft, InStep, IntelaTrac, InTouch,
PIPEPHASE, PRiSM, PRO/II, PROV IS IO N, ROMeo, SIM4ME, SimCentral, SimSci, Skelta,
SmartGlance, Spiral Software, Termis, WindowMaker, WindowViewer, and Wonderware are trademarks
of AVEVA and/or its subsidiaries. An extensive listing of AVEVA trademarks can be found at:
https://sw.aveva.com/legal. All other brands may be trademarks of their respective owners.
Publication date: 7/2/2018
Contact Information
AVEVA Group plc
High Cross
Madingley Road
Cambridge
CB3 OHB. UK
https://sw.aveva.com/
For information on how to cont act sales, customer training, and technical support, see
https://sw.aveva.com/contact.
MES Virtual Environment Implementation Guide
Contents
Chapter 1 Getting Started..................................................................................................... 5
Virtualization Overview .............................................................................................................. 5
Using This Guide ...................................................................................................................... 5
Understanding Virtualization ...................................................................................................... 5
Definitions........................................................................................................................... 6
Version 6.0 3
MES Virtual Environment Implementation Guide Contents
4 Version 6.0
MES Virtual Environment Implementation Guide
C HAPTER 1
Getting Started
In This Chapter
Virtualization Overview.................................................................................................................... 5
Using This Guide ............................................................................................................................ 5
Understanding Virtualization ............................................................................................................ 5
Virtualization Overview
Virtualization technologies are becoming a high priority for IT administrat ors and managers, software
and systems engineers, plant managers, software developers, and system integrat ors.
Mission-critical operations in both small- and large-scale organizations demand availability—defined as
the ability of the user community to access the system—along with dependable recovery from natural or
man-made disasters. Virtualization technologies provide a plat form for High A vailability solutions.
Understanding Virtualization
Virtualization is the creation of an abstracted or simulated—virtual, rather than actual—version of
something, such as an operating system, server, net work resource, or storage device. Virtualization
technology abstracts the hardware from the software, extending the life cycle of a software platform.
In virtualization, a single piece of hardware, such as a server, hosts and coordinates multiple guest
operating systems. No guest operating system is aware that it is sharing res ources and running on a
layer of virtualization software rather than directly on the host hardware. Each guest operating system
appears as a complete, hardware-based OS to the applications running on it.
Version 6.0 5
MES Virtual Environment Implementation Guide Getting Started
Definitions
This implementation guide assumes that you and your organization have done the necessary research
and analysis and have made the decision to implement MES in a virtualized environment that will replace
the need for physical comput ers and instead run them in a virtualized environment. Such an environment
can take advant age of advanced virtualization features, including High A vailability. In that context, we’ll
define the terms as follows:
Virtualization in MES terms can be defined as creating a virtual, rather than real, server for one of the
MES servers, including the Middleware server and the Application Object server, as well as the
network resources to support the server.
High A vailability in MES can be defined as the way to make any of the MES servers available without
interruption during production operations. In this virtualization testing, we chose the most critical
server—t he MES Middleware server.
While these definitions are general and allow for a variety of High A vailability designs, this
implementation guide focuses on virtualization as an indispensable element in creating the redundancy
necessary for highly available MES solutions.
The virtualized environment described in this guide is based on Microsoft Hyper-V technology
incorporated in the Windows Server 2008 R2 operating system, and on VMware’s virtualization platform
vSphere 5.0.
MES is tested on newer versions of vS phere as updates are released by VMware. Microsoft Hyper-V is
not routinely tested. As the virtualization technology is invisible to the MES software, there is no specific
version support statement for VMware vSphere or Microsoft Hyper -V. The timing tests as described in
this document have not been re-run on newer versions of either virtualization platform.
6 Version 6.0
MES Virtual Environment Implementation Guide
C HAPTER 2
Application Configuration
To test the MES High A vailability implementation in a virtualized environment, a predictable MES
application was developed. A description of the application used for the validation and testing of the
virtualized environments described in this guide —a machine shop application—is provided in the
following topics.
In This Chapter
E vents ........................................................................................................................................... 7
Application Server Configuration Details ........................................................................................... 8
Events
The machine shop application was based on a set of ArchestrA object templates with pre -configured
scripts that trigger events at regular intervals. This made the events, and thereby the data logged in the
MES databas e, predictable. Any values related to events (e.g., good production) are set to a consta nt
value.
The following events were triggered by the machine shop application.
Hourly Events
Change state of mac hine to a running state
Create job
Receive inventory
Stop job
Reset event commands if in error
Reset stop job commands if in error
Start job
Change state of mac hine to a running state
Minutely Events
Add good production
Add bad production
Add consumption
Reset various event commands if in error
This configuration guarantees that at any given second one of each minutely event will be triggered if
there is a corresponding MES object for that second. This ensures that when migration (Live
Migration/failover) is triggered, the configured events are being exercised.
Also, the MES database was pre-populated with one year’s wort h of data prior to running through the
various scenarios to ensure that the test scenarios were run on a typically loaded system.
Version 6.0 7
MES Virtual Environment Implementation Guide Application Configuration
8 Version 6.0
MES Virtual Environment Implementation Guide
C HAPTER 3
Recovery Time Objective and Recovery
Point Objective
In This Chapter
Key Performance Indicators Used .................................................................................................... 9
RTO Measurement ......................................................................................................................... 9
RPO Measurement ......................................................................................................................... 9
RTO Measurement
The RTO measurement is the time interval during which the MES Middleware server is not available:
RTO = RTO (T2) – RTO (T1)
where
T1 = The time-stamp when the MES Middleware service cannot be detected. This also can be the
time-stamp for when the "fault" is introduced into the system (e.g., Live Migration cutover, power off,
network disabled or an operating system failure).
T2 = The time-stamp when the MES Middleware service is detected on the target machine (by using
the servic es log on the target machine to determine T2).
If more than one clock is involved, be sure to either synchronize the clocks or understand the offset. In
addition, MES Middleware service was monitored using Windows Performance Monitor (P erfMon).
RPO Measurement
The RP O measurement is the time interval during which there is no data being entered in the MES
database.
RPO = RPO (T2) – RPO (T1)
where
Version 6.0 9
MES Virtual Environment Implementation Guide Recovery Time Objective and Recovery Point Objective
T1 = The time-stamp of the last record entered in the MES database at or before RTO (T1); that is,
(RPO (T1) <= RTO (T1)).
T2 = The time-stamp of the first record entered in the MES database at or after RTO (T2); that is,
(RPO (T2) >= RTO (T2)).
It is possible that RPO is less than RTO, since there is a client middleware layer that allows for
serialization of events while the MES Middleware service is unavailable. If RPO = 0, this implies that no
events were missed.
Custom SQL queries were developed and run against the MES database to determine the time-stamp of
the "last" record before an HA DR fault and the "first" record following an HA DR fault.
10 Version 6.0
MES Virtual Environment Implementation Guide
C HAPTER 4
Configuration Information for a VMware
Environment
In This Chapter
VMware Environment Configuration Overview ................................................................................ 11
Host Hardware for VMware Servers ............................................................................................... 11
Guest Operating Systems (Virtual Machines) .................................................................................. 12
Shared Disk Storage/SAN ............................................................................................................. 12
Physical Machines (External)......................................................................................................... 12
Memory 48 GB
Storage SAN with a 1-TB storage disk (single SCS I Disk with virtual partition)
Version 6.0 11
MES Virtual Environment Implementation Guide Configuration Information for a VMware Environment
Memory 4 GB
Storage 39.8 GB
Product Installed MES 2012 (Middleware, MES Service, MES Client, MES Configurator, MES
Operator, and MES Supervisor)
Memory 4 GB
Storage 39.9 GB
Product Installed Application Server - ASP2012 - IDE and Bootstrap, Applic ation Server Runtime,
and deployed MES application objects
Processor Intel Core 2 Quad CPU Q9400 @ 2.66 GHz, 2.66 GHz
Memory 4 GB
Storage 149 GB
Network LAN
Product Installed Application Server – ASP2012, Galaxy Repository, IDE and SQL Server 2008
SP3
12 Version 6.0
Configuration Information for a VMware Environment MES Virtual Environment Implementation Guide
Memory 128 GB
Network LAN
Version 6.0 13
MES Virtual Environment Implementation Guide
C HAPTER 5
Scenarios and Observations in the VMware
Virtual Environment
In This Chapter
VMware Scenarios and Obs ervations Overview .............................................................................. 15
Live Migration (Using vMotion)....................................................................................................... 15
Observations
After the migration, the MES transactions continued to execute correctly.
Version 6.0 15
MES Virtual Environment Implementation Guide Scenarios and Observations in the VMware Virtual Environment
There were no errors/warnings in the SMC, Windows E vents Log, or SQL Server loggers after Live
Migration.
There was no transaction/data loss after Live Migration.
Both MES Client and the objects retained the connection to the MES middleware after the Live
Migration.
MES Service and Middleware server Top of the minute 138 180
MES Service and Middleware server When an Arc hive job is in 145 180
progress
MES Service and Middleware server When a Purge job is in 138 240
progress
Application Server hosting the mac hine During normal execution 138 300
shop application
MES Service, Middleware server, and During normal execution 144 (A ppObject) *
Application Server hosting the mac hine
308 (middleware)
shop application
*The RPO is undefined as the Application Server Objects had to be redeployed before they were functional.
Notes
The RTO is also influenced by the Windows delay during start up, during which the system waits
approximately 30 seconds for the user to select normal or safe mode start up.
Observations
While the MES middleware alone fails over, the ArchestrA log on the Application Server machine logs
warnings and errors indicating the inability of the objects to communicate with the middleware while the
middleware is unavailable. However, the MES objects recover after the failover and no more
errors/warnings are reported in the Logger.
The Archive operation is aborted aft er the middleware failover and the same is indicated against the
job in the client and the ArchestrA log on the Middleware server.
The Purge operation is aborted after the middleware failover and the same is indicated against the
job in the client and the ArchestrA log on the Middleware server.
The databas e is left undisturbed.
When the Application Server alone fails over, the middleware logs Lost connection to subscription
warnings. There are a few warnings and errors in the Application Server machine; however, the
Application Server stabilizes event ually (as defined by the RTO).
When bot h the Application Server and the middleware fail over, the middleware comes back online
and does not log any warnings or errors. However, the Application Server is unable to recover wit h
multiple warnings and errors logged to the Arc hestrA logger.
16 Version 6.0
MES Virtual Environment Implementation Guide
C HAPTER 6
Configuration Information for a Hyper-V
Environment
In This Chapter
Hyper-V Environment Configuration Overview ................................................................................ 17
Host Machines.............................................................................................................................. 17
Virtual Machines ........................................................................................................................... 18
Physical Machines ........................................................................................................................ 18
Host Machines
Primary Hyper-V Machine
Memory 96 GB
Network 4 network adaptors (2 public/ virtual net works; 1 iSCSI; and 1 private)
Version 6.0 17
MES Virtual Environment Implementation Guide Configuration Information for a Hyper-V En vironment
Network 4 network adapters (2 public/ virtual net works; 1 iSCSI; and 1 private)
Notes
For the Hyper-V hosts to function optimally, the server should have the same processor, RAM,
storage, and service pack level. Though the different host machine configurations are supported,
they will impact the performance during failovers.
Two Hyper-V hosts were used to configure the failover cluster.
Virtual Machines
Virtual Machine 1: MES Middleware Server
Memory 8 GB
Storage 80 GB
Products Installed MES 2012 (MES Middleware, MES Service, MES Client, MES Configurator,
MES Operator, and MES Supervis or)
Memory 4 GB
Storage 100 GB
Products Installed Application Server – ASP2012 R2 - IDE & Bootstrap, GR and Application Server
Runtime, and deployed MES application objects
Physical Machines
The configurations of the physical machines that were used are provided in the following topics.
Virtualization of thes e machines could have been done, but it was decided to keep them on real
machines to concentrate the qualification on the virtualized MES functionality.
Physical Machine 1: Galaxy Repository
Processor Intel Core2 Quad CPU Q9400 @2.66 GHz, 2.66 GHz
Memory 4 GB
Storage 150 GB
18 Version 6.0
Configuration Information for a Hyper-V En vironment MES Virtual Environment Implementation Guide
Network LAN
Products Installed ASP2012 R2, Galaxy Repository, IDE, and SQL Server 2008 R2 SP1
Memory 128 GB
Storage 1 TB
Network LAN
Version 6.0 19
MES Virtual Environment Implementation Guide
C HAPTER 7
Scenarios and Observations in the HyperV
Virtual Environment
In This Chapter
Hyper-V Scenarios and Obs ervations Overview.............................................................................. 21
Live Migration ............................................................................................................................... 21
Live Migration
Version 6.0 21
MES Virtual Environment Implementation Guide Scenarios and Observations in the HyperV Virtual Environment
Observations
It is recommended that you segregate the network traffic bet ween the failover cluster and the virtual
machines.
The MES objects recovered and managed to reestablish connectivity with the MES Middleware
server on the remote node and resumed normal operational rhythm.
The following repres entative log entries indicate communication issues observed after the failover
and before the objects recovered:
o Data cannot be read from the MES Middleware, the communications to the MES Middleware
could be brok en.
o Proxy failed to connect to Middleware server service endpoint.
o Counter: JobPos0.Prod.Bad1 encountered an error calling AddProd, error:Proxy failed to
connect to Middleware server service endpoint.
Application Server hosting the mac hine shop During normal execution N/A 240
application
Observations
All jobs were terminated following the Application Server VM failover event and due to the application
design. Full operation required one hour to resume.
The following log entry was made for each entity indicating communication issues relat ed to the
failover of the Application Server VM:
22 Version 6.0
Scenarios and Observations in the HyperV Virtual Environment MES Virtual Environment Implementation Guide
Version 6.0 23