Building a high availability

ArcSight solution
Paul Brettle – Presales Manager, Americas Pacific Region
#HPProtect

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
What is high availability?

High availability system design approach and associated service implementation

that ensures a prearranged level of operational performance will be met during a
contractual measurement period.
1. Elimination of single points of failure. This means adding redundancy to the system so that failure of a
component does not mean failure of the entire system.
2. Reliable crossover. In multithreaded systems, the crossover point itself tends to become a single point of
failure. High availability engineering must provide for reliable crossover.
3. Detection of failures as they occur. If the two principles above are observed, then a user may never see a
failure. But the maintenance activity must.

3 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
What is disaster recovery?

Disaster recovery (DR) involves a set of policies and procedures to enable the
recovery or continuation of vital technology infrastructure and systems following
a natural or human-induced disaster.
• [1] Disaster recovery focuses on the IT or technology systems supporting critical business functions
Critical differentiation
• What do I need?
• How do I approach it?
• What is the minimum that I will accept?

4 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
But what is high availability?

Understand what is required, approach and differences

• Data
• Systems
• Usage
• Resilience
• Processing
Understand differences between hot, warm, and cold!

5 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Prioritize and organize

What are the drivers for this?

• Regulation?
• Legislation?
• Compliance?
• Good governance/best practice?
Start examining the critical components
Look at systems, processes and models to assist you
• More on this later!

6 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
What do I get by default?

Communications
• Reliable communications
Cache
• Built in once collected for all SmartConnectors
Commit
• Commit model for storage of data (SmartConnector -> ESM)
Recovery
• Archive files
Hardware
• Dual power supply, reliable hardware, hot swap components and storage

7 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
ArcSight Architecture
ArcSight Logger Instance

Analysts will leverage the Logger SAN

ArcSight Console or a web (Optional)
browser to access ESM,
Logger, and CA. Enriched events from ESM will be forwarded
to Logger for long-term event storage.

ArcSight ESM Instance

Connector
Appliance
Analysts (Optional)
Manager Database SAN

All SmartConnectors are

managed remotely via the
ArcSight Connector Appliance
Events from all or ESM Manager.
SmartConnectors will be ArcSight ArcSight ArcSight
forwarded to the ESM Instance.
SmartConnectors
8 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
ArcSight Architecture
ArcSight ESM Instance
AUP Master

Analysts will leverage the

ArcSight Console or a web Manager Database SAN

browser to access ESM, Events of interest will be forwarded from Logger to ESM
Logger, and CA. for real-time correlation. Correlated events will be
forwarded back to Logger for long-term storage.
ArcSight Logger Instances (2+)

Connector
Logger Logger
Analysts Appliance
Loggers are configured in a Peer Network.
Events from all
SmartConnectors will be
forwarded to separate Loggers
for load balancing purposes.
All SmartConnectors are
ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight managed remotely via the
AUP Master ArcSight Connector Appliance.
SmartConnectors SmartConnectors SmartConnectors

9 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
ArcSight Architecture
Analysts will leverage the ArcSight Console Globally correlated and base events will be
or a web browser to access the Global or forwarded from the Global ESM Instance to All SmartConnectors are managed remotely
Regional ESM and Logger Instances. Logger for long-term storage. via the ArcSight Connector Appliance.

Global ESM Instance ArcSight Logger Instances

Analysts Logger (Optional) Connector Appliance

Manager Database SAN (Optional)

Correlated and the base events will be forwarded

Loggers can be configured in a Peer Network for
from each Regional ESM Instance to the Global ESM
a holistic view of all events in the environment.
Instance for Global Correlation.

Regional ESM Instance Regional ESM Instance Regional ESM Instance

Events of interest Events of interest

will be forwarded will be forwarded
from Logger to ESM from Logger to ESM
Manager Database SAN for real-time Manager Database SAN for real-time Manager Database SAN
correlation. correlation.
Correlated events Correlated events
will be forwarded will be forwarded
back to Logger for back to Logger for
ArcSight Logger Instances long-term storage. ArcSight Logger Instances long-term storage. ArcSight Logger Instances

Events from all Events from all

Loggers Connector Appliance SmartConnectors Loggers Connector Appliance SmartConnectors Loggers Connector Appliance
will be forwarded to will be forwarded to
the Regional ESM the Regional ESM
Instances. Instances.
ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight ArcSight

SmartConnectors SmartConnectors SmartConnectors SmartConnectors SmartConnectors SmartConnectors

Connector layer
ArcSight Logger/Express/ESM

• Push connector type

• Load balanced Each SmartConnector forwards on the
encrypted, compressed and processed
• Needs consistency events to the ArcSight solution.

• Typically used for Here a two-node load

balancing solution can be
– Syslog Node 1 Node 2 deployed. The load balancing
system can be used to spread
– Large volumes Session information the load between two or more
shared for load nodes for processing. There is
balancing only. no need for clustering here as
we simply want to process the
logs and events and this
The source devices send their logs
represents the most efficient
and events directly to the load
method to do this.
balancing IP address using their
native protocol, such as Syslog.

11 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Connector layer
ArcSight Logger/Express/ESM

• Pull connector type

• Log messages not lost SmartConnector forwards on the
encrypted, compressed and processed
• Active HA needed events to the ArcSight solution.

– Require consistency Two-node active/passive

cluster for the
• Typically not implemented Active Passive SmartConnector. Should the
node node active node fail for any
reason, the passive node can
Shared disk continue where it left off.
Since the shared disk is used,
all current events are
SmartConnector connects to processed with no loss
the sources directly from the or duplication.
active node. All processing is
done by the active node but
state information stored on
shared drive.

12 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Log storage layer
Logger DR site Main Logger

• Dual feed strategy

• Duplicate in two Loggers
• No replication needed
Connector receives/pulls the
events and forwards on to Connector
configured Loggers.

Devices send/receive their logs and events to

and from the Connector in their native formats
as required. Typically this will be via Syslog,
which uses UDP.

13 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Log storage layer
Storage device used for
Logger DR site
archived daily logs. Secondary
• Warm standby model Logger can retrieve archives
as necessary.
• Backup configuration
Configuration restored to
• Access archives access stored data and Main Logger
assume role of main Logger.
• Provide cache at connectors

Devices send/receive their logs and events to

and from the shared IP in their native formats
as required. Typically this will be via Syslog,
which uses UDP.

14 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Log storage layer

• Most effective solution Logger DR site Main Logger

• Dual feed and dual archive
Loggers auto-archive to
– Easy to restore storage system for resilient
long-term storage.
– Little impact Connector receives/pulls the
events and forwards on to
– No replication needed configured Loggers.

• Be aware of network
Connector
Connector

Devices send/receive their logs and events to

and from the Connector in their native formats
as required. Typically this will be via Syslog,
which uses UDP. Remote site

15 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Correlation layer
Here a primary Manager is used
Primary as the single processing server
• ESM with Oracle Manager for the correlation etc. of the ESM
solution. All communications to
– Simple fail-over to single DB the database come from the
single primary Manager.
– Use commercial solutions
– Tried and trusted

• Replicate database Heartbeat

– Several technologies available

ArcSight Console Oracle
• Fail-over manager starts database
• Console re-connects
Fail-over
Manager

16 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Correlation layer

• ESM with CORRe

Primary CORR
– No one single DB database
Manager
– Need to replicate DB
– Consider options

• Consider ESM/Express
Replication
• Look at options
ArcSight Console
• Work out difference
– HA or DR
Fail-over CORR
Manager database

17 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Options?

Hardware
• Power
• Disk
• Network
Software
• HA/fail-over/cluster software
Operating system
• HA/fail-over/cluster software
Virtualization
• Don’t forget what you can get here
• Usually a cost option

18 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Summary

Lots of options
• Consider what is needed and how to address
HA deployed at a lot of customers
• Using in-built and external technologies

Only as strong as weakest link

Plan and understand issues

19 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Please give me your feedback
Session TT3058 Speaker Paul Brettle

Please fill out a survey.

Hand it to the door monitor on your way out.
Thank you for providing your feedback, which
helps us enhance content for future events.

20 © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you

© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.