Revolutionary next generation

human computer interaction

 Emotiv Brain Computer Interface

Emotiv Systems Inc. is a neuroengineering

company developing the next phase of
man-machine interface

We are releasing a consumer product that

offers a uniquely personal experience in
the areas of:

•Games & Entertainment

•MMOGs

•Online Virtual Worlds

•Accessibility Emotiv’s breakthrough neurotechnology
•Semantic Application Interface platform enables a whole new realm of
interaction
 New Interface for the Virtual World

60-100 million people globally are

participating in virtual worlds

Average age of users: 26 years old

(Second Life: 33), only 25% are teens

Virtual worlds are on the verge of going

mainstream

In 2008/09 Virtual Worlds

•Will become more like social networks
•Social networks will become social
experiences
•Convergence consoles and PCs in
online space Facial expression & emotional feedback
are critical hurdles to mainstream
adoption of 3D virtual worlds
 Enhanced Experience

Our headset enables interact directly with elements on

screen using the power of your brain

Facial expressions are detected naturally, in real-time –

allowing Avatars come to life without conscious
involvement

By understanding the emotional experience of the user,

content can be adjusted dynamically to reflect
excitement, engagement, calmness, tension, and
frustration

Player’s learn to control objects by thought – giving them

the sensation of “the Force”

Designed to complement existing controllers, Emotiv

provides a intuitive, highly immersive and natural extension
of the user experience
 Consumer Product Offering

In 2008, Emotiv will release the Epoc

Neuroheadset

Complete, out-of-the-box
entertainment experience

Control the universe with your mind

through a game uniquely designed
for the Emotiv headset

Extend the headset features to any

game in your existing collection of
PC games through Emokey software

Download and experience new

detections for the headset through Consumer product includes a game
our web platform designed for the headset as well as
tools to allow it to be used with existing
game and applications
 Emotiv Neuroheadset
Combines low powered wireless
communication with a high fidelity
signal acquisition system to produce
a practical, light weight consumer
product

Self adjusts to fit most head shapes

and sizes

Features optimal sensor positioning

that offers accurate, spatial resolution
of the brain

Incorporates high performance

wireless connectivity Stylish, wireless, high fidelity, consumer
EEG signal acquisition headset

A gyroscope to control avatar head

or cursor movement
 Era of Graphics
For the past 30 years game
development has been focused on
improved graphics and sound

Graphics continue to evolve but with

disproportional effort and diminished
return on value

Computer interface has hardly

progressed

Until recently, innovations in other

areas have suffered

Graphic realism has been the focus of

innovation in game design
 Innovation in Graphics vs. Interface

Comparison Timeline :

.... | . . . . . . . . . . . . | . . . . . . . . . . .. | . . . . . . . . . . | . . . . . . . . . .. . | . . . .
PAST PRESENT

Innovation in gaming interface has not kept pace with graphics evolution
 Development in Semantic Interface

Dance Dance Revolution – 1999

‘Buttons’ are located on the floor, allow player to
act out moves

EyeToy - 2003
Camera captures the player’s movement by
stripping video image from background and
allows interaction via edge detection

Guitar Hero - 2005

Like DDR, player participates in a way more
consistent with the fantasy
 Semantic Interface
Nintendo Wii – Introduced Fall 2006

Interactivity trumps graphics

performance

Natural interface demonstrates mass

market appeal

Sales outperforms competition and

device continues to be out of stock in
stores 1 ½ years after introduction
 BCI as Semantic Interface
Brain Computer Interface is the
ultimate semantic device

Taps into the human ‘control center’

Emotions and non-concious thoughts

can be detected before you’re even
aware

Multichannel Human Computer

Interface allows traditional input of
conscious control while providing a
feedback loop based on user
perception

Information about the user’s

experience can be used to modify
content and tailor experience
 Science behind Emotiv

EEG (Electroencephalography)

Low latency

Portable

Inexpensive

Passive & Non-invasive

 Emotiv Class of Detections

Emotiv has released three classes of detection using our

neuroheadset

Expressiv – Identifies facial expressions

Affectiv – Detects emotional states

Cognitiv – Classifies conscious, active intent

 Classes of Detections

Emotiv Detection Suites:

EXPRESSIV: facial expressions

MMOG, Social Networks, AI characters --- become more humanly
interactive

AFFECTIV: feelings & emotions

Dynamic difficulty adjustment, feedback loop --- enables media to
react to individual emotional engagement and adjust play accordingly

COGNITIV: telekinesis
Lift, rotate, push… --- fulfills a powerful fantasy that crosses all cultures;
beyond anything previously thought possible
 Emotiv Software Development Kit

Emotiv SDK

Enables game developers to deliver content for

the Emotiv platform

Emulation tools allow for rapid prototyping of

existing applications

Composer allows for off-line simulation of

headset input

Downloadable SDKLite provides immediate

access to Emulation and Composition tools
 Emotiv SDK

Levels of Software Integration :

Rapid integration into existing and emerging
applications via traditional input device emulation
• Detection outputs (or combinations of outputs) are mapped
to keystrokes which trigger events in the application

• User setup is performed outside the target application

• Initial integration in just minutes

Complete integration via API (edk.dll)

• The technology is fully integrated via a dynamically linked
library and accessed thru a concise set of function calls (api)

• Setup can be integrated into the application

Example: instead of training in the control panel, Harry

Potter would go to wizard school and learn his spells to
train the cognitiv actions

Profiles can be managed by the application’s own user

profile
 Emotiv SDK

Emotiv Beta SDK

Hardware

Headset

Charger (power outlet or USB)

Headset battery life greater than 10 hours

Software CD
• User manual
• Emotiv Development DLL (edk.dll)
• API reference documentation
• Example projects ( c++ source code)

• Control Panel

Supports 2 players

Includes hardware status and user profile
management

Exposes detection suites in action

• Development Tools

EmoKey – Headset interface via keyboard
emulation

EmoComposer – Brain state emulation

EmoScript examples
 Emotiv SDK – Rapid Integration

development & testing platform

applicatio
EmoKey
n

timed keystrokes
mapping profile

standard input
receiver
wireless
 Emotiv SDK

Rapid Integration

EmoKey
• Converts detection events into signals that emulate traditional input devices

Simple events: Smile > 50% triggers “:)” keystrokes

Compound events: Smile > 50% + wink triggers “;)”

Simulated keystrokes can be triggered:

Once only (both hold and release time may be specified)

Repeatedly, while a mapping condition is met (both hold and release

times between strokes may be specified)

• Emokey profiles can be saved for each application

• Common modifiers (shift, alt, ctrl) can also be used o create complex keystroke
combinations.

• Implementation time in the order of minutes

 Emotiv SDK – Complete Integration

host platform

applicatio
emotiv sdk library(edk.dll)
n
device driver

emoengine

api inerface

api calls
receiver
wireless
 Emotiv SDK – Complete Integratration

Complete Integration:
Hardware and user management
• Integrated into the menu system of the application

• Combined with other service routines

• Controller configurations
• Difficulty settings

Detections and Training

• Detections matched to the content and fantasy of the application

• Unified train and play environments facilitate more rapid and entertaining
training

• Inclusion of new content and interactivity enabled by access to the

detection suite output, for example:
• Dynamic difficulty adjustment (Affectiv)
• Real time personalized avatars (Expressiv)
• New and more amazing super powers, mind powers or abilities (Cognitiv)

Development and testing tools

• EmoKey
• EmoComposer
 Emotiv SDK - Multiplayer

host platform

wireless receiver
emotiv sdk applicatio
library(edk.dll) n
wireless receiver

user id
 Emotiv SDK – API Function Detail

Emotiv SDK Function Classes

EE_ Prefix
• Status, calibration &
profile functions Profile commands
Calibration commands
ES_ Prefix Detection queries
Detection suite access

api functions
•

Emotiv SDK Core Objects

EmoEngine Event Object System status

• Alerts to events such as new
detection states being Detection results
available

EmoEngine State Object

• Contains the status of
hardware, software and
detection suite outputs.
 Emotiv SDK – User Profiles

Each user has a unique profile to store:

• Expressiv sensitivities and ranges

• Sensitivity specifies ease to trigger a detection

• For example, smile may be triggered when the user is just

starting to smile, or when they have a large grin

• Range specifies how broad the detection is after onset

• Low range: detections slightly above onset go to full scale

• High Range: detections must be triggered far above the

onset level to trigger full scale

• Cognitiv training data

• Data from training to classify their individual brainwave

‘signature’, for each focussed and intent thought trained, as well
as neutral
 Emotiv SDK – API Usage

create core objects connect to the emoengine

ES_Create
EE_EngineConnect
EE_EmoEngineEventCreate

respond to events query for new events

Examples: New EmotState EE_EngineGetNextEvent

New user EE_EmotEngineEventGetType
Profile selection EE_EmotEngineEventGetUserId

disconnect from the free core objects

emoengine
ES_Free
EE_EngineDisconnect
EE_EmoEngineEventFree
 Emotiv SDK – Event Types

Events in the Emotiv SDK

EE_UserAdded / EE_UserRemoved
• New user registered with or removed from the
emoengine, respectively.

EE_EmotEngineEventGetUserId
query for new events

EE_EmotEngineEventGetType
EE_EngineGetNextEvent
EE_ ProfileEvent
• Profile access events, such as the return of the
requested profile for a new user.

EE_CognitivEvent
• Dedicaed to events in the cognitiv suite, most
importantly control of training

EE_EmoStateUpdated
• New detection event, such as a new facial
expression.
 Detection Result Types

Detection EmoState Query Result Type

Blink ES_ExpressivIsBlink Binary (integer 0 or 1)

ES_ExpressivIsLeftWink /
Wink (Left or Right) Binary (integer 0 or 1)
ES_ExpressivIsRightWink

Horizontal Eye Movement ES_ExpressivIsLookingLeft /

Binary (integer 0 or 1)
(Left or Right) ES_ExpressivIsLookingRight

Continuous (float)
Smile ES_ExpressivGetSmileExtent
(101 increments, from 0.00 to 1.00)

Continuous (float)
Clench ES_ExpressivGetClenchExtent
(101 increments, from 0.00 to 1.00)

Continuous (float)
Eyebrow movement ES_ExpressivGetEyebrowExtent
(101 increments, from 0.00 to 1.00)

Short Term Continuous (float) (101 increments, from

ES_AffectivGetExcitementShortTermScore
Excitement / Calmness most calm at 0.00 to most excited at 1.00)

Long Term Continuous (float) (101 increments, from

ES_AffectivGetExcitementLongTermScore
Excitement / Calmness most calm at 0.00 to most excited at 1.00)

Continuous (float) (101 increments, from

Cognitiv Action (Up to 4) ES_CognitivGetCurrentAction
low power at 0.00 to full strength at 1.00)

EMOTIV REVOLUTION
 Emotiv SDK – Development Tools

Hardware and Detection Emulation

EmoComposer
• Hardware emulator which can be used in place of the headset to simulate
both the presence of the hardware as well as detection suite outputs

• Two modes of operation:

Interactive mode: events triggered and adjusted via on-screen
controls. Events of all types including detection outputs, training
sequences and profile management are supported

• Script mode: timed sequence of events played back from pre-defined

descriptions written in EmoScript, a simple markup language file

• Removes the need for each developer/designer to have a headset, thereby

removing the potential bottleneck on headset access

• Enables rapid and automated testing for both development and quality
assurance. Extensive scripting options are supported to facilitate rigorous
testing of common and boundary conditions

• Interaction mirrors the real api architecture in calling convention and

response.
 Emotiv SDK – Phased Development

Sample Development Process – Phase 1

development & testing platform

applicatio
EmoCompose
n
r
EmoScript (eml)

EmoStates

edk.dll
 Emotiv SDK – Phased Development

Sample Development Process – Phase 2

development & testing platform

applicatio
n
receiver
wireless

edk.dll
 Emotiv SDK – Training Interface

Training the Cognitiv Suite

The Cognitiv suite is trained by recording:
• Neutral: a background state, recorded when the user is not performing
expressions or cognitiv actions

• Actions: recordings of the user focusing their intent on an action for about 8
seconds

Subsequent training helps the system learn to identify the user’s

brainwaves, and helps the user learn to focus on the actions

An application must alert the EmoEngine when to start recording

the user’s brainwaves, and which with cognitiv state to associate
them. A dedicated event class returns training status

If the user is distracted, they may discard their last recording. If the
signals are noisy, it is automatically discarded

Up to four actions are available at any one time, the number

available is referred to as the current level of the user
 Cognitiv Training Process
Training the Cognitiv Suite

start training
EE_CognitivSetTrainingAction
EE_CognitivTrainingControl(userId, COG_START)

await completion
EE_CognitivTrainingFailed
EE_CognitivTrainingSucceeded

accept training reject training

EE_CognitivTrainingControl(userId, COG_ACCEPT) EE_CognitivTrainingControl(userId, COG_REJECT)

receive acknowledgement
EE_CognitivTrainingCompleted
 Emotiv Systems

For a copy of the SDKLite

Visit our website at www.emotiv.com
Select Developers on the menu bar
Select Download SDKLite from the side bar

Thank you!
 Emotiv SDK - EmoEngine

Game Process

EmoEngine (EDK.dll)

EmoState and Game Input

EmoState
EmoEvent query Loop
Buffer
handling

Emotiv
API

Game AI

EEG and Gyro Control

Emotiv Epoc
Post-Processing Logic

Game UI
 Emotiv SDK – API Usage

Initialization

EE_EngineConnect()

EE_EngineRemoteConnect()

Event Polling
EE_EngineGetNextEvent(EmoEngineEventHandle hEvent)
 Emotiv SDK – API Usage

Detection Tuning

EE_ExpressivSetThreshold(…)

EE_CognitivSetCurrentLevel(…)

Detection Training
EE_CognitivSetTrainingAction(…)

EE_CognitivSetTrainingControl(…)
 Emotiv SDK – API Usage

User Profile Management

EE_GetUserProfile(unsigned int userId, EmoEngineEventHandle hProfile)

EE_GetUserProfileBytes(EmoEngineEventHandle hEvt, … )

EE_SetUserProfile(unsigned int userId, … )

• User profile stores per-user detection settings and training data

• Allows game to manage user profile data in manner that is ideal
for that application or platform
Emotiv SDK - API Usage
 Emotiv SDK – EmoEngine Events

• EE_UserAdded
• Generated when Emotiv USB dongle is plugged into the computer.
User ID is assigned at this time

• EE_UserRemoved

• EE_EmoStateUpdated
• A detection has updated the user’s EmoState
• Call EE_EmoEngineEventGetEmoState to retrieve EmoState handle
 Emotiv SDK – EmoState Events
•Getter functions to access detection results
•Detection results can be binary (0/1) or continuous (0..1)

Detection Result Type

Blink Binary (0 or 1)
Wink Left Binary (0 or 1)
Wink Right Binary (0 or 1)
Horizontal Eye Movement (Left) Binary (0 or 1)
Horizontal Eye Movement (Right) Binary (0 or 1)
Smile Continuous (101 increments, from 0.00 to 1.00)
Clenched teeth / grimace Continuous (101 increments, from 0.00 to 1.00)
Eyebrow movement Continuous (101 increments, from 0.00 to 1.00)
Excitement / Calmness Continuous scale (101 increments, from most calm at 0.00
to most excited at 1.00)
Engagement (Focus) Continuous (101 increments, from 0.00 to 1.00)
Frustration (coming soon) Continuous (101 increments, from 0.00 to 1.00)
Cognitiv Actions 1-4 Continuous (101 increments, from 0.00 to 1.00)
 Emotiv SDK – EmoEngine Events

• EE_CognitivEvent
• Informs application of the status of Cognitiv training

• EE_CognitivTrainingStarted

• EE_CognitivTrainingSucceeded

• EE_CognitivTrainingFailed

• EE_ExpressivEvent
 Emotiv SDK – Control Panel

Demonstrate, tune, train and test detections

 Emotiv SDK – Control Panel

• EmoEngine proxy: Use EE_EngineRemoteConnect to connect

EDK.dll to Emotiv Control Panel
• Allows developers to start game integration and testing
without having to build complete UI and game logic to
support training, profile mgmt, detection tuning
 Emotiv SDK - Composer

• EmoEngine and Headset emulator

• Generates EmoEngine events

• Reponds to training and profile mgmt requests

 Emotiv SDK - EmoKey

• An utility that maps detection results to keystroke sequences

• EmoEngine detections can be used with existing games/

application