U.P.B. Sci. Bull., Series C, Vol. 75, Iss.

4, 2013 ISSN 2286 – 3540

A COMPUTER VISION APPROACH FOR THE EYE

ACCESSING CUE MODEL USED IN NEURO-LINGUISTIC
PROGRAMMING

Ruxandra VRÂNCEANU1, Laura FLOREA2, Corneliu FLOREA3

This paper investigates the Eye Accessing Cue (EAC) model used in Neuro-
Linguistic Programming (NLP) and shows how image processing techniques can be
used to improve the interpretation of this model. An experiment was carried out to
validate the model by inducing certain eye cues using a set of questions. A simple
and efficient method is proposed for automatically locating the eyes and the
correspondent EAC. The relative position between the iris and the sclera is
determined using a fast mechanism, based on the analysis of integral projections
inside the bounding box of the eye.

Keywords: Neuro-Linguistic Programming, Eye Detection, Eye Gaze

1. Introduction

The progress made in image processing and the increase of computational

capabilities of machines over the past decades has led to new opportunities for
human-computer interactions and to the development of systems capable of
automatically interpreting the facial attributes of a person. Such algorithms are
used in the field of people identification and description, in applications that
attempt to identify the human facial expressions [1, 2], and ultimately better
describe the internal emotional and psychological mechanisms of a person.
The most distinctive of the facial features are the eyes, whose movement
and physical properties have been a constant source of information in multiple
applications such as accurately identifying the facial expressions [3], developing
eye-gaze communication solutions [4], or describing persons in terms of physical
attributes such as race, age [5], or level of fatigue [6].
The movements of the eyes have been the subject of intensive
psychological research and have been classified according to everyday visual
tasks performed by a person [7]. Furthermore, non-visual movements of the eyes

1
PhD. Student, The Image Processing and Analysis Laboratory, LAPI, University
POLITEHNICA of Bucharest, Romania, e-mail: rvranceanu@alpha.imag.pub.ro
2
Lecturer, The Image Processing and Analysis Laboratory, LAPI, University POLITEHNICA of
Bucharesti, Romania, e-mail: lflorea@alpha.imag.pub.ro
3
Lecturer, The Image Processing and Analysis Laboratory, LAPI, University POLITEHNICA of
Bucharesti, Romania, e-mail: cflorea@alpha.imag.pub.ro
80 Ruxandra Vrânceanu, Laura Florea, Corneliu Florea

were analyzed by Bandler and Grinder, who introduced the Eye Accessing Cues
(EAC) model as part of the Neuro-Linguistic Programming (NLP) theory [8].
NLP proposes a set of models used to describe the behavior patterns
created by the interaction between the brain, the language and the body. The EAC
model describes the eye-movements that are not used for visual tasks and suggests
that the direction of gaze in this case can be an indicator for the internal
representational system used by a person, who may think in visual, auditory or
kinesthetic terms, as well as for the mental activity of that person, of
remembering, imagining or having an internal dialog.
Automatic eye gaze detection increases the accuracy and flexibility of the
tiresome manual labeling and is a necessary improvement in the process of
psychological analyzes. But despite the extensive existing state of the art, the
problem of finding a real time solution for accurately detecting the gaze direction
is still unsolved, especially when considering the extra constraint of using
noninvasive technology, highly recommended in psychological experiments.
This paper analyzes the problem of eye position detection in the context of
the NLP EAC model. The model is described in section 2, where we also present
our experiment that attempts to validate this highly controversial theory. In
section 3 we propose a fast and accurate method of eye detection, which is then
used in section 4 for a sclera and iris position analysis to determine the EAC. The
performance of the method is discussed in section 5 and the paper ends with some
conclusion and future direction of research.

2. NLP-EAC Model Experiment

The idea that eye movements are related to the internal representational
system of a person was first introduced in [9]. A thorough study was conducted by
Dilts in 1977 [10], which used electrodes to track both the eye movements and the
brain wave characteristics of people, while asking them questions meant to
stimulate the sense of sight, hearing and feeling with tasks involving both memory
and mental construction. The results of these tests lead to the identification of the
basic NLP EACs, which can be seen in Fig. 1.
Since its introduction, several studies have attempted to validate this
model but the results have been inconsistent. Most of the research has provided
unsupportive conclusion [11, 12, 13, 14, 15], while several studies have shown a
correlation between the representational systems and the direction of eye gaze,
mostly for the visual system [16, 17, 18, 19]. Most of these studies correlate the
appearance of certain predicates, words associated to one of the representational
systems, with the first EAC that follows them. However the cognitive response
can differ from person to person and all the eye movements in response to a
question should be considered.
A computer vision approach for the eye accessing cue model used in neuro-linguistic progr. … 81

Fig. 1. Eye Accessing Cues (EACs).

Considering these observations, we have design our own experiment: a set

of 29 questions has been constructed to induce certain EACs - 5 questions for VR,
VC, AC and K respectively, 6 questions for AR and 3 questions for ID. The
questions were recorded and played back to 10 subjects who had the flexibility of
moving to the next question in their own time, while a video camera recorded
them. No interviewer was present since the subject tends to look at the person
asking questions, limiting the appearance of EACs. The movies were then
cropped and only the sequences corresponding to the answer time were kept for
analysis. These sequences were then played back 4 times slower and all the EACs
were manually marked according to their type and their duration: normal visible
movement, quick barely visible movement (that lasts a few frames) and pausing
for a longer period in a position (for more than 3 seconds).
Table 1 shows the overall appearance of the EACs. The duration of the
gaze was taken into consideration when computing the percentage, giving higher
weight to longer pauses in a certain position. It can be seen that the VD EAC has a
very high percentage. Although this cue is usually associated with a visual
internal representation, it can in fact belong to the preferential system of the
person or suggest a quick access of almost any sensory information. Based on
these observations, the VD EAC was left out of further statistical analysis.
Table 1
EAC - Overall Appearance
EAC Category VR VC AR AC K ID VD
Percentage [%] 7.05 10.40 9.01 11.25 11.66 14.92 35.71

For each question the mean percentage of every EAC was computed. The
sum on each row is 100%. A visual representation of this information can be seen
in Fig. 2, where lighter areas indicate higher percentages. Analyzing the
information on each row, which shows every EAC percentage for a certain
question, higher values should appear for the EAC group to which the question
belongs. The expected EAC for each question group is highlighted in the figure.
However, it can be noticed that the highlighted areas do not have much higher
values, and the separation of the EAC groups is not that clear for all the cues.
82 Ruxandra Vrânceanu, Laura Florea, Corneliu Florea

Fig. 2. Normalized (sums to 100% on rows) appearance of the EACs (columns) for each question
(rows). The correspondence between gray levels and percentages can be seen in the colorbar on
the left of the image: white means higher values, black means lower values.

Fig. 3. (a) Appearance of the EACs for each question (V = VR + VC; A = AR + AC); (b)
Maximum values on each line in (a); (c) Appearance of the EACs for each question (R = VR +
AR; C = VC + AC); (d) Maximum values on each line in (c).

The next step was to evaluate the separation between the representational
systems alone, without distinguishing between the mental activities of
remembering and constructing. The VR/VC cues, and respectively AR/AC cues,
were merged and the percentages were recomputed. In Fig. 3.(a) it can be seen
that the separation is slightly more visible. Fig. 3.(b) shows only the highest
percentage on each line, corresponding to the EAC that appears most often for a
question. Most of the questions appear to have the expected EAC. The same
strategy was applied for seeing the separation between constructing and
remembering, merging VR/AR, and respectively VC/AC. A slight separation is
visible in Fig. 3.(c)/(d), mainly for the visual cues. However, for most of the
questions the construction activity appears to be dominant.
 A computer vision approach for the eye accessing cue model used in neuro-linguistic progr. … 83

Some studies have shown that eye movement patterns are not universal but
specific for every person [20]. Furthermore, the NLP theory claims that each
person has a preferential representation system, which he may access even when
asked a question that refers to another system. Based on these observations, we
have attempted to see how each of the EACs is distributed over all the questions,
independently of the other cues. The results can be seen in Fig. 4.(a) which shows
for every EAC in what percentage it appears for every question. The sum on each
column is 100%. Higher values should appear for the questions in the group of
that particular cue.
We also repeated the merging process to see where is each representational
system concentrated over all the questions, Fig. 4.(b). For a better visibility, in
Fig. 4.(c), the percentages of the questions where a certain cue does not appear
very often were discarded. Despite different eye patterns for each person, each eye
cue seems to appear more frequently for the questions designed to access the
respective cue.

Fig. 4. (a) Distribution of each EAC over all questions (sum to 100% on columns); (b)
Distribution of each EAC over all questions (V = VR + VC; A = AR + AC); (d) The higher
values on each column in (b).

The results of our experiment have not entirely confirmed the NLP theory
but a correlation between eye movements and NLP EACs was observed. The high
variance of EACs over a question can be due to the complexity of the question,
determining the subject to access multiple representational systems and to vary
the mental activity between imagining and remembering, or carrying an internal
dialog to clarify the question. A larger set of questions could be used to determine
which questions are more precise and statistics should be done with more test
subjects. However this is a very complex task, tiresome and prone to errors, when
manual labeling of eye movements is involved. An automatic solution capable of
identifying EACs is indispensable for a thorough analysis. To the author's best
84 Ruxandra Vrânceanu, Laura Florea, Corneliu Florea

information the other only known attempt for such a solution is found in [21],
where the camera used to record the eye movements is mounted on the head of the
subject. This is not a very practical solution and can interfere in the experiment as
a distraction to the subjects. In our experiments we have used a camera mounted
on a tripod, positioned at about 2 meters from the subject. In order to
automatically detect the EACs, the face crop was detected [22] and the face image
is then analyzed to detect the eyes and the gaze direction using a refined version
of the authors’ previous work [23].

3. Eye Detection

The first step in the EAC analysis is locating the eyes. For this task the
method uses the structure of the eye: a darker area, the iris, and a whiter one, the
sclera. To enhance and speed up the process, only the upper part of the face is
analyzed. For locating the eyes two maps are formed, both using the blue (B) and
the red (R) plane. On the blue plane the contrast between the sclera and the
surrounding area is the best, as can be seen in Fig. 5(b). On the red plane the best
highlight of the iris can be found (Fig. 5(c)).
The first eye map M1 is formed as:
B ⊕ Se1
M1 = (1)
RΘSe1 + 1
where Se1 is a disk-shaped structuring element used for dilation ( ⊕ ) and erosion
( Θ ). The result can be seen in Fig. 5(d).
The second map M2 is obtained by:
M 2 = ( B − R ) − ( B − R ) D Se 2 (2)
where ( D ) represents the morphological closing. The structuring element Se1 is a
larger disk-shaped kernel.

Fig 5. (a) Original image; (b) Dilatation of B plane; (c) Erosion of R plane; (d) Eye map using
Eq.1; (e) Final eye map.
 A computer vision approach for the eye accessing cue model used in neuro-linguistic progr. … 85

Fig. 6. Eye Detection Examples.

The final map M is formed by pixel-wise multiplication of the two maps:

M (i, j ) = M 1 (i, j ) ⋅ M 2 (i, j ) 2 , ∀i, j (3)
Only the maximum value given by Eq.( 3) for each eye will be kept. The
result can be seen in Fig. 5(e). Fig. 6 shows some examples for the eye detection,
when people are looking in different directions.
The analysis is done on a smaller scale of the image. This makes the
algorithm not only simple, but also fast.

4. EAC Detection

After finding a point inside each eye, the bounding box of the eye can be
determined and then used for analyzing the position of the iris inside the sclera.
The search area is first chosen large enough to contain the whole eye, as shown in
Fig. 7(a). Inside this search area the horizontal projection, introduced in [24], is
computed on eMap, a processed version of the image, highlighting the eyes:
⎛ R ⎞
eMap = 255 ⋅ ⎜1 − Eye ⎟ (4)
⎜ R ⎟
⎝ Eye ⎠
where REye is the search area's red channel and REye is its average.
As can be seen in Fig. 7(c) and (d), the projections have higher values in
the eye area and the local minimums are searched for in order to find the upper
and lower limits of the bounding box of the eye. After cropping based on limits of
the horizontal projection, the same process is done on the vertical projection in
order to find the right and left limits. The resulting bounding box can be seen in
Fig. 7(b).
The map of the sclera sMap is formed based on the transformed blue plane
BEye:
BEye − BEye
sMap = 255 ⋅
(
max BEye − BEye ) (5)
86 Ruxandra Vrânceanu, Laura Florea, Corneliu Florea

where BEye is the mean of the blue component of the search area. This operation
highlights the sclera region, while the surrounding area is darkened.

Fig. 7. (a) Highlighted eye; (b) Bounding box of the eye; (c) Horizontal projection for the eye
crop; (d) Vertical projection for the eye crop.

Fig. 8. (a) Vertical projection for the sclera of a centered eye; (b) Vertical projection for the
sclera of an eye looking sideways.

Based on this information, the sclera is extracted using a morphological

opening with a disk-shaped structuring element. The opening is needed in order to
remove the artifacts caused by glint. Then the vertical projection inside the
bounding box can be an indicator for the size and form of the sclera:
• If the sclera occupies a very small region of the eye crop, below 5%, then the
eye is looking down. In this case a further analysis is needed to find out the
horizontal position of the iris.
• If the iris has a center position, the sclera is separated into two distinct regions,
as seen in Fig. 8(a).
• If the sclera is a single region, the horizontal and vertical positions of the iris
must be determined.
When necessary, the overall position of the sclera inside the eye bounding
box, (sX,sY), is determined as the weighting centers of the projections:
1 ∑ x =1 x ⋅ PV ( x )
H

sX = ⋅ (6)
H ∑H PV ( x )
x =1
 A computer vision approach for the eye accessing cue model used in neuro-linguistic progr. … 87

1 ∑ y ⋅ PH ( y )
W

sY = ⋅ y =W1 (7)
W ∑ PH ( y )
y =1

where PV and PH are the vertical and horizontal integral projections for the
bounding box, while H and W are the sizes of the box. We note that image
projections are known to be very useful in localization of the eye center [25].

Fig. 9. Removing symmetrical objects.

A new analysis, similar to the one used for analyzing the sclera, is done on
the eMap given by Eq.(4) to determine the position of the iris. To reduce the
influence of the eyelashes on the vertical projection, a vertically reflected image is
used and the symmetrical elements in the crop are removed, while preserving the
asymmetrical ones as shown in Fig. 9. If the iris has a central position, the vertical
position is not relevant.
At the end, the average position from both eyes is computed.

5. Results

In order to evaluate the accuracy of our eye detection algorithm we have

used the AR Database, [26]. For 126 subjects the database provides a set of RGB
images with different facial expressions, illumination conditions and occlusions,
such as sun glasses, for which the eye positions have been manually marked. The
accuracy measure to indicate the error of estimating the centers of the eyes is the
normalized error, introduced in [27]:
max(d left , d right )
ε= (8)
d eyes
where d left and d right are the Euclidian distances between the located eyes and the
ones in the ground truth, and d eyes is the Euclidian distance between the true eyes.
A value of ε ≤ 0.25 corresponds to the distance between the eye center
and the eye corners, and the detection is inside the eye. In this case the detection
rate was 93.18%. For a value ε ≤ 0.1, the detection is inside the iris and the
detection rate of the method was 79.42%.
In order to test the EAC detection method, a set of frontal face images has
been selected from ten subjects, the set consisting in five images for each of the
88 Ruxandra Vrânceanu, Laura Florea, Corneliu Florea

seven EACs. The faces did not contain any occlusions in the eye area, like
spectacles, strands of hair or heavy shadows.
The detection of the eye position was evaluated based on the horizontal
and the vertical detection rate, Table 2. It can be seen that a good separation of the
eye positions can be achieved horizontally, where the right and left positions are
less confused with each other. However, the vertical positions are less separated,
and a high confusion rate appears between the center position and the up position.
This can be interpreted, from the EAC point of view, as a good separation
between the mental activities of remembering and constructing, and a less
accurate separation between the visual and the auditory internal representational
system. These observations can be found as well in Table 3 that shows the
detection rates for the different EACs and which pairs are more likely to be
confused with each other.
Table 2
Detection Rate for Horizontal and Vertical Positions
Horizontal Center Right Left Vertical Center Up Down
Position [%] [%] [%] Position [%] [%] [%]

Center 62 15 23 Center 62 27 11
Right 6 80 14 Up 22 78 0
Left 8 28 64 Down 31 9 60

Table 3
Detection Rate for Visual Accessing Cues
Detection rate
Accessing VR VC AR AC ID K VD
Cues [%] [%] [%] [%] [%] [%] [%]

VR 72 16 9 0 0 0 3
VC 6 62 0 24 0 0 8
AR 25 3 54 6 3 0 9
Ground truth

AC 2 23 0 64 0 2 9
ID 2 0 23 5 50 12 8
K 3 13 0 17 3 55 9
VD 6 14 9 8 0 1 62
 A computer vision approach for the eye accessing cue model used in neuro-linguistic progr. … 89

6. Conclusions

This paper has investigated the EAC model used in NLP to detect the
internal representational system and the mental activity of a person, based on the
position of the eyes. An experiment was carried out to evaluate this theory and the
results showed there is some correlation between the eye movements and mental
processes. Furthermore we have highlighted the importance of an automatic
solution in analyzing eye cues and we have implemented a simple and accurate
eye detection method, based on which a more detailed investigation of the eye
area was possible. This analysis attempted to separate the horizontal and vertical
position of an eye based on the iris and the sclera relative position. A good level
of separation has been obtained for the horizontal positions of the eye, the
separation of the vertical positions being slightly less accurate. Further research
will be made to add an eye tracking step and include in the analysis the movement
of the eyes, an important factor in interpreting the visual accessing cues. Once an
automatic eye cue tracker is developed a more thorough analysis can be made to
see the extent of the EAC that can be determined and induced for a person, and
how such information could be used in applications such as improving interaction
and learning, or overcoming psychological traumas.

Acknowledgement

This work has been co-funded by the Sectoral Operational Program

Human Resources Development (SOP HRD) 2007-2013, financed from the
European Social Fund and by the Romanian Government under the contract
number POSDRU/107/1.5/S/76903 and POSDRU/89/1.5/S/62557.

REFERENCES

[1] Zeng, Z.; Pantic, M; Roisman, G.I.; Huang, T.S. ”A Survey of Affect Recognition Methods:
Audio, Visual, and Spontaneous Expressions,” In IEEE Trans. Pattern Analysis Machine
Intelligence, Vol. 31, Nr. 1, pp.39-58, 2009
[2] Salah, A.A.; Sebe, N.; Gevers, T. ”Communication and Automatic Interpretation of Affect from
Facial Expressions,” Affective Computing and Interaction: Psychological, Cognitive and
Neuroscientific Perspectives. IGI Global, pp. 157-183, 2011.
[3] Ying-Li Tian; Kanade, T.; Cohn, J.F. ”Recognizing upper face action units for facial
expression analysis,” Proc. IEEE Conf. on Computer Vision and Pattern Recognition, vol.1,
pp.294-301, 2000
[4] Hansen, D.W.; Qiang Ji ”In the Eye of the Beholder: A Survey of Models for Eyes and Gaze,”
IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.32, no.3, pp.478-500, 2010
[5] Lanitis, A. ”On the significance of different facial parts for automatic age estimation,” Int.
Conf. on Digital Signal Processing, vol.2, pp. 1027- 1030, 2002
[6] Devi, M.S.; Bajaj, P.R. ”Driver Fatigue Detection Based on Eye Tracking,” Int. Conf. on
Emerging Trends in Engineering and Technology, pp.649-652, 2008
90 Ruxandra Vrânceanu, Laura Florea, Corneliu Florea

[7] Carpenter, R. H. S. ”Movements of the Eyes,” 2nd ed. London: Pion, 1988
[8] Bandler, R.; Grinder, J. ”Frogs into Princes: Neuro Linguistic Programming,” Moab, UT: Real
People Press, pp. 149, 1979
[9] James, W. ”The Principles of Psychology,” Harvard University Press, 1890
[10] Dilts, R.B.; Grinder, J.; Bandler, R. ; DeLozier, J. ”Neuro-Linguistic programming: Volume
1: the study of the structure of subjective experience,” Meta Publications: Cupertino CA,
1980
[11] Elich M.; Thompson R.W.; Miller L. ”Mental imagery as revealed by eye movements and
spoken predicates: a test of Neurolinguistic Programming,” Journal of Counseling
Psychology, vol.32, no.4, pp.622-625, 1985
[12] Farmer, A.; Rooney, R.; Cunningham, J.R. ”Hypothesized eye movements of Neurolinguistic
Programming: a statistical artifact,” Perceptual and Motor Skills, vol.61, pp.717-718, 1985
[13] Hogans, K. ”NLP Eye Accessing Cues: Uncovering the Myth,” taken from
http://www.kevinhogan.com/NLPeyeaccess.htm
[14] Poffel, S.A.; Cross, H.J. ”Neurolinguistic programming: a test of the eye movement
hypothesis,” Perceptual and Motor skills, vol.61, no.3, 1985
[15] Thomason, T.C.; Arbuckle, T.; Cady,D. ”Test of the eye movement hypothesis of
Neurolinguistic Programming,” Perceptual and Motor Skills, vol.51, 1980
[16] Baddeley, M.; Predebon, J. ”Do the eyes have it?: A test of neurolinguistic programming’s
eye movement hypothesis,” Australian Journal of Clinical Hypnotherapy and Hypnosis,
vol.12, no.1. pp.1-23, 1991
[17] Beck, C.E.; Beck, E.A. ”Test of the eye movement hypothesis of Neurolinguistic
Programming: a rebuttal of conclusions,” Perceptual and Motor Skills, vol.58, no.1, pp.175-
176, 1984
[18] Buckner, M.; Meara, N.M.; Reese, E.J.; Reese, M. ”Eye movement as an indicator of sensory
components in thought,” Journal of Counseling Psychology, vol.34, no.3, pp.283-287, 1987
[19] Hernandez, V.O. “A study of eye movement patterns in the Neurolinguistic Programming
model,” Dissertation Abstracts International, vol.42, no.4, 1981
[20] Burke, D. T.; Meleger, A.; Schneider, J.C.; Snyder, J.; Dorvlo, A.S.; Al-Adawi, S. ”Eye-
movements and ongoing task processing,” Perceptual and Motor Skills, vol.96, no.3,
pp.1330-1338, 2003
[21] Diamantopoulos, G. ”Novel eye feature extraction and tracking for non-visual eye- movement
applications,” Ph.D. Thesis, University of Birmingham, 2010
[22] Viola, P.;Jones, M. ”Robust real-time face detection,” Int. Journal of Computer Vision.
vol.57, no.2, pp. 137-154, 2004
[23] Vranceanu, R.; Vertan, C.; Condorovici, R.; Florea, L.; Florea, C. ”A Fast Method for
Detecting Eye Accessing Cues used in Neuro-Linguistic Programming,” IEEE Int. Conf. on
Intelligent Computer Communication and Processing, pp.225-230, 2011
[24] Kanade, T. ”Picture processing by computer complex and recognition of human faces,”
Technical Report, Kyoto University, Department of Information Science, 1973
[25] Zhou, Z.-H.; Geng X., Projection functions for eye detection, In Pattern Recognition,Vol. 37,
Nr. 5, pp.1049 – 1056, 2004.
[26] Martinez, A.M.; Benavente, R. ”The AR Face Database,” CVC Technical Report, vol.24, 1998
[27] Jesorsky, O.; Kirchberg, K.J.; Frischholz, R. ”Robust Face Detection Using the Hausdorff
Distance,” Proc. Int. Conf. on Audio- and Video-Based Biometric Person Authenfitication,
pp.90-95, 2001