American Journal of Internal Medicine

2019; 7(1): 5-8

http://www.sciencepublishinggroup.com/j/ajim
doi: 10.11648/j.ajim.20190701.12
ISSN: 2330-4316 (Print); ISSN: 2330-4324 (Online)

Inter-Rater Reliability of ST-Segment Measurement at a

University Hospital in Argentina
Ignacio Martín Santarelli1, Diego Costa2, Sandra Swieszkowski2, Ricardo Perez de La Hoz2
1
Department of Medicine, Hospital de Clínicas “José de San Martín”, Buenos Aires, Argentina
2
Cardiovascular Intensive Care Unit, Hospital de Clínicas “José de San Martín”, Buenos Aires, Argentina

Email address:

To cite this article:

Ignacio Martín Santarelli, Diego Costa, Sandra Swieszkowski, Ricardo Perez de La Hoz. Inter-Rater Reliability of ST-Segment
Measurement at a University Hospital in Argentina. American Journal of Internal Medicine. Vol. 7, No. 1, 2019, pp. 5-8.
doi: 10.11648/j.ajim.20190701.12

Received: December 14, 2018; Accepted: January 15, 2018; Published: January 31, 2019

Abstract: The accurate determination of the electrocardiographic ST-segment elevation in a patient with clinical suspicion
of acute coronary syndrome is essential for treatment with urgent myocardial reperfusion. The aim of this study was to
determine the inter-rater reliability of ischemic and non-ischemic ST-segment elevation measurement among physicians with
different specialties and experience. We performed an observational, cross-sectional study, with a comparative correlation and
paired sampling. 56 physicians from a university hospital in Buenos Aires city were included: Cardiologists from the Coronary
Care Unit (CCU) and Cardiology Division, Internal Medicine physicians from the hospitalization, ambulatory care and
Emergency divisions, and third- and fourth-year Internal Medicine residents. Each participant analyzed 6 electrocardiograms
and was asked to determine the magnitude of the ST-segment elevation at the J-point, and the corresponding diagnosis. The
inter-rater coefficient was lower than 0.2, and the global kappa coefficient was 0.06 (p < 0.001). The global correct
interpretations were: inferior wall myocardial infarction (MI): 89.3%; anterior wall MI: 51.8%; lateral wall MI: 75%; left
bundle branch block: 91.1%; left ventricle hypertrophy: 44.6%; acute pericarditis: 25%. We believe that the low correlation
was probably due to the difficulty in determining the J-point. These findings could suggest the need to strengthen the
electrocardiographic concepts of ischemia, and the differentiation between ischemic and non-ischemic ST-segment elevations.

Keywords: J Point, Electrocardiogram, Ischemia, Infarct, ST Segment

where nor how to measure the ST-segment elevation [3].

1. Introduction Some researchers have used the J-point, while others used a
The precise determination of an ST-segment elevation, on 60 or 80 msec distance from the J-point [3-5]. Research
the electrocardiogram of a patient in whom an acute coronary conducted at Emergency Departments have reported between
syndrome is suspected, is the key to correctly applying the 1.9 and 8% of all patients who attend because of chest pain
electrocardiographic criteria for urgent myocardial caused by acute myocardial infarction are mistakenly
reperfusion. According to the fourth definition of acute discharged with an incorrect diagnosis, probably due to
myocardial infarction [1] and the guidelines from the overlooking the ST-segment elevation [6]. On the other hand,
American Heart Association [2], a diagnostic ST-segment QRS repolarization abnormalities may be responsible for
elevation is defined as the elevation of the J point in, at least, STEMI false-positives, therefore limiting the usefulness of
two adjacent leads ≥ 1mm in all leads other than V2-V3, ECG as a sole diagnostic tool [7].
where the cutting point is ≥ 2 mm in men older than 40 years, The general objective of this work was to determine the
≥ 2.5 m in men under 40 years, and ≥ 1.5 mm in women inter-rater reliability between physicians of different
regardless of age. The American Heart Association defined specialties and expertise in determining the ST-segment
these findings as class I and “A” level of evidence for urgent elevation due to ischemic and non-ischemic causes, and in
myocardial reperfusion on the appropriate clinical context. stablishing an electrocardiographic diagnosis.
Despite these specific criteria, it has not been defined
6 Ignacio Martín Santarelli et al.: Inter-Rater Reliability of ST-Segment Measurement at a
University Hospital in Argentina

2. Materials and Methods Kappa coefficient was calculated to determine the global
correlation between the participants. In order to compare the
We designed a cross-sectional, observational, comparative percentage values, the chi-square test was used. A statistical
correlational study, using paired samples. The following significance level of 0.05 was established. The statistic
physicians from a University Hospital from Buenos Aires, software used was Stata 12.0.
Argentina, were enrolled: 1) Cardiologists from the
Cardiovascular Intensive Care Unit and Cardiology Division,
2) Clinicians who work in the Internal Medicine Department 3. Results
and Ambulatory Care division, 3) third- and fourth-year We analyzed data obtained from 56 physicians. 44.64%
Internal Medicine Residents, and 4) Internists who work in were Clinicians, 19.64% were Cardiologists, 26.79% were
the Emergency Department. Written authorization from each fourth-year Internal Medicine residents, and 8.93% were
chief of Department and from the Bioethics Committee was third-year Internal Medicine residents. 58.18% of the
obtained. This protocol was conducted according to the participants performed their activities on critical areas, such
Declaration of Helsinki. Data collection was carried out from as the CCU and the Emergency Department.
October 23rd to December 31st, 2015. Intraclass correlation coefficient according to specialty
Each participant was requested to analyze 6 was lower than 0.2, with no statistical significance. Global
electrocardiograms, which were obtained from the website kappa coefficient was 0.006 (p < 0.001).
CardioNetworks (http://www.cardionetworks.org) and Life In Global successful diagnostic percentages for each ECG
the Fast Lane (http://lifeinthefastlane.com). They are Open presenting an ischemic ST-elevation cause were: inferior
Access, according to each website’s declaration. The wall MI: 89.3%, anterior wall MI: 51.8%; lateral wall MI:
following diagnoses were included: Anterior acute 75%. For the remaining diagnosis, results were: left-bundle
myocardial infarction (MI), acute pericarditis, left bundle branch block: 91.1%; left ventricle hypertrophy: 44.6%;
branch block, lateral acute MI, left ventricle hypertrophy, acute pericarditis: 25%. Regarding the latter
inferior acute MI. They were exhibited on A4 size paper, and electrocardiogram, 30% of participants informed the
for each of them, through a questionnaire, the participants alternative diagnosis of early repolarization, ventricular
were requested to determine the magnitude in millimeters of extrasystoles, left bundle branch block, among others. In
the ST-segment elevation at the J-point, and the more than half of the cases (55.4%), left ventricle
electrocardiographic diagnosis. No clinical data was provided hypertrophy was mistaken for left ventricle overload, lateral
for the cases, permitting the participants to concentrate solely wall MI and inferior wall MI.
on the electrocardiograms. The identity of both the patients to During analysis by medical specialty, we found the
whom the electrocardiograms belong and that of the greatest percentage of correct diagnoses was among
participants was preserved. Cardiologists, as stated in table 1. For lateral wall diagnosis,
The intraclass correlation coefficient was determined for the difference between Cardiologist and Clinicians was
each observation, and for each previously defined subgroup. statistically significant (p= 0.045).
Table 1. Correct electrocardiographic diagnosis according to specialty [%].

Cardiologists Clinicians 4th year residents 3rd year residents p

Anterior wall MI 81.82 56 26.67 40 0.040
Lateral wall MI 100 72 60 80 0.13
Inferior wall MI 100 100 66.67 80 0.004
Acute pericarditis 63.64 20 13.33 0 0.006
Left-bundle branch block 90.91 84 100 100 0.33
Left ventricle hypertrophy 54.55 40 40 60 0.7

As for correct diagnoses according to seniority in the medical practice, tendency to provide correct answers was greater in
those physicians with over 5 years of experience, especially among electrocardiograms with signs of ischemia. Data is
presented in table 2.

Table 2. Correct electrocardiographic diagnosis according to experience [%].

Over 5 years Less than 5 years p

Anterior wall MI 66.67 38.71 0.040
Lateral wall MI 75 74.19 0.946
Inferior wall MI 100 80.65 0.022
Acute pericarditis 29.17 19.35 0.396
Left-bundle branch block 83.33 96.77 0.086
Left ventricle hypertrophy 41.67 45.16 0.796

Finally, after comparing answers from physicians belonging to critical care units to those who work at non-critical units,
there were no statistically significant differences (table 3).
 American Journal of Internal Medicine 2019; 7(1): 5-8 7

Table 3. Correct electrocardiographic diagnosis comparing critical-care units and non-critical care units [%].

Critical care units Non-critical care units p

Anterior wall MI 47.83 54.55 0.698
Lateral wall MI 82.61 69.70 0.244
Inferior wall MI 95.65 84.85 0.186
Acute pericarditis 34.78 18.18 0.099
Left-bundle branch block 86.96 93.94 0.387
Left ventricle hypertrophy 30.43 54.55 0.094

fulfilled criteria for ST-segment elevation [10]. A different

4. Discussion study compared inter-rater variability between 20 Emergency
As already mentioned, the recently published Fourth physicians, 20 Emergency Medicine residents and 20 fourth-
Definition of Myocardial Infarction clearly establishes the year Medicine students (60 participants in all), analyzing
criteria a J point elevation must meet [1]. Nevertheless, 2,070 ST-segments. There were no statistically significant
determining the J point can be difficult due to preexisting or differences using 2 mm diagnostic criteria [11]. We cannot
ischemia-induced conduction disturbances or early determine whether this difference between this study and our
repolarization [8]. results lies in the participant’s performance, or in the fact that
A Multidisciplinary Standardized Reporting Criteria Task the former was based on a much greater
Force recommends using the TP-segment as a reference for electrocardiogram/participant ratio than ours. Lastly,
the isoelectric baseline (unless tachycardia or artifacts McCabe, et al. [10] conducted a cross-sectional survey
preclude this measurement) and measuring ST-segment among 124 physicians (Emergency Physicians, Cardiologists
elevation 0.06 seconds from the J point [9]. The lack of a and Interventional Cardiologists), consisting of 36
universalized practice when determine the ST-segment deidentified ECGs that had previously resulted in putative
elevation is probably responsible for the inter-rater variance, STEMI diagnoses (the reference standard for STEMI
which has been addressed in reduced groups and among diagnosis was subsequent emergent coronary arteriography).
experienced Emergency Medicine physicians. Participants were asked the following: “Based on the ECG
A case-control study conducted in The Netherlands [8] above, is there a blocked coronary artery present causing a
analyzed 53 ECGs recorded preceding emergency STEMI?”. Inter-reader agreement (kappa) for ECG
catheterization of acute coronary syndrome patients with a interpretation was 0.33. The sensitivity to identify real
completely occluded culprit artery and 88 control elective STEMIs was 65% and specificity was 79%. They found no
ECGs recorded in the cardiology outpatient clinic, using a significant difference by specialty. An Italian research group
software (LEADS) which performs baseline corrections, and the Italian Society of Emergency Medicine (SIMEU)
calculates an averaged beat and computes global onset-Q, J- reproduced the same survey three years later and determined
point and end-of-T landmarks. The authors obtained the an 85.2% positive predictive value of a STEMI interpretation
highest sensibility and specificity (94.3 and 93.2% among all readers, and a 52.8% negative predictive value.
respectively) for ST-elevation myocardial infarction Overall accuracy (the ability to discriminate true STEMI
(STEMI) when ST-elevation was measured 10 mseg after the pattern from false STEMI electrocardiographic changes) was
J-point. 69.1%. The authors concluded ECG interpretation lacks the
Our study resulted in a much lower inter-rater reliability necessary sensitivity and specificity to be considered a
than others. Lim et al. [3] recruited 30 physicians from an reliable single diagnostic test [13].
Emergency Department in Singapur, including 6 consultant To further complicate matters, there are non-ischemic
Emergency physicians, 2 senior Emergency Medicine causes of ST-segment elevation. These patients, despite
residents, and 22 physicians trained in specialties different fulfilling the already mentioned criteria from the American
from Emergency Medicine. They found an excellent level of Heart Association [2], should not be subjected to urgent
agreement among physicians from different specialties myocardial reperfusion. A study conducted by Brady, et al.
(intraclass correlation coefficient= 0.85), but no comparison [13] at a University Hospital tested the determination of ST-
between experienced and less-experienced physicians could segment elevation at the J-point and its etiology on
be made because the number of the former was small. Erling experienced Emergency physicians, retrospectively
et al. [6] found the smallest the ST-segment elevation, the comparing it to Cardiologists who had evaluated patients
greater the discordance rate among experienced Emergency with chest pain. They concluded that the performance of
physicians: no discordance when ST-segment elevation was Emergency Physicians was: acute pericarditis: 100%; left
over 10 mm (kappa coefficient= 1), and significant bundle branch block: 97%; left ventricle hypertrophy: 96%,
discordance when ST-segment elevation was less than 5 mm among other diagnoses we did not include. These results
(kappa coefficient= 0.48). It has also been described ST- differ from ours, compared to those obtained in the Critical
segment elevation magnitude differs according to whether it Care Unit subgroup (34.78%, 86.96% and 30.43%
is determined at the J point, or 60 milliseconds after it: in the respectively). It is important to highlight the statistically
first case, less patients with retrospective diagnosis of MI significant difference obtained between the diagnosis of
anterior and inferior wall MI, both according to specialty and
8 Ignacio Martín Santarelli et al.: Inter-Rater Reliability of ST-Segment Measurement at a
University Hospital in Argentina

experience. Regarding early repolarization, an Med J 2015 Jan 23. pii: emermed-2014-204102. doi:
electrocardiographic pattern which may mimic an ischemic 10.1136/emermed-2014-204102. [Epub ahead of print].
ST-segment elevation, a different study observed it was [4] Koren G, Weiss AT, Hasin Y, et al. Prevention of Myocardial
correctly interpreted by 90% of cardiologists and 81% of Damage in Acute Myocardial Ischemia by Early Treatment
Emergency physicians. Moreover, overdiagnosis (that is, with Intravenous Streptokinase. N Engl J Med 1985; 313:
incorrect diagnosis of MI rather than early repolarization) 1384-9.
was greater in the Emergency physician group (27.6%) than [5] Verstraete M, Bernard R, Bory M, et al. Randomised trial of
in the Cardiologist group (17.3%). Underdiagnosis (mistaken intravenous recombinant tissue-type plasminogen activator
diagnosis of early repolarization over MI) was lower in the versus intravenous streptokinase in acute myocardial
Cardiologist group (2.8% against 9.7%). All differences were infarction. Report from the European Cooperative Study
Group for Recombinant Tissue-type Plasminogen Activator.
statistically significant [15]. We must remember the Lancet 1985; 1: 842-7.
participants of our study were not provided with the medical
records of the patients whose electrocardiograms were [6] Erling BF, Perron AD, Brady WJ. Disagreement in the
analyzed. Even though this lack might have conditioned to interpretation of electrocardiographic ST segment elevation: a
source of error for emergency physicians? Am J Emerg Med
some extent the electrocardiographic diagnosis, we do not 2004; 22: 65-70.
believe it accounts for the inter-rater variability of the J-point
elevation since this is an objective determination. [7] Tanguay A, Lebon J, Brassard E, et al. Diagnostic accuracy of
prehospital electrocardiograms interpreted remotely by
emergency physicians in myocardial infarction patients. Am J
5. Conclusion Emerg Med. 2018 Sep 6. pii: S0735-6757 (18)30741-1.

In this study, we found a low inter-rater correlation in [8] Man S, Ter Haar CC, de Jongh MC, et al. Position of ST-
determining the ST-segment elevation at the J point, deviation measurements relative to the J-point: Impact for
ischemia detection. J Electrocardiol. 2017 Jan - Feb; 50 (1):
evidencing a difficulty in establishing the latter. Moreover, 82-89.
the percentage of correct answers for each
electrocardiographic diagnosis was variable, both within each [9] Multidisciplinary Standardized Reporting Criteria Task Force,
specialty and between them. Experienced cardiologist had the Hollander, J. E., Blomkalns, A. L., et al. Standardized
reporting guidelines for studies evaluating risk stratification of
most ability to establish a correct diagnosis. This could imply emergency department patients with potential acute coronary
the need to strengthen, especially among trainee-physicians, syndromes. Acad Emerg Med. 2004 Dec; 11 (12): 1331-40.
the electrocardiographic signs of ischemia, and to
differentiate ischemic and non-ischemic causes of ST- [10] Smith SW. ST segment elevation differs depending on the
method of measurement. Acad Emerg Med 2006; 13: 406-12.
segment elevation.
[11] Tandberg D, Kastendieck KD, Meskin S. Observer variation
in measured ST-segment elevation. Ann Emerg Med 1999; 34:
Conflict of Interest 448-52.
None to declare. [12] McCabe, JM, Armstrong, EJ, Ku, I., et al. Physician Accuracy
in Interpreting Potential ST-Segment Elevation Myocardial
Infarction Electrocardiograms. J Am Heart Assoc. 2013 Oct 4;
2 (5): e000268.
References
[13] Veronese G, Germini F, Ingrassia S, et al. Emergency
[1] Thygesen K, Alpert JS, Jaffe AS, et al.; Fourth Universal physician accuracy in interpreting electrocardiograms with
Definition of Myocardial Infarction (2018). Circulation. 2018 potential ST-segment elevation myocardial infarction: Is it
Nov 13; 138 (20): e618-e651. enough? Acute Card Care. 2016 Mar; 18 (1): 7-10.
[2] O'Gara PT, Kushner FG, Ascheim DD, et al. 2013 [14] Brady WJ, Perron A, Ullman E. Errors in emergency
ACCF/AHA Guideline for the Management of ST-Elevation physician interpretation of ST-segment elevation in
Myocardial Infarction: A Report of the American College of emergency department chest pain patients. Acad Emerg Med
Cardiology Foundation/American Heart Association Task 2000; 7: 1256-60.
Force on Practice Guidelines. J Am Coll Cardiol 2013; 61:
485-510. [15] Turnipseed SD, Bair AE, Kirk JD, Diercks DB, Tabar P,
Amsterdam EA. Electrocardiogram Differentiation of Benign
[3] Lim HC, Salandanan EA, Phillips R, Tan JG, Hezan MA. Early Repolarization Versus Acute Myocardial Infarction by
Inter-rater reliability of J-point location and measurement of Emergency Physicians and Cardiologists. Acad Emerg Med.
the magnitude of ST segment elevation at the J-point on ECGs 2006 Sep; 13 (9): 961-6.
of STEMI patients by emergency department doctors. Emerg