EM 1110-2-1009 1 Jun 02

Chapter 4 Sources of Measurement Error and Instrument Calibrations

pointing error centering error leveling error reading error An expected variance can be calculated for each type of error component. These are added together to estimate the measurement standard deviation. Measurement precision is critical information for analyzing positioning accuracy through survey preanalysis and for assigning appropriate weights to measurements in a least squares network adjustment. b. Systematic error. Systematic error is caused by deficiencies in the physical or computation model used to process the measurements. Systematic error can produce a biased value for the estimated mean (i.e., it is significantly different than the desired true value). For example, systematic errors in EDM instruments include:

EDM/prism zero error EDM scale error EDM signal refraction error EDM cyclic error Corrections to these systematic errors are determined by various instrument calibration techniques. Error is eliminated from the data during post-processing by applying the resulting correction values to the measurements. Where systematic errors cannot be adequately determined, sometimes their effects can be randomized with specialized observation procedures. For example, if the surveying measurements are repeated under different field conditions--e.g., at different times of the day, under different weather conditions, and even on different days. c. Instrument calibrations. Measuring equipment used for deformation monitoring surveys must be maintained in adjustment and undergo periodic calibration so as to minimize systematic error. Manufacturer specifications and recommendations will be used as the basis for any internal instrument adjustments. EDM instruments must be calibrated for the instrument/prism zero correction and for scale error, as these may change with time. EDMs should be calibrated at least once a year for engineering projects of high precision, or before and after an important project. The calibration for zero error must account for all actual combinations of EDM-reflector pairings that are used on the survey since each reflector may produce a different error value (i.e., there is a different constant for each pairing).

4-1

EM 1110-2-1009 1 Jun 02

4-2. Optical Pointing Error a. Optical pointing. All measurements with optical theodolites are subject to pointing error due to such factors as: target design, prevailing atmospheric conditions, operator bias, and focusing. The approximate magnitude of a single-direction pointing error (i.e., standard deviation) is directly related to the magnification of the theodolite telescope. s p = 45" / M (Eq 4-1) where s p = instrument pointing error (arcseconds) M = objective lens magnification For example, an objective lens with 30 times magnification would have a pointing error (s p ) of approximately 1.5 arcseconds (one-sigma) for a single direction. Taking repeated measurements from the same setup reduces the standard deviation by a factor of 1/sqrt (n), with n being the number of repetitions (see Figure 4.1). The standard deviation (due to pointing error) of a single direction measured by repetition in (n) sets can be determined from the following formula: s = s p / sqrt (n) (Eq 4-2) where s = single direction pointing error (arcseconds) s p = instrument pointing error n = number of repeated sets This result assumes that each pointing to the backsight and foresight has the same precision. b. Empirical determination of pointing error. Pointing error can be determined for a given instrument by making direct observations to a target point (for use with an optical theodolite with micrometer scales). First, the operator sets and levels the instrument and target according to standard techniques. Next, the operator points the theodolite crosshairs to the target and records the direction reading. Repeat the pointing procedure at least twenty times to gather a sufficient number of direction readings for calculating a mean error value from the data. Then compute the standard deviation of the resulting test data (in arcseconds). The result is an empirical pointing uncertainty for the instrument. The reading error component must be subtracted from the combined pointing and reading error. s p = sqrt [ (s p + s r )2 - (s r )2 ] (Eq 4-3) where s p = instrument pointing error (arcseconds) (s p + s r ) = combined pointing and reading error s r = instrument reading error (arcseconds) Reading error ( s r ) is determined independently from either the standard deviation of a series of twenty readings of the theodolite scales with the instruments motion locked, or from the instrument specifications. c. Minimization of pointing error. Instrument pointing error degrades the precision of horizontal and vertical angle measurements to a greater degree on long baselines. It can be minimized by observing
4-2

EM 1110-2-1009 1 Jun 02

survey targets under high magnification. Some instruments, such as the Leica T3000 Electronic precision theodolite, are equipped with interchangeable eyepieces that provide up to 59 times magnification (compared to 30-43 times when focused to infinity with a standard eyepiece). The technique of averaging repeated sets of angles is used to reduce instrument pointing error when greater lens magnification is not available. d. Atmospheric disturbance. Air turbulence can greatly interfere with instrument pointing accuracy. This is especially true on structures exposed to direct sunlight (e.g., dams, concrete lock walls). Very little can be done to eliminate these effects except to observe under more favorable conditions, such as in the early morning hours, on overcast days, and during cooler seasons. Repeated sets of angles will again reduce instrument pointing error with poor observing conditions. The spread of repeated measurements (final standard deviation after repetitions) should be checked to see whether it exceeds the required measurement accuracy.

Figure 4-1. Standard error decreases as repeated number of directions increases

e. Electronic pointing . Precision EDM instruments use electronic pointing instead of optical pointing to retrieve the EDM signal. Electronic pointing is a trial-and-error targeting procedure used to find the peak reading of the EDM signal strength indicator as the instrument fine motion screws are adjusted. A stronger signal return produces more accurate distance measurements and minimizes the use of anomalous phase patterns near the return signal beam edges. This is especially critical when making measurements over short distances (i.e., less than 20 m), where the EDM 'phase inhomogeneity' effect will be most pronounced. f. Minimization of electrical pointing error. The modulation wave front issuing from a properly designed and operating instrument is at all points equidistant from the center of the instrument. However, the wave front may be distorted in passing through the modulator, and then a portion of the wave may be ahead or behind the remainder. An instrument senses both reflectors as equidistant because the phase of the modulated wave is the same for both. If the instrument is moved in azimuth slightly, the distance that it reads would change. This type of error may be detected simply by multiple pointings at a reflector. If different pointings yield different results, it may be necessary to take several readings in the field, swinging off the target and then back until two or three sets of readings agree well. Practice in the field
4-3

EM 1110-2-1009 1 Jun 02

may help eliminate this problem as an experienced operator tends to point and adjust an instrument in the same way for each measurement. 4-3. Instrument Leveling Error a. General. Even when exercising great care in setting up the instrument, there is a definite limit to the ability to level the instrument due to the sensitivity of the plate level bubble. Therefore, at least some inclination of the instrument's vertical axis with respect to the plumb line is unavoidable. Theodolite inclination error is computed (for the purpose of measurement design) as: s I = (0.2) (bubble sensitivity per division) or at five times less than the level bubble sensitivity of the instrument (e.g., for a thirty-second bubble one gets approximately 30"/5 = 6" leveling error). The effect of vertical axis inclination error (s I ) on the measured horizontal angle, is computed as: s L = (s I ) / [tan (Z)] (Eq 4-4) where s L = leveling error (arcseconds) s I = inclination error Z = measured zenith angle Leveling errors affect the accuracy of horizontal angle measurements mainly when observing over steep vertical angles. This situation is common in monitoring embankment dams where reference monuments set on the crest of the dam are used to observe monuments set at the toe of the structure (or vice-versa). As a result the measurements between the backsight and foresight stations will be subject to 'standing axis' error causing incorrect horizontal circle readings. b. Standing axis correction. The inclination of the instruments vertical axis can be measured and corrected for with a special observing technique. First, with the instrument properly leveled and centered, the inclined target (above or below) is directly sighted through the scope. With the theodolite vertical circle clamp locked, the vertical angle (VTAR) is read and recorded, then the horizontal clamp is loosened. Next, the instrument is turned 90 degrees to the right of the line-of-sight and the vertical circle (still locked) is again read and recorded ( VR ). Next, the instrument is rotated 90 degrees to the left of the line-of-sight and the vertical circle reading (still locked) in its new position is also recorded ( VL ). The maximum value of the leveling error correction for a single pointing is half the difference of the right and left vertical circle readings multiplied by the tangent of the vertical angle initially measured to the inclined target. LC = [ (VR - V L ) / 2 ] tan (VTAR) (Eq 4-5) where LC = leveling correction (arcseconds) VR = recorded vertical angle right, VL = recorded vertical angle left, VTAR = vertical angle to target. The horizontal circle reading is corrected by this amount right or left according to its sign (i.e., with a positive inclination value, the correction is added as a right deflection).

4-4

EM 1110-2-1009 1 Jun 02

c. Predetermined tolerance. Not every horizontal direction will need to be corrected for leveling error. The standing axis correction needs to be computed only for steep lines of sight. A vertical angle threshold value can be pre-determined from the leveling sensitivity of the instrument to decide if the correction needs to be applied on a given target sighting. This tolerance angle is found by solving the above correction equation for the VTAR term with a known inclination error for the instrument. For example, with an instrument plate bubble sensitivity of 30 arcseconds (taken from manufacturers instrument specifications), and a maximum allowable tolerance for the final correction value of one (1) arcsecond; vertical angles greater than 10 degrees from horizontal would require the correction. On embankment dams, toe monument stations observed from reference stations on the dam crest at the opposite end of the structure will typically have vertical angles less than 10 degrees. d. Internal bi-axial compensator. Modern instruments, such as the Leica TC2002 Total Station and the T3000 Electronic precision theodolite, can correct horizontal angle readings for slight mislevelment error by employing a bi-axial compensator. The compensator senses the degree of nonverticality of the vertical axis using two liquid sensors mounted along perpendicular horizontal axes within the instrument. Some instruments equipped with a bi-axial compensator will automatically compute and apply corrections to the horizontal circle reading. 4-4. Instrument Centering Error a. General. Either forced centering or an optical plummet built in to the tribrach are standard means for centering during instrument/target setups. Centering errors are caused when the vertical axis of the instrument (or target/prism) is not coincident with (i.e., collimated above) the reference mark on the control point monument. b. Short baselines. The uncertainty (standard deviation) of an angle measurement due to centering error can be approximated for the case where centering methods and distances between backsight and foresight stations are similar: s b = sqrt [ ( s c 2 ) ( 2 ) ( 4 / D 2 ) ] (Eq 4-6) where s b = angle uncertainty due to centering error s c = centering standard deviation = 206264.8 D = distance between stations (mm) The centering standard deviation (sc ) is computed as: s c = ( 0.5mm ) ( HI ) for tribrach optical plummets, and, s c = ( 0.1mm ) ( HI ) for forced centering. In each case the height of instrument (HI) is measured in meters. For example, a measurement made over a distance of 100 m using standard tripods and tribrachs (with optical plummet), can introduce as much as 3.1 arcseconds error in the horizontal angle. Therefore, taking repeated sets of angles (re-centering

4-5

EM 1110-2-1009 1 Jun 02

between sets) is recommended to reduce the effects of centering error. As the distance increases between the instrument and target stations, the influence of centering error is reduced. Therefore, centering errors will be especially dangerous for horizontal angle measurements that are made over very short baselines. c. Optical plummets. Nadir and zenith plummet surveying instruments (Figure 4-2) are specially designed for precise centering and collimation. These types of instruments are available commercially for geodetic, deformation monitoring, and mining surveying applications. Their use has been recommended for deformation surveys because high centering tolerances are required to ensure survey repeatability.

Figure 4-2. Nadir plummet instrument from Leica Co. Nadir plummets operate like an automatic level but establish a vertical line of sight. The tribrach is centered over the surveyors mark using the nadir plummet. Then a theodolite or a GPS antenna is placed in the tribrach by forced centering

Figure 4-3. Comparison of optical plummet centering devices. The left-hand image shows the field-of-view and centering circle for a standard optical plummet fixed-mounted in a tribrach. The right hand image shows the greater magnification and centering ability provided by a nadir plummet instrument. Both images show the same brass disk at an instrument height of approximately 1.5 meters

4-6

EM 1110-2-1009 1 Jun 02

d. Reference marks. Ground based survey control monuments should be designed with a welldefined reference mark--see Figure 4-3. This visual feature will minimize difficulty in re-establishing an instrument or target over the reference point, thus, ensuring greater repeatability in centering. Further information on monuments and targets for deformation surveys can be found in Chapter 2 and in the references listed in Appendix A. 4-5. Horizontal Angle Measurement Error a. General. The precision (standard deviation) of horizontal angle measurements made with a theodolite can be predicted by a summation of variances as follows: s angl = sqrt [ s2 p + s2 r + s2b +s2 L ] (Eq 4-7) where s angl = horizontal angle error (arcseconds) s p = pointing error s r = reading error s b = centering error s L = leveling error The above variance components can be determined from the formulas presented in Sections 4-2 thru 4-4. b. Example horizontal angle error. Using the above formulas, for an objective lens magnification equal to 30 times, and centering with a tribrach optical plummet with a 1.5 m HI, and a level bubble sensitivity of 20"/2 mm with a zenith angle of 85 degrees, and a 100 meter slope distance to the backsight and foresight stations, the precision of the horizontal angle is estimated to be approximately 3.5 arcsec. Using this same example with two independent sets of repeated measurements, the standard deviation of the angle measurement is approximately 2.4 arcseconds (at the one-sigma level). Using an electronic theodolite equipped with biaxial compensator, leveling and reading error are negligible. 4-6. Electronic Distance Measurement Error a. General. The precision (standard deviation) of distance measurements made with EDM instruments may be expressed in a general form as: s = sqrt [ a 2 + b2 S2 ] (Eq 4-8) where a = errors of the phase measurement, centering, and calibration errors b = scale error due to uncertainties in the determination of the refractive index and the calibration of the modulation frequency, S = measured distance. The standard deviation for near infrared (NIR) and lightwave carrier EDM instruments can be determined by the following summation of variance components: sS = where sqrt [ s2 res + s2 c + s2 cal + s2 ref ] (Eq 4-9)

4-7

EM 1110-2-1009 1 Jun 02

sS = s res = sc = s cal = s ref =

slope distance measurement error resolution of instrument centering error calibration error refractive index correction error

(1) Resolution (s res). EDM Measurement resolution varies according to each specific type of instrument, but is generally a function of both the modulated wavelength and its sensitivity to detecting signal phase difference. The constant value used for EDM resolution (sres) is normally given in the manufacturers equipment specifications. (2) Refraction correction error (s ref). The accuracy of the computed refractive index correction depends on the accuracy of the temperature and pressure values input into the refraction correction formula. To compute the effect of inaccurate temperature and pressure on the corrected distance the following approximation can be used: s ref = sqrt [ [ ( s2N ) / N2 ] (S2 ) ] (Eq 4-10) where s ref sN N S = = = = error in refraction correction determination error in refractive index determination estimated refractive index slope distance

For a simplified estimate of the refraction correction error, an approximate formula for the above quantities can be used. The following refraction correction error equations neglect the partial water vapor pressure without creating an extreme distortion to the distance error estimation: N = 1 + [ ( N G ) ( P ) ] / [ (3.709) ( T ) ] / (1106 ) (Eq 4-11) s N = sqrt [ (A ) (s where
2 2 Temp

) + ( B ) (s

2 Press

) ] / [110 ]
12

(Eq 4-12) NG = (287.604) + [(4.8864) / (2 )] + [(0.068) / (4 )] = EDM carrier frequency wavelength (micrometers) T = 273.15 + t t = temperature in C (std. = 15C) P = pressure in mbar (std. = 1013.25 mb) A = [ (-N G / 3.709) (P / T2 ) ] 2 B = [ (N G / 3.709) / ( T ) ]2 s Temp = temperature measurement uncertainty s Press = pressure measurement uncertainty

Relative humidity is the least critical parameter for determination of refractive index for light and NIR source EDM instruments. Temperature differences between stations can be substituted for temperature measurement uncertainty to give more conservative estimates of the refraction correction error. The error also can be determined sufficiently by simply multiplying the distance by 1 ppm for every C of temperature measurement error to roughly obtain the standard deviation of the refraction correction term.

4-8

EM 1110-2-1009 1 Jun 02

(3) Centering error (s c ). The estimated centering error for distance measurements (s c ) can be calculated according to formulas presented earlier in this chapter. For distance measurements the centering error from both the instrument and target are combined: s c = sqrt (s 2 inst + s 2 tar ) (Eq 4-13) where s inst = instrument centering error s tar = target centering error (4) Calibration error (s cal). Calibration error refers to the precision (standard deviation) of the correction constants determined from instrument calibrations. These are standard outputs of statistical tests performed during the calibration data reduction process (typically less than instrument resolution). (5) Slope-to-Horizontal distance error. Uncertainty in horizontal distances stem from both the precision of the height difference determination and the precision of the slope distance measurement. s horz = ( S / H ) sqrt ( s S 2 + s 2 hdiff ) (Eq 4-14) where s horz = horizontal distance error s S = slope distance error s hdiff = height difference error S = slope distance H = horizontal distance b. Example EDM distance error. With an EDM instrument resolution of 3.0 mm, carrier wavelength of 0.850 micrometer, and centering with a tribrach optical plummet with a 1.5 m HI for both the instrument and target stations, the resulting distance determination will have an uncertainty of approximately 3.2 mm (one-sigma level, at 15C, 1013 mb, over a distance of 200 m), assuming temperature and pressure were measured to 1C and 3 mb, respectively. If a temperature difference of 7C is substituted for temperature error, the distance error estimate increases to 3.5 mm. Using the example values above and a height difference uncertainty of 3 mm, station height difference reductions add approximately 1 mm (one-sigma error) to the horizontal distance over a 20 m height difference. 4-7. Zenith Angle Measurement Error a. General. Zenith angle measurements are determined by the difference of two direction measurements, with one direction defined by the vertical axis of the theodolite. Theodolite-based zenith angle measurement precision can be predicted as follows: s zen = sqrt (s2 p + s2 r + s2 I + s ref ) (Eq 4-15) where s zen = zenith angle error (arcseconds) s p = pointing error s I = inclination error s r = reading error s ref = refraction error

4-9

EM 1110-2-1009 1 Jun 02

The above variance components can be determined from the formulas presented in sections 4-2 thru 4-3. One of the main sources of systematic error in zenith angle measurement is due to atmospheric refraction. Zenith angle error can be roughly determined as a function of the slope distance: s ref = sqrt [ ( S2 ) / ( 4 R2 ) ( 2 ) (4 ) ] (Eq 4-16) where s ref = refraction error (arcseconds) S = measured slope distance (m) R = mean radius of the earth (~ 6374000 m) = 206264.8 For example, over a distance of 100 m, the expected error due to refraction would be approximately 3 arcseconds. b. Trigonometric height traversing. Station height differences determined from zenith angle and horizontal or slope distance measurements are not always as accurate as differential leveling. Methods of trigonometric heighting are warranted where differential leveling would accumulate excessive random error. If differential leveling is attempted over steep slopes between crest and toe stations on a dam, then most of the total error in the height difference is introduced during the numerous instrument set-ups. In cases like this, trigonometric height traversing can directly substitute for differential leveling methods. 4-8. Refraction of Optical Lines of Sight a. General. All types of measurements with optical instruments are affected by atmospheric refraction. The line is refracted when the air temperature is not homogeneous but varies across the line of sight with a gradient of dT/dy. Refraction effects are most pronounced in leveling and zenith angle measurements when the line of sight is near the ground surface (e.g., 2 meters or less), and has a significantly different temperature than the layers of air above the surface. The horizontal effects of refraction may also be dangerous if the line of sight of the observed horizontal direction runs parallel and very close to prolonged objects of a different temperature, such as walls in tunnels, galleries of long dams, or rows of transformers or turbines at a different temperature than air flowing in the center of the gallery. b. Refraction effects. If the temperature gradient (dT/dy) across the line of sight is constant at all points of the line, then the line is refracted along a circular curve (Figure 4-4) producing an error ('e') of pointing to a survey target.
Y

S A

B
e

B' X
Figure 4-4. Refracted line of sight

4-10

EM 1110-2-1009 1 Jun 02

The value of 'e' can be estimated from: e = (k)(S2 ) / (2R) (Eq 4-17) where k is a coefficient of refraction, S is the distance to the target, R is the earth radius. The coefficient of refraction (k) is a function of temperature gradient (dT/dy) and it can be approximated (for the average wavelength of the optical spectrum) by: k = 508.8 (P/T2 )(dT/dy) (Eq 4-18) where P is barometric pressure [in mb], T is the average (absolute) air temperature in Kelvin, t in deg C and T in Kelvin are related by T = 273.15 + t. By taking the average radius of earth (R = 6371 km), and substituting (Eq 4-17) into (Eq 4-18), the pointing error may be expressed as a function of the gradient of temperature: e = 3.9 (PS2 /T2 )(dT/dy) 10-5 (Eq 4-19) For example, given a line of sight S = 200 m; temperature of air t = +30 deg C (i.e., T = 303.15 Kelvin); barometric pressure 1000 mb, and a constant gradient of temperature across the line of sight, dT/dy = 0.5 deg C/m. From (Eq 4-19) above, we have: k = 2.8 and e = 8.5 mm. Usually, the temperature gradient differs from one point to another, producing an irregular shape of the refracted line of sight (Figure 4-5). In this case, the gradient of temperature and the coefficient of refraction also change along the line of sight (x direction) and k is a function of position [k(x)]. The pointing error should be calculated from: e = (1/R) integral [ k(S-x) dx ]; from 0 to S (Eq 4-20) If gradients of temperature are measured at discrete points, say in the middle of each segment si in Figure 4-5, then the integral (Eq 4-20) can be solved using, for example, Simpson's rule, to obtain: e = 1/2R {si [k1 S+k2 (S-si )]+s2 [k2 (S-s1 )+k3 (S-si -s2 )]+... ...+s n-1 [kn -1(S-s1 -s2...s n-2)+kn (S-s 1 -s2 ...s n-2)]+s n-1[kn (S-s1 ...sn-1 )]} (Eq 4-21)

4-11

EM 1110-2-1009 1 Jun 02

B' A
e

B k1 0 k2 k3 S X

Figure 4-5. Effects of changeable gradients of temperature

For example, let us take S = 200 m divided into 4 segments of si = 50 m each, and consider two cases: (I) For k1 = 2.8 (same as previous example) where k2 = k3 = k4 = 0. In this first case, we assume that the refraction takes place only within the first 50 m from the instrument, while there is no refraction (dT/dy = 0) over the rest of the line. (II) For k4 = 2.8 where k1 = k2 = k3 = 0. In this second case, the refraction takes place only in the last segment, near the survey target. For both cases, t = 30 deg C P = 1000 mb R = 6371 km. From Equation 4-21 we have: For Case I: For Case II: e = 3.8 mm e = 1.9 mm

The effects of refraction are more dangerous near the instrument than near the target (see Figure 4-6). Thus, instruments should be located as far away as possible from any surfaces having different temperature than the surrounding air. c. Effects on alignment measurements. In alignment surveys between two fixed stations A and B (Figure 4-7), the line of sight from A is constrained to point to a target at B. If the gradient of temperature across the line of sight is constant between A and B, then the alignment reference line will be refracted along a circular path with the largest error of alignment being in the middle between A and B. Even when the alignment surveys would be performed in segments (i.e., resetting the alignment telescope or laser in

4-12

EM 1110-2-1009 1 Jun 02

Y
REFRACTION ZONE

Y
REFRACTION ZONE

refracted path refracted path

e
target instrument

e
instrument target

refraction near instrument

refraction near target

Figure 4-6. Effects of refraction near instrument vs. near target

steps between A and B), the whole survey will follow the same refracted circular curve. For example, when using a HeNe laser (l = 0.63 mm) the maximum error (Dy) in the center of the refracted path can be calculated from: Dy = [(PS2 )/(101760 T2 )](dT/dy) (Eq 4-22) where P is barometric pressure in [mb], T is temperature in Kelvin, S is total length of the alignment line. With example values of S = 500 m, T = 300 K, P = 1000 mb, and dT/dy = 0.2 deg C. The error of alignment is Dy = 5.4 mm.
Y

X
Figure 4-7. Refraction effects on alignment survey

d. Methods of reducing refraction effects. The effects of refraction can be reduced by: (1) avoiding lines of sight running closer than one meter from any prolonged surface that may have a different temperature than the surrounding air, (2) Measuring the gradients of temperature with several temperature sensors of high precision (resolution of at least 0.05 deg C) and calculating the refraction correction, or
4-13

EM 1110-2-1009 1 Jun 02

(3) Using two sources of radiation of a different wavelength (i.e., dispersion method). The first method is the most practical, but it does not assure a refraction-free line of sight. The second method, though applicable in practice, requires special instrumentation and tedious measurements. The third method requires very expensive and difficult to acquire instrumentation and it is applicable only in scientific measurements of the highest precision (e.g., in industrial metrology). In deformation surveys only the first method seems to be feasible. If both walls in narrow galleries are exposed to different temperature, e.g., one wall being exposed to water and another to the sun radiation, even placing the alignment line in the center of the gallery may not sufficiently reduce the refraction effect. In the latter case, non-optical methods may be used in the displacement measurements. e. Effects on direction measurements. Refraction occurs in deflection angle and direction measurements with optical theodolites. With a uniform temperature gradient over the length (S) of the line of sight, the refraction error (eref ) in arcseconds of the observed direction may be approximated by: eref = ( 8" ) [ ( S ) ( P ) / ( T 2 ) ] ( dT/dx ) (Eq 4-23) where S = distance between stations (m) P = barometric pressure (mb) T = temperature in Kelvin ( T = 273.15 + t C ) dT/dx = temperature gradient For example, If a gradient of only 0.1C/m persists over a distance of 500 m at P = 1000 mb and t = 27C, it will cause a directional error of 4.4 arcseconds.

Figure 4-8. Aspirating Psychrometer from Belfort Instrument Co. Psychrometers are equipped with two thermometers, one with a wet bulb and one with a dry bulb. The wet bulb is covered with a wick that must be saturated with distilled water. A battery-operated fan in the aspirating psychrometer draws air over the bulbs. The cooling effect of evaporation produces a lower reading on the wet bulb, which is then used to determine the relative humidity. The dry bulb is read directly from the dry thermometer. These readings are used to compute the refractive index correction for the atmosphere in precise distance measurement.

4-14

EM 1110-2-1009 1 Jun 02

4-9. Theodolite System Error a. General. Theodolite instruments used for angle measurement can have small misalignments in the system attributed to its manufacturing or normal wear from repeated use and handling. Theodolite misadjustments produce systematic error in angle measurements. For example:

Trunnion axis not perpendicular to line-of-sight Horizontal axis not perpendicular to vertical axis Vertical circle index error Inclined cross-hairs Plate eccentric to rotation axis b. Elimination of instrument errors. The main technique used to eliminate the above instrument biases is to observe the target in both the direct and reverse positions of the theodolite at all times. All of the systematic errors noted above will cancel when measurements are made in two positions of the theodolite (e.g., double centering). 4-10. Reflector Alignment Error a. General. Older model reflector prisms can introduce small errors in both distance and direction measurements due to mis-orientation. The magnitude of the error is based on factors such as the EDM wavelength, the prism dimensions, the refractive index of the prism glass, and the horizontal and vertical misalignment angle (non-perpendicular to the line-of-sight). It has been determined for older prism designs that less than 0.5 mm error in distance will be caused by a misalignment angle under 10 degrees from the line-of-sight, which is well within the normal ability to point the reflector toward the EDM. With directions (horizontal and vertical) the error can be limited to less than 1 arcsecond (over 500 m) with a misalignment of less than 10 degrees. The angular error is highly dependent on the distance from the instrument in that shorter distances will produce larger errors. b. Modern reflector design. Reflector prisms in use today have been specifically re-designed to minimize the influence of misalignments on distance measurement accuracy. With standard prisms the resulting distance errors are negligible for small misalignments. However, attention to consistent and accurate pointing of the prism toward the instrument is still recommended practice. 4-11. EDM Scale Error a. Temperature frequency drift. Short term drift of the oscillator frequency in EDM instruments is most likely to occur during the warm-up of its internal electronic components. If a frequency drift persists throughout the course of a survey, then the measurements will contain a time dependent bias that can reach a maximum drift value of up to 3 ppm (depending on the instrument and environment). b. Frequency drift due to aging. Frequency drift can also occur as a result of mechanical aging of the oscillator crystal over time, so that the EDM develops a bias in its internal measuring scale (in some instruments less than 1 ppm per year). This can be a critical factor in deformation surveys when the same EDM instrument is used for repeated surveys and the resulting observations are compared over time. c. Scale error calibration. For precise surveying applications, it is recommended that the EDM be calibrated for scale error at least once a year. The most common method for EDM scale factor calibration is from measurements made over a certified calibration baseline. Scale error is determined by comparing a series of distances measured along a linear array of stations where the station coordinates are
4-15

EM 1110-2-1009 1 Jun 02

precisely known. The known distances between stations are differenced from the product of the measured distance and an unknown scale factor, as shown in the following linear equation: [( k ) ( S )] - ( D ) = 0 (Eq 4-24) where k = unknown scale factor S = measured distance D = fixed distance (known) The horizontal distance measurements (S) are processed together in a linear least squares adjustment to solve for the unknown scale factor parameter (k). Once the scale error has been determined, all subsequently measured distances are multiplied by the constant scale factor (k) to yield a corrected distance. d. Frequency counter methods. An alternative to using a baseline for EDM scale calibration is to send the instrument back to the manufacturer for a direct reading of its oscillator frequency. This laboratory procedure uses a high grade electronic frequency counter to compare the actual and reference frequencies of the EDM under controlled conditions. The instrument must have a built-in port connector so that the frequency may be sampled to solve for the instrument scale factor. 4-12. EDM Prism Zero Error a. Additive constant or zero error. The additive constant is an unknown systematic bias that is present in all distance measurements made with a particular EDM instrument-prism combination. The bias is usually small for instruments using light waves. The bias is an absolute constant offset that exists between the optical and mechanical centers of the reflector prism and the electrical center of the EDM instrument when centered over the setup station. Distances uncorrected for zero error will produce discrepancies in the final station coordinates of survey points. These can be detected from check measurements; such as multiple observations made over a network of points or when more than two distance intersections are compared as a check on the monument positions. b. Correction determination. Measured distances can be corrected for zero error by a determination of the additive constant of the instrument-prism combination on a calibration baseline. The calibration process relies on comparing distances that are measured over a set of fixed stations. The four station baseline and observing configuration shown in figure 4-9 allows each of the six measured distances (m i ) to be written as the sum of each true distance (d i ) and one unknown constant bias term (z). m i = ( d i + z ) for i = 1 thru 6 This basic equation is repeated for each observed station-pair (p i ) along the baseline and the method of parametric linear least squares is used to solve for the constant term (z). For example each measured and true distance can be expressed using station coordinates as: m1 m2 m3 m4 m5 m6 = = = = = = (p2 - p1 ) (p3 - p1 ) (p4 - p1 ) (p3 - p2 ) (p4 - p2 ) (p4 - p3 ) + + + + + + z z z z z z i.e., i.e., i.e., i.e., i.e., i.e., d1 d2 d3 d4 d5 d6 = = = = = = p2 - p1 p3 - p1 p4 - p1 p3 - p2 p4 - p2 p4 - p3

4-16

EM 1110-2-1009 1 Jun 02

Since m1 m2 m3 m4 m5 m6 = - p1 = - p1 = - p1 = 0 = 0 = 0 + p2 +0 +0 - p2 - p2 +0 +0 + p3 +0 + p3 +0 - p3 +0 +0 + p4 +0 + p4 + p4 +z +z +z +z +z +z

By setting the coordinate of the initial point (p1 ) to zero, leaving three unknown coordinates (p2 , p3 , and p4) and one unknown constant (z), this system of equations can be represented in matrix form as:

Ax=b
where +1 0 0 -1 -1 0 [p2 [m1 0 +1 0 +1 0 -1 p3 m2 0 0 +1 0 +1 +1 p4 m3 +1 +1 +1 +1 +1 +1 z ]T m4 m5 m6 ]T

A =

x = b =

The unknown parameter (z) is common to each measurement so an over determined set of measurements is used to detect the discrepancy it causes between measurements. The least squares solution:

x = (AT P A) -1 AT P b
uses a diagonal weight matrix (P) populated with the inverse of the variances computed for each distance measurement. In practice, 5 to 7 fixed points are needed, which adds to the observing time but also provides sufficient redundancy for statistically testing the significance of the additive constant parameter. National Geodetic Survey (NGS) standards for baseline calibration recommend a four-station baseline to simultaneously solve for both the scale factor and the additive constant, but this requires measurements both forward and backward from each station and uses known distances between stations. In either case, standard data reductions must be applied to the measurements (e.g., refractive index and slope distance reductions) before solving for the additive constant. Software applications are available from NGS that cover all aspects of EDM baseline calibration including data collection and associated least squares computations. The residuals from the data adjustment should be plotted and examined visually for any obvious trends that would indicate there are systematic errors remaining in the measurements. The resulting calibration constant is added to each measured distance, with an opposite algebraic sign, to obtain the corrected distance. The accuracy of the correction itself will depend on the number of observations made and their precision.

4-17

EM 1110-2-1009 1 Jun 02

m 5

m4 m 6

2
m1 m2

m 3

Figure 4-9. Zero error calibration baseline concept

4-13. EDM Cyclic Error Stray radiation and signal interference inside the EDM unit can occur at the same phase as the internal reference signal. A sinusoidal pattern of deviations in the measured phase can systematically change the distance measurements. The stability of the EDM internal electronics can also vary with age, therefore, the cyclic error can change significantly over time. Cyclic error is inversely proportional to the strength of the returned signal, so its effects will increase with increasing distance (i.e., low signal return strength). Calibration procedures exist to determine the EDM cyclic error that consist of taking bench measurements through one full EDM modulation wavelength, and then comparing these values to known distances and modeling any cyclic trends found in the discrepancies. This procedure requires a specialized calibration baseline designed to detect the presence of cyclic error from the spacing of the measurement intervals. 4-14. Calibration Baselines a. General. Construction of an EDM calibration baseline requires preparation for the design, layout, monumentation, and proposed calibration techniques. It should be noted that establishing a calibration baseline and keeping it in good order can be expensive and time consuming when maintenance is considered. Use of an established baseline that is available within the local area would be far more economical than to build a new facility. b. Standard baselines. A standard baseline design recommended by the National Geodetic Survey should be used for EDM instrument calibrations. Guidance is provided on EDM calibrations and baselines in the following publications:

NOAA Technical Memorandum NOS NGS-8, Establishment of Calibration Base Lines, NOAA Technical Memorandum NOS NGS-10, Use of Calibration Base Lines. These documents shall be consulted before building a new calibration baseline and for conducting EDM instrument calibrations following NGS methods.

4-18

EM 1110-2-1009 1 Jun 02

c. Specialized baselines. NGS calibration baselines have at least four monuments set in an alignment that deviates no more than 5 degrees of arc, which provides 12 measured distances in a complete calibration session--see Figure 4-9. NGS conventions for baseline design have monuments set out at 0, 150, 400-430, and 1,000-1,400 m along a straight line. If this spacing is not convenient to the local terrain, it is recommended to place the intermediate monuments at an even multiple of the unit wavelength of the EDM instrument to be calibrated (e.g., multiples of 10 m). This 'multiple of 10' rule of thumb is meant to ensure that the EDM phase detector will sample the return signal at the same point along the carrier wave for each measurement (i.e., resolve the partial wavelength at the same point along the carrier wave to avoid the effects of cyclic error in the calibration process). Alternative designs for calibration baselines can be developed for special purposes (see Rueger, 1990). A determination of monument spacing for specialized calibration baselines is made using baseline design formulas with the following input parameters:

Total length of baseline EDM instrument reference wavelength Total number of baseline monuments Trial and error combinations of different values for the above design parameters can be tested and the results examined to evaluate each baseline configuration. Designing a specialized baseline is warranted where unique EDM resolution, signal frequency, and range limits are encountered. A baseline design should provide distance combinations that evenly span the working range of the EDM, and the length of the baseline should be optimized at the minimum range of the EDM when operating under poor atmospheric conditions with a single reflector prism. d. Tripod method. The EDM/prism calibration process (i.e., additive constant determination) can be made using a very short (30-50 m) alignment of at least five tripods instead of on a permanent baseline. Good results have been obtained by this method; however, forced centering on tripods is less stable than on pillars. For longer baselines, the stations should be no more than four minutes of arc out of line, and the tops of the tripods should be set in as straight a vertical alignment as possible relative to each other (within a few cm or less). Accurate tripod arrangement minimizes deviations in the line-of-sight from any single instrument setup station and will speed up the field observations. Precise leveling is used to determine the elevation of each ground point (monument) and fixed height offset measurements are made to each (tripod mounted) prism. Elevation and offset measurements are made to reduce the slope distances to horizontal distances. e. Stability. If pillars are selected as monuments, it should be recognized that their stability can be influenced by various types of movements, namely, those due to external forces, settlement and tilt, dry shrinkage of concrete, swelling, and thermal expansion. The horizontal instability of concrete pillars due to thermal expansion is based on the operating height, width, and temperature change. Pillars should be set below the frost line and insulated in cold regions. The initial observation of baseline distances is delayed until the concrete has set through at least one freeze-thaw cycle. f. Site selection. Permanent baselines should be easily accessible allowing transport and setup of the instrument and prisms. A roadway alongside the baseline will speed-up the movement of equipment and personnel between stations. Stations should have about 20 ft clearance on either side, and are set back 1/4 mile from high voltage lines and transmitters, and 100 feet from metal fences. Reconnaissance of the proposed baseline location (site visit, aerial photographs, topographic, geologic maps, etc.) is recommended to investigate soil type, relief, atmospheric conditions (avoid completely unvegetated areas with no shade). Once a suitable location has been found, a preliminary survey and stake out of temporary points can be made at the proposed distances.

4-19

EM 1110-2-1009 1 Jun 02

4-15. Equipment for Baseline Calibration A list of the equipment required for establishing a calibration baseline is provided below. Excluding the first two items (theodolite and EDM) these are also recommended for conducting calibration sessions.

1 Wild T2000 Theodolite 2 Wild DI2000 Electronic Distance Meter PC type laptop computer NGS calibration software 1 Nadir Plummet instrument 1 calibrated prism and mounting bracket tribrachs and tripods for each station 2 communication radios 2 psychrometers 2 barometers 2 (fan driven) thermistors 2 tripods and poles for thermistors 2 thermistor recording and reading units 12 volt battery power supply 1 hand tape measure 2 shade umbrellas

Calibration baselines are established with an instrument that has a higher precision than the instruments that are intended for calibration (ideally by an order of magnitude), however this is not always feasible for modern EDM instruments because of their extremely high precision. 4-16. Procedures for Baseline Calibration a. General. Procedures for conducting measurements on a calibration baseline consist of setting the instrument at the baseline initial station and recording the distances to all of the others in sequence. The instrument is then moved up to the next pillar in line and the process is repeated until all of the baseline stations have been occupied (all distance combinations forward and backward are usually observed from each instrument setup). With a four-monument baseline, a total of 12 distances are used to solve four unknown baseline coordinates and each calibration parameter. b. Preparation. First, set up the tripods and tribrachs over each station on the baseline. The tripods should be close to the same elevation (i.e., in an alignment or with a slightly up-sloping tilt away from the zero station to maintain visibility throughout the length of the baseline). The tripod is leveled by eye so that the tripod head on which the tribrach is mounted is as near level as possible. The tripod head is further leveled by mounting a calibrated Wild type target (with precise level bubble) into the tribrach, and then adjusting the legs to the position where the bubble is level in four positions under rotation. c. Collimation. Accurate centering is critical for the measurement of baseline distances. After the tripod head is level, the target is replaced by a precision Nadir Plummet having an internal level compensator. The tribrach and Nadir Plummet assembly is then translated (without rotation) until it is centered. The collimation is then checked in four positions (90 degree rotations) around the center point. If the centering is not well-established, then slight adjustments are made to the centering and leveling of the tribrach until it is collimated. The leveling screws of the tribrach should be moved as little as possible because the offset from the tripod head to the top of the tribrach mounting plate needs to be measured and recorded, so if it is kept constant, a pre-calibrated value for the prism height above the tripod head can be used (reference marks are sometimes painted on the leveling screws and on the upper tribrach housing to
4-20

EM 1110-2-1009 1 Jun 02

verify the offset is correct for the setup). The height of the tripod head is measured in four positions using the hand tape measure and a mean value is added to the constant height offset value between the theodolite optical center and the base of the tribrach (known beforehand), giving the height of instrument above the monument. This value is then added to the known elevation of the baseline monument. d. Calibration procedures. The procedures for the baseline calibration presented below are to be repeated for each instrument setup. The observing procedures used to establish coordinates on a new baseline follow the same basic methodology used for actual instrument calibration. (1) The instrument is placed at the baseline initial point and powered on while the prism is set up at the next station along the baseline. Thermistors are mounted on a vertical 3 meter pole, one at the top, and one at the level of the instrument, perpendicular to the direction of the sun, and oriented so that the front end faces into the prevailing wind. The two thermistors measure the temperature gradient, and are capable of reading to a tenth of a degree Celsius. A barometer is placed nearby the instrument and is capable of reading to one hundredth of an inch Hg. A similar thermistor and barometer arrangement is set up at the prism station. A psychrometer reading (Figure 4-8) is initially recorded, and thereafter, repeated only when the instrument is moved to the next setup. The top/bottom thermistor and barometer readings are recorded on both ends of the measured line at the start and finish of the measurements for each EDM setup (two different EDM instruments are used at each station when establishing a new baseline). (2) Mount the first EDM to the theodolite and point it in the direct position at the center of the forward target (Note: it is important that the EDM ppm value is set to zero). Obtain adequate return signal strength and then measure five consecutive distances to the forward target. The values are recorded to the tenth of a millimeter. Reverse the scope and read five more distances. At the conclusion of the set, read and record the temperatures and pressure at both the instrument and prism stations, (the exact same procedure is then followed for the second EDM instrument when establishing a new baseline). (3) Once the measurements for one baseline distance segment are finished (i.e., one station pair), the instrument stays at the same station and the prism moves to the next point along the baseline. (4) At each instrument setup distances are measured to every other monument forward and back. (5) If computed values for the instrument-prism constant exceed instrument tolerances, then the measurements must be repeated. e. Calibration results. Field book records should include the following information:

distance measurements, from station name, to station name, instrument serial number, prism serial number, date and time of observation, height of instrument/prism, meteorological observations, units of measurement, geometrical reductions, calculation of calibration parameters,

This information is compiled into a calibration report for use in data reductions.

4-21

EM 1110-2-1009 1 Jun 02

4-17. Mandatory Requirements There are no mandatory requirements in this chapter.

4-22