Engineering Mathematics I: Measurements, Errors, Most Probable Value and Least Squares Adjustment

Part A of the Geomatics Engineering (GE) paper opens with an Engineering Mathematics heading that is not calculus at all: it is the theory of errors a surveyor lives by. It names surveying measurements, accuracy, precision, the most probable value, errors and their adjustment, and least squares adjustment, then regression, correlation, significance and the chi-square test (the next chapter). This chapter takes the first half. It separates accuracy from precision and the three kinds of error; computes the standard deviation, the standard error of the mean and the probable error; propagates errors through sums, products and general functions; weights observations and finds the weighted most probable value; distributes a misclosure in station and figure adjustment; and sets up observation and condition equations and the normal equations of least squares. Almost every question is a short computation whose trap is a divisor — n or n − 1, σ or σ/√n, 1/w or w.

1. Measurements, accuracy, precision and the three kinds of error

No measurement is exact. The true error of an observation is observed − true value, and the true value is never known; what can be computed is the residual v, the difference between an observation and the best estimate of the quantity. In this chapter v = adjusted (most probable) value − observed value, so that observation + v = adjusted value, the sign convention least squares uses. Accuracy is closeness to the true value; precision is closeness of repeated observations to one another, i.e. small spread. A tape 2 cm short gives readings that agree to a millimetre — precise and inaccurate. Precision is usually stated as a relative precision, e.g. an error of 0.05 m in 500 m is 1 in 10,000.

The three kinds of error, and the one remedy that works on each
KindBehaviourGrowth over n measurementsRemedy
Gross error (blunder)A mistake: misread staff, transposed digits, wrong stationNot a statistical quantity at allIndependent checks; detect and reject (e.g. a residual beyond 3σ)
Systematic error (cumulative)Same sign and size each time, by a law: wrong tape length, temperature, sag, collimationProportional to nCalibrate and apply a correction; observing procedure (face left and right). Averaging does nothing.
Random error (accidental, compensating)Small, either sign, equally likely: estimation of the last digit, pointing, bubble centringProportional to √nRepeat and adjust by least squares
⚠️ n versus √n is the whole question
A 30 m tape with ±5 mm per length laid 20 times over 600 m accumulates ±5√20 = ±22.4 mm if the error is random, but 5 × 20 = 100 mm if it is systematic. Most “which statement is true” questions on error types turn on this difference.

2. Measures of precision and the propagation of errors

For n equally reliable observations xᵢ with mean x̄ and residuals vᵢ = x̄ − xᵢ (so Σv = 0): standard deviation of one observation σ = √(Σv²/(n − 1)); standard error of the mean σₘ = σ/√n; probable error PE = 0.6745σ (half the observations are expected to lie within ±PE, since 0.6745 is the 50 % point of the normal distribution), and PE of the mean = 0.6745σ/√n. The divisor is n − 1, not n, because one degree of freedom was spent computing x̄. The normal-distribution ranges are worth knowing exactly: ±1σ holds 68.27 %, ±2σ holds 95.45 %, ±3σ holds 99.73 %, and ±1.96σ holds 95 %.

  • Worked: a distance measured five times, 100.02, 100.04, 99.98, 100.00 and 100.06 m. Mean 100.02 m; residuals 0, −0.02, +0.04, +0.02, −0.04; Σv² = 0.0040 m². σ = √(0.0040/4) = 0.0316 m; σₘ = 0.0316/√5 = 0.0141 m; PE of the mean = 0.6745 × 0.0141 = 0.0095 m.
  • Sum or difference z = x ± y of independent quantities: σ_z² = σₓ² + σ_y². For n equal parts, σ_total = σ√n.
  • Multiple z = kx: σ_z = kσₓ. Product A = xy (area of a rectangle): σ_A² = (yσₓ)² + (xσ_y)². For 50 ± 0.02 m by 30 ± 0.01 m, σ_A = √((30 × 0.02)² + (50 × 0.01)²) = √(0.36 + 0.25) = 0.781 m².
  • General law: for z = f(x₁, …, xₙ) with independent xᵢ, σ_z² = Σ(∂f/∂xᵢ)² σᵢ². Angles enter in radians (1″ = 1/206 265 rad).
🧠 Squares add, errors do not
Independent random errors combine in quadrature. Adding ±3 mm and ±4 mm gives ±5 mm, not ±7 mm. The plain sum is right only for systematic errors of the same sign.

3. Weights, the most probable value and the adjustment of a misclosure

The weight of an observation measures its relative reliability: w ∝ 1/σ². An angle that is the mean of four equally good readings has weight 4; a level line's weight is inversely proportional to its length, because the variance of a levelled height difference grows with distance. The most probable value (MPV) of a quantity observed several times is the weighted mean x̂ = Σwx/Σw, and its weight is Σw. The laws of weights: the weight of kx is w/k²; the weight of a sum x + y satisfies 1/w = 1/wₓ + 1/w_y.

  • Worked: a line measured as 250.30 m (weight 2), 250.36 m (weight 3) and 250.33 m (weight 5). MPV = (500.60 + 751.08 + 1251.65)/10 = 250.333 m.
  • Worked: a bench mark levelled by three routes of 2, 4 and 1 km gave 100.250, 100.270 and 100.240 m. Weights ∝ 1/L = 1/2 : 1/4 : 1, i.e. 2 : 1 : 4. MPV = (200.500 + 100.270 + 400.960)/7 = 100.247 m.

Station adjustment makes the angles at one station consistent (the angles closing the horizon must total 360°); figure adjustment makes a figure consistent (the three angles of a plane triangle total 180°, a spherical triangle 180° plus its spherical excess). With a single condition, least squares distributes the misclosure in inverse proportion to the weights: correction to angle i = −e × (1/wᵢ)/Σ(1/w). With equal weights each of n angles receives −e/n. A triangle with angles of weights 1, 2 and 3 that closes 18″ too large receives corrections in the ratio 1 : 1/2 : 1/3 = 6 : 3 : 2, i.e. −9.82″, −4.91″ and −3.27″.

⚠️ The heavier observation moves less
Distributing a misclosure in proportion to the weights is the classic slip: it moves the most reliable angle the most. The correction goes as 1/w. In a weighted mean, by contrast, the value is pulled toward the heavy observation — the same principle seen from the other side.

4. Least squares adjustment: observation equations, condition equations and normal equations

The principle of least squares: the most probable values are those that make the sum of the weighted squares of the residuals, Σwv², a minimum. For normally distributed errors this is the maximum-likelihood estimate, and for a single quantity observed directly it reduces to the (weighted) mean. When there are more observations n than unknowns u, the redundancy (degrees of freedom) is r = n − u.

The two formulations
MethodSet-upSolution
Observation equations (parametric)One equation per observation in the u unknowns: l + v = Ax (linearised if needed)Normal equations (AᵀWA)x = AᵀWl — u equations, one per unknown; covariance of x is σ₀²(AᵀWA)⁻¹
Condition equations (correlates)One equation per geometric condition the adjusted observations must satisfy: r = n − u conditionsLagrange multipliers (correlates) give v; with one condition the misclosure is shared as 1/w

The normal-equation rule in words: to form the normal equation for an unknown, multiply each observation equation by the coefficient of that unknown in it and by its weight, and add. Worked: angles A = 30°15′20″, B = 45°20′10″ and A + B = 75°35′40″ are observed with equal weight. A + B from the parts is 75°35′30″, so the misclosure is 10″. The one condition shares it equally, 10″/3 = 3.33″ each: A = 30°15′23.33″, B = 45°20′13.33″, A + B = 75°35′36.67″, which is consistent. The same answer follows from minimising v_A² + v_B² + (v_A + v_B + 10″)², which is the check that the rule and the principle agree.

🎯 Why the reference variance matters
After adjustment the a-posteriori variance factor σ̂₀² = vᵀWv/r estimates how well the weights described the real errors. If it differs significantly from the a-priori value, either the weights were wrong or a blunder remains — and that is exactly what the chi-square test of the next chapter decides.

Key takeaways

  • Accuracy is closeness to truth, precision is agreement among repeats; systematic errors grow as n and are removed by correction, random errors grow as √n and are reduced by repetition and least squares.
  • σ = √(Σv²/(n − 1)) for one observation, σ/√n for the mean, and 0.6745σ for the probable error.
  • Independent errors propagate in quadrature: σ_z² = Σ(∂f/∂xᵢ)²σᵢ².
  • Weight ∝ 1/σ² (∝ number of repeats, ∝ 1/length of a level line); the MPV is Σwx/Σw with weight Σw.
  • Least squares minimises Σwv²; with one condition the misclosure is distributed in proportion to 1/w, and redundancy r = n − u is the number of conditions.

Practice questions (14)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. A steel tape that is 2 cm too short is used carefully, and ten repeated measurements of a line agree to within 1 mm. The measurements are best described as

    1. precise but not accurate
    2. accurate but not precise
    3. both accurate and precise
    4. neither accurate nor precise
    Show answer

    Answer: A — precise but not accurate

    The readings agree closely with one another, which is precision, but every one carries the same systematic error from the short tape, so they are all far from the true length. Repetition cannot reveal a systematic error; only calibration can, which is why “agrees to 1 mm” does not mean accurate.
  2. A 30 m tape carries an error of 5 mm per tape length and is laid 20 times to measure 600 m. If the error is systematic the total error is 100 mm. If instead it is purely random, the expected total error (in mm) is

    1. 22.4
    2. 100
    3. 5
    4. 44.7
    Show answer

    Answer: A — 22.4

    Random errors accumulate as the square root of the number of occurrences: 5√20 = 5 × 4.472 = 22.4 mm. The 100 mm figure is 5 × 20, correct only for a systematic error; 44.7 mm doubles the random result by mistake.
  3. A distance is measured five times: 100.02, 100.04, 99.98, 100.00 and 100.06 m. The standard deviation of a single observation, in millimetres (to one decimal place), is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 31.6

    Mean = 500.10/5 = 100.02 m. Residuals 0, −0.02, +0.04, +0.02, −0.04 m give Σv² = 0 + 0.0004 + 0.0016 + 0.0004 + 0.0016 = 0.0040 m². σ = √(0.0040/(5 − 1)) = √0.0010 = 0.0316 m = 31.6 mm. Dividing by n instead of n − 1 gives 28.3 mm, the usual slip.
  4. For the five measurements 100.02, 100.04, 99.98, 100.00 and 100.06 m, the probable error of the mean, in millimetres (to one decimal place), is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 9.5

    σ = √(0.0040/4) = 0.03162 m. Standard error of the mean σₘ = 0.03162/√5 = 0.01414 m. Probable error of the mean = 0.6745 × 14.14 mm = 9.54 mm, i.e. 9.5 mm. Forgetting the √5 gives the probable error of one observation, 21.3 mm.
  5. A line is measured as 250.30 m with weight 2, 250.36 m with weight 3 and 250.33 m with weight 5. Its most probable value, in metres (to three decimal places), is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 250.333

    MPV = Σwx/Σw = (2 × 250.30 + 3 × 250.36 + 5 × 250.33)/(2 + 3 + 5) = (500.60 + 751.08 + 1251.65)/10 = 2503.33/10 = 250.333 m. The unweighted mean, 250.330 m, gives the weight-2 reading of 250.30 m more influence than it has earned.
  6. The reduced level of a bench mark is carried by three level routes of lengths 2 km, 4 km and 1 km, giving 100.250 m, 100.270 m and 100.240 m. The most probable reduced level, in metres (to three decimal places), is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 100.247

    In levelling, weight ∝ 1/route length: 1/2 : 1/4 : 1 = 2 : 1 : 4. MPV = (2 × 100.250 + 1 × 100.270 + 4 × 100.240)/7 = (200.500 + 100.270 + 400.960)/7 = 701.730/7 = 100.247 m. Weighting in proportion to length instead gives 100.260 m, which favours the least reliable route.
  7. The sides of a rectangular plot are measured as 50.00 ± 0.02 m and 30.00 ± 0.01 m (standard errors, independent). The standard error of its computed area, in m² (to two decimal places), is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 0.78

    A = xy, so ∂A/∂x = y = 30 and ∂A/∂y = x = 50. σ_A = √((30 × 0.02)² + (50 × 0.01)²) = √(0.36 + 0.25) = √0.61 = 0.781 m², i.e. 0.78. Adding the two terms linearly (0.6 + 0.5 = 1.1 m²) is correct only if both errors were systematic.
  8. The three angles of a plane triangle are observed with weights 1, 2 and 3 respectively, and their sum is 180°00′18″. By least squares, the magnitude of the correction applied to the weight-1 angle, in seconds (to two decimal places), is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 9.82

    Misclosure e = +18″ must be removed. Corrections are ∝ 1/w: 1 : 1/2 : 1/3 = 6 : 3 : 2, total 11 parts. Weight-1 angle: 18 × 6/11 = 9.82″ (subtracted); the others get 4.91″ and 3.27″, and 9.82 + 4.91 + 3.27 = 18.00″. Sharing in proportion to w would give the weight-1 angle only 3″, backwards.
  9. Angles A = 30°15′20″, B = 45°20′10″ and (A + B) = 75°35′40″ are measured with equal weights. After least squares adjustment, the seconds part of the adjusted value of A (to two decimal places) is ____.

    Numerical answer — type the value.

    Show answer

    Answer: 23.33

    A + B from the parts = 75°35′30″, 10″ less than observed. The single condition shares the 10″ equally among the three observations: A and B each increase by 3.33″ and (A + B) decreases by 3.33″. So A = 30°15′23.33″. Giving the whole 10″ to (A + B) alone ignores that A and B are equally uncertain.
  10. In a least squares adjustment with 12 observations and 5 unknown parameters, the number of degrees of freedom (redundancy) is

    1. 7
    2. 17
    3. 5
    4. 12
    Show answer

    Answer: A — 7

    Redundancy r = n − u = 12 − 5 = 7, which is also the number of independent condition equations. It is the divisor in σ̂₀² = vᵀWv/r; adding the two counts (17) has no meaning.
  11. In the observation-equation method with design matrix A, weight matrix W and observation vector l, the normal equations for the unknowns x are

    1. (AᵀWA)x = AᵀWl
    2. (AWAᵀ)x = Wl
    3. (AᵀA)x = W⁻¹l
    4. Ax = l exactly
    Show answer

    Answer: A — (AᵀWA)x = AᵀWl

    Minimising vᵀWv with v = Ax − l gives ∂/∂x = 2AᵀW(Ax − l) = 0, i.e. (AᵀWA)x = AᵀWl — one equation per unknown. Ax = l has no solution when there is redundancy, which is why the adjustment exists at all.
  12. Which of the following are systematic errors in surveying measurements?

    1. Sag of a tape suspended between supports
    2. Taping at a temperature different from the standardisation temperature
    3. Estimating the last millimetre on a levelling staff
    4. Booking 2.345 as 2.435
    Show answer

    Answer: A — Sag of a tape suspended between supports; B — Taping at a temperature different from the standardisation temperature

    Sag always shortens the horizontal span by w²L³/(24P²) and a temperature difference always changes the tape by αLΔT — each follows a law with a fixed sign, so both are systematic and corrected by formula. Estimating the last digit is random; transposing digits is a gross error, detected by checks.
  13. Which of the following statements about weights and the most probable value are correct?

    1. The weight of an observation is inversely proportional to its variance
    2. If a quantity has weight w, the weight of 3 times that quantity is w/9
    3. The weight of a weighted mean equals the sum of the individual weights
    4. In a single-condition figure adjustment the heaviest angle receives the largest correction
    Show answer

    Answer: A — The weight of an observation is inversely proportional to its variance; B — If a quantity has weight w, the weight of 3 times that quantity is w/9; C — The weight of a weighted mean equals the sum of the individual weights

    w ∝ 1/σ²; σ(3x) = 3σ so its weight is w/3² = w/9; and the weight of Σwx/Σw is Σw. The last statement is backwards: corrections go as 1/w, so the heaviest (most reliable) angle moves least.
  14. Which of the following are true of a least squares adjustment of normally distributed observations?

    1. It minimises Σwv², the weighted sum of squared residuals
    2. For one quantity measured directly several times it gives the weighted arithmetic mean
    3. The number of independent condition equations equals the number of observations minus the number of unknowns
    4. It removes systematic errors from the observations
    Show answer

    Answer: A — It minimises Σwv², the weighted sum of squared residuals; B — For one quantity measured directly several times it gives the weighted arithmetic mean; C — The number of independent condition equations equals the number of observations minus the number of unknowns

    The principle is min Σwv²; for a single directly observed quantity, setting d/dx Σw(x − xᵢ)² = 0 gives x = Σwxᵢ/Σw; and the redundancy n − u counts the independent conditions. Least squares assumes the errors are random: a systematic error left in the observations is spread through the solution, not removed — it must be corrected first.