In ordinary linear regression, the goal is to minimize the sum of the squared vertical distances between the y data values and the corresponding y values on the fitted line. In orthogonal regression the goal is to minimize the orthogonal (perpendicular) distances from the data points to the fitted line.

The slope-intercept equation for a line is:

Y = m*X + b

where m is the slope and b is the intercept.

A line perpendicular to this line will have -(1/m) slope, so the equation will be:

Y' = -X/m + b'

If this line passes through some data point (X0,Y0), its equation will be:

Y' = -X/m + (X0/m + Y0)

The perpendicular line will intersect the fitted line at a point (Xi,Yi) where Xi and Yi are defined by:

Xi = (X0 + m*Y0 - m*b) / (m^2 + 1)
Yi = m*Xi + b

So the orthogonal distance from (X0,X0) to the fitted line is the distance between (X0,Y0) and (Xi,Yi) which is computed as:

distance = sqrt((X0-Xi)^2 + (Y0-Yi)^2)

So the goal of the NLREG program is to minimize the sum of these orthogonal distances. Here is a NLREG program that does this:

Title "Fit a line to data points minimizing orthogonal distances";
Variables X0, Y0;
Parameters m, b;
Double Xi, Yi, distance;
Xi = (X0 + m*Y0 - m*b) / (m^2 + 1);
Yi = m*Xi + b;
distance = sqrt((X0-Xi)^2 + (Y0-Yi)^2);
Function distance;
Data;

For an example of a NLREG program that performs orthogonal regression to a 3D plane, please click here.