The fitting to a line is done in one of three ways: best line, best line parallel to coordinate plane, and best line parallel to coordinate axis. All the currently selected points are processed by a least squares fitting algorithm that determines the best line in 3D space that passes through these points. When that line has been determined, all the points are dropped to that line at the closest point. The end result is a set of points that all perfectly lie on a 3D line. If you choose a set of points that are not very line like (i.e. the corners of cube) it will still try its best to fit a line to them and then modify all the points so they lie on that line.
When line fitting is complete a dialog will appear showing you the fitting error in the current project units. This error is the root mean squared error (sort of like an exaggerated average error) between the computed line and the 3D points. If this error is too large the fitting might not have worked as expected (perhaps some incorrect points are selected). In this case you can cancel and no changes will be made to the selected points.
The "best line parallel to a coordinate plane" is similar to the best fit except that an additional constraint is added that the fitted line should lie parallel to one of the XY, YZ or XZ planes. The "best line parallel to a coordinate axis" has the constraint that the fitted line should be parallel to one of the X, Y or Z axes.