Nano Laser Interferometer: Non-linear Error Symptoms, Causes, Correction
If your linear axis passes the backlash check but still fails a positioning test, the fault is often periodic. This page is for engineers and metrology staff who calibrate CNC axes with a single-frequency nano laser interferometer. Read it to tell optical, thermal and electronic sources apart, and to pick the right correction before you touch a compensation table.

Symptom, likely cause, first action
Read the left column first. A symptom with a clean sinusoid is rarely a screw problem.
| Symptom on the readout | Most likely cause | First action |
|---|---|---|
| Error repeats once per 633 nm fringe | Periodic non-linearity in the receiver | Log the period, then swap the receiver head |
| Error repeats once per screw turn | Screw pitch error or a bent lead screw | Compare at two feed rates before touching anything |
| Slow drift over 30-60 minutes | Thermal growth in the optic mount | Soak the setup for 1 hour and repeat the run |
| Spikes that last one sample | Cable or connector noise | Re-route the cable away from drives |
| Error grows with travel length | Dead-path or Abbe offset error | Re-measure with the optic on the axis centerline |
| Error appears only above 2,000 mm/min | Servo lag or vibration, not optics | Drop the feed rate and re-run the same test |
| Error differs between forward and reverse | Backlash or reversal error | Check mechanically before applying compensation |
When to correct and when to fix the machine
Correct the interferometer only when the curve is periodic, speed-independent and unchanged by Abbe offset. If the curve moves with feed rate, travel length or temperature, fix the machine or the environment first; a compensation table written from bad data makes the axis worse.
How a nano laser interferometer produces non-linear error
A single-frequency nano laser interferometer counts interference fringes to resolve displacement down to the nanometer. The count should be a straight line against true travel. Non-linear error is the small curve on top of that line: a periodic deviation that repeats with the optical path rather than with the distance moved. On a 633 nm helium-neon source the classic period is one fringe, about 316 nm of displacement, though the amplitude you see depends on the receiver, the polarizing optics and the alignment.
The error is not random. If you plot deviation against position, a genuine non-linear source gives a repeatable sine-like shape. That shape is the reason the fault survives a fresh calibration. You can zero the axis, re-run the test and get the same curve within a few nanometers. Random scatter means something else is wrong, usually noise or vibration.
In practice you meet this error in two places. First, when a machine is bought off after a linear positioning test and the numbers look worse than the machine's own repeatability. Second, when a compensation table is built from bad data and then applied to the control. The second case is worse, because you are now correcting a measurement artifact with a real axis move.
Keep the scale in mind. Non-linear error on a healthy single-frequency system sits in the low nanometer range over a short travel. If your curve is tens of micrometers, you are not looking at non-linearity. You are looking at a mechanical or thermal problem that the interferometer is faithfully reporting.
- 1Periodic, not randomThe deviation repeats with optical path, so a re-run gives the same curve.
- 2Independent of distanceAmplitude stays roughly flat as travel grows; a pitch error does not.
- 3Small in absolute termsLow nanometer amplitude on a healthy receiver, not tens of micrometers.
Optical, electronic and environmental sources
Optical sources sit at the top of the list. A receiver that mixes the two polarizations imperfectly turns phase into amplitude, and the fringe count drifts as the beam intensity changes. Misalignment of the polarizing beam splitter or a waveplate rotated away from its design angle adds a second harmonic to the curve, so you see two cycles per fringe instead of one. Dirty optics do the same thing in a less repeatable way.
Electronic sources are easier to fix but harder to spot. Comparator threshold drift in the counter, limited bandwidth in the amplifier, and a reference clock that is not locked to the source all show up as period error. Cable noise adds spikes rather than a smooth curve, so you can usually separate the two by looking at the shape of the data rather than its size.
Environmental sources behave differently. Air temperature changes the refractive index of the beam path, which shifts the apparent wavelength. The result is a slow drift, not a periodic wobble, and it grows with the length of the beam in air. A 1 °C change over a 1 m path is enough to move the reading by roughly 1 μm, which will bury nanometer-level non-linearity in thermal noise.
For CNC work the practical split matters. Optical and electronic causes need the interferometer fixed. Environmental causes need the shop fixed: soak time, air shielding, and a stable foundation. Trying to correct a thermal drift with a compensation table on the control is a dead end, because the drift is not repeatable from morning to afternoon.
How to separate non-linear error from machine error
Run the same test twice, once with the optic mounted on the axis centerline and once offset by a known distance. Abbe offset error scales with the offset; true non-linearity does not care where the optic sits. This single test removes the most common false alarm on long beds, where the optic is often clamped to the table edge for convenience.
Then change the feed rate. Non-linear error from the receiver is independent of speed. A screw pitch error, a servo tuning problem or a resonance in the structure all change when you change speed. If the curve moves when you change feed, look at the machine first and the interferometer second.
Compare forward and reverse passes. A stable gap between the two directions points to backlash or reversal error. A curve that flips sign between directions points to a cosine or sign error in the interferometer or its setup, not to lost motion in the screw.
Finally, shorten the travel. Non-linear error keeps its shape over a short run; dead-path error and thermal drift usually shrink or vanish. If a 100 mm run is clean and a 1,000 mm run is not, you are dealing with the long path, not the optics.
Write all four results down before you change anything. Most bad compensation tables come from a technician who adjusted the machine on the first suspicious curve instead of running the checks that identify it.
Step by step: correcting non-linear error
Work in this order. Each step removes one variable, so a later change is easier to trust.
- 11. Record the baselineLog position, deviation, feed rate and air temperature over at least three full runs. Keep the raw files. You need the shape, not a single peak-to-peak number.
- 22. Soak the setupLet the interferometer and the machine sit in the same air for 1 hour before the run. Shorten the beam path in air where you can, and shield the beam from direct airflow and lamps.
- 33. Clean and re-align the opticsClean every surface with lint-free wipes and reagent-grade solvent. Check the polarizing beam splitter and waveplate angles against the datasheet, and confirm the return beam lands on the receiver aperture center.
- 44. Re-seat cables and check the counterRoute signal cable away from servo drives and spindle power. Tighten connectors, then verify the counter reads zero over a static path for 10 minutes; drift here is electronic, not optical.
- 55. Repeat the offset and feed-rate testsRe-run with a 50 mm Abbe offset and at two feed rates. If the curve is unchanged, the interferometer is now the only remaining suspect.
- 66. Apply the correction at the sourceUse the manufacturer's non-linearity correction, or map the residual curve into a compensation table in the control. Cap the compensation at 2-3 μm per 100 mm; larger values mean the root cause is still in the machine.
- 77. Verify with an independent methodConfirm the corrected axis with a second technique, such as a ballbar or a step gauge. Agreement within ±0.005 mm over the working range is the usual acceptance window in our shop.
Questions engineers ask
What amplitude of non-linear error is normal on a single-frequency system?
On a well-aligned receiver from a reputable maker, expect low single-digit nanometers over a short travel. The number that matters more is repeatability: three runs should overlay within a nanometer or two.
If you are seeing tens of nanometers, check alignment and the polarizing optics before you blame the machine.
Can I fix non-linear error with the machine compensation table?
Only the residual part. The compensation table moves the axis, so it can cancel a repeatable curve, but it cannot cancel a curve that changes with temperature or speed.
Keep the applied compensation small, a few micrometers per 100 mm at most. Large entries usually hide a mechanical fault.
Why does my error look worse in summer?
Air refractive index changes with temperature and pressure. A warmer shop shifts the apparent wavelength and the reading drifts with it.
Record air temperature and pressure during the run. If the curve follows the temperature trace, the problem is the environment, not the interferometer.
Does a shorter beam path help?
Yes. The dead path in air is the part of the beam that does not move with the axis. Every degree of temperature change over that path adds error.
Move the interferometer closer, or use a tracking reflector setup so less of the beam sits in open air.
How often should the interferometer be checked?
Check the setup before any calibration that will feed a compensation table. Re-verify the receiver after transport, after a firmware update, or after any optic is removed and refitted.
There is no fixed interval that fits every shop. The trigger is the consequence of a bad measurement, not the calendar.
Can we do this in-house on a 4,000 mm axis?
Yes, if you can keep the beam path stable. Long beds make thermal drift the dominant term, so soak time and air shielding matter more than the instrument.
For parts up to 4,000 mm we machine on 5-axis centers with a Ø400 mm rotary table and inspect with 100% checks before shipment. Interferometer data from the customer is welcome when it changes the setup.
Send us the curve, we will read it with you
Share your interferometer log and drawing. We return a quotation and a free DFM analysis within 12 hours, and we can start production within 24 hours once the setup is agreed.
12-hour quote100% inspectionNo minimum order quantity