How to Ensure CNC Machining Accuracy: A Shop-Floor Routine
A practical routine for machine operators who need parts to hit ±0.005 mm and stay there across a shift. We cover calibration, tool setting, probing, and thermal drift, with the numbers and the mistakes that break each one.
Key takeaways
What accuracy means before you touch an offset
Accuracy is the ability of the machine and setup to land a feature where the drawing says it should be. Repeatability is the ability to land there again on the next part. Operators mix the two up constantly. A machine can repeat within 0.003 mm and still cut every part 0.03 mm off nominal because a single offset is wrong.
Tolerances split into two families. Dimensional tolerances control size: bore diameter, slot width, step height. Geometric tolerances control form and relationship: flatness, perpendicularity, position, concentricity. A part can pass every size check and still fail assembly because the position tolerance was never verified.
Match the measurement tool to the tolerance band. A caliper reads to 0.02 mm at best, so it cannot judge a ±0.005 mm bore. Use a bore gauge, micrometer, or CMM for anything tighter than ±0.02 mm. The 10:1 rule still holds: your gauge should resolve one tenth of the tolerance you are checking.
Shop temperature belongs in this conversation too. Aluminum expands about 23 μm per meter per °C, steel about 12 μm. A 5 °C swing across a shift moves a 300 mm steel part roughly 0.018 mm. That is larger than the tolerance on many of our jobs. Measure at the same part temperature every time, and let parts cool before final inspection.
Calibrate the machine before the first cut
Start every campaign with a geometric check, not a warm-up jog. Ballbar testing gives you circularity and backlash in one run. A healthy VMC holds circular deviation under 10 μm over a 100 mm circle. If the plot shows a step at each axis reversal, backlash is present and the ballscrew or thrust bearing needs attention.
Squareness and parallelism come next. Indicate a granite square along X and Y; a 300 mm span should not show more than 0.01 mm of deviation on a well-kept machine. Spindle taper runout should sit under 0.005 mm when you indicate a test bar 100 mm from the gauge line. More than that and every tool you load inherits the error.
Thermal compensation is not a substitute for a cold machine. Let the spindle run 20 to 30 minutes at working speed before you trust the first offset. On machines without scale feedback, the headstock grows 10 to 20 μm in that window. Take your master gauge reading after the warm-up, never before.
Log what you find. Date, ambient temperature, ballbar result, backlash values, and the offset shift you applied. When a job drifts three weeks later, the log tells you whether the machine moved or the process did. Without it you are guessing.
Set tools so runout does not eat your tolerance
Tool runout is the quiet error. A 12 mm end mill with 0.03 mm TIR cuts one flute deeper than the others, and the resulting surface finish reads Ra 2.5 μm instead of the Ra 0.8–1.6 μm you planned. Check TIR with a dial indicator on the flute, not on the shank. Aim for under 0.01 mm on finishing tools, under 0.02 mm on roughing.
Clean the taper every single change. A chip or a film of coolant on the CAT 40 or HSK interface tilts the holder and adds 0.01 to 0.02 mm of runout instantly. Wipe the taper, blow out the retention knob area, and seat the holder by hand before the drawbar pulls it in.
Preset tools offline when the machine allows it. A tool presetter measures length and diameter to 0.002 mm in a controlled environment, so the operator only verifies the offset instead of discovering it. In-machine touch-off with a spindle probe is the next best option. Paper shims and feeler gauges are not measurement tools at this tolerance level.
Watch tool wear by the numbers. Flank wear past 0.2 mm on carbide changes cutting forces and pushes dimensions. Track parts per edge and swap on count, not on sound. For aluminum at 6061 with a 10 mm carbide end mill, 3,000 to 5,000 parts per edge is a reasonable starting budget for finishing passes, and you adjust from there.
Step by step: the six checks that hold accuracy
Run these in order at the start of a job and again after any crash, tool change, or long idle period.
- 11. Warm up and verify geometryRun the spindle 20–30 minutes at working rpm. Then check ballbar circularity (target under 10 μm on a 100 mm circle) and squareness over 300 mm (under 0.01 mm). Log ambient temperature with the result.
- 22. Check tool runout and seat the taperIndicate each finishing tool on the flute. Hold TIR under 0.01 mm for finishing, under 0.02 mm for roughing. Wipe the taper clean at every change; a chip adds 0.01–0.02 mm instantly.
- 33. Set work offsets from a known datumTouch off on the vise jaw or a dedicated datum feature, never on a rough face. Verify with a second touch on a different feature. Anything over 0.01 mm disagreement means the part moved or the setup is soft.
- 44. Probe the first part in-processMeasure the critical features while the part is still clamped. Correct the offset and re-cut if the deviation is under 0.03 mm. If it is larger, stop and find the cause instead of chasing it with offsets.
- 55. Control the thermal environmentKeep the shop within ±2 °C during the run if the tolerance is ±0.005 mm. Point fans away from the machine. Never open a bay door onto a hot machine mid-job; the column bends before the control notices.
- 66. Inspect after cooldown and recordLet parts reach room temperature before final measurement. Record the offset shift, tool count, and any out-of-tolerance feature. Feed those numbers back into the next setup sheet rather than starting from zero.
Which control matters for which tolerance band
Use this to decide where to spend setup time. Tighter bands need more controls, not more operator attention.
| Tolerance band | Dominant error source | Control to apply | Check frequency |
|---|---|---|---|
| ±0.1 mm and looser | Workholding and chip load | Standard vise, moderate feeds | Start of job |
| ±0.05 mm | Tool wear and offsets | Tool presetter, edge count tracking | Every 50 parts |
| ±0.02 mm | Tool runout and thermal growth | TIR under 0.01 mm, 30 min warm-up | Every 20 parts |
| ±0.01 mm | Spindle and ballscrew drift | Ballbar check, scale feedback | Daily |
| ±0.005 mm | Temperature and clamping stress | ±2 °C shop, light clamping, probing | Every 5 parts |
Common questions from the floor
Does a newer machine remove the need for calibration checks?
No. A new machine holds geometry better out of the crate, but it still drifts with temperature and wear. We ballbar every machine on a schedule regardless of age.
The schedule changes with age, not the practice. A two-year-old VMC may need a quarterly check; a ten-year-old one may need it monthly.
How often should offsets be re-verified during a long run?
For tolerances of ±0.02 mm or tighter, probe the first part of every few hours and after any pause longer than 30 minutes. Thermal growth accumulates fastest in the first two hours after a cold start.
For looser work, once per shift is usually enough. The decision should come from your own drift data, not from a fixed rule someone handed you.
Can coolant temperature affect dimensional accuracy?
Yes, and it is often overlooked. Coolant that warms 8 °C over a shift transfers heat into the part and the fixture, which moves dimensions on thin walls and long parts.
On tight jobs, chill the coolant or at least monitor its temperature. If the return temperature climbs more than 5 °C from start to end of shift, expect the part to move with it.
What causes a part to measure good on the machine and bad at inspection?
Usually temperature. The part is warm and expanded when you measure it on the machine, then contracts in the inspection room. A 300 mm aluminum part can shrink about 0.03 mm over a 4 °C difference.
The second cause is clamping stress. A part released from a vise springs back, especially on thin sections. Probe before unclamping, then confirm after release to separate the two effects.
When should we stop adjusting offsets and investigate instead?
When a single correction larger than 0.03 mm does not hold on the next part. That pattern means the error is mechanical, thermal, or workholding-related, not an offset problem.
Keep adjusting and you will chase the error around the part. Stop, indicate the setup, check runout, and look at the fixture before touching the control again.
Do we need a CMM to hold ±0.005 mm?
Not always, but you need a gauge that resolves to 0.0005 mm, and hand tools rarely get there. A good micrometer and bore gauge can cover simple features. Position and form usually need a CMM or a vision system.
Whatever you use, keep it in the same thermal environment as the parts. A CMM in an air-conditioned room measuring parts straight off a warm machine reports the temperature difference, not the part.
Send us the drawing and the tolerance callouts
We run 127 high-precision CNC machines and inspect 100% of parts before shipment. Upload your files and we return a quotation with free DFM analysis within 12 hours.
12-hour quote±0.005 mm100% inspectionISO 9001:2015
More from the shop floor
We publish setup notes, tooling trials and inspection data from the factory floor.
