How to Judge CNC Machining Center Precision
A machine's brochure tolerance and its real cutting accuracy are two different numbers. This guide shows the checks we run on the shop floor to separate them. Read it before you accept a new machine, buy a used one, or quote a tight-tolerance part.
In this article
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
Key takeaways
What Machine Tool Precision Actually Means
Machine tool precision is not one number. It is four separate behaviors stacked together: geometric accuracy of the axes, positioning repeatability, dynamic accuracy while cutting, and thermal stability over time. A brochure usually quotes only the second one, because repeatability is the easiest to measure and the easiest to make look good.
Geometric accuracy covers straightness, squareness, parallelism, and spindle orientation. If the X axis bows 8 μm over 500 mm, no amount of servo tuning will fix the parts that come off it. Repeatability covers how well the machine returns to the same commanded point. Dynamic accuracy covers what happens when the axes move at 10 m/min with a tool in the cut. Thermal stability covers how all three shift as the spindle and ballscrews heat up.
When you judge CNC machining center precision, the order matters. Check geometry first, repeatability second, dynamics third, and thermal drift last. A machine that fails geometry cannot be rescued by the other three. A machine that passes all four will hold ±0.005 mm on aluminum and steel parts within its work envelope.
The envelope size changes the target. A 500 × 500 × 450 mm machine and a 4,000 × 400 × 150 mm machine cannot be held to the same straightness figure. Longer travel means more accumulated error. Ask for the accuracy figure at a stated travel length, not a single blanket number.
- 1Geometry firstStraightness, squareness, and spindle orientation set the floor.
- 2Repeatability secondHow tightly the machine returns to the same point.
- 3Dynamics thirdAccuracy while the axes are actually moving under load.
- 4Thermal lastHow the first three shift across a full shift of cutting.
How to Judge CNC Machining Center Precision Before You Cut Metal
Start with the machine cold. Cold means at least 8 hours since the last run, with the spindle stopped. Mount a dial test indicator on a magnetic base and sweep the table surface. Look for a total spread under 10 μm across a 500 mm span on a new machine. On a used machine, 20 μm is workable if you know where the low spots are.
Next, check squareness between X and Y. Clamp a granite square or a certified cylindrical square to the table and sweep one face with the indicator. A 300 mm square should read under 10 μm end to end for a new machine. Over 20 μm and you will fight taper on every pocket you cut.
Spindle runout comes next. Put a 0.002 mm indicator on the spindle taper and rotate it by hand. Taper runout should stay under 3 μm. Then insert a test bar and check runout 300 mm from the gauge line. Under 8 μm is good; over 15 μm means the spindle or the taper needs work.
Finally, check backlash on each axis. Command a 0.010 mm move and measure the actual table movement. Backlash over 5 μm on a ballscrew axis will show up as witness marks on every climb-milled wall. If the machine has linear scales, backlash is mostly removed from the control loop, but the mechanical play is still there and still wears.
- 1Table flatnessUnder 10 μm over 500 mm on a new machine.
- 2X-Y squarenessUnder 10 μm over a 300 mm square.
- 3Spindle taper runoutUnder 3 μm measured at the taper.
- 4Axis backlashUnder 5 μm on a ballscrew axis.
Thermal Drift: The Number Brochures Leave Out
A machine can be perfect when cold and useless by lunch. The spindle grows as it heats, ballscrews stretch, and the column leans. On a machine without thermal compensation, expect 10–25 μm of drift in the first two hours of cutting. On a machine with compensation, expect 3–8 μm if the sensors are working.
Measure it properly. Indicate a point on the table cold. Run the spindle at 8,000 rpm for 30 minutes with no cut. Re-indicate the same point. Repeat at 60 and 120 minutes. Plot the three readings. A flat curve is what you want. A curve that keeps climbing after 60 minutes means the cooling system is undersized.
The cutting load matters more than the spindle speed. Roughing aluminum at 12 mm depth of cut pushes far more heat into the structure than a finishing pass. If your shop does heavy roughing, run the thermal test with a real roughing load, not an air cut.
This is also the check most buyers skip, because it takes half a day. It is also the check that explains why a machine that passed acceptance cuts out of tolerance on the third shift.
- 1Test with load, not airA real roughing pass heats the frame differently.
- 2Plot at 0, 60, and 120 minutesA flat curve is the target.
Dynamic Accuracy and the Cutting Test
Static checks tell you the machine is aligned. They do not tell you how it behaves when the axes move at speed. For that, cut a test part. The standard geometry is a circle-diamond-square pattern, 100 mm across, cut in aluminum 6061 at a feed rate you actually use.
Measure the circle with a CMM. Roundness should stay within 10 μm on a healthy machine. A circle that comes out oval points to servo mismatch between the two axes. A circle with flat spots points to backlash or a tight spot in the ballscrew. A square with rounded corners points to acceleration limits set too low.
Then check the diamond. The four faces should match within 8 μm. If one face is off, the machine has a directional error, often from a worn guide or a loose preload. This is the kind of fault that only shows up in cutting, never in a static test.
Finally, repeat the test part three times without changing offsets. The three parts should stack within 5 μm. If they do not, look for thermal drift, chip buildup on the fixture, or a warm spindle. This repeatability across parts is what your customer actually feels.
- 1RoundnessWithin 10 μm on a 100 mm circle.
- 2Diamond face matchWithin 8 μm across the four faces.
- 3Part-to-part repeatWithin 5 μm across three consecutive parts.
Matching Machine Class to Part Tolerance
Not every part needs a 5-axis machine with linear scales. A bracket with a ±0.1 mm tolerance runs fine on a 3-axis mill that holds ±0.02 mm. Spending machine time on a tight-tolerance machine for a loose part is wasted capacity, and it slows your whole shop down.
The reverse mistake costs more. Putting a ±0.010 mm medical part on a machine that holds ±0.03 mm will produce scrap at every setup. Match the machine to the tolerance band, and leave a safety margin. If the part needs ±0.010 mm, use a machine that holds ±0.005 mm.
Feature geometry matters as much as the tolerance number. A deep pocket with a 4:1 depth-to-diameter ratio needs a rigid spindle and good chip evacuation more than it needs a tight positional tolerance. A thin-wall part needs thermal stability and light finishing passes.
At GreatLight we run 16 simultaneous 5-axis centers, 12 four-axis mills, 27 three-axis machines, and 16 mill-turn centers. That spread lets us put each part on the class of machine that fits it, instead of forcing every job onto the tightest machine in the building.
- 1Loose tolerance, small machine±0.1 mm parts belong on a stable 3-axis mill.
- 2Tight tolerance, tight machine±0.010 mm parts need a machine that holds ±0.005 mm.
Step-by-Step Acceptance Procedure
Run these in order. Stop at the first failure and fix it before moving on.
- 1Level and clean the machineCheck the foundation bolts and level the bed to 0.02 mm/m in both directions. Dirt under the pads will show up as a squareness error later.
- 2Sweep the table coldDial indicator on a magnetic base, 500 mm span. Target under 10 μm total spread. Record the low spots.
- 3Check X-Y and Y-Z squarenessUse a certified 300 mm square. Target under 10 μm end to end. Over 20 μm, adjust the column before anything else.
- 4Measure spindle taper runout0.002 mm indicator at the taper, hand rotation. Target under 3 μm. Then check a 300 mm test bar, target under 8 μm.
- 5Check backlash on every axisCommand a 0.010 mm move and read the table. Target under 5 μm. Note any axis over 8 μm for preload adjustment.
- 6Warm up and re-checkRun the spindle at 8,000 rpm for 30 minutes. Re-sweep the table and re-check squareness. Record the shift.
- 7Cut the circle-diamond-square partAluminum 6061, 100 mm pattern, production feed rate. Measure roundness (under 10 μm) and face match (under 8 μm).
- 8Repeat the part three timesSame program, same offsets. Stack the three parts within 5 μm. If not, recheck thermal state and fixture cleanliness.
Precision Checks and Target Values
Values assume a machine within a 750 × 1,150 × 550 mm envelope. Scale targets up for larger travel.
| Check | Method | Target (new) | Action if failed |
|---|---|---|---|
| Table flatness | Dial indicator, 500 mm sweep | Under 10 μm | Re-level the bed and pads |
| X-Y squareness | 300 mm certified square | Under 10 μm | Adjust column, recheck level |
| Spindle taper runout | 0.002 mm indicator, hand rotation | Under 3 μm | Inspect taper, regrind if worn |
| Test bar runout | Indicator at 300 mm | Under 8 μm | Check drawbar force and taper |
| Axis backlash | 0.010 mm commanded move | Under 5 μm | Adjust ballscrew preload |
| Thermal drift (2 h) | Indicate after warm-up cycle | Under 8 μm | Check chiller and compensation |
| Circle roundness | CMM on 100 mm test part | Under 10 μm | Tune servo mismatch |
| Part-to-part repeat | Three parts, same offsets | Within 5 μm | Check thermal state and fixture |
Frequently Asked Questions
How often should I re-check machine precision?
Run the full procedure once a year, and the cutting test every quarter. After a crash, after a spindle replacement, or after moving the machine, run the full procedure again before releasing production parts.
Daily checks can be much shorter. Sweep one reference point on the table and cut one test feature. If both stay in range, the machine is healthy.
Do linear scales remove the need for mechanical checks?
No. Linear scales close the loop on position, so they hide ballscrew wear and backlash from the control. The mechanical wear is still there and still affects surface finish, tool life, and vibration.
You still need to check squareness, spindle runout, and thermal drift. Scales do not correct any of those.
What tolerance can I expect on a well-maintained machine?
On aluminum and steel parts within the work envelope, a healthy machine holds ±0.005 mm and finishes at Ra 0.8–1.6 μm. Tighter finishes down to Ra 0.2–0.8 μm are possible with the right tooling and light finishing passes.
Larger parts lose some of that. A 4,000 mm part will not hold the same figure as a 500 mm part, because error accumulates along the travel.
Is a ballbar test enough on its own?
A ballbar is a good dynamic check, but it only measures the path it is given. It will not catch spindle runout, taper damage, or table flatness errors.
Use it as one input, not the whole answer. Pair it with a physical cutting test and a thermal drift run.
What causes a machine to drift out of tolerance between shifts?
Thermal growth is the usual cause. The spindle, ballscrews, and column all expand at different rates. A machine without working compensation will drift 10–25 μm over two hours.
The second cause is chip and coolant buildup on the fixture. It changes the clamping height by a few microns and shows up as a Z error on every part.
Send Us Your Tight-Tolerance Part
We quote and return a free DFM analysis within 12 hours, run 100% inspection before shipment, and hold ±0.005 mm on aluminum and steel parts.
12-hour quote100% inspection±0.005 mmNo MOQ
More from the shop floor
We publish setup notes, tooling trials and inspection data from the factory floor.
