GreatLight CNC Machining Factory logo
CNC Machining
Rapid Prototyping
Materials
Industries
News
About GL

Get Instant Quote

CNC machining knowledge

How to Guarantee the Machining Precision of Machine Tools

This guide is written for engineers and buyers who must hold tight tolerances on real parts, not on a spec sheet. It covers six proven steps, from machine geometry checks to final probing, plus the parameter ranges and mistakes that cause scrap. After reading, you can tell whether a shop can actually guarantee the machining precision it quotes.

±0.005 mm toleranceRa 0.2–0.8 μm finish100% inspectionISO 9001 / IATF 16949
Precision CNC machining setup used to guarantee the machining precision of machine tools
Key takeaways

Six things that decide accuracy

Thermal drift comes firstA spindle warming from 20 °C to 30 °C can move the tool 20–40 μm. Let the machine idle 20–30 minutes before the first finish pass.
Geometry is checked, not assumedSquareness, straightness and spindle runout need a log. A machine that has not been checked in 12 months is a risk on any ±0.005 mm job.
Runout under 5 μmTool holder and cutter runout above 10 μm shows up directly in hole diameter and wall thickness.
Workholding moves partsClamping force can distort a thin wall by more than the tolerance band. Support the part, do not just squeeze it.
Probing closes the loopMeasure on the machine after roughing, then correct the offset before finishing.
Inspection proves itThe final report matters more than the claim. Ask which features were measured and with what equipment.
Step 1–2

Machine condition: what to check before a tight job

A machine tool does not hold tolerance because it is new. It holds tolerance because its errors are known and controlled. Before a ±0.005 mm job, the shop should verify geometric accuracy: squareness between X, Y and Z, straightness over the full travel, and spindle axial and radial runout. On a 4,000 mm machine, straightness over the whole stroke matters more than a short-travel check, because the part sees the full envelope.

Spindle runout is the number that hits the part directly. On a healthy spindle, radial runout at the taper should stay under 2–3 μm. Once it passes 5 μm, hole diameters start to scatter, and operators often compensate by tweaking offsets instead of fixing the spindle. That hides the problem and creates a machine that cannot repeat tomorrow.

Backlash and servo tuning belong in the same check. A ball screw with 8–10 μm of lost motion will produce different dimensions depending on approach direction. If a bore measures 0.01 mm larger when approached from +X than from −X, backlash is the first suspect, not the cutter.

Keep a simple log. Date, ambient temperature, spindle runout, squareness result, and any correction made. A machine with a 12-month history of stable readings is a different risk from one with no records at all.

  • 1
    Spindle runoutTarget under 2–3 μm at the taper; investigate above 5 μm.
  • 2
    SquarenessCheck X–Y, X–Z and Y–Z, not just one pair.
  • 3
    BacklashCompare dimensions from opposite approach directions.
  • 4
    RecordsOne page per machine, updated every 3–6 months.
Step 3

Thermal control: the quietest source of error

Heat moves metal. A spindle that rises 10 °C during a long roughing cycle will push the tool downward and outward. On aluminum, a 100 mm steel part can grow roughly 1–2 μm per degree Celsius, and the machine structure moves too. The combined effect often reaches 20–40 μm over a morning.

The fix is boring but effective: warm up. Run the spindle at 40–60% of maximum speed for 20–30 minutes before the first finishing cut. Keep the coolant at a stable temperature. If the shop has a temperature-controlled room, hold it at 20 ± 2 °C and let parts equalize before measuring.

Do not measure a hot part. A part pulled straight from the machine and checked on a cold granite table will read small, and the operator will chase a phantom error. Let it cool to room temperature, then measure.

For long runs, check the first part, the middle part and the last part of the batch. Drift shows up as a trend across the run, not as random scatter.

Step 4

Tooling and workholding: where the last 20 μm live

Tool holders and cutters are the shortest path from error to part. Measure total runout with the tool mounted, not the holder alone. A holder that reads 3 μm on the taper can read 15 μm once a collet and cutter are added. Above 10 μm, re-seat the tool, change the collet, or change the holder.

Cutting parameters matter as much as the hardware. In aluminum 6061, a 10 mm carbide end mill at 8,000–12,000 rpm and 0.05–0.10 mm per tooth keeps radial forces low. Push the feed too hard and the tool deflects; cut too light and the tool rubs, which raises Ra and shortens life.

Workholding is where thin parts get lost. A 2 mm wall clamped at 40 bar will spring back after unclamping, and the finished bore will be out of round. Support the part from below, use soft jaws machined to the part profile, and reduce clamping pressure to the minimum that holds the cut.

For five-sided work, a Ø400 mm rotary table with a matched fixture removes one re-clamping step. Every re-clamp adds a new error source.

  • 1
    Runout limitKeep total tool runout under 5 μm for finishing.
  • 2
    Soft jawsMachine them to the part profile, not to a generic size.
  • 3
    Clamp pressureLowest value that holds the part without slipping.
Step 5–6

In-process probing and final inspection

Probing on the machine turns the machine into its own inspector. After roughing, measure two or three datums, then adjust the workpiece offset. This catches fixture shift, thermal growth and tool wear in one pass. On a five-axis job with a rotary table, probing also confirms the rotary center before the finishing path runs.

Probe accuracy has limits. A typical touch probe repeats within 1–2 μm, but the machine's own positioning error sits on top. Treat probe results as a trend, not as absolute truth. If the probe and the CMM disagree by more than 5 μm, find out why before shipping.

Final inspection should measure the features the customer cares about: bore diameters, flatness, position and surface finish. On our jobs we inspect 100% of parts before shipment and can supply reports on request. Typical achievable values are ±0.005 mm tolerance and Ra 0.8–1.6 μm as machined, with Ra 0.2–0.8 μm when a fine finish is specified.

Inspection equipment matters. A caliper cannot resolve 5 μm. Use a micrometer, bore gauge, height gauge or CMM matched to the tolerance. If the measurement tool's uncertainty is half the tolerance band, the reading tells you little.

When it is hard

When tight tolerance is the wrong goal

Not every part needs ±0.005 mm. If a bracket only locates a cover, a ±0.05 mm band is enough, and tightening it adds cost without adding function. Ask which features actually control fit and function, then apply tight tolerance only there.

Some geometries resist tight tolerance no matter how good the machine is. A 0.8 mm wall on a 200 mm part will move during and after machining. Deep pockets with small tools deflect. In those cases, change the design: add a rib, thicken the wall, or split the part into two pieces that are easier to hold.

Material choice also limits the result. Aluminum 6061 machines cleanly and holds size well. Titanium TC4 and Inconel move more, wear tools faster and need slower feeds. Expect wider process windows and more frequent in-process checks.

The honest answer is that precision comes from the whole system: machine, tooling, fixture, thermal control, probing and inspection. A single strong element does not compensate for a weak one.

  • 1
    Apply tolerance where it mattersTighten only the features that control fit.
  • 2
    Design for holdabilityThin walls and deep pockets fight the process.
  • 3
    Match material to the targetAluminum holds size more easily than titanium or Inconel.
Step by step

How to guarantee the machining precision of machine tools on a job

Follow this order on the shop floor. Skipping a step usually moves the error somewhere harder to find.

  • 1
    Warm the machineRun the spindle at 40–60% of max speed for 20–30 minutes. Check ambient temperature; hold 20 ± 2 °C if the room allows.
  • 2
    Verify the setup frameProbe the fixture datums. Confirm squareness and flatness within 5 μm before the first cut. Re-probe if the fixture was moved.
  • 3
    Measure tool runoutMount the cutter and measure total runout. Under 5 μm is good for finishing; above 10 μm, re-seat or replace the collet.
  • 4
    Rough with marginLeave 0.3–0.5 mm radial and 0.1–0.2 mm axial stock for finishing. Roughing heat is expected; the margin absorbs it.
  • 5
    Cool and re-probeLet the part settle, then probe critical datums. Update workpiece offsets from the probe results, not from the drawing.
  • 6
    Finish with light passesUse 0.05–0.15 mm radial depth for finishing. Constant feed, no dwell. Stop the cut if chatter appears.
  • 7
    Measure on the machineCheck key features with the probe. If a bore is 0.01 mm off across the batch, correct the cutter offset before the next part.
  • 8
    Record and inspectLog the offsets used. Send the part for final inspection with a report listing the measured features and equipment.
Reference

Error source, symptom and correction

Use this table when a part is out of tolerance and you need to narrow the cause fast.

Error sourceTypical symptomFirst correction
Thermal driftBores shrink across the runWarm up 20–30 min, control room temp
Spindle runoutHole diameter scatter, poor RaRe-seat tool, replace collet or holder
BacklashSize differs by approach directionCheck ball screw, re-tune servo
Workholding distortionOut-of-round after unclampingSoft jaws, lower clamp pressure
Tool deflectionTapered walls, chatter marksReduce radial depth, raise stiffness
Fixture shiftPosition error on second sideRe-probe datums, re-clamp before finish
Probe offset errorConsistent shift on all featuresRe-calibrate probe stylus and offsets

Precision is a system, not a setting

If you need tight tolerance on a real part, check the machine log, the thermal routine, the tooling runout and the inspection report before you trust the number. That is the difference between a claim and a result.

FAQs

Questions engineers ask about machining precision

What tolerance can a CNC machine actually hold in production?

On a well-maintained machine with a stable setup, ±0.005 mm is achievable on critical features, and ±0.0002 in in imperial terms. The number applies to the feature, not to every dimension on the drawing.

Wider bands like ±0.02 mm or ±0.05 mm are cheaper and faster. Reserve the tight band for the features that control fit and function.

How often should a machine be checked for geometric accuracy?

Every 3–6 months for machines running tight work, and immediately after a crash, a move, or any unexplained size shift. Keep the records with the machine.

Spindle runout should be checked more often, especially after a tool holder change or a heavy interrupted cut.

Does coolant affect machining precision?

Yes, but mostly through temperature. Coolant carries heat away from the cut and keeps the part and the spindle more stable. A consistent coolant temperature is more useful than a colder one.

Flood coolant also clears chips. Recutting chips raises cutting force and spoils surface finish, especially in deep pockets.

How do you handle a part that moves after unclamping?

Reduce clamping pressure first. Then support the part from below and machine soft jaws to the part profile so the load spreads over more area.

If the wall is very thin, rough, stress-relieve, then finish in a second operation. One heavy pass on a flexible wall rarely holds size.

What inspection data should come with the parts?

A report listing the measured features, the nominal and actual values, and the equipment used. For critical parts, include the measurement uncertainty.

We inspect 100% of parts before shipment and can supply reports on request. Ask for the report before placing the order if the part is critical.

Can probing replace a CMM?

No. Probing is fast and closes the loop inside the machine, which is valuable for offset correction. But the machine's own positioning error is included in the result.

Use probing for process control and a CMM or dedicated gauge for final verification. If the two disagree, investigate the difference before shipping.

Send your drawing and get a DFM review within 12 hours

Upload a STEP file and we will review tolerances, tooling and setup before quoting. No minimum order quantity, from one prototype to 10,000+ parts, with 100% inspection before shipment.

12-hour quote±0.005 mm tolerance100% inspectionNDA on request

Follow GreatLight

More CNC machining knowledge

We publish setup notes, tooling trials and inspection data from the factory floor.

FacebookTikTokYouTubeLinkedInInstagramThreadsPinterest

Trusted by engineers and manufacturers worldwide

Tesla Ford Motor Company BYD Auto Denso Magna International Boeing Airbus Medtronic KUKA FANUC