Machining a Complete Knowledge of the Precision of Machined Parts

• 1
  What Machining Precision Actually Measures
• 2
  Tolerance Grades and Why IT Numbers Matter
• 3
  Where Machining Error Comes From
• 4
  Surface Finish, Cutting Data, and the Ra You Get
• 5
  GD&T: Position, Form, and the Datum Chain
• 6
  Material Behavior and Setup Choices
• 7
  How Precision Is Verified
• 8
  Process Capability Against Tolerance and Finish

Machining precision is the gap between the geometry a drawing asks for and the geometry the cutter leaves behind. The drawing states an ideal size, an ideal form, and an ideal position. The finished part has a real size, a real form, and a real position. Precision is how close the real one sits to the ideal one.

That gap has a name: machining error. Error is not a defect by itself. Every process leaves some. Turning, milling, grinding, and EDM all produce a spread around the target value, and the width of that spread is what sets the achievable tolerance for a given feature.

Precision is therefore a range, not a number. Two shops can both hit ±0.005 mm on a 20 mm bore and still deliver different parts, because one holds that band over 500 pieces and the other holds it over 5. Repeatability is the part of precision that shows up in a production run.

A practical way to read it: precision describes the process, tolerance describes the requirement. The drawing sets the requirement. The shop has to pick a process whose natural spread sits comfortably inside it.

ISO 286 defines tolerance grades from IT01 up to IT18. The lower the number, the tighter the band. IT01 is used for gauge blocks and reference masters. IT7 to IT9 covers most fits in general machinery. IT11 to IT14 is where welded frames and cast housings usually land.

The grade is not an absolute size. It is a fraction of the nominal dimension, so a 10 mm shaft at IT7 has a tighter band in micrometers than a 400 mm shaft at the same grade. This is why copying a tolerance callout from a small part onto a large one often creates an unbuildable print.

For CNC work, a useful reference: turning and milling on rigid machines routinely hold IT7 to IT8 without special effort. Grinding pushes into IT5 to IT6. Fine boring and lapping go lower. Each step down costs time, fixtures, and inspection effort.

When a designer writes ±0.005 mm on a 300 mm aluminum bracket, the band is roughly IT5 territory. That is not impossible, but it usually means temperature control, a stress-relieved blank, and a CMM check. Cheaper to loosen the callout than to pay for the process.

Error sources fall into a few families: the machine, the tool, the workpiece, the fixture, and the environment. Machine errors include spindle runout, axis straightness, and thermal drift in the ballscrews. A spindle that has run for four hours is not the same machine it was at startup.

Tool errors come from wear, deflection, and runout in the holder. A 6 mm end mill sticking 40 mm out of the collet will bend under load. The cut goes shallow at the bottom of the wall. Reduce the stickout or take lighter radial passes and the wall straightens out.

Workpiece errors are often thermal or residual stress. A 7075 block machined from plate will move after the skin is removed because the internal stress balance changes. Rough, stress relieve, then finish. That sequence costs a day and saves a scrapped batch.

Fixture errors are subtler. Clamping force deforms thin walls before the cutter touches them. A 2 mm aluminum wall clamped at 300 N bows inward, gets machined straight in the clamped state, and springs back curved when released. Soft jaws or vacuum fixturing avoid this.

Environment rounds it out. A 1 °C change moves a 500 mm steel part about 0.006 mm. For work at ±0.005 mm, the shop floor temperature band matters as much as the machine spec sheet.

Surface finish and dimensional precision are separate requirements, but they interact. A rough surface is a field of tiny peaks and valleys. Measure across it and the local size varies by roughly the peak-to-valley height. If Ra is 3.2 μm on a bore, a micrometer reading can shift by a few micrometers depending on where the anvil lands.

The main lever on Ra in milling is feed per tooth. Theoretical peak-to-valley height scales roughly with feed squared divided by eight times the tool radius. Halve the feed per tooth and the scallop height drops about four times. Halve the tool radius and it doubles.

Spindle speed and coolant matter too. Aluminum at 6061-T6 cuts clean at high speed with good chip evacuation. 316L stainless work-hardens if the feed is too light, so a timid pass often produces a worse finish than a firm one.

Vibration is the wildcard. Chatter leaves a regular pattern of marks and can push Ra from 0.8 μm to 3.2 μm in a single pass without changing any program value. Shorter tools, more rigid setups, and tuned speeds fix it more reliably than polishing the part afterward.

GD&T separates four things that a ± callout mixes together: size, form, orientation, and location. A position tolerance at MMC on a bolt circle controls the hole pattern. Flatness controls the surface itself. They are independent and both can be tight or loose.

The datum chain decides how the part is held for inspection. If the drawing calls A, B, C but the shop clamps on a different face, the measured position values will not match the print even when the part is good. Agree on datums before the first cut, not after.

Bonus tolerance is often wasted. Position at MMC gains extra tolerance as the hole grows from its minimum size. A hole drilled to the high end of its size range can pass a position callout that the nominal hole would fail. Useful on clearance holes, not on press fits.

Where GD&T is overkill: simple turned bushings with one diameter and a length. A pair of ± callouts and a finish note says everything. Adding a feature control frame there just slows the drawing review.

Material choice moves the achievable band more than most people expect. Aluminum 6061 machines freely and holds tight sizes. Titanium Ti-6Al-4V generates heat at the cutting edge, springs back under the tool, and needs lower surface speeds. The same tolerance costs more in titanium.

Thin walls are the classic constraint. As a rule of thumb, a wall thinner than about 20 times the cutter diameter is where deflection starts to dominate. Below that, plan on multiple light finishing passes, or support the wall from behind.

Hardened steel above 45 HRC usually goes to grinding or EDM for the final size. Milling it to ±0.005 mm is possible with the right tooling, but the tool life cost is high and the process is less forgiving of a small program error.

Setup count drives precision as much as the machine. Every refixture adds a new stack of positional error. A part that needs four setups accumulates four times the locating error of a part that needs one. Five-axis work exists mostly to remove setups.

You cannot claim precision you do not measure. Calipers read to about 0.02 mm and are fine for rough checks, not for a ±0.005 mm callout. Micrometers, bore gauges, and height gauges cover most shop-floor verification. A CMM covers position and form.

Temperature is part of the measurement. ISO 1 defines 20 °C as the reference. A part measured at 26 °C in an uncontrolled room reads larger than it is, and a steel gauge block reads different again. For tight work, let the part soak to room temperature before the final check.

Sampling strategy matters. Checking one part per batch proves the process worked once, not that it holds. On a first article, check every dimension on the print. On a running job, monitor the dimensions that drive the fit and let the rest ride.

Documentation closes the loop. Material certs, in-process readings, and a final inspection report let the buyer confirm what was delivered. Ask for the report up front rather than after the parts arrive.