I was reading a post on the forum the other day and was impressed with the posters ability to measure to .0001. I thought to myself Holy crap he must have some very accurate measuring tools. There was a picture included and the dial indicator was in the picture. I thought to myself I want one of those that goes to the 4th decimal point, and right now. So I blew up the picture as large as my computer would allow to get the model number, and to my surprise the manufacturer of the indicator had printed right on the face of the indicator that it was only capable of .001, or 3 decimal points of accuracy not 4 so how does this poster get the extra accuracy he is claiming? Is there some dial indicator guru like SParker with scales that fixes them, or something I don't understand? Please help I want the .0001 so I can make the best ammo I possibly can.
Howdy !
Back,when I was an aircraft mechanic working in the Hydraulic Shop at the local Air National Guard, we were tasked to measure F-16 Main landing gear “ drag braces.
We were required by the applicable tech order to make length measurement on a
“ calibrated “ machined piece of steel plate… that featured 2 vertical steel pins set for an allowed spacing distance. For the desired level of accuracy and precision in measurement, the Air Force set a certain level of “ flatness “ the top surface of the plate had to maintain, and could not exceed. The limit was called out within some ten thousandths of an inch over the entire length of the plate.
The Air Force’ regional Precision Measurement Equipment Lab would not ok periodic
re-inspection and re-certification of the test jig as approved for use, unless the specified flatness level of the base plate’s top surface was within AF established spec.
How do you “ calibrate “ ( Air Force PMEL description ) a flat steel plate ?!?
You uniform the surface by a performing an intricate machining/polishing of the surface
to get the plate back into spec..and THEN send it to the PMEL lab for re-certification.
To maintain the specified tolerance on “ flatness “, the machine processes had to be performed in a room within an allowable limited temp span. The kicker…. a-soon-as the machined plate left the temp controlled room, the temperature swing was enough to once again put the plate “ out of spec “. Temperature change had effect of the metal down to 10 thousandths of an Inch level. The PMEL lab was some 2hr away from our base, and the jig was treated to a covered pickup truck ride… both ways.
Best we could do was find out what temp the PMEL lab was maintained at, and pass that info on to the machine shop… ostensibly for them to duplicate that temp while they worked on the steel plate. Once PMEL ok’d the jig for use ( certified as calibrated ), best we could do once it made it back to our Base’s Hydraulic Shop; was to make the critical drag brace measurement only when we first had the shop set to the same room temp as the machine shop used.
And, when the fixture once again came up for “ cal “, we did the whole goat rope all over again.
My point:
Don’t….. Ah Say…. DON’T drive measurements down to the .0001 level IF you don’t have to.
That way lies madness.
With regards,
357Mag