Let me begin by apologizing for the oversight in seeing and responding to your message. I thought I had the correct notifications set up to see discussions posted.
In response to your question, first let me commend you on your diligence, integrity, and your desire to improve your measurements.
Based on what you've said that when your items, like vernier calipers, get close to tolerance limits that they should be changed to "Reference Only". That is a good strategy to reduce the risk of passing product that actually fails. What you want to avoid, and it seems that you understand this, is using a measurement tool that is not accurate enough.
What are you using for standards used to check the calipers? If they are gage blocks, make sure they are calibrated (ideally ISO/IEC 17025 accredited calibration). Gage blocks, calipers, and other hand measurement tools are popular and there are plentiful accredited calibration labs.
The gage blocks should at least have an accredited calibration. You can then calibrate your calipers & micrometers yourself.
The gage blocks should have accuracy and measurement uncertainty that is much smaller than the accuracy needed for your calipers & micrometers.
For example, gage blocks may have a calibration measurement uncertainty of 3 micro-inches. Assigning a tolerance to your calipers of 2x the resolution is an acceptable practice. You can check the impact of that tolerance on your product by determining if the product were out by that tolerance, what would be the effect on its risk to the customer?
Depending on what you are measuring for product; the material, critical dimensions, and the environment in which it is manufactured, tested, and used are some of the factors to consider. There are published standards available, such as in ASTM and with ASME B891.14 (specifically for calipers). https://www.asme.org/codes-standards/find-codes-standards/b89-1-14-calipers
YouTube also has a lot of useful tutorial videos. I recommend using a quality set of calipers and reaching out to reputable manufacturers of these products.
Please let me know if these help clear up your questions and if you have further questions.
Heather A Wade