QC by the numbers

^Full disclosure. That's not me. That's George Clooney. Although I look at least that dorky while I'm doing QC.

I’m writing this at the end of a long day doing quality control, or “QC” inspections on a new model, the NTH DevilRay. 

The topic of QC came up recently in a spirited discussion among some enthusiasts on Facebook, and I thought I should take the opportunity to make a blog post about it, while I'm thinking about it, within the context of today's results.

First, let me dispel a myth – there is no universal standard for QC. Every company has their own process, and their own standards. 

If that’s true, and I assure you it is, then it stands to reason that customers aren’t in a position to say what should or should not have passed QC, unless they are intimately familiar with that specific company's QC standards, which seems very unlikely in most scenarios.

In case the point above is too subtle, let me drive it home with some emphasis – it’s not for customers to say what is or is not within QC standards. That is the sole domain of the company doing the selling. It is only for customers to decide whether or not they are satisfied, and they should decide that before accepting the watch, by which I mean, before they unwrap and wear it, which is an implicit acceptance of its condition, according to most companies' return policies. 

The “Rolex Standard"

I’ve been fortunate to know several watchmakers who attended the watchmaking school sponsored by Rolex, as well as watchmakers who went through WOSTEP in Switzerland, and some other watchmakers with unassailable qualifications. 

What vexes me is the varying answers I’ve gotten when I asked them about QC standards, which only goes to show that those standards are a fungible thing, not something which is agreed upon industry-wide.

However, I have found some small amount of consistency among them, and it seems as if most of the industry uses the same standards Rolex does, at least when it comes to stuff under the crystal. 

In a nutshell, what I’ve frequently heard is that anything which can’t be seen with a 3x loupe simply “isn’t there”, and even at that, “micro-dust” (tiny particles in the air) may still be visible.

But there’s more to QC than just stuff under the dial, and as such, I and every other person in charge of QC at any company, have had to come up with our own standards for every aspect of operation, fit, finish, and alignment. 

The “Vail Standard" – what do I look for?

My QC process has evolved over time, but currently, this is an as-concise-as-I-can-make-it summary:

1. Visual inspection of case, to include caseback and bezel, looking for any scratches or dings, or uneven patches in the finish. 

Any visible marks are an automatic QC failure. I’ll either send those pieces back to the factory for replacement or refinishing, or I’ll use those pieces as parts donors, or I’ll sell them as “irregulars” – pieces sold at a discount, as-is, with the imperfections disclosed prior to sale.

2. Check to make sure the clasp is installed correctly (right-side-up, with logo facing the right direction). 

It’s a stupid little thing, but every so often, we’ll find a clasp attached to the bracelet with the logo upside down. It’s an easy fix when it happens.

3. If there’s a rotating bezel – inspect the alignment of the insert and check the bezel action. 

4. Check the hands alignment, i.e., how does the hour hand line up with each hour marker when the minute hand is at 12?

Here again, the watch’s hands are installed by human hands, and while we can often get perfect alignment, perfection isn’t the standard. Generally, if I think a customer would complain about the alignment, or if I’m not happy with it being considered a true representation of our quality, it gets set aside for my watchmaker to reset the hands.

5. Assuming there is a date window, check the date change time and mechanism.

All I’m doing here is making sure the date change happens close to midnight, then checking to make sure the quick-set date change works as it should.

6. Inspect for debris under the crystal, any imperfections in the crystal itself, or its anti-reflective (“AR”) coating, and look for any clear imperfections in the dial or hands.

This is where the jeweler’s loupe comes in. Anything visible under the crystal will need to come out, if it’s a speck of dust or a fiber. 

If there’s any imperfection in the crystal itself, or the AR coating, we’ll re-assemble the case with a new crystal, or re-apply the AR. 

Likewise, the dial or a hand may need to be replaced, if I find an unacceptable imperfection in the part. Applied hour markers are installed by hand, and lume is applied by hand, so again, there can be a small degree of human error.

What’s “acceptable”? It’s a judgment call, but here again, if I think a customer is likely to complain about anything, or if I’m not happy having it considered representative of the quality we deliver, we’ll look to replace the part. Thankfully, it’s rare that I find any reason to replace a dial or hand.

7. Shake the watch, listening for the rotor. 

8. Wind the crown, a lot, and listen to the movement as I do.

9.Check the crown’s engagement, and if applicable, the screw-down action. 

Here in step 9, as well as steps 4 & 5, I check to make sure the crown positively engages in time-setting, date-setting, and winding positions. 

Sometimes the crown will slip between positions. It may require a bit of working back-and-forth for the lubricants within the movement to get evenly distributed. Other times, the crown stem can be cut a little too long or too short, which can be the cause of the crown not engaging correctly. 

If I find that the crown doesn’t operate as it should, even after working it a bit, I’ll set it aside for my watchmaker to look at, and possibly re-cut the crown stem, or install a new one. Otherwise, I just return the crown to its winding position or screw it down, as applicable.

10.Check the timekeeping and amplitude are within manufacturer’s specs. 

So…what are the numbers, or, what’s our defect rate?

The process I outlined above probably sounds fairly thorough, and I’d argue that it is, even though I understand other brands may have even more thorough QC processes. 

What’s important for people to understand is that every part of the watch goes through at least 3 rounds of QC prior to being sent for assembly, the movements will be adjusted to within spec (if need be), and each watch goes through at least 2 rounds of QC after being assembled (including timekeeping tests), BEFORE it gets to me.

The QC I do is the 5^th, if not the 6^th round, and by the time I get the watches, I shouldn’t be finding ANY mechanical or assembly defects. I shouldn’t find ANY watches not running within spec.

Going further, the movements most microbrands like mine use are usually “off-the-shelf” components from large suppliers, with low defect rates. Very often, we're doing no customization of the movements, which can sometimes lead to defects. As such, we’re not expecting to pull many, if any watches out for mechanical issues.

And therein lies the rub. When you’re not expecting to find many, if any defects, it’s easy to let your guard down, and of course that’s when you get a spike in your return for defect rate. It’s a rookie mistake, one I’m willing to admit even I have made.

It’s not uncommon for me to get a 300-500 piece production without any issues at all, when I won’t pull a single piece out for any reason whatsoever. That’s always a good result, but generally, I assume I’ll pull 1% out for some issue, be it quality, assembly, or operation. 

Within that 1% number, most issues – about 90% - can be fixed before sending the watch to a customer or putting it into our inventory. Perhaps only 1 in 1,000 pieces (0.1% of our total production, across all models and all production runs) will have an issue we can’t sort out, leading us to use that watch as a parts donor or sell it as an irregular. 

But that's assuming we even spot the issue, and we may not, since we’re talking about very small, tiny issues, things which have already made it through multiple rounds of QC before I get the watches. Some of those will end up in my personal collection, if the issue is something small enough that I can overlook it, as many of them are.

We sometimes find that a model has a high rate of one particular assembly issue. For instance, we might find that 5% of the crown stems were cut the wrong length, or the clearances between two parts are so tight that 6% of those parts exceed the specified tolerance, which can lead to some latent defects, only revealed through customer use. 

With the movements, the numbers can vary from one batch to the next. 

The defect rate I’ve seen with the Seiko NH35 is about 0.2% (1 movement replaced for every 500 used). With the Miyota 9015, it’s been as low as 0.1% to as high as 3%. With both Swiss and Chinese movements, I’ve seen higher rates, from 10% to 30%, though most watchmakers I’ve asked have said 5%-6% is to be expected, and acceptable, for Swiss movements.

The killer combo is when we get a higher-than-expected defect rate in the movement, combined with an unexpectedly high number of assembly errors. Dealing with that sort of double-whammy is no day at the spa. 

It can and does happen (it happens to me about once in five production cycles, or 20% of the time), and so it’s important for companies to be prepared to deal with it, not only by doing good QC, but also by having robust post-sale capabilities in place. It’s good for companies to maintain relationships with multiple watchmakers in various regions around the world. 

Last round – the customer’s inspection

As I said above, because there is no universally accepted standard for QC, each company must come up with their own process and standard, and therefore, no customer can say what is or is not within that standard. That decision is at the sole discretion of the company, not the customer.

That said, and for that reason, it’s up to the customer to do the last round of QC for themselves, before accepting the watch. 

That means the customer – that’s YOU – must make the effort to inspect new arrivals carefully, looking for anything amiss, BEFORE you do anything which will prevent you from being able to return it.

Most returns policies stipulate that a watch must be in as-delivered condition for the company to accept its return for a refund or replacement. Once you’ve unwrapped and begun wearing a watch, most companies won’t offer a refund or replacement if you find something “wrong” with it later. 

At that point, they may only offer to repair or correct the “defect”, and only IF they agree that what you’ve found IS a defect at all, and not something within their QC standards.

Understanding that last point is important. 

I said above that I generally assume we’ll pull 1% out during QC, which is the 5^th or 6^th round. That means, if you find something “wrong” with a watch we sent you, the odds of it being something that’s NOT within our QC standards are pretty slim. 

More likely, it’s something we deemed to be within our acceptable range, or it’s within that 1%, but it just happened to slip through five or six rounds of QC. If it’s within the 0.1% of issues we can’t sort out, but you’ve accepted the watch by wearing it, we may not be able to resolve the issue to your satisfaction.

Bottom line – while we do our best to conduct thorough QC, the last round is up to our customers.

Chris Vail is the lead designer and majority owner of Janis Trading company. He's QC'd over 3,000 watches, and blames his wife for most issues that slip through.