I have my doubts about your conclusions given your statements. It seems you have 3 steps in your interview process: code assessment, architecture assessment, and CTO interview. And the candidate who does well on all three gets the job? So, does your funnel filter to exactly one candidate every time? And if it does not, then what criterion are used to funnel all candidates who finished this process successfully? Perhaps, you will argue it comes down to whomever did better on one of the exercises. But that is never a guarantee and would seem a trivial differentiation if multiple candidates had viable and correct solutions.
So, genuine question then: What are the final and most important criterion that would be used to differentiate between two completely successful candidates being equal in their performance in your pipeline?
I think the answer to that will give more credit to the comment you are trying to negate.
In my experience, if there are multiple qualified candidates it usually comes down to things like who lives closest (so we don't have to pay to move them here)? Who was the most friendly/likeable? Who might have a skill that we might need in the future? Etc.
Asking candidates where they live is illegal. You can not discriminate based on exact location. (It’s okay to ask if they need relocation of course, but you can’t decide based on who is “the closest”)
I don't need to ask them. I've never ever seen a resume (and I've seen thousands of resumes) that doesn't have the candidates home address at the very top.
Have you ever seen a resume that doesn't have the candidates address at the top of the page? I haven't. We typically receive hundreds of applications for any one job. We phone interview maybe 20 of them and in person interview maybe 10 of them. Those who we don't hire are never given a reason why we chose someone else. They don't know if it is because they don't have the requisite skills or because they live 2000 miles away and we would have to pay to move them to our state.
This isn't to say that we never hire anyone from out of state. We have done so on many occasions. But if there are 2 equally qualified candidates, we are more likely to choose the one that already lives close by and can start immediately.
So, genuine question then: What are the final and most important criterion that would be used to differentiate between two completely successful candidates being equal in their performance in your pipeline?
I think the answer to that will give more credit to the comment you are trying to negate.