Google's interview training is pretty specific about what is expected of an interviewer, and the interviewer has to write up fairly comprehensive reports about the questions that were asked, and how the candidate answers them, what hints were given, etc. Most interviewers ask questions that they have asked multiple times before, so they will include things like, "The candidate (TC) took twice as long to code this a very basic warmup question that was designed to motivate a more in-depth distributed computing question; unfortuantely, what a strong candidate could do in ten minutes, TC couldn't get working C code in 45 minutes". (Current best practice is to try to avoid revealing the gender of the candidate in the interview notes, hence the use of TC instead of he or she. The goal is to try to avoid triggering any unconscious bias on the part of the members of the hiring committee.)

Now, there are some screening questions that are asked by recruiters as part of an initial phone screen when they are hiring for SRE candidates. Those questions are multiple choice and are really basic questions designed to screen out old-school operators who are quite good at mounting tapes (for example) but don't have any understanding of what TCP might be, and who think an SRE job is no different from a traditional system administrator position. Those are not asked by an engineer, but by a recruiter; the goal is to avoid wasting everybody's time.

Most interview systems don't have a systematic, ongoing way for accounting for this especially given that interviewers are self reporting.

In my opinion, most interviewers are undertrained/underskilled at asking specific questions and at interviewing in general. They don't invest the resources to systematically improve but rather treat interviewing as a burden to be minimized.

There is still a lot play in systems that allow for borderline smart ass behavior. Do you think an interviewer will write up that they sneered at the candidate?

The interview process is a factory. The company is mostly concerned with reaching outputs. The candidate is just a happenstance casualty.

Of course, that would be rediculously expensive, so at best some canned training is used instead. Still, it might make sense for higher value teams.

Afterwards, the new interviewer has to do a non-trivial number of interviews before they are considered "calibrated". When an interviewer is uncalibrated, their score won't be given much weight, and the hiring committee can see how their interview reports and scores compare against more experienced interviewers. This also gives an opportunity for the hiring committee to send interview feedback (e.g., you're asking a banned question; the coding question is too simple, so it's not providing enough of a useful signal; don't ask "trick questions" which again don't provide much useful signal whether or not the candidate can answer it correctly, etc.)

So there is certainly ways in which interviewers do get suggestions for improvement. And it doesn't have to be _that_ expensive. It's just a matter of making sure you don't have more than one uncalibrated interviewer per panel.

It won't be possible to extract any useful information from such interviews. But I guess it's possible to convince employees to haze candidates this way. You should feel bad for doing this to people though.

Empirically this isn't the case. It would be much more interesting if you took the time to elaborate on what about this process is cultish, why it won't provide useful signal. Without that, it just comes across as a mean-spirited complaint.

For a design question, the interviewer will write up a sketch of the design, what tradeoffs were identified by the interviewee; what hints, if any, were needed, etc.

Then the interviewer will rate candidate on various technical dimensions (coding efficiency, design, etc.) and non-technical dimensions (communication, leadership, etc.) For each of these ratings the interviewer has to justify the rating, by pointing at examples from the interview notes.

Finally, the interviewer will be asked to score the candidate along a dimension of "strong hire" to "strong no-hire" and again, the score must be justified with a paragraph. For people on the hiring committee, the justification for the scores are often far more important than the actual rating given by the interviewer.

The hire/no-hire decision is not up to the interviewers; the hiring committee is composed of a different panel of engineers who review the interview reports from the interview panel; and the members of the interview panel write their interview reports without getting to see or hear from the other members of the interview panel.

The system is not perfect (what system is?) but as someone who has conducted tens of interviews as a Googler and at other tech companies, I can say it's one of the most rigorous and fairest systems I've seen so far. Interviewers are trained to try and get candidates to a 'win', trained to be aware of their unconscious biases, and the hiring committee process reduces the significance of any one vote. In general, the process is designed to select the best candidates and give all candidates a positive experience.

It's not perfect -- indeed, I'm sure I've given 'failing-grade' interviews as an interviewer here and there -- but it's one of the least-worst human systems that I've encountered, from both sides of the process.

Besides being wrong and pathetic, why would anybody want to do that? The goal of maybe 90% of Google engineers is to spend as little time as the company allows on interviewing and writing feedbacks, and instead spend time on actually working on their projects.