I would add a 4th also based on extensive experience, can the users do anything at all about the situation? Being able to order someone to generate numbers doesn't mean the recipients are permitted or capable of doing anything at all. It shows they "care" but a random number generator is more effective WRT dev time than trying to generate meaningful numbers.
I've seen dashboards get tied up in internal politics. The list of recipients of "quality" dashboard is a line in the sand vs the "quantity" dashboard political group. The recipient list and who controls it is far more important than the data contained in the report. Whats important is who reports to who, and why.
A fifth question is does anyone in the chain of command even remotely understand basic statistics like error bars and standard deviations? If the only purpose of the report is to loudly trumpet when pet division B beats divisions A and C thru G, then a very high std deviation / error rate makes stack ranking give the predetermined "correct" result more often. I've seen this personally in "metrics as a teambuilding exercise" where whats actually produced is a weekly report that every division will get to stack ranking win at least once a quarter to meet the Morale Improvement goal on some exec's goal list. Again a PRNG gives better data than real data, if the goal is to give everyone a participation trophy.
I've seen dashboards get tied up in internal politics. The list of recipients of "quality" dashboard is a line in the sand vs the "quantity" dashboard political group. The recipient list and who controls it is far more important than the data contained in the report. Whats important is who reports to who, and why.
A fifth question is does anyone in the chain of command even remotely understand basic statistics like error bars and standard deviations? If the only purpose of the report is to loudly trumpet when pet division B beats divisions A and C thru G, then a very high std deviation / error rate makes stack ranking give the predetermined "correct" result more often. I've seen this personally in "metrics as a teambuilding exercise" where whats actually produced is a weekly report that every division will get to stack ranking win at least once a quarter to meet the Morale Improvement goal on some exec's goal list. Again a PRNG gives better data than real data, if the goal is to give everyone a participation trophy.