Questions to ask MIS, and what a good answer looks like

You do not need to understand the ILR to hold it to account. You need to know which questions expose a problem.

The best questions are about what is known to be wrong, not about whether everything is right. Nobody can answer the second one honestly.

A question like 'is our data accurate?' invites reassurance. A question like 'how many learners are recorded as in learning past their planned end date, and what is the largest reason?' invites a number, and the number is either small and explained or it is a problem.

The pattern to look for in an answer is specificity. A good answer names a figure, a cause and an owner. A weak answer describes a process.

The questions below map onto measurable positions in the return, so each can be answered from a report rather than from opinion. Where a college cannot produce the number, that is itself the finding.

Four questions cover most of what can go wrong. Ask the same ones every month so the trend is visible rather than the snapshot.

• How many learners are recorded as in learning who have not attended recently?
• How many aims have ended with no outcome recorded?
• What moved in earnings since last month, and why?
• What is outstanding on the data quality and monitoring reports?

Each of these has an observable answer that comes from a report rather than from an opinion, which is what makes them useful. A college that cannot produce the number has told you something important without meaning to.

Asking the same four every month turns them into a trend. A count that rises month on month is a control failure, and the trend makes that visible in a way a single figure never does.

The fourth question is the one that most often uncovers something. Reports that arrive and are not worked accumulate quietly, and they test exactly what an audit will test.

The first two map to standing reports over the current return: learning deliveries with no actual end date whose planned end date has passed, and learning deliveries with an actual end date and no outcome recorded.

The third is the period-on-period comparison of the funding reports, worked at total level then by funding model then by curriculum area.

The fourth is the exception log: every flagged record with its outcome recorded as corrected, evidenced, or awaiting evidence.

Weak answers describe a process. Good answers give a number and name a person.

None of this means anybody is being evasive. Most weak answers come from people who have never been asked for the number before.

"We run a report on that" tells you a report exists, not what it says. "It is under control" is a judgement, not a measure. "The auditors did not raise it" describes a sample at a point in time.

A good answer has three parts: the number, the reason, and who owns it. If any part is missing, the follow-up question writes itself.

Where the answer genuinely is not known, that is useful information too. It usually means nobody has been asked to produce it, which is a fixable problem rather than a concealed one.

Build the four measures into a standing one-page report with the count, the trend and the named owner for each. That converts the monthly conversation from an investigation into a review.

Report by curriculum area rather than in total. A college-level figure hides the one area where the process has broken, which is exactly the thing worth finding.