TEC is supposed to release its educational performance information on Wednesday and I’ll be making some comments on that for ED Blog and ED Insider. Before I do that, I thought I’d use this post to jot down my approach to the figures.

• There is an enormous amount of data available on tertiary education inputs (eg enrolments), but very little on outputs (course or qual completions) or outcomes (did people get a job, make a living, enrol for another qual, etc). TEC’s data release will help a lot with filling in the outputs area, but graduate outcome data will be the next step.
• The data that will be released will have various errors, stemming from two main issues.
  1. The first is that tertiary education organisations (TEOs) have had no good incentive to tidy up their data until now and it will take time to fix things. That is not a reason to delay publication any further as TEOs will continue to delay until data is published – it’s a natural reaction. Exactly the same arguments about poor quality data were raised in 2003 when then-Minister Steve Maharey proposed releasing similar data.
  2. The second is that TEC has produced data using techniques (and sometimes raw data) that TEOs do not understand or agree with. The TEC has, in response, delayed publication several times to fix things, which is good. I suspect, however, that TEC has made things too complicated and will continue to face reasonable criticism over their techniques. TEC will need to improve things before funding is based on their data, but I’m not convinced that the data should be withheld.
• Data on whether students stay in the course (retention) are important because they give us a sense of whether the course material and teaching is engaging and whether students were appropriately assessed before enrolment. If students are bored or overwhelmed they are likely to leave early.
• Data on whether students succeed in the course (course completion) are important. Most people should pass a well-targeted course and low pass rates will usually point to lax entry standards or poor teaching (or an earthquake in some rare cases).
• Data on whether students gain the qualification (qualification completion) are important because the government’s policy is to encourage qualification completion, as that is associated with higher incomes. TEOs will argue that many students enrol just to get a course – that’s true, but the government has been clear for a long time that it does not want to fund or encourage that. If TEOs have been ignoring that position for many years, then it is hard to complain when the government reinforces it through funding shifts. (NB I am supportive of course enrolments being funded outside of wider qualifications, as a segment of overall funding.)
• Data on whether students enrol in a further qualification (progression) are useful, but are less clear-cut than the other measures. The utility of further quals will vary by industry/subject and I suspect this measure will be the least used of the four to be published.
• Every TEO will come up different reasons for why their enrolments are affected by student factors, but they should mostly be disregarded unless they are accompanied with a serious research report. The fact is that every TEO will have a good number of students shifting, getting pregnant, losing a job, going to jail, getting bored or playing X-Box. At an individual course level those factors will skew results, but at an organisational level, the results will be a fair reflection of the enrolment, delivery and student support policies and practices of an organisation.
• There is a reasonable argument that enrolling some population groups may depress completion rates, given those groups’ worse prior academic achievement. But that is a difficult issue, as some TEOs do quite well with groups that otherwise have poor performance – one good thing about the release of stats is that we will  get a clearer picture of who does well and who doesn’t with different groups.
• Publishing performance data may lead people to game the system and artificially increase pass rates, but I think the more likely response that professionals in the tertiary education sector will take is solid work to help improve students’ performance. I expect that the attention to detail will far outweigh the gaming, and that gaming will often (but not always) be picked up at the margins.

Overall, I’m looking forward to this data helping students to choose their courses and redirecting providers’ efforts. Students who enrol and do not stay in the course, do not pass their courses or do not get a qual are a silent group in education. They usually do not get surveyed on their experience (that is left to the graduates) but they are saddled with debt, foregone earnings and quite probably discouraged from further education. I support publication of performance stats for the impact it may have on those students.