It’s just a short little blurb, but report is that starting this year the MCATs are going to be graded automatically using an artificial intelligence system, rather than human graders. The MCAT, like the GRE, moved to a computerized format recently to allow more frequent administration. The essay section, though, was graded in much the same way with multiple readers scoring the writing samples and a degree of consensus being required for the final score. The claim is that the AI system is more consistent than human scorers. I’ve certainly read that human scoring can vary widely. The AI system, IntelliMetric has been used in a variety of settings so far, including a range of domains. I’ve not read through the papers on their site in detail, but it seems to be a basic learning algorithm that works from a corpus of pre-graded writing samples to match essays to be graded to various scoring catagories. I’ll definitely be reading more about this, both for reference for my own research and possibly for inclusion in my spring class. I want to look into what types of errors they are getting compared to the mistakes made by expert raters, if they have that data available….