CMU School of Drama


Tuesday, September 09, 2014

Robo-readers, robo-graders: Why students prefer to learn from a machine.

www.slate.com: In April 2012, Mark D. Shermis, then the dean of the College of Education at the University of Akron, made a striking claim: “Automated essay scoring systems” were capable of evaluating student writing just as well as human readers. Shermis’ research, presented at a meeting of the National Council on Measurement in Education, created a sensation in the world of education—among those who see such “robo-graders” as the future of assessment, and those who believe robo-graders are worse than useless.

8 comments:

Unknown said...

Wow, I am seriously impressed. I initially read the article because I thought it might be about the online/computer based classes that are becoming more popular, which I have not had a good experience with. I a sense, I think I might be reversed of most people, in my experience, I abhor taking math or science classes online, because there is no easy way to engage in discussion or ask questions about a different way of understanding something, and I would have been perfectly happy a grammar or language class online. I can completely see how the inhibition and sloth that prevents people from having another person revise their work would disappear if it is done by computer. The School of Drama should subscribe to this service.

Jason Cohen said...

I can't believe that this is true. I would hate to have a robot as a teacher. There is a special connection that is formed between a student and a teacher that I'm not sure that can be accomplished with a robot. For example, I probably would not let a robot read my Hawk vs. Handsaw journal. I feel that they would store my personal writings in their system creating a chance for it then to get out some where. Another reason that I believe that this is not the greatest idea is because they robot would be programed to answer questions in a certain way. However, what happens when the robot can't answer the question that it is asked? That's my two cents.

Sarah Keller said...

Based on the title of the article, I was ready to write a rant about how computers could never help a student's education in the way that the teacher does. However, the actual subject of the article really surprised and fascinated me. Of course it would be ridiculous to replace teachers with grading computers (essays are at their heart a way of measuring how well a student understands a topic or how well they can construct an argument, not how well they can write long sentences with lots of clunky vocabulary). However, using computers to help a student write better (while not claiming to address the actual content of the essay) is a great idea. It makes complete sense that students would be more comfortable with having their work proofread by a computer- it makes me nervous giving my writing to my friend or parents to proofread, let alone having a teacher check it. I don't want someone who's opinion I care about to read less than my best work, even if it would lead to it being better in the future. In addition, editing is an awful task for anyone, and I know many students skip that step entirely, preferring to just hand in a less-than-perfect draft. Simply sending it to a computer, which would spit out some helpful comments and corrections with no judgement attached, sounds quite wonderful. If it was available, I would definitely try this program out.

Unknown said...

Although I can see why having computers grade essays makes some people hesitate, I can see a good case to be made for them. I would argue that on standardized tests like the SAT, having a computer grade all the essays would help level the playing field. Instead of your essay perhaps getting graded differently by a human, all essays are looked at by the same set of eyes. All the grading criteria could be given the exact same set of priorities by the grader every single time.

But yes, in a lot of other situations, I definitely would find it quite upsetting to know my essay had been evaluated by a computer.

Zoe Clayton said...

It is the fear of judgment that makes robots preferable to human teachers and readers. There is no real judgment in a robot or computer, and when there is anything that urks you about a machine's response, you can simply turn it off. You can't do that with a human.

It is inevitable that our children will learn from machines as a main source of education, because we are living in a technologically advanced age. It is inevitable. But we can also put it to good use.

Humans provide excellent feedback in terms of cultivating ideas. So why not have the robots check the work for grammatical mistakes and what not, while humans relay the more meaningful information. After all, robots and machines excel at what they are made on: programming and formulas.

Unknown said...

There's something inherently harrowing about submitting your work to another person, whether it be teacher or peer or someone else entirely. There's a judgement variable that I think everyone fears. But computers - especially for people of my generation and younger - have been with us for so long that they are a comfortable interface. I think using computer graders encourages kids to revise their essays in the same way I prefer to text rather than voice call someone. That said, the consequences of computer are far reaching. While kids might be escaping that "judgement factor", they lose the ability to constructively discuss their work with someone. No matter what you do in life, working with people is inescapable, and I worry that this life-skill accumulation as a side effect of academic learning will be impaired.

Cathy Schwartz said...

And yet we still can’t have computers write a coherent story. This, while it keeps authors in business, is a bit of a shame, as I would love to read a story written by a computer. It would be an interesting perspective. Using computers as proofreaders, and a bit as editors, would actually be a brilliant idea which would then allow the teachers to focus their attentions on the actual content of the essay, rather than getting bogged down with the minutiae of the grammar and spelling. Plus, we already have computers serving as proofreader and editors in the form of programs like spellcheck. It is interesting to look at the psychology of humans’ interactions with computers, especially how people react differently to a computer correcting them than they tend to do to a human. I wonder how much of that also has to do with a lack of a sense of bias. If a computer tells you you’re wrong or could use improvement, than it feels like concrete fact, whereas with another person, it feels like a personal attack, and there is the sense that the person could also be wrong.

K G said...

I am not opposed to the integration of electronics in the classroom, but the replacement of teachers with robots is a bit much. We are already beginning to replace jobs for humans with robots, such as self checkout lines at stores. If we continue this trend, robots will just be doing all of our jobs. Also, there is something to be said about learning to interact with other humans in a classroom setting. Although you will not always have a teacher with authority over you, it is unlikely there will never be anybody in your life who has some authority over you. Learning to deal with authority, however you may deal with it, from a young age, is not only a good thing, but probably a vital thing for paving the way for things such as employer employee relationships in the future.