Students taking the STAAR exam throughout Texas take a written exam. Answers are automatically scored By artificial intelligence. This development was brought about by some people concerned about the potential of AI. have a major impact on education In the near future.
This exam, offered by the Texas Education Agency, uses an “automated scoring engine” to score open-ended questions on the Texas Academic Assessment in Writing, Reading, Science, and Social Studies. This move is a step up from simply using machines to grade multiple-choice answers or Scantron sheets.
The state appears poised to save $15 million to $20 million a year by significantly reducing the number of humans who have to score tests.plan of Plans to hire just 2,000 graders by 2024This represents a significant reduction from the 6,000 human graders that will be hired in 2023.
The technology utilized to score long-form answers uses natural language processing. This is just one of the artificial intelligence components that popular chatbots use. GPT-4.
Blaze News reached out to John Simmons, professor of philosophy and director of the Center for Cybersocial Dynamics at the University of Kansas. He seemed optimistic about AI’s ability to effectively score STAAR exams without major problems.
“We’re basically looking for a threshold, how can kids reach that threshold? And I think, you know, the modern LLM.” [large language models] You will be able to catch most of these cases and handle them without any problems. ”
Since the STAAR exam was restructured in 2023, there are fewer multiple-choice questions and more free-response questions. These open-ended questions are also known as structured response items.
“We wanted to keep as many open-ended responses as possible, but grading them is incredibly time-consuming,” said Jose Rios, director of student assessment for the Texas Education Agency.
Before AI technology could accurately grade student exams, the Texas Education Agency needed to develop a scoring system. The scoring system was developed by the agency, which collected 3,000 responses and was scored twice by humans.
After a human scores an answer, an automated scoring engine can now learn certain characteristics of the answer and assign the same score that a human would give.
When asked about the potential for AI to make mistakes, Simmons said the technology being used is not advanced enough to identify the answer in the same way a human would find a particular shade on a color wheel. I said that there is. Between red and blue on the color wheel, there are dozens, even hundreds, of shades of blue and red.
“So if we think about something like a color wheel, and we think about the position of a color within the color wheel, we can say that this particular shade of red has a particular coordinate within this color wheel, right? “It’s a number,” Simmons said.
“Now imagine making it three-dimensional. So not only the color, but also the tint and brightness of the color. You can also add other dimensions, such as the warmth of the color. The end result is a multidimensional color space.”
“And what are these [AI] What a system can do is use concepts to do that. ”
Despite the advanced nature of AI, it is still possible for the assigned score to produce an “unreliable” response. If this happens, your score will be double-checked by a human. The same is true if the AI answers a question that the programming doesn’t recognize, such as an answer that uses slang or words that don’t exist in English.
Chris Roznick, director of assessment development for the Texas Education Agency, said the agency has “always had a very robust human quality control process,” and the same goes for technology that replaces humans in the grading process. Stated.
In addition to humans assisting AI in grading, a random sampling of student responses is automatically pushed to humans to double-check whether the technology is on the right track.
Mike Wacker, a software engineer and technologist, told Blaze News that AI should not suddenly “replace humans…unless there is evidence that AI will perform as well as humans. “You can’t trust an evaluation just because it functions as well as a human.”
However, the Texas Education Agency has pushed back against the idea of using AI to grade exams. The agency acknowledged that although automated scoring machines use technology similar to GPT-4 and Google’s Gemini, humans will still maintain significant oversight over the process.
“We’re a long way from being autonomous and being able to think for ourselves,” Roznick said.
Although the agency claims the technology is a reliable grading mechanism, educators across the state have expressed pessimism about the potential for machines to grade children’s work.
“There’s some consensus as to whether this is a good thing or a bad thing, fair or unfair,” said Kevin Brown, executive director of the Texas Association of School Administrators. It should be done,” he said. ”
Carrie Griffiths, a policy expert with the Texas Teachers Association, seemed to echo Brown’s sentiments, saying, “There’s a sense that everything is not about what happens to students, schools, and teachers; There’s always that.”
STAAR exam results are a vital accountability system used by the Texas Education Agency to determine academic proficiency at school districts and individual campuses across the state.
Each school district and individual campus is rated on the AF scale. If a district or individual campus is found to be underperforming on a test, the Texas State Board of Education is obligated to intervene.
The stakes are high for campuses and school districts across the state, which is why some are nervous about putting student testing in the hands of advanced technology.
It remains to be seen how advanced technology will handle scoring state-mandated exams and the unintended consequences they bring.
Do you like Blaze News? Avoid censorship and sign up for our newsletter to get articles like this delivered straight to your inbox. Please register here!
