research paper of education pdf

Help | Advanced Search

Computer Science > Computation and Language

Title: realm: reference resolution as language modeling.

Abstract: Reference resolution is an important problem, one that is essential to understand and successfully handle context of different kinds. This context includes both previous turns and context that pertains to non-conversational entities, such as entities on the user's screen or those running in the background. While LLMs have been shown to be extremely powerful for a variety of tasks, their use in reference resolution, particularly for non-conversational entities, remains underutilized. This paper demonstrates how LLMs can be used to create an extremely effective system to resolve references of various types, by showing how reference resolution can be converted into a language modeling problem, despite involving forms of entities like those on screen that are not traditionally conducive to being reduced to a text-only modality. We demonstrate large improvements over an existing system with similar functionality across different types of references, with our smallest model obtaining absolute gains of over 5% for on-screen references. We also benchmark against GPT-3.5 and GPT-4, with our smallest model achieving performance comparable to that of GPT-4, and our larger models substantially outperforming it.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

CNN values your feedback

Fear & Greed Index

Latest Market News

Teachers are using AI to grade essays. But some experts are raising ethical concerns

When Diane Gayeski, a professor of strategic communications at Ithaca College, receives an essay from one of her students, she runs part of it through ChatGPT, asking the AI tool to critique and suggest how to improve the work.

“The best way to look at AI for grading is as a teaching assistant or research assistant who might do a first pass … and it does a pretty good job at that,” she told CNN.

She shows her students the feedback from ChatGPT and how the tool rewrote their essay. “I’ll share what I think about their intro, too, and we’ll talk about it,” she said.

Gayeski requires her class of 15 students to do the same: run their draft through ChatGPT to see where they can make improvements.

The emergence of AI is reshaping education, presenting real benefits, such as automating some tasks to free up time for more personalized instruction, but also some big hazards, from issues around accuracy and plagiarism to maintaining integrity.

Both teachers and students are using the new technology. A report by strategy consultant firm Tyton Partners, sponsored by plagiarism detection platform Turnitin, found half of college students used AI tools in Fall 2023. Meanwhile, while fewer faculty members used AI, the percentage grew to 22% of faculty members in the fall of 2023, up from 9% in spring 2023.

Teachers are turning to AI tools and platforms — such as ChatGPT, Writable, Grammarly and EssayGrader — to assist with grading papers, writing feedback, developing lesson plans and creating assignments. They’re also using the burgeoning tools to create quizzes, polls, videos and interactives to up the ante” for what’s expected in the classroom.

Students, on the other hand, are leaning on tools such as ChatGPT and Microsoft CoPilot — which is built into Word, PowerPoint and other products.

But while some schools have formed policies on how students can or can’t use AI for schoolwork, many do not have guidelines for teachers. The practice of using AI for writing feedback or grading assignments also raises ethical considerations. And parents and students who are already spending hundreds of thousands of dollars on tuition may wonder if an endless feedback loop of AI-generated and AI-graded content in college is worth the time and money.

“If teachers use it solely to grade, and the students are using it solely to produce a final product, it’s not going to work,” said Gayeski.

The time and place for AI

How teachers use AI depends on many factors, particularly when it comes to grading, according to Dorothy Leidner, a professor of business ethics at the University of Virginia. If the material being tested in a large class is largely declarative knowledge — so there is a clear right and wrong — then a teacher grading using the AI “might be even superior to human grading,” she told CNN.

AI would allow teachers to grade papers faster and more consistently and avoid fatigue or boredom, she said.

But Leidner noted when it comes to smaller classes or assignments with less definitive answers, grading should remain personalized so teachers can provide more specific feedback and get to know a student’s work, and, therefore, progress over time.

“A teacher should be responsible for grading but can give some responsibility to the AI,” she said.

She suggested teachers use AI to look at certain metrics — such as structure, language use and grammar — and give a numerical score on those figures. But teachers should then grade students’ work themselves when looking for novelty, creativity and depth of insight.

Leslie Layne, who has been teaching ChatGPT best practices in her writing workshop at the University of Lynchburg in Virginia, said she sees the advantages for teachers but also sees drawbacks.

“Using feedback that is not truly from me seems like it is shortchanging that relationship a little,” she said.

She also sees uploading a student’s work to ChatGPT as a “huge ethical consideration” and potentially a breach of their intellectual property. AI tools like ChatGPT use such entries to train their algorithms on everything from patterns of speech to how to make sentences to facts and figures.

Ethics professor Leidner agreed, saying this should particularly be avoided for doctoral dissertations and master’s theses because the student might hope to publish the work.

“It would not be right to upload the material into the AI without making the students aware of this in advance,” she said. “And maybe students should need to provide consent.”

Some teachers are leaning on software called Writable that uses ChatGPT to help grade papers but is “tokenized,” so essays do not include any personal information, and it’s not shared directly with the system.

Teachers upload essays to the platform, which was recently acquired by education company Houghton Mifflin Harcourt, which then provides suggested feedback for students.

Other educators are using platforms such as  Turnitin  that boast plagiarism detection tools to help teachers identify when assignments are written by ChatGPT and other AI. But these types of detection tools are far from foolproof; OpenAI shut down its own AI-detection tool last year due to what the company called a “low rate of accuracy.”

Setting standards

Some schools are actively working on policies for both teachers and students. Alan Reid, a research associate in the Center for Research and Reform in Education (CRRE) at Johns Hopkins University, said he recently spent time working with K-12 educators who use GPT tools to create end-of-quarter personalized comments on report cards.

But like Layne, he acknowledged the technology’s ability to write insightful feedback remains “limited.”

He currently sits on a committee at his college that’s authoring an AI policy for faculty and staff; discussions are ongoing, not just for how teachers use AI in the classroom but how it’s used by educators in general.

He acknowledges schools are having conversations about using generative AI tools to create things like promotion and tenure files, performance reviews, and job postings.”

Nicolas Frank, an associate professor of philosophy at University of Lynchburg, said universities and professors need to be on the same page when it comes to policies but need to stay cautious .

“There is a lot of danger in making policies about AI at this stage,” he said.

He worries it’s still too early to understand how AI will be integrated into everyday life. He is also concerned that some administrators who don’t teach in classrooms may craft policy that misses nuances of instruction.

“That may create a danger of oversimplifying the problems with AI use in grading and instruction,” he said. “Oversimplification is how bad policy is made.”

To start, he said educators can identify clear abuses of AI and begin policy-making around those.

Leidner, meanwhile, said universities can be very high level with their guidance, such as making transparency a priority — so students have a right to know when AI is being used to grade their work — and identifying what types of information should never be uploaded into an AI or asked of an AI.

But she said universities must also be open to “regularly reevaluating as the technology and uses evolve.”

CNN Business Videos

Show all

'.concat(e,"

'.concat(i,"

\n Find more topics that matter to you on your Follow page. Browse, add, or remove topics for a\n personalized experience.\n

",16,{name:"compare",hash:{},data:o,blockParams:r,loc:{start:{line:12,column:78},end:{line:13,column:31}}}),{name:"if",hash:{},fn:l.program(3,o,0,r,t),inverse:l.noop,data:o,blockParams:r,loc:{start:{line:12,column:72},end:{line:13,column:81}}}))?c:"")+'">'+s(i(null!=(c=r[0][0])?u(c,"label"):c,a))+"

To revisit this article, visit My Profile, then View saved stories .

• Backchannel
• Newsletters
• WIRED Insider
• WIRED Consulting

Amanda Hoover

Students Are Likely Writing Millions of Papers With AI

Students have submitted more than 22 million papers that may have used generative AI in the past year, new data released by plagiarism detection company Turnitin shows.

A year ago, Turnitin rolled out an AI writing detection tool that was trained on its trove of papers written by students as well as other AI-generated texts. Since then, more than 200 million papers have been reviewed by the detector, predominantly written by high school and college students. Turnitin found that 11 percent may contain AI-written language in 20 percent of its content, with 3 percent of the total papers reviewed getting flagged for having 80 percent or more AI writing. (Turnitin is owned by Advance, which also owns Condé Nast, publisher of WIRED.) Turnitin says its detector has a false positive rate of less than 1 percent when analyzing full documents.

ChatGPT’s launch was met with knee-jerk fears that the English class essay would die . The chatbot can synthesize information and distill it near-instantly—but that doesn’t mean it always gets it right. Generative AI has been known to hallucinate , creating its own facts and citing academic references that don’t actually exist. Generative AI chatbots have also been caught spitting out biased text on gender and race . Despite those flaws, students have used chatbots for research, organizing ideas, and as a ghostwriter . Traces of chatbots have even been found in peer-reviewed, published academic writing .

Teachers understandably want to hold students accountable for using generative AI without permission or disclosure. But that requires a reliable way to prove AI was used in a given assignment. Instructors have tried at times to find their own solutions to detecting AI in writing, using messy, untested methods to enforce rules , and distressing students. Further complicating the issue, some teachers are even using generative AI in their grading processes.

Detecting the use of gen AI is tricky. It’s not as easy as flagging plagiarism, because generated text is still original text. Plus, there’s nuance to how students use gen AI; some may ask chatbots to write their papers for them in large chunks or in full, while others may use the tools as an aid or a brainstorm partner.

Students also aren't tempted by only ChatGPT and similar large language models. So-called word spinners are another type of AI software that rewrites text, and may make it less obvious to a teacher that work was plagiarized or generated by AI. Turnitin’s AI detector has also been updated to detect word spinners, says Annie Chechitelli, the company’s chief product officer. It can also flag work that was rewritten by services like spell checker Grammarly, which now has its own generative AI tool . As familiar software increasingly adds generative AI components, what students can and can’t use becomes more muddled.

Detection tools themselves have a risk of bias. English language learners may be more likely to set them off; a 2023 study found a 61.3 percent false positive rate when evaluating Test of English as a Foreign Language (TOEFL) exams with seven different AI detectors. The study did not examine Turnitin’s version. The company says it has trained its detector on writing from English language learners as well as native English speakers. A study published in October found that Turnitin was among the most accurate of 16 AI language detectors in a test that had the tool examine undergraduate papers and AI-generated papers.

Lauren Goode

Karen Williams

William Turton

Schools that use Turnitin had access to the AI detection software for a free pilot period, which ended at the start of this year. Chechitelli says a majority of the service’s clients have opted to purchase the AI detection. But the risks of false positives and bias against English learners have led some universities to ditch the tools for now. Montclair State University in New Jersey announced in November that it would pause use of Turnitin’s AI detector. Vanderbilt University and Northwestern University did the same last summer.

“This is hard. I understand why people want a tool,” says Emily Isaacs, executive director of the Office of Faculty Excellence at Montclair State. But Isaacs says the university is concerned about potentially biased results from AI detectors, as well as the fact that the tools can’t provide confirmation the way they can with plagiarism. Plus, Montclair State doesn’t want to put a blanket ban on AI, which will have some place in academia. With time and more trust in the tools, the policies could change. “It’s not a forever decision, it’s a now decision,” Isaacs says.

Chechitelli says the Turnitin tool shouldn’t be the only consideration in passing or failing a student. Instead, it’s a chance for teachers to start conversations with students that touch on all of the nuance in using generative AI. “People don’t really know where that line should be,” she says.

You Might Also Like …

In your inbox: The best and weirdest stories from WIRED’s archive

Jeffrey Epstein’s island visitors exposed by data broker

8 Google employees invented modern AI. Here’s the inside story

The crypto fraud kingpin who almost got away

It's shadow time! How to view the solar eclipse, online and in person

Steven Levy

Kate Knibbs

Will Knight

Matt Burgess

Benj Edwards, Ars Technica