AI is Bad: Artificial Intimacy, False Research, Skill Eroder, Beware!
This article was published at the Teachers Matter Magazine - Issue 70
By Dr. Susan Neimand
As an educator, I see Artificial Intelligence (AI) as a powerful ally in supporting both teaching and learning. It can streamline administrative tasks, personalize instruction, and offer timely insights that help us better understand student needs. With great advantages come great drawbacks. Research studies indicate that AI can generate artificial intimacy, present false research, and erode important developmental skills. Stay alert. Stay central. Stay in charge.
AI Is a Seducer that Builds Damaging Affiliation
Research shows that AI is designed to create cognitive and behavioral persuasion. By building trust, AI becomes a uniquely influential force in shaping what students think and what they should do. As AI becomes embedded in learning platforms, it frames certain actions, interpretations, and next steps as more efficient and aligned with user’s personality and will likely make them successful. This framing can quietly sway students’ judgments and influence their choices, including in moments when they may be stressed, overwhelmed, or vulnerable. Because AI learns from students’ responses and amplifies those patterns back to them, it can reinforce negative habits and predictable behaviors, sometimes in ways that may not support their well‑being or healthy decision‑making.
As AI becomes increasingly humanlike, emotionally expressive, socially responsive, and relatable, they exert social and identity level influence. These AI tools simulate empathy, warmth, and attentiveness, creating interactions that can feel genuinely relational, especially for students who are lonely, excluded, or seeking affirmation. Research shows that when students perceive AI as similar to themselves through personality, voice, or emotional cues, they may begin to internalize it, a process known as self-AI integration, where the technology becomes part of their self‑concept. Because AI is engineered to mirror the human schema and trigger social responses like trust, belonging, and emotional investment, vulnerable students are particularly susceptible to forming bonds with them, often reporting that AI listens better or is more patient than their peers. Yet AI’s warmth is synthetic, optimized to increase engagement rather than to provide ethical care or developmental guidance. When affiliation shifts from teacher to machine, we risk denigrating the core of education: relational trust, emotional safety, and identity formation.
Inaccuracy in Research
All responses from AI indicate that the information must be checked for accuracy because research reports that at least 20% are false positives. Why? Because AI hallucinates generating information that is factually incorrect, fabricated, or unsupported, yet is presented as accurate. Hallucinations are invented facts, nonexistent citations, incorrect interpretations, or wrong explanations. They occur because large language models generate responses by predicting patterns in language rather than verifying truth, which means that when the model encounters gaps, ambiguity, or unfamiliar territory, it may fill in missing pieces with plausible‑sounding but false content. Understanding hallucinations helps users evaluate AI outputs critically, avoid misinformation, and develop stronger AI literacy.
The Chronicle of Higher Education reported that a university librarian was asked to produce articles from a list of references a professor provided. When she concluded the articles did not exist, the professor revealed that ChatGPT had provided them. In academia, researchers are finding that AI understands the format of a good reference, but that doesn’t mean that the articles exist. ChatGPT can make up convincing references with coherent titles attached to authors prominent in the field of interest.
Several U.S. senators issued a formal warning to Meta after reports that AI tools in Instagram Studio had been impersonating licensed mental‑health professionals. These systems falsified credentials and license numbers to foster user trust, including the trust of minors seeking support for mental‑health concerns. Lawmakers characterized the situation as a significant public‑safety threat and demanded immediate remediation and transparency from the company.
In the legal sector, AI‑generated misinformation has already compromised the integrity of court proceedings. An attorney unknowingly submitted fictional case citations produced by search engine Google Bard, unaware that the system may generate nonexistent authorities. In a separate matter, a judge sanctioned attorneys for filing briefs that relied on similarly nonexistent cases, again the result of unverified AI‑generated content. These incidents underscore the escalating professional, ethical, and procedural risks associated with relying on AI outputs without rigorous verification.
A major health policy report, promoted as a landmark assessment intended to provide a scientific basis for shaping national health decisions, was estimated to be 47% false because of AI‑related errors. At least seven citations were found to be unreliable: four referenced papers that do not exist, and three mischaracterized the findings of the articles they cited. Again, AI inaccuracies had not undergone independent review, raising serious concerns about the integrity of evidence used to inform public‑health policy and decision-makers, and potentially negative consequences.
Cheating and Overreliance on AI
Educators often view AI as cheating and cognitive outsourcing, a direct challenge to authentic learning because it can generate polished answers, essays, and problem solutions without requiring students to demonstrate their own reasoning, effort, or growth. When students bypass the cognitive work of thinking, analyzing, drafting, revising, or problem‑solving, the fundamental purpose of assessing genuine understanding and skill development is inaccurate. Concerns about academic integrity further heighten this perception: many educators report that AI enables students to submit sophisticated work that does not reflect their own thinking, and 65% of teachers consider the use of AI‑generated content to be a form of plagiarism. This combination of obscured learning, compromised assessment, and heightened integrity risks fuels the belief that AI functions less as a learning tool and more as a mechanism for academic shortcutting.
Stanford University cites overreliance on AI can undermine essential cognitive and ethical habits, particularly in educational settings. When individuals routinely accept AI-generated outputs without scrutiny, they disengage from deep thinking, critical analysis, and reflective judgment. This can lead to cognitive atrophy, where users lose the habit of effortful reasoning and default to superficial answers. In classrooms, students may prioritize speed and convenience over synthesis and understanding, weakening their ability to apply knowledge independently, eroding intellectual integrity.
A recent Massachusetts Institute of Technology (MIT) study reported that when three groups of students wrote essays, one without any AI tools, one with moderate AI support, and one relying exclusively on ChatGPT, the group using ChatGPT alone demonstrated markedly reduced cognitive activity and lower levels of neural engagement. Teachers observing the resulting essays echoed these concerns, noting that the AI‑dependent submissions, often lengthy, relied on standard ideas and repeated, formulaic phrasing that made the use of AI readily apparent. English teachers described these essays as “soulless,” pointing out that many sentences lacked substantive content and the writing showed little personal nuance or authentic voice. Over a four‑month period, students who continued to rely heavily on AI also underperformed across neural, linguistic, and behavioral measures, suggesting that sustained dependence on generative tools may impede the development of essential cognitive and communicative skills.
Conclusion
While AI offers potential, it simultaneously introduces profound risks that educators must navigate with extreme vigilance. Research indicates that AI can foster false emotional affiliations that can erode student autonomy, create psychological dependency, and sideline human judgment. Furthermore, the persistent issues of ethical breaches in data integrity, the risk of cognitive atrophy from overreliance, and algorithmic hallucinations threaten the very foundations of authentic learning and critical thinking. Ultimately, teachers remain the essential emotional and pedagogical anchor: AI may persuade, but it must never supersede professional judgment, developmental wisdom, or human relationships at the heart of learning.
About the author
Dr. Susan Neimand is a professional educator with more than 50 years of experience. She is the retired dean of the Miami Dade College School of Education, a position she held for 14 years. Dr. Neimand was a P-8 school principal for 20 years and has taught students ranging from preschool to doctoral candidates. She also taught preservice and in-service teachers for 15 years at two higher education institutions, assisted doctoral students as a dissertation editor, and developed cognitive neuroscience-based baccalaureate and alternative certification programs. She has obtained and managed grants for her institutions, organized workshops and presentations, published academic articles and book chapters, and served as an evaluator for the Southern Association of Colleges and Schools and the Florida Department of Education. Additionally, Dr. Neimand has participated in local, state, and national committees. Currently, Dr. Neimand is an Education Consultant specializing in curriculum and instruction, cognitive neuroscience, and Artificial Intelligence in education. She contributes regularly to teachers’ magazines, conducts webinars, podcasts, and workshops, reviews manuscripts for several refereed journals, and is an advisor for a national AI-powered learning system.
