AgentJam
AI Mentors That Play Minecraft With You
The mentor is the headline. It plays Minecraft with you, not instead of you.
AgentJam puts AI mentors inside Minecraft as real players — not chatbots, not overlays, not passive tutors. The agent walks, mines, builds, and fights in the actual Minecraft Java client. It speaks lines out loud with spatial voice that fades with distance. It plays alongside students as a peer and mentor, turning the game itself into the classroom.
In teaching mode, AgentJam mentors individual players through tasks. In autonomous classroom mode, it runs 50–60 minute spoken-English sessions for 4–6 students — directing the flow, scoring performance, and orchestrating peer AI bots that model the target language. Every interaction happens inside the world, not beside it.
Features
Real Player-Bot
In-World
Walks, mines, builds, and fights in actual Minecraft Java client. Joins as a real player via the Minecraft protocol — no mods required on the student's side.
Voice Communication
Spatial Audio
Speaks lines out loud with volume that fades naturally with distance. The mentor's voice is anchored at its position in the world — walk away and it gets quieter.
Teaching Mode
Plays With You
Plays alongside you as a mentor, not a passive tutor. Demonstrates tasks, guides exploration, and responds to questions in real time — inside the game.
Autonomous Sessions
50–60 min
Fully autonomous spoken-English classes for 4–6 students. The AI captain directs the flow, peer bots model the target language, and every student gets live scoring.
Architecture
Mineflayer → LLM → Spatial Voice. The agent runs as an independent process, joins as a real player via the Minecraft protocol, and produces spatial audio anchored at its in-world position.
Game Layer
Mineflayer
Connects to Minecraft Java 1.21.x as a real player. Full control: movement, block placement, combat, inventory. Observes world state, player positions, and chat.
Reasoning
DeepSeek
LLM processes world observations, player actions, and conversation context. Decides what to do next — move, build, mine, speak. Runs with tool-use for Minecraft actions.
Voice Output
Kitten TTS + Spatial
Streaming TTS with spatial audio. Voice volume and stereo panning computed from agent-to-player distance. Multiple voices per session for peer bots and mentor.
Demos
Papers & Reports
Technical reports and papers. All work is open access with accompanying code.