When ChatGPT first dropped I was saying that one of the best uses of this technology would be in NPC dialogue/ control.
Imagine a game with NPCs who could actually dynamically respond in character.
Of course, if you're still using a paid model to achieve this effect as a developer you're playing checkers while everyone else is playing chess, but still.
Unfortunately I think the technology hasn't gotten that far. Especially if you have long conversations, you'll definitely start to see issues where things fall out of the context bin, instructions get deprioritized or over-prioritized, or the original prompt gets lost.
Even relying on more modern tricks like leveraging a persistent external store for information eventually seems to grow the information beyond a model's ability to handle well, which makes sense. Conversations can traverse a pretty infinite space.
I think it could work okay for very limited interactions. I think it'll be super disappointing for art.
I think it could be interesting to treat AI NPC’s context window as their memory. It makes sense that characters forget things you say to them after a while, so the game could be designed to make the context window less game breaking or immersion breaking
It's not a terrible idea, but I think what's likely is that the npc is supposed to know things like "hide that you're a spy", and suddenly they forget that they're supposed to hide it, but their historical prompts are full of messages about "You're a spy, but you didn't mention it."
Or "You know about X thing happening because you're a spy, but you ask about it anyways."
That's a real problem in AI and it was solved pretty early on in the piece.
When people say "context" what they mean is prompt + conversation history + user's latest message, fed as one big "model input" block.
Every single time you run the model it only "remembers" what you feed it, and the conversation history is the part that gets compacted. The model always has the full accurate prompt + message, so the NPC will never forget it is a spy and that it wants to try hide that from you. With respect to an actual currently-operating model there is no "historical prompt", there's only what it currently has been fed for this run.
REAL iteration on this front would involve doing some super smart stuff with progress gates and quest flags and stuff that modify the prompt component of the context so that if you discover they're a spy, their prompt changes to say "you're a spy, but you were uncovered (X ingame days ago) when the player accused you with the evidence they found (at Y location)" or something.
Janitor is a website with a script setup that I think could be conceptually adapted to something like that. And there are chatbot roleplay platforms (not j ai) that have chats that go on for pretty long durations. Small details get missed, but I feel like it could work. You'd have to figure out the computational power, though. Would have to be a subscription model or something.
You say that, but that also definitely both runs counter to my experience, and runs counter to (e.g.) jailbreaking models.
I don't doubt the model gets the full context. I have doubts that it's able to execute on it in a way that would be deemed successful for moderately long conversations
It's because you don't explicitly re-state the original prompt every time when you chat with AI. When design your LLM API integration in your app, you can append your original prompt such as "you are a spy" in addition to the chat history to the new prompt every turn. This is something you as a user don't see
The "breaks down on long conversations" problem has two vectors.
The first vector is the prompt being compacted/dropping off. The solution: ensure the prompt is preserved fresh and complete on every input. Very easy to do.
The second vector is if the size of the historical context is so great that the model starts to fail to weigh the prompt and the latest message correctly. This is the part where the model breaks down entirely - not just in terms of failing to keep the secret prompt in mind, but also in terms of just outright failing to respond to what the latest input actually says because all of its attention is on history.
The second vector has two solutions. Model training is the first one. Newer, larger, and particularly closed frontier models tend to have leaps and strides in training specifically to rigidly adhere to system prompt and to stay on task for the latest user input, that's just a huge part of the underlying training. So you make sure your model is up to the task.
The second is that you benchmark the total context window/ratio of history to prompt+message where things start to break down.. and you limit the context window before that and keep compacting history more and more, progressively, as it approaches that limit. Yes, it will forget details about interactions long ago. You will too; that's a very human thing. The trick, like I said, is saving the actual important persistent changes via game flags affecting the prompt, which is never compacted or left off. Exactly like a human: long term recollection (important stuff committed to memory) is good for important details, short term memory is pretty good for acting on what just happened, and recent memory slowly degrades until you can't remember what you had for lunch two weeks ago.
It's 100% possible. The reason we haven't seen it yet is because it's still financially inadvisable for a game developer to actually try it... But as we can see, it's starting.
Sure, I can believe both. Which does handle creating contradictions over long conversations. _Butttttt_ I really am still pretty unconfident that it'll create a particularly great experience. I am deeply aware human writers can be pretty terrible, but my experience with LLMs is that I can expect them to be better than a person off the streets, but I have generally never found them to be exceptional at things. They're worse than what I can do in my fields of speciality, but better than what I can do in arbitrary other fields.
I think they're going to appeal to different kinds of players, personally. Some are going to feel like the character in the game is really responding to them, personally, and that customised experience is going to be more meaningful to them than one that is more polished.
Others are going to respond more positively to outright better, human, creatively written dialogue.
When it comes to AAA games especially we're not really comparing this to your average Joe writing dialogue, we're comparing this to industry professional writers who are usually very good at what they do. They're going to gap any kind of model efficient enough to do this kind of thing at scale.
But, yeah. Some people are definitely going to not even notice that gap if they're just talking to a character and that character is just responding to them in a personal way.
As for me... I'm not entirely unskeptical, I just believe it can absolutely technologically be done with the tech we have today, it's more down to how it's done and whether it makes any financial sense to do.
As long as the game runs a custom model, they can control the context tokens to always keep crucial information like that.
I do agree with you that with today’s technology it would likely cause unintentional game breaking situations. But it’s theoretically possible to have an immersive game like this
On the scale of how large games are, a couple gigs for a decent LLM don't really match up to the cost of high-resolution images, music, and 3d models... at least, not if you expect it to be able to run in real time for the average user who doesn't have a super high-powered PC.
It would probably work best for something always-online with a subscription fee that can help cover the ongoing costs of tokens. MMO-style, host your own models and serve them to the player, so you're not bottlenecked by their computer's power
But the model won't need to be just for the game, in the same way that every unity game requires you to have .Net runtime, but you don't have an instance of the runtime for every unity game you have installed.
Things like Player2 already exist and modders are using them to retrofit it into games. In my experience it's still pretty janky, but it's pretty clear it's just a matter of maturity
Yeah I had a discord bot that was programmed to respond as a certain character and bound by that, it was fun and worked really well. It kept going offline because of the free tier limits lol
Skyrim with ai powered npc that can answer any questions about current game events and thungs they should know by their character. Interact in character with the player is definitely the future.
Skyrim actually has that already and has done since it came out, it's just a somewhat limited model with a very small context where the only thing they know by their character is that they used to be an adventurer, and the only game event they know is that they were hit in the knee by an arrow.
There’s a few of these out already and they’re generally pretty neat, but basically all tech demo type stuff. It’s difficult to design a real game around “promp user to talk to a chatbot and hope they have fun” all you can really do is theme it well and add setdressing and hope the gimmick carries enough to make it worth the money. All of them are basically just “roleplay improv but you aren’t talking to a real person”
Having a real rpg game where you connect an AI to each npc is still simply too much to ask for, until we are at the point where every players computer can run the AI model locally. Even then it’s still overkill. Will probably happen one day though
Mark my words by the time we get GTA7 or 8 we will see at least some side quests entirely generated by AI. Possibly entire side storylines. NPCs, dialog, events, stories, etc.
I can't see them doing a main storyline like this since you do want a carefully crafted narrative there. But someone else will make such a game I am sure.
98
u/domthebomb2 1d ago
When ChatGPT first dropped I was saying that one of the best uses of this technology would be in NPC dialogue/ control.
Imagine a game with NPCs who could actually dynamically respond in character.
Of course, if you're still using a paid model to achieve this effect as a developer you're playing checkers while everyone else is playing chess, but still.