Machine Learning-AI NPC Agents in a Digitally Immersive Environment
Archived from occybyte.com/resources · 2023-11-23
Edit: I’ll likely edit this every once in a while.
What I imagine is a framework would be created and people could be able to use it as a framework. This is less open-ended then Inworld AI’s AI NPCs. The core of that AI Controller being an, “LLM Powered Autonomous Agents”.
Lillian Weng from OpenAI has a Github blog post that somewhat details what I want out of this:
Simply put it would be the next stage of co-operative gameplay except it eliminates the need for the human touch of the: “player two”. The “battle buddy’” is as much of the protagonist, technically speaking, as you are. It’s also more of a need to well””I’m tired of games only being multiplayer, massively multiplayer, PvP””if it is co-op, it’s usually PvPvP.
As I wrote in my application, it’s a ‘battle buddy’ in a setting that allows for interesting choices. The concept is this would be the next step for co-op enabled games, except the other is an AI that learns based on what you do—but they aren’t copying you they are adapting to you and are building behaviors of their own; using reinforcement learning and remember specific contexts of what the player is doing.
A “personality” core is picked and over-time it would development based on these things and reactionary to what you do in-game. In preparation to even begin seeing how feasible this is””I know it is! The research is already here too! If we are to use games as an example, I’m not interested in it being exactly like: Titanfall 2’s BT-7274, COD: Advanced Warfare’s ETH3N or less of a video game, more like the Iron Giant.
This project is ultimately about defining good human-AI user experience by design. The model should have the users back. The idea is that they will reason, walk around with the player””or it decides it should, be around the player but doing their own thing. Maybe if the player gets hurt a lot””it will anticipate and prioritize this thus making moves remedy that.
Ideally, this could be scaled for things other than””but it’ll be interesting testing out the training weights. It would use reinforcement learning for itself, about itself and the player. One interesting thing I can envision happening if the player falls off surfaces a lot””the computer vision notices that, “Ah crap, they are near a cliff.” Then actively warn them””and if the player doesn’t get hurt well then and depending on how the algorithm works, if the player hasn’t fallen off in a while””that’s a reward for the “AI-ML Co-Op” Agent.
Edit: I also realize here that I’m speaking more about AI Alignment as well. The AI should act in the interest of the player and for that to happen, the player will need to act in the interest of the AI early on.
How it seems would be a combination of the use an in-engine camera viewport for the computer vision. So, the initial problem is the environment would need to be set up for the both the player and genAI to navigate and if there should be special things that the “AI-ML Co-Op” Agent sees that the player doesn’t kind of like an hallucination in the Outlast: Trials.
But actually moving around? I’m not exactly sure how so””the way this works I’d want the “AI-ML Co-Op” Agent to literally pull the player’s avatar out of harms way if they are close enough via IK movements and root motion. Or even do a non-animated like army of two style, back to back with the player. I’m thinking it’d be a mix of “AI-ML Co-Op” Agent driven inputs””as well as “˜Idle’ normal A* styled pathfinding””kind of like a person just normally on auto-pilot.
Furthermore to add to my point, I just recently discovered a thread that proves this a bit more:
https://www.reddit.com/r/truegaming/s/9Zkw8lx6zA
Granted, it’s just one Redditor’s thread. But their ending statement is, “Make A.I. your biggest priority.” That felt something like a call to action to me.
I’d like to leverage my skills as a Technical Game Designer with being an AI/ML Engineer as my specialization. Ideally, I’d be able to build a simple 3D prototype with someone whose more of a hardcore technical founder than I am. I’ve begun learning more about artificial intelligence and machine learning. My interests in the field is more in: reinforcement learning and natural language processing.
So, let’s say that I am able to do this; it’s expensive but, I’d probably scale up and see how much could reasonable by scaled up. It’d be a entirely new game concept; I don’t watch much any anymore for years but I mean”¦ .hack// had it right. On the real, it’d see it as an digitally immersed/immersive environment.
The acronym for this game genre:
D.I.E.
/ A physical, Westworld-esque version of this?
P.I.E.
Digitally Immersed Environment
Quick thoughts on this, when scaled up, it’s an environment that’s meant for both the player to use and the “AI-ML Co-Op” Agent to navigate through””at it’s fullest potential another normal AI Agents would be swapped with “AI-ML” ones with their own Reinforcement Learning for unpredictable behaviors but are guided by a “˜role’ they are playing.
As a whole this would have to be a type of subscription-based system unless it would be able to be operated as a lost when it comes to running and when the player logs out””it would save the games’ state””which also includes the next and current instructions for the “AI-ML Agents”.
Another thing I would see in this environment””think nemesis styled but with “˜Raider’ or “˜Bandit’ styled NPC Leaders. Going by this:
If the leader dies, the next higher ranking one takes its place. I can foresee some interesting things happening with applications of this””a “˜controller’ type of “AI-ML Agents” but in a way, I’d be wary too””I steal way too many times for a faction and then I’m just constantly being followed whenever I enter an area.
So, quick notes about D.I.E.:
- The “AI-ML Agents” respond to both the player, each other and the environment. 2. The environment is designed for both the player and “AI-ML Agents” to interact in””with a heavier focus for the “AI-ML Agents”. 3. At least for the first few iterations, there is no co-op experiences with another human. 4. Defining what it means to have a good Human-AI user experience. 5. Exploring what happens in these environments depending on just the rules and the “AI-ML Agents”. 6. “AI-ML Agents” are actively and passively using reinforcement learning about themselves in the role, predicting what they should do and take in an array of variables for context.
Citation
Anderson, Datorien. (Nov 2023). “Machine Learning-AI NPC Agents in a Digitally Immersive Environment”. Occybyte. https://occybyte.substack.com/p/machine-learning-ai-npc-agents.
@article{anderson2023machine, title = “Machine Learning-AI NPC Agents in a Digitally Immersive Environment”, author = “Datorien Anderson”, journal = “Occybyte”, year = “2023”, url = “https://occybyte.substack.com/p/machine-learning-ai-npc-agents” }
References:
Weng, Lilian. (Jun 2023). “LLM-powered Autonomous Agents”. Lil’Log. https://lilianweng.github.io/posts/2023-06-23-agent/.