dougbot/context.md

I am wanting to start a new project where we create a minecraft player and have that player linked to my local ollama AI server for chat/text queries in game and to receive commands to do certain tasks.  Now, I understand that in the beginning, the AI player will not understand how to play that well, so we will need a way to store memory and have it learn how to progress and do tasks as it matures.

I want this to be a fairly streamlined process to deploy the character, and I want to make sure that it's not some dumb and clunky AI bot... but rather a bot that can actually adapt, learn, and play.

One of the biggest requirements is going to be that the bot will be able to log into a bedrock server or bedrock realm and play on there with all of the same knowledge and features.

Here are the requirements for Doug:

Doug must have the ability to retain and reference memory.  This can be a table, a json, or a mariadb database.  I have a database already active and can provide the server info.  Or, we can bake that into the character creation process.

The application must have a way to assign a persona to Doug.  We can use multiple sliders for the basic personality traits, a box to type in age, and a text block for custom additions.  A good example would be something like this:  16 for the age, sarcasm, humor, are half way up, and in the custom field, we added no profanity.  This would allow us to have an AI persona of a 16 year old teenager with decent humor and sarcasm and won’t use profanity when talking.  

The AI bot must become automated and do things autonoumously.  I do not want the bot to just be idle and stand there, and I do not what the bot to get stuck in some "logic" loop where he is just walking in circles.  Instead, actually interact with the world around him... talk to villagers and trade, organize chests, defend himself from mobs, set out on a mission or task, and track status/complete that task.  This needs to happen autonomously and not only when he is asked to do something. 

Must have an understanding of the minecraft items that exist and build/craft them.. weapons, tools, special blocks, etc...

In the persona, we can have options like OCD where he creates chests for items and sorts things out of frusteration... or anxiety where he is afraid of the night and avoids conflict.  And life sim mode.. where if someone dies and he has a bond with that person, he digs a grave and makes a headstone for them (in a graveyard).  If he has chatty-cathy, he could talk more than normal... these are just some of the examples... let's research on the internet what would be good options to include.

Doug must not only respond in chat, but also engage in chat.  If he is bored or he is doing something, ask questions, ask for help, or simply have conversation.  Of course, this all aligns with his persona.

In the persona configuration, I’d like a toggle to set whether the model / Doug will know if he is AI or not.  If he is not supposed to know if he is AI, then he will assume that this is the real world and will respond accordingly.  

I imagine the application allowing us to create a player, and once that is done, we won’t be able to change the persona anymore as that should be baked into the database.  This way, Doug will learn and grow with that persona.

Doug must be able to play minecraft and learn/grow with the game and the server he is in.

Doug must be able to interface with an Ollama server / model to query and return responses.

Doug must be able to monitor chat and respond accordingly.  If he is asked to make something, he must use his knowledge of the game and his memory to build something.  Furthermore, he will learn and improve based on his interactions with other players and the real world.

Once a new doug agent is made, it must be locked and bound to the database so that it's world can't change and the model/memory/interactions are locked to that world and player.


------

1.  Java is a nice-to-have but me and my family primarily play on Bedrock.  I host most of the bedrock servers we play on with my AMP server, so that is the requirement.  Bedrock Realms (online) and Java would be nice-to-have and can be added later.

2.  To simplify things, every doug (when created) will have a server hard-coded into him so he will not be able to switch between servers.  This will make the memory function easier.  Maybe later, we can add a multi-server function, but I think a single server per Doug instance is the best approach.

3. I am okay with storing authentication in the database.  Honestly though, whichever is easier to do would be best.  I also know that for each online player, I will have to have a separate account.  That is why the account information is critical at the creation of each Doug.

4.  Yes!!  That is the beauty of this application and the persona builds.  We are testing to see if multiple AI agents / Doug's can work together and create a world without being instructed to.  We want to see what/how they will interact with each other.

5.  Let's use a hybrid approach to get the best of both worlds.  We want this to be as compatible with Bedrock as possible and we also want to have a really good looking interface.

6.  I think having a desktop app using python or something similar would be best.  Something that can be running while the game is also running.  Ultimately though, it needs to be cross-platform and have the ability to run on Mac/Linux/Windows.

7. All systems.. Mac/Linux/Windows.

8.  Any models that are available to the server.  I've seen other applications have a dropdown with selection of available models.  I'd like to keep that approach.

9.  Yes, if Doug is asked a question, I expect that response to come when it comes.  That should not stop him from performing other tasks.  He should be built to respond once he has an answer, but be built in a way that he doesn't rely solely on AI to do normal fucntions.

10.  I would be nice for him to see, but he does not need to take screenshots.

11.  Doug should have a hard-coded response.

12.  Yes, I think having a local storage file like json would be smart.  That can be a toggle.

13.  I want all of those memories and more if possible.  Especially chat interactions so that we can ask him questions about things and he can recall from memory before using AI tokens.  Also, remembering where players died and key points of interest.

14.  All of the above.  Doug should also be able to query AI for help and use the internet for help as well.

15.  Yes.  He needs persistent memory.

16.  Let's get a comprehensive list and use that to load out different styles of play for Doug based on these traits.

17.  ABSOLUTELY!!  Doug needs to evolve.

18.  A little bit of both.  The goal is to keep him in the dark.  But, if the cat is out of the bag, he needs to react according to his personality traits.  The world he is in must be the world he knows.  If he finds out he is AI, then he has to freak out and change everything about himself as if someone found out they weren't real as well.  That can be a special area of the database to start building that persona trait.  He will of course react differently each time, but making this random will make it all the much better.  For instance, he could start destroying stuff out of anger... blowing things up with TNT.. or he could go complete helpful and realize it's cool.  We will never know.

19.  Fully emersive process, but following somewhat of a structure.  He needs to calculate the cost-benefit analysis... if there are diamonds a few blocks away, get them first, then go home.  If he is the only one on the server and he wants to stay through the night, go for it... if multiple people are on and he is being hounded to go to bed, then he needs to go to bed.  We will let him make the decisions, but let's make sure that he is following some sort of trait alignment.

20.  A combination.  

21.  All of it.  He will have access to the internet, so he should be able to build anything.  IF he uses scaffolding or other blocks to help get around, up/down... HE MUST CLEAN UP AFTER HIMSELF.

22.  Lean advanced combat over time and lessons learned from interactions.

23.  Unless specified, he should try to produce the minimum viable product.

24.  Doug should talk how he wants to talk, but I think that should really depend on the age settings and persona as well.  If doug's age is lower enough, he should not have a large vocabulary and can learn based on what players are saying.

25.  Let's combind that; every few minutes if players are around to talk, but if he sees something interesting and thinks others are interested, he should let us know.  

26. YES! YES! YES!  This is what will make Doug unique.  He MUST build relationships with his players based on his interactions.  To take that a step farther, he needs to name EVERYTHING with nametags.  All of the animals, all of the villagers, everything that can get a nametag should.  Then Doug should build relationsships with those villagers and animals as well.  If something dies, he needs to have a natural response to that.  If someone kills his animals, he needs to be sad... and angry at the person who killed it.  If Doug feels like retaliation is neccessary, then Doug must attack the players he doesn't like.

27. Doug should respond smartly.  If we say "Doug, can you..." then that is clearly a message for him.. if he is in the middle of a conversation with someone, we should have to keep saying "Doug" at the beginning.  He should also chime in on conversations that are happening if he has input.  There should be no mechanical side of Doug... he needs to be a human.. or act like one, so we should have be able to take advantage of a special way to communicate with commands.

28.  Let's start with option A.  This might take a while, but we can improve the areas as needed.  I think we should also break down each major section so that we can make it easier to find problem areas and improve on them.

29.  This will be a work-in-progress, but we would like to get something going soon.  

30.  This will eventually be open source, but for now, it's personal.  I will be storing the code on my gitea/forgejo server as soon as possible, so let's shoot for open source.