Happy Sunday! ☀️
The latest update introduced the automatic mode for GPU layers. This is also planned for the chat format, which should make things a lot easier (as promised in the Early Access description).
If you run into issues with automatic mode, let me know, it will likely become the default in later versions. Also I'm curious: How are the new Qwen3.5 models performing in non-English languages? I was positively surprised how they performed in my image interpretation tests.
This devlog is about the memory functionality, which is a bit of a complex topic and the access to it a little hidden, so let's look at how it works.
🧠AI That Remembers
Imagine talking to an AI Pal that remembers details about you. Not just within a single conversation, but across long chats, roleplays or documents. That’s the idea behind the memory system.
The Context Window Problem
If you've used AI and AI Pals Engine, you know the context window: the maximum amount of text a model can process at once.
A few years ago, getting 4096 tokens in a local model was already impressive. Today we have much larger contexts, but they still have limits and roleplay world descriptions, documents and long conversations can exceed them (and of course your hardware).
Instead of trying to fit everything into the context, the memory functionality uses a different approach.
Memory as Search
The system works similarly to the web search tool we look at in this Devlog, but instead of searching the internet, it searches a local memory database.
Example
User question:
"What was the name of the city my character came from again?"
Due to semantic search the system can retrieve relevant entries, even if the original text looked like:
"Elira was born in the coastal city of Vaelthorne."
Why This Matters
The AI Model itself is static and can't add information on the fly. Using the memory approach allows the AI to access information that goes far beyond the model's context window. Without memory systems, long-running interactions eventually lose important details.
Practical benefits:
Conversations stay coherent over time – even very long chats won't lose key information or earlier decisions.
World lore and documents stay accessible – roleplay settings, guides, manuals or research notes can all be referenced during chat.
Remembers important details – characters, story events, personal preferences or ongoing projects can come up again later.
Large documents become usable in chat – instead of pasting them repeatedly, the AI can retrieve relevant sections when needed.
Your own knowledge base – you can build a personal database of information the AI can search and use as a general reference.
Less repetition – you don’t have to keep re-explaining the same context or background information.
✅ Can I Use Memory in AI Pals Engine?
Yes, you can enable Memory in the Char's Config. Your input will then be compared with entries in the database using embedding similarity.
At the moment:
Each Widget uses a shared memory (Char-independent)
The chatview/testview has memory per Char
You can access the memory view in the sidepanel, where you can:
Paste in text
Load documents
Manipulate current memory entries
You can also store conversations that are removed when the context becomes too long, this is configurable in the Char's Config.
When memory is enabled, the {{memory}} token is automatically replaced with actual memory entries.
Example
This Char's memory includes the entire American Bible Standard Edition, which is a little less than one million tokens of text.
For the test we'll add additionally: the secret word is 'Curiosity'
👉 Out of 5578 text chunks, it still is able to retrieve the right information to phrase the correct answer to "What is the secret?"
🔍 How Could We Further Improve the Memory Functionality?
The current system uses similarity-based comparison, which means you might not always get the most relevant entries, just like how web search doesn't always return perfect results. So, there's always room to improve lookup accuracy.
Even more importantly: it would be powerful to build functionality on top of memory entries. Imagine an AI that:
Continuously transforms new memory entries into meaningful insights
Compresses information so you can inject more (or all) information
Learns about your RP history or personal patterns
Analyzes your behavior
Right now, you can experiment with this manually:
Multi-select messages → copy → transform via AI → re-enter into memory
Enable “Auto-load the last conversation” in the widget settings to ensure the conversation grows long enough for messages to be transferred to memory.
❗As with any of the features, Memory will become more integrated over time and easier to use. Let me know if you think it's worth adding an automatic layer, maybe using a fast, lightweight model to handle tasks like summarization or compression automatically!
🌅What's on the Horizon for AI Pals Engine?
Speech-To-Text Wake Word
Better Audio integration
More of the following: Help pages, Internationalization, Fixes, QoL Improvements
📝Changelog V0.8.18.0
Summarized from smaller updates deployed since the last Post:
✨ New Features
Presets - GPU Auto Mode: Auto GPU Layer Selection + MoE Speed Optimizations (when enabled)
Presets: Basic Device Support (GPU selection)
Extended Model Support: Added Qwen35 and other hybrid model support
Char Config: Available Dynamic Tokens Overview
Char Config: ST Card Exporter/Importer
🔧 Improvements
Char Config: Increased text limits
Char Config: Aiprepend/Userappend now parse dynamic tokens
Char Overview: "Load in Widget" now temp-loads and starts a fresh conversation (instead of just selecting it in the widget settings)
TTS: Better handling of non-English-only sentences
Memory: Multiselect + enhanced info & usability
Help: Group Chat and the Memory pages have been improved
🐞 Fixes
Fixed Widget Temporary Char dependency loading
Fixed duplicate key error on context overflow and chat memory enabled