Hey storytellers, quick dev update!
We’re still hard at work — and while the new settings menu is still in beta, there’s a good reason:
🔧 We’ve been busy upgrading the llama-cpp backend DLL to support the latest models.
Why now? Because the new settings system makes it super easy to load your own custom models and we wanted to make sure the latest models work right out of the box.
Maximum compatibility = maximum creative freedom.
✅ This updated backend DLL is now live in the Open Beta branch and ready for testing. Try it out with your favorite GGUF models and let us know how it goes!
P.S. We’ve also included a new optional model file in this beta: LFM2, a brand-new release from Liquid.ai. It’s impressively lightweight at just 1.2GB, making it a great option for those running on 6GB GPUs or even 8GB GPUs with other apps running in the background. It also runs surprisingly fast with CPU based inference! If you’ve struggled with getting 4GB models like Phi-3.5 to run smoothly, LFM2 should offer a fast, responsive alternative that still delivers fairly solid results.
That said, due to its small size and limited context window, LFM2 may occasionally repeat page content or miss instruction cues. It’s only about three weeks old, so there's a good chance the creators are actively improving its instruction-following behavior. In the meantime, it’s still a fun model to experiment with while we continue to investigate these quirks and complete our rollout of full cloud support. Due to it's quirks it is not enabled or used by default, you will need to add it via the settings menu's Text-Generation tab.
Regardless of the model selected for general generation, exit choice generation will continue to use the default Phi-3.5 model for now, thanks to its strong and consistent instruction-following performance.
🌩 Cloud Support News
We’re also making great progress on a modular cloud provider system, which will allow us to plug in different cloud services for those of you who prefer SaaS-based (cloud service) storytelling.
Right now, we have support for:
🦙 Ollama
🧠 Gemini
More are coming over the next two weeks, and we’re planning another Open Beta update soon that adds this cloud support for everyone to try. This modular approach means easier maintenance, better performance, and more choices for you.
Here's a sample video showing the speed you can expect when using Gemini to generate the story page, exits, and image: https://imgur.com/a/9QMEiZD Since all the 'heavy lifting' is done by a cloud provider, it should run this fast on almost any hardware setup.
Thanks again for being part of this journey! Your feedback and support make it all possible. If you haven’t tried the beta branch yet, now’s a great time to jump in and help us shape the next evolution of local (and soon cloud-powered) storytelling.
As always, happy adventuring
Chris