Building a 24/7 Multimodal Personal Agent, Part 2 of 3: Always On and Learning

​This is the second session of a three-part coding series on building a personal agent that listens, sees, remembers and acts, entirely on-device with open models like Gemma 4 running through llama.cpp. Building on Part 1, this session turns the agent from something you call into something that runs continuously and improves over time.

​We cover:

• ​Running a background process with scheduling and triggers

• ​Recovering from crashes using checkpoints

• ​Consolidating raw memories into summaries and facts

• ​Self-evolution, where the agent rewrites its own prompts and memory based on your feedback

​Each component comes with its own evaluation, including held-out tests for self-improvement. Live tasks: an ambient listener that recalls earlier conversations, and a daily brief that adapts to your feedback.