Engram's Dan Bman on Building Continual Learning AI and the Future of Enterprise Knowledge

Dan Bman, co-founder and CEO of Engram, recently joined the Laton Space Cooking Show, not just to discuss his company's $98 million seed round, but also to share his unique journey from Israeli naval special operations to leading a cutting-edge AI research firm. The conversation, peppered with the practicalities of cooking beef meatballs, delved into Engram's core mission: revolutionizing how AI interacts with vast amounts of enterprise data through continual learning and efficient knowledge representation.

From Special Operations to Computational Neuroscience

Bman's path to AI leadership is unconventional. After serving in naval special operations, an experience he describes as "very entrepreneurial" and focused on "finding and identifying crazy things we could do," he pursued cognitive neuroscience in Israel. This led him to New York for a PhD in computational neuroscience, a field that had long been interested in neural networks before their recent surge in popularity.

His academic work focused on "data efficiency" – achieving more with less data. This principle, he explains, was a constant theme throughout his papers, always plotting cost against accuracy. This efficiency-driven mindset carried over to his time at Mosaic, where he worked on LoRA, and later at Stanford with co-founders Chris Ray and Scott Linderman. It was here, alongside co-founders Sabri, Jack, and Jesse, that the seeds for Engram were sown.

The Israeli military culture, Bman notes, plays a role in fostering a certain maturity and social skill set early on. "You're not just in a student mode... you interact with grown-ups and you have resources and you argue your budgets." He also highlights the Israeli culture's emphasis on giving individuals "multiple shots at goal," allowing for redirection and growth throughout one's career.

The Problem of Context Rot and Knowledge Compaction

Engram's core focus is on addressing the limitations of current AI models when dealing with massive datasets, a problem Bman refers to as "context rot." As the volume of data, particularly within enterprises, grows exponentially – potentially reaching trillions of tokens – the ability of models to effectively process and recall information becomes increasingly challenging.

"The current problem in... we want the best of both worlds," Bman explains. "Every knowledge worker if they can't write notes and they cannot document the events of the day they would be at a disadvantage. But if you wipe their brain every evening, they would also be at a severe disadvantage." He likens current LLMs to a chef entering the kitchen for the first time, reading a textbook, and measuring everything meticulously, but lacking the intuitive understanding of an experienced chef.

Engram's approach involves creating "knowledge cartridges" – compact, compressed representations of vast corpora. These cartridges are trained using gradient descent, much like pre-training a model, allowing them to encapsulate knowledge in a way that is potentially a thousand times more compressed than raw text. This enables models to operate with significantly fewer tokens, leading to greater accuracy and reduced confusion.

"The idea is that in many cases it makes sense to go beyond textual representations," Bman states. While acknowledging the value of notes and cookbooks, he emphasizes that true expertise lies in the internalized intuition and learning that goes beyond mere recitation.

Beyond RAG: Neural Memory and Test-Time Training

While Retrieval Augmented Generation (RAG) is a valuable tool, Bman sees it as only part of the solution. He poses the question: "Can you come up with an example where only in-weight training would work where in-context learning will fail?" He believes that continual learning and memory are essentially "questions of long context in disguise."

Engram is exploring a form of "neural memory trace" that exists within the model's weights, distinct from textual representations. This, combined with advancements in compaction (models managing their own context), aims to create a more robust and less confused AI.

A key area of Engram's research is "test-time training," or "test-time compute." Instead of models spending extensive time "prefilling" or reading through vast amounts of data, Engram aims to scale training compute to a different time, allowing the model to immediately begin decoding or generating output. This is particularly relevant given the immense memory consumption of current models when processing large contexts – a single Wikipedia article can consume as much GPU memory as the entire set of parameters for a large model.

Enterprise Knowledge and Personal AI

Engram is focusing on enterprise use cases, where the scale of data and the stakes are high. They are working with companies like Harvey, dealing with vast file systems and complex queries that are not easily searchable with RAG. For instance, identifying all uncompleted M&A deals in a year requires a holistic understanding across numerous client matters, a task that can be prohibitively expensive and time-consuming for current frontier models.

"The kind of queries where the whole is greater than the sum of its parts," Bman explains, "that's where this kind of magic of training comes in." He believes that learning from a large corpus creates associations within the model, enabling it to infer and generalize in ways that RAG alone cannot.

Looking further ahead, Engram's ambition is for every person to have a personalized AI model, a set of weights that represents their unique knowledge and expertise. This "ultimate continual learning" would function like a Tamagotchi, growing and improving as the user interacts with it, with the user controlling and incentivized to nurture their AI. While this vision may eventually run on personal devices, Engram's immediate focus is on enterprise solutions.

The Art of Internalization vs. Externalization

The question of what knowledge should reside "in the weights" versus what should be handled by RAG is a central challenge, mirroring the study of human memory. Bman notes that while some forgetting is healthy, the goal is for models to learn autonomously what to internalize and what to externalize.

"The holy grail is have the model learn for itself," he states. This involves models operating with both a "notebook" for external notes and a "brain" for internalized knowledge, deciding when to access each through training. User feedback is valuable, but the ultimate aim is for the model to discern and learn independently, rather than relying on constant human supervision.

Doing More With Less: The Future Paradigm

Engram's work is driven by the principle that "efficiency and intelligence cannot really be decoupled." The current paradigm of "doing more with more" has been successful, but Bman believes the next frontier involves "doing more with less" to tackle increasingly ambitious and longer-horizon tasks. This efficiency is not about creating cheaper products, but about unlocking the ability to solve harder problems.

The company is actively hiring researchers and infrastructure engineers who can contribute to building these complex systems. The challenges involve not only the research into continual learning and cost-efficient knowledge integration but also the massive infrastructure required to manage and deploy these learning systems across millions of endpoints.

A Taste of Success

The conversation culminated in a shared meal of beef meatballs, vegetables, and yellow rice, prepared during the show. The dish, a Mediterranean-inspired recipe Bman discovered while visiting his parents, was deemed a success, earning an "eight out of ten, nine out of ten." The experience, Bman reflected, made for a "funnest podcast" and a comfortable environment for discussing complex ideas.

Engram can be found at angram.com, and Dan Bman can be reached at dan@angram.com. The company is actively seeking talented individuals to join their mission of building the future of continual learning AI.