Part 7/11:
Optimizing for Cost-Effective Long-Term Memory
One of the project's central needs is long-term, scalable memory. Shapiro discusses techniques such as:
Selecting relevant memories: Using machine learning (e.g., classification models, SVMs) to identify which past conversations are most pertinent.
Data sampling: Handling vast datasets by selecting representative subsets for training relevance models.
Recursive summarization: Continuously compressing conversations into summaries or knowledge graphs to support quick retrieval.
He cautions that indiscriminate retrieval and summarization can waste computational resources, suggesting careful tuning and strategic data management are essential.