Pixelagent is a reference implementation for a multimodal agent framework to show that an agent class is easy to build and users should be empowered to build their own from scratch for their use cases.
Regarding Memory, to me it's just about Data Storage, Indexing, Orchestration, and Retrieval and I don't know why we should abstract Memory away from users. Memory will mean so many different things for many use cases.
Let's say you want:
- Working memory: Holds current context and immediate interaction history within the agent's context window -> this is just about passing Q&A pairs to maintain context alongside with roles.
- Episodic memory: Stores specific past experiences and interactions -> this is just about indexing past exchanges and having semantic search on it.
- Semantic memory: Organizes specific knowledge in structured formats -> this is just about building a custom logic (udf) to decide how and what to extract insight from and then retrieve it.
Pixelagent is a reference implementation for a multimodal agent framework to show that an agent class is easy to build and users should be empowered to build their own from scratch for their use cases.
Regarding Memory, to me it's just about Data Storage, Indexing, Orchestration, and Retrieval and I don't know why we should abstract Memory away from users. Memory will mean so many different things for many use cases.
Let's say you want:
- Working memory: Holds current context and immediate interaction history within the agent's context window -> this is just about passing Q&A pairs to maintain context alongside with roles.
- Episodic memory: Stores specific past experiences and interactions -> this is just about indexing past exchanges and having semantic search on it.
- Semantic memory: Organizes specific knowledge in structured formats -> this is just about building a custom logic (udf) to decide how and what to extract insight from and then retrieve it.
I've implemented them all in this example: https://github.com/pixeltable/pixelbot