Spaces:
Running
Running
Kunal Pai
commited on
Commit
·
545222f
1
Parent(s):
0581f43
Add case studies demonstrating HASHIRU's self-improvement capabilities and tool integration
Browse files- paper/conference_101719.tex +13 -0
paper/conference_101719.tex
CHANGED
@@ -193,6 +193,19 @@ To enable HASHIRU agents to learn from past interactions and rectify previous er
|
|
193 |
|
194 |
Memory retrieval is based on semantic similarity between the current context (e.g., task description, recent actions, error messages) and the stored memory entries. We utilize embeddings generated by the \textbf{all-MiniLM-L6-v2} model \cite{wang2020minilmdeepselfattentiondistillation} to represent both the query and the stored memories in a high-dimensional vector space. Relevance is determined by calculating the \textbf{cosine similarity} between the query embedding and each memory embedding. Memories exceeding a predefined similarity threshold are retrieved and provided to the CEO agent (or relevant Employee agents) as contextual information. This allows the system to draw upon past experiences, understand why previous approaches failed, and potentially adjust its strategy to avoid repeating mistakes, thereby improving performance and efficiency over time.
|
195 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
196 |
\section{Experimental Setup}
|
197 |
\label{sec:experiments}
|
198 |
|
|
|
193 |
|
194 |
Memory retrieval is based on semantic similarity between the current context (e.g., task description, recent actions, error messages) and the stored memory entries. We utilize embeddings generated by the \textbf{all-MiniLM-L6-v2} model \cite{wang2020minilmdeepselfattentiondistillation} to represent both the query and the stored memories in a high-dimensional vector space. Relevance is determined by calculating the \textbf{cosine similarity} between the query embedding and each memory embedding. Memories exceeding a predefined similarity threshold are retrieved and provided to the CEO agent (or relevant Employee agents) as contextual information. This allows the system to draw upon past experiences, understand why previous approaches failed, and potentially adjust its strategy to avoid repeating mistakes, thereby improving performance and efficiency over time.
|
195 |
|
196 |
+
\section{Case Studies}
|
197 |
+
\label{sec:casestudies}
|
198 |
+
This section presents two case studies demonstrating HASHIRU's self-improvement capabilities in practical settings. We highlight two instances where HASHIRU enhanced its own architecture and functionality: (1) by generating a comprehensive cost model for base models suitable for specialized agent creation, and (2) by autonomously integrating new tools for the CEO agent.
|
199 |
+
|
200 |
+
\subsection{Case Study 1: Self-Generating the Cost Model for Agent Specialization}
|
201 |
+
\label{sec:casestudy1_costmodel}
|
202 |
+
An accurate cost model is essential for optimizing resource allocation and ensuring the efficiency of specialized agents within HASHIRU. Traditionally, constructing this model involves manual research into local model performance relative to hardware (e.g., 16~GiB VRAM) and the API costs of cloud-hosted alternatives. HASHIRU automated this labor-intensive process by leveraging its web search capabilities to autonomously identify and incorporate the necessary cost data into its internal model. The results were successfully committed to the codebase\footnote{\url{https://github.com/kunpai/HASHIRU/commit/70dc268b121cbd7c50c6691645d8a99912766965}}.
|
203 |
+
|
204 |
+
\subsection{Case Study 2: Autonomous Tool Integration for the CEO Agent}
|
205 |
+
\label{sec:casestudy2_tools}
|
206 |
+
Extending the CEO agent's capabilities through tool integration is vital for broadening HASHIRU's operational scope. Manual tool development typically requires detailed analysis of existing tool schemas and diligent code implementation. HASHIRU streamlined this process by employing a few-shot learning approach, using an existing tool as a template to guide the autonomous creation of new tools~\cite{brown2020language}. The newly generated tools were directly integrated into the codebase\footnote{\url{https://github.com/kunpai/HASHIRU/commit/193e10b2b00917256b7cc01cb3aa5ac7b6a6c174}}.
|
207 |
+
|
208 |
+
|
209 |
\section{Experimental Setup}
|
210 |
\label{sec:experiments}
|
211 |
|