Itz-Amethyst commited on
Commit
b750052
·
unverified ·
1 Parent(s): 1706d7c

feat: add guide for later usecases

Browse files
Files changed (3) hide show
  1. guide.md +80 -0
  2. metadata.jsonl +0 -0
  3. test.ipynb +0 -0
guide.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Gaia Agent Evaluation Guide
2
+
3
+ This guide will walk you through the setup process for running the sample code and evaluating your agent using Gaia results.
4
+
5
+ ## Step 1: Configure API Keys
6
+
7
+ Before anything else, make sure you configure your secret keys in the **Space Settings** section.
8
+
9
+ - Log into each required platform.
10
+ - Locate and input your API keys in the designated fields.
11
+
12
+ ## Step 2: Set Up Supabase
13
+
14
+ 1. **Log in to Supabase**.
15
+ 2. Navigate to your **space**, then go to your **project**.
16
+ 3. Open the **SQL Editor**, paste the SQL code below, and run it to create the necessary table and function.
17
+
18
+ ### 📦 SQL Code – Creating Tables and Functions
19
+
20
+ ```sql
21
+ -- Enable pgvector if not already enabled
22
+ create extension if not exists vector;
23
+
24
+ -- Create the documents table (if not already done)
25
+ create table if not exists documents (
26
+ id bigserial primary key,
27
+ content text,
28
+ metadata jsonb,
29
+ embedding vector(768) -- Make sure this matches your model's embedding dimension
30
+ );
31
+
32
+ -- Create the match_documents function
33
+ create or replace function match_documents (
34
+ query_embedding vector(768),
35
+ match_count int default 5,
36
+ filter jsonb default '{}'
37
+ )
38
+ returns table (
39
+ id bigint,
40
+ content text,
41
+ metadata jsonb,
42
+ similarity float
43
+ )
44
+ language plpgsql
45
+ as $$
46
+ begin
47
+ return query
48
+ select
49
+ id,
50
+ content,
51
+ metadata,
52
+ 1 - (embedding <=> query_embedding) as similarity
53
+ from documents
54
+ where metadata @> filter
55
+ order by embedding <=> query_embedding
56
+ limit match_count;
57
+ end;
58
+ $$;
59
+ ```
60
+ 4. After running the above, execute this command to ensure Supabase’s API layer (PostgREST) refreshes its internal schema cache:
61
+ ```sql
62
+ NOTIFY pgrst, 'reload config';
63
+ ```
64
+ ## Step 3: Populate the Database
65
+
66
+ To enable document retrieval, you need to populate the database with example entries:
67
+
68
+ - Open and run the **test.ipynb** Jupyter notebook.
69
+
70
+ - This script reads from the **metadata.jsonl** file and inserts the examples into the documents table.
71
+
72
+ - This adds a Basic Retrieval capability to your agent, enhancing its performance.
73
+
74
+ ## Step 4: Run the Evaluation
75
+
76
+ Once the database is set up and filled with data:
77
+
78
+ - Proceed to the Evaluation section in your project.
79
+
80
+ - Run the evaluation script to test and score your agent’s performance.
metadata.jsonl ADDED
The diff for this file is too large to render. See raw diff
 
test.ipynb ADDED
The diff for this file is too large to render. See raw diff