Spaces:

yzhuang
/

MixtureOfInputs

Sleeping

yzhuang commited on May 22

Commit

ce39d2b

verified ·

1 Parent(s): 3f54d28

Update server.py

Files changed (1) hide show

server.py CHANGED Viewed

@@ -22,7 +22,7 @@ def launch_vllm_server(beta=1.0):
         "--enforce-eager",
         "--max-model-len", "2048",
         "--max-seq-len-to-capture", "2048",
-        "--max-num-seqs", "1",
         "--host", "0.0.0.0",
         "--port", "8000",
         "--api-key", "EMPTY"

         "--enforce-eager",
         "--max-model-len", "2048",
         "--max-seq-len-to-capture", "2048",
+        "--max-num-seqs", "36",
         "--host", "0.0.0.0",
         "--port", "8000",
         "--api-key", "EMPTY"