matt-bcny
/

llama-3b-instruct-ft-function-call

Text Generation

Model card Files Files and versions Community

matt-bcny commited on Mar 17

Commit

fdfb021

·

verified ·

1 Parent(s): ee4f160

Update handler.py

Files changed (1) hide show

handler.py +1 -0

handler.py CHANGED Viewed

@@ -464,6 +464,7 @@ Question:
         input_length = inputs_dict["input_ids"].shape[1]
         generated_text = ""
         if stream:
             # Use streaming for interactive responses
             streamer = TextIteratorStreamer(self.tokenizer, skip_prompt=True, skip_special_tokens=True)

         input_length = inputs_dict["input_ids"].shape[1]
         generated_text = ""
+        stream = False
         if stream:
             # Use streaming for interactive responses
             streamer = TextIteratorStreamer(self.tokenizer, skip_prompt=True, skip_special_tokens=True)