ekurtic commited on
Commit
be00b3b
·
verified ·
1 Parent(s): 92a6f45

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -1
README.md CHANGED
@@ -120,5 +120,17 @@ guidellm benchmark \
120
  --rate-type sweep \
121
  --max-seconds 600 \
122
  --output-path "Qwen235B-HumanEval.json" \
123
-
 
 
 
 
 
 
 
 
 
 
 
 
124
  </details>
 
120
  --rate-type sweep \
121
  --max-seconds 600 \
122
  --output-path "Qwen235B-HumanEval.json" \
123
+ ```
124
+ GuideLLM interface changed, so for compatibility with the latest version (v0.6.0), please use the following command:
125
+ ```bash
126
+ GUIDELLM__PREFERRED_ROUTE="chat_completions" \
127
+ guidellm benchmark \
128
+ --target "http://localhost:8000/v1" \
129
+ --data "RedHatAI/speculator_benchmarks" \
130
+ --data-args '{"data_files": "HumanEval.jsonl"}' \
131
+ --profile sweep \
132
+ --max-seconds 1800 \
133
+ --output-path "my_output.json" \
134
+ --backend-args '{"extras": {"body": {"temperature":0.6, "top_p":0.95, "top_k":20}}}'
135
+ ```
136
  </details>