Update README.md
Browse files
README.md
CHANGED
|
@@ -120,5 +120,17 @@ guidellm benchmark \
|
|
| 120 |
--rate-type sweep \
|
| 121 |
--max-seconds 600 \
|
| 122 |
--output-path "Qwen235B-HumanEval.json" \
|
| 123 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
</details>
|
|
|
|
| 120 |
--rate-type sweep \
|
| 121 |
--max-seconds 600 \
|
| 122 |
--output-path "Qwen235B-HumanEval.json" \
|
| 123 |
+
```
|
| 124 |
+
GuideLLM interface changed, so for compatibility with the latest version (v0.6.0), please use the following command:
|
| 125 |
+
```bash
|
| 126 |
+
GUIDELLM__PREFERRED_ROUTE="chat_completions" \
|
| 127 |
+
guidellm benchmark \
|
| 128 |
+
--target "http://localhost:8000/v1" \
|
| 129 |
+
--data "RedHatAI/speculator_benchmarks" \
|
| 130 |
+
--data-args '{"data_files": "HumanEval.jsonl"}' \
|
| 131 |
+
--profile sweep \
|
| 132 |
+
--max-seconds 1800 \
|
| 133 |
+
--output-path "my_output.json" \
|
| 134 |
+
--backend-args '{"extras": {"body": {"temperature":0.6, "top_p":0.95, "top_k":20}}}'
|
| 135 |
+
```
|
| 136 |
</details>
|