Pin model weights, tokenizer and chat template before comparing local servers
A model label is not a reproducible configuration. Keep a small artifact record so a server upgrade does not become an accidental model comparison.
Two local servers can answer to the same model name while using different files or formatting the conversation differently. Before comparing their speed or answers, record what each one actually loads.
Start with three identities: the weights, the tokenizer and the chat template. Add the engine build and its settings beside them. This takes more detail than a model name, but it makes a later difference easier to investigate.
Use a repository commit, not just a moving branch
Hugging Face’s download documentation describes selecting a repository revision and notes that downloads otherwise use the latest main revision. For a comparison, retain the full resolved commit identifier and the filenames used. If you use a repackaged or quantized checkpoint, identify that repository too.
A revision identifies repository content. It does not prove that your process loaded those files. Keep the server’s startup record and an artifact manifest alongside it. For local copies, record file checksums when you can verify them; a filename and file size alone are weaker evidence.
Do not replace a missing identity with a guess. Mark it unknown and resolve it before describing two runs as the same model configuration.
Keep the tokenizer and conversation format in the record
A chat template turns role-labelled messages into the token sequence a model sees. Hugging Face’s chat-template guide explains that models derived from the same base can use different conversation formats.
Record the tokenizer source and revision separately when they differ from the weights. Keep the exact template file or its checksum, including an explicit override. Record formatting options used by the application as well. Sending the same visible message text does not establish identical model input.
The vLLM 0.32.0 model configuration source has separate model and tokenizer revision handling. Check the behavior of the build you actually run rather than assume one pinned field covers every external repository.
Treat the API model name as a label
The vLLM serving reference permits API-facing model names. A familiar response label therefore cannot replace the artifact record above. This is a configuration check, not evidence of a faulty server.
Keep precision, quantization, context limit, parallelism, cache settings, sampling and output allowance with the engine version. Change one intended factor at a time. A container upgrade can change several packaged components even when the requested model stays fixed.
Make one comparison record before generating results
For each arm, retain:
- Artifact repository, full revision and local file checksums where verified.
- Tokenizer revision, template identity and any formatting overrides.
- Engine/container identity and the complete effective settings.
- Exact input fixture, measurement boundaries and error policy.
- Startup evidence showing the intended configuration loaded.
If an answer changes, first compare these records. Matching them does not guarantee identical output; it removes several avoidable ambiguities. Only then is a measured difference useful evidence about the factor you intended to test.
No model was downloaded, server changed or inference run for this guide. It is a reproducibility checklist, not a LocalXPU compatibility or performance result.