WER (Word Error Rate)¶
API reference for the wer module. For metric and evaluation documentation, see: WER
evaluatio.metrics.wer
¶
Word-level error metrics
This module provides utilities to compute word error rate (WER) and
word-level edit distance between reference and hypothesis text sequences.
All computations operate on whitespace-tokenized words. If you need more
complex tokenizing, please see metrics.uer.
The functions accept any iterable of strings and internally convert them to a format compatible with the underlying native bindings.
Notes
- If a reference string is empty or contains no tokens, the corresponding
WER is defined as
inf. - These functions are thin wrappers around optimized native implementations.
word_error_rate_per_pair ¶
Compute word error rate (WER) for each reference-hypothesis pair.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
references
|
Iterable[str]
|
Iterable of reference strings. |
required |
hypotheses
|
Iterable[str]
|
Iterable of hypothesis strings. Must be the same length as
|
required |
Returns:
| Type | Description |
|---|---|
List[float]
|
Word error rate for each pair of reference and hypothesis. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the lists are of different lengths. |
See Also
metrics.uer.universal_error_rate_per_pair : Type-agnostic version.
Notes
- Tokenization is performed by splitting on whitespace.
- If a reference string is empty or contains no tokens, the resulting
WER is
inf.
word_edit_distance_per_pair ¶
Compute word-level edit distance for each reference-hypothesis pair.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
references
|
Iterable[str]
|
Iterable of reference strings. |
required |
hypotheses
|
Iterable[str]
|
Iterable of hypothesis strings. Must be the same length as
|
required |
Returns:
| Type | Description |
|---|---|
List[int]
|
Word-level edit distance for each pair. |
See Also
metrics.uer.universal_edit_distance_per_pair : Type-agnostic version.
Notes
- Tokenization is performed by splitting on whitespace.
word_error_rate ¶
Compute the corpus level word error rate (WER) over all pairs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
references
|
Iterable[str]
|
Iterable of reference strings. |
required |
hypotheses
|
Iterable[str]
|
Iterable of hypothesis strings. Must be the same length as
|
required |
Returns:
| Type | Description |
|---|---|
float
|
Corpus level word error rate across all pairs. |
Notes
- Tokenization is performed by splitting on whitespace.
- Equivalent to common WER implementations (e.g.,
jiwer-based metrics). - If all reference strings are empty or contains no tokens, the resulting
WER is
inf.
word_error_rate_ci ¶
Estimate a confidence interval for the word error rate via bootstrapping.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
references
|
Iterable[str]
|
Iterable of reference strings. |
required |
hypotheses
|
Iterable[str]
|
Iterable of hypothesis strings. Must be the same length as
|
required |
iterations
|
int
|
Number of bootstrap iterations. |
required |
alpha
|
float
|
Significance level for the confidence interval. |
required |
Returns:
| Type | Description |
|---|---|
ConfidenceInterval
|
Estimated confidence interval for the corpus level word error rate. |
Notes
- The bootstrapped metric corresponds to
word_error_rate. - Tokenization is performed by splitting on whitespace.
- If any reference string is empty or contains no tokens, the resulting
WER can be
inf.