11 Annex 1: AMR Performance Requirements

10.703GPPGSM Adaptive Multi-Rate Speech Codec (AMR)Project schedule and open issues for AMRTS

11.1 Static conditions

This document presents performance requirements and objectives for the speech quality of the GSM AMR system under static test conditions.

The half-rate and full-rate channels will be assessed separately. For each channel the speech quality of the codec modes associated with that channel will be assessed over a range of C/I and background noise conditions to provide a ‘family’ of performance curves.

Separate requirements and objectives are specified for clean speech and background noise. The type of the background noise, e.g. babble or vehicle, and signal-to-noise ratio (SNR) are specified in the associated subjective test plan.

The requirements and objectives for the full-rate and half-rate traffic channels under static test conditions are specified in Table 1. The following notes apply:

Note 1: ‘Ideal case performance’ assumes that optimum mode selection has occurred for the given channel, and is defined as the performance of the codec mode which provides the best subjective performance under the specified C/I and background noise condition.

Note 2: ‘Worst case performance’ assumes that the mode selection algorithm has made the poorest choice for the given channel, and is defined as the performance of the codec mode which provides the worst subjective performance under the specified C/I and background noise condition.

Note 3: Unless otherwise stated, the requirement shall be assumed to be ‘not worse than’, interpreted statistically at the 95% confidence level. In the case where two reference codecs are specified, the candidate codec must meet the stated requirement for both reference codecs.

Full-Rate Channel

Half-Rate Channel

C/I

Ideal case performance (requirement)

Worst case performance (objective)

Ideal case performance (requirement)

Worst case performance (objective)

No errors

EFR no errors

G.728 no errors

G.728 no errors

FR no errors

19 dB

EFR no errors

G.728 no errors

G.728 no errors

FR no errors

16 dB

EFR no errors

G.728 no errors

G.728 no errors

FR at 10 dB

13 dB

EFR no errors

G.728 no errors

FR at 13 dB

FR at 7 dB

10 dB

G.728 no errors

EFR at 10 dB

FR at 10 dB

FR at 4 dB

7 dB

G.728 no errors

EFR at 7 dB

FR at 7 dB

4 dB

EFR at 10 dB

EFR at 4 dB

FR at 4 dB

Table 1a: Clean speech requirements and objectives under static test conditions.

Full-Rate Channel

Half-Rate Channel

C/I

Ideal case performance (requirement)

Worst case performance (objective)

Ideal case performance (requirement)

Worst case performance (objective)

No errors

EFR no errors

G.729 and FR

no errors

EFR no errors

G.729 and FR

no errors

19 dB

EFR no errors

G.729 and FR

no errors

G.729 and FR

no errors

G.729 and FR

no errors

16 dB

EFR no errors

G.729 and FR

no errors

G.729 and FR

no errors

FR at 10 dB

13 dB

EFR no errors

G.729 and FR

no errors

FR at 13 dB

FR at 7 dB

10 dB

G.729 and FR

no errors

FR at 10 dB

FR at 10 dB

FR at 4 dB

7 dB

G.729 and FR

no errors

FR at 7 dB

FR at 7 dB

4 dB

FR at 10 dB

FR at 4 dB

FR at 4 dB

Table 1b: Background noise requirements and objectives under static test conditions:

11.2 Dynamic conditions

Full-Rate Channel

Requirement

Same or better than the EFR under the same conditions, and also the same or better than all the AMR full rate tested modes under the same conditions

Objective 1

Same or better than the EFR using the error pattern + 3 dB

Objective 2

Same or better than the EFR using the error pattern + 6 dB

Table 2a: Requirements and objectives under dynamic test conditions for the full-rate channel

Half-Rate Channel

Requirement

Same or better than the FR under the same conditions, and also the same or better than all the AMR half rate tested modes under the same conditions

Objective 1

Same or better than the FR on a full rate channel using the error pattern + 3 dB

Objective 2

Same or better than the FR on a full rate channel using the error pattern + 6 dB

Table 2b: Requirements and objectives under dynamic test conditions for the half-rate channel

Note 1: It is not yet agreed, whether mixed FR/HR case is needed in the qualification phase. Should it be needed, requirements are left for further study.

Note 2: The above requirements are for clean conditions. AMR#7 agreed that dynamic performance requirements need not be set for background noise case since background noise performance will be already assessed in the static tests in qualification tests. However, it is noted that to cover codec algorithms where mode adaptation is based on the background noise content of the source signal, it could be useful to define requirements also for dynamic conditions under background noise.

Note 3: 4-6 error patterns are needed for each channel. In order to initialize the system, a [5]-second preamble will be used. Each profile having duration of 2 minutes.

11.3 Additional speech codec performance requirements and objectives

In addition to the static and dynamic performance requirements and objectives, there are a number of other performance criteria which must be specified for the AMR speech codec. The reference speech codecs for the performance under tandeming and talker, level and language dependency are specified in Table 3. The performance requirements and objectives for DTMF, information tones and idle noise are specified in Table 4.

Tandeming performance and level dependency will be evaluated in the qualification phase. It is anticipated that the other additional requirements will be evaluated in the characterization phase.

Full-Rate Channel

Half-Rate Channel

Condition

Reference for highest bit-rate

(requirement)

Reference for lowest bit-rate

(objective)

Reference for highest bit-rate

(requirement)

Reference for lowest bit-rate

(objective)

Tandeming

EFR

G.729 and FR

G.729 and FR

G.729 and FR

Talker dependency

EFR

G.729 and FR

G.729 and FR

G.729 and FR

Level dependency

EFR

G.729 and FR

G.729 and FR

G.729 and FR

Language dependency

EFR

G.729 and FR

G.729 and FR

G.729 and FR

Table 3: Reference codecs for additional speech signal performance requirements.

The following notes apply to Table 3:

Note 1: The bit-rates tested will be those normally used within the given channel, i.e. excluding ‘special’ modes such as TFO. In the case of a discrepancy, ‘highest bit-rate’ shall be defined as the bit rate that gives the best performance for the given condition, ‘lowest bit-rate’ refers to the bit-rate that demonstrates the worst performance, i.e. all bit-rates will be tested for the given channel.

Note 2: Unless otherwise stated the performance criterion shall be assumed to be ‘not worse than’. In the case where two reference codecs are specified, the candidate codec must meet the stated requirement for both reference codecs. The method of comparison will be defined in the appropriate test plan.

Note 3: In the tandeming case, each AMR codec mode will be tested in tandem with itself (i.e. both codecs in the tandem chain will operate at the same rate).

Note 4: The level dependency shall be measured over -16dBov (-10dBmO), -26dBov and -36dBov.

Note 5: All requirements are defined under error free conditions.

Condition

Requirement

Objective

DTMF

Transparent transmission of DTMF.

Information tones

Recognizable as given information tone.

Output noise with idle input.

No annoying artifacts.

Table 4: Requirements and objectives for speech codec performance with non-speech inputs.