TR-398

dc:ef:09:e3:b8:7d  dc:ef:09:e3:b8:7b  

Objective

The TR-398 WiFi Performance test plan by the Broadband forum provides a comprehensive set of tests to qualify the performance of WiFi access points (APs) designed for residential and small office environments. Radio performance, Throughput, Connection Stability, Airtime Fairness, AP Co-existence, Mu_MIMO Performance, Spatial Consistency and Long-term Stability are some of the test areas covered in this test plan. The test plan is designed for service providers deploying in home WiFi APs to qualify the APs in the lab before deployment and for equipment makers to test during the development of the APs. Candela Technologies offers a fully automated TR-398 test system. The user can select from the list of 11 tests available in the GUI and all selected tests are run fully automated at one click of a button. Measurements are made and compared to the specified PASS/FAIL criteria in the TR-398 test plan and this report will show the summary PASS/FAIL results followed more detailed results for each test.

Add your notes below:

Setup is similar to what is described here:
https://www.candelatech.com/lf_tr398_testing.php

Summary Results

Calibrate Zero Attenuation RSSI

Summary

Calibrate the Zero attenuation settings for 2.4 and 5Ghz

Test Procedure

1. Create an /a/b/g (legacy) station on each radio.
2. Set attenuators to 0.
3. Create download connections for each station and run them for 30 seconds.
4. Record Received Signal Strength (RSSI) for each station.
5. Change attenuation to 15, 25, and 35 and re-run download test and record new RSSI.
6. Record the zero-attenuation RSSI, based on the 25 attenuation step, in the TR-398 Automation setup window.

Pass/Fail Criteria

1. Ensure that relative RSSI is within 2 for each of the different attenuation values.

Calibrate Zero Attenuation RSSI Results

Attenuator Calibration Snapshot for band: 2.4Ghz: Attenuation Set to: 0

Attenuator Calibration Snapshot for band: 2.4Ghz: Attenuation Set to: 15

Attenuator Calibration Snapshot for band: 2.4Ghz: Attenuation Set to: 25

Attenuator Calibration Snapshot for band: 2.4Ghz: Attenuation Set to: 35

Attenuator Calibration Snapshot for band: 5Ghz: Attenuation Set to: 0

Attenuator Calibration Snapshot for band: 5Ghz: Attenuation Set to: 15

Attenuator Calibration Snapshot for band: 5Ghz: Attenuation Set to: 25

Attenuator Calibration Snapshot for band: 5Ghz: Attenuation Set to: 35

6.1.1 Receiver Sensitivity Test

Summary

Receiver Sensitivity is a receiver’s ability to receive and correctly demodulate weak signals. This test provides a simplified measurement of the receiver’s sensitivity, relative to the total attenuation inserted between the DUT and the STA. As that attenuation is increased, the STA is limited to a single coding scheme, eventually causing the connection to degrade. The point at which the connection degrades represents the receiver’s approximate sensitivity. This is an approximate measurement only, where a detailed receiver sensitivity measurement would typically be performed in a conducted test environment with calibrated transmitter power levels. The test is repeated with multiple coding schemes, ensuring the DUT should smoothly transition between coding schedules as the attenuation increases in normal operation.

Test Procedure

1. Initialize attenuation to starting value (10 db stronger than passing value).
2. Configure the Traffic Generator to generate UDP traffic at 65% of theoretical rate.
3. Configure the attenuators to emulate 2-meter distance. This is considered 'zero' attenuation.
4. Increment attenuation by 1 dB each step. For each attenuation step, run traffic for 20 seconds. Record the packet-error rate (PER) If the PER is greater than 10%, then consider that the sensitivity cutoff and move to next iteration.
5. These test steps are run for MCS 0 and 7 at 20Mhz on 2.4Ghz, and MCS 0 and 9, 80 Mhz, and spatial-streams of one.
6. These test steps are run for each 45-degree spatial rotation.

Pass/Fail Criteria

6.1.1 Receiver Sensitivity Test Results

6.2.1 Maximum Connection Test (32-STA)

Summary

The Maximum Connection test intends to verify that the Wi-Fi AP can support 32 STAs simultaneously connected with minimal packet loss and no disassociations taking place.

Test Procedure

1. Establish the LAN connection, create 32 stations and allow the 32 stations to associate with the DUT.
2. Measure the downlink UDP packet loss, using a test time of 120 seconds and a traffic rate of 2 Mbps for 802.11n or 8 Mbps for 802.11ac, through each STA concurrently. Record the number of packets transmitted and received to calculate the packet error rate.
3. Do same test in the upload direction.

Pass/Fail Criteria

1. For each of the test configuration, Packet Error Rate (PER) for each STA SHALL achieve less than 1%.
2. For 2.4Ghz, summed upload throughput shall be at least (64Mbps * 0.99)
3. For 2.4Ghz, summed download throughput shall be at least (64Mbps * 0.99)
4. For 5Ghz, summed upload throughput shall be at least (256Mbps * 0.99)
5. For 5Ghz, summed download throughput shall be at least (256Mbps * 0.99)

6.2.1 Maximum Connection Test (32-STA) Results

Max-Cx-Test: Snapshot 5Ghz Download

Max-Cx-Test: Snapshot 5Ghz Upload

6.2.2 Maximum TCP Throughput Test

Summary

The Maximum throughput test intends to measure the maximum throughput performance of the DUT with a single station active. The test uses TCP connections and the attenuation is adjusted to emulate a distance of 2 meters. The 2.4Ghz station is configured for 20Mhz bandwidth and the 5Ghz station is configured for 80Mhz bandwidth. In both cases the station is configured for a maximum of two spatial streams.

Test Procedure

1. Establish the LAN connection, create 1 station and allow the station to associate with the DUT.
2. Measure the downlink TCP throughput, using a test time of 120 seconds.
3. Measure the uplink TCP throughput, using a test time of 120 seconds.

Pass/Fail Criteria

1. For each of the test configuration, Packet Error Rate (PER) for each STA SHALL achieve less than 1%.
2. For 2.4Ghz, upload rate must be at least 100Mbps
3. For 2.4Ghz, download rate must be at least 100Mbps
4. For 5Ghz, upload rate must be at least 560Mbps
5. For 5Ghz, download rate must be at least 560Mbps

6.2.2 Maximum TCP Throughput Test Results

6.2.3 Airtime Fairness Test

Summary

Airtime Fairness Test intends to verify the capability of Wi-Fi device to ensure the fairness of airtime usage.

Test Procedure

1. Establish the setup using default configuration.
2. Associate STA1 and STA2 with DUT. Establish the LAN connection and wait for 10 seconds.
3. Measure the downlink TCP throughput to each STA1 and STA2, using a test time of 120 seconds. Record this as STA1 _throughput_1 and STA2 _throughput_1.
4. Move STA2 to a medium distance to the DUT (equivalent to 38 dB@2.4GHz and 32 dB @5GHz attenuation between DUT and STA2). Wait for 10 seconds.
5. Measure the downlink TCP throughput to STA 1 and STA2, using a test time of 120 seconds.
Record this as STA1 _throughput_2 and STA2 _throughput_2.
6. Disassociate STA2 with the DUT. Replace STA 2 by STA 3 and remove the attenuation. STA3 is configured to support only a 2.4 GHz connection. Establish the Wi-Fi connection between STA3 and DUT and wait for 10 seconds.
7. Measure the downlink TCP throughput to STA 1 and STA3, using a test time of 120 seconds. Record this as STA1 _throughput_3 and STA3 _throughput_3.
8. Replace STA 3 with a STA that uses only 802.11a. Set the DUT to operating frequency band of 5 GHz. Repeat Step 2 to 7.

Pass/Fail Criteria

1. For the test in 2.4 GHz frequency band:
   1. STA1_throughput_1 SHALL be within (1±5%)*Mean(STA2_throughput_1, STA1_throughput_1).
   2. STA2_throughput_1 SHALL be within (1±5%)*Mean(STA2_throughput_1, STA1_throughput_1).
   3. STA1_throughput_2 SHALL be within (1±15%)*Mean(STA2_throughput_1, STA1_throughput_1).
   4. STA1_throughput_3 SHALL be within (1±15%)*Mean(STA2_throughput_1, STA1_throughput_1).
2. For the test in 5 GHz frequency band:
   1. STA1_throughput_1 SHALL be within (1±5%)*Mean(STA2_throughput_1, STA1_throughput_1).
   2. STA2_throughput_1 SHALL be within (1±5%)*Mean(STA2_throughput_1, STA1_throughput_1).
   3. STA1_throughput_2 SHALL be within (1±25%)*Mean(STA2_throughput_1, STA1_throughput_1).
   4. STA1_throughput_3 SHALL be within (1±25%)*Mean(STA2_throughput_1, STA1_throughput_1).
3. For the test in 2.4 GHz frequency band:
   1. The summation of STA1_throughput_1 and STA2_throughput_1 SHALL be larger than 80 Mbps.
   2. The summation of STA1_throughput_2 and STA2_throughput_2 SHALL be larger than 54 Mbps.
   3. The summation of STA1_throughput_3 and STA3_throughput_3 SHALL be larger than 50 Mbps.
4. For the test in 5 GHz frequency band:
   1. The summation of STA1_throughput_1 and STA2_throughput_1 SHALL be larger than 475 Mbps.
   2. The summation of STA1_throughput_2 and STA2_throughput_2 SHALL be larger than 280 Mbps.
   3. The summation of STA1_throughput_3 and STA3_throughput_3 SHALL be larger than 230 Mbps.

6.2.3 Airtime Fairness Test Results

ATF: Run #1 Snapshot 2.4Ghz

ATF: Run #2 Snapshot 2.4Ghz

ATF: Run #3 Snapshot 2.4Ghz

ATF: Run #1 Snapshot 5Ghz

ATF: Run #2 Snapshot 5Ghz

ATF: Run #3 Snapshot 5Ghz

6.3.1 Range Versus Rate Test

Summary

Range versus rate test intends to measure the rate-control, baseband and RF chain performance of Wi-Fi device at different signal levels. The attenuation of signals due to range increase is achieved by adjusting the attenuator.

Test Procedure

1. Configure the system to emulate a 2-meter distance. This is the baseline '0' attenuation.
2. Establish the LAN connection, create 1 station and allow the station to associate with the DUT.
3. Measure the downlink TCP throughput, using a test time of 120 seconds.
4. Measure the uplink TCP throughput, using a test time of 120 seconds.
5. For each of these attenuations on top of the baseline attenuation, repeat the steps above:
   For 2.4Ghz: 0, 10, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 51, 54, 57, 60, 63
   For 5Ghz: 10, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 51, 54

Pass/Fail Criteria

1. For 2.4Ghz, upload rate: 100, 100, 100, 100, 100, 100, 100, 95, 80, 75, 50, 31, 24, 17, 12, 7, 4
2. For 2.4Ghz, download rate: 100, 100, 100, 100, 100, 100, 100, 95, 80, 75, 50, 45, 35, 25, 14, 9, 8
3. For 5Ghz, upload rate: 560, 530, 420, 400, 360, 300, 220, 150, 125, 100, 45, 25, 5, 1
4. For 5Ghz, download rate: 560, 530, 420, 400, 360, 300, 220, 150, 125, 100, 45, 25, 5, 1

6.3.1 Range Versus Rate Test Results

6.3.2 Spatial Consistency Test

Summary

Spatial consistency test intends to verify the Wi-Fi signal consistency in spatial domain by testing throughput with DUT at different angles to the Station and at different emulated distance. The angles are changed with a programmable turn-table. The distance is emulated by adjusting attenuation.

Test Procedure

1. Configure the system to emulate a 2-meter distance.
2. Establish the LAN connection, create 1 station and allow the station to associate with the DUT.
3. Measure the downlink TCP throughput, using a test time of 60 seconds.
4. Measure the uplink TCP throughput, using a test time of 60 seconds.
5. Change attenuation by adding these values to the baseline 2-meter attenuation:
   For 2.4 Ghz band: 10, 38, 48
   For 5 Ghz band: 10, 32, 42
   
6. Rotate the AP platform by 30 degrees and repeat steps above until the AP has been rotated a full 360 degrees.

Pass/Fail Criteria

1. For 2.4Ghz, upload rate: 90, 70, 35
2. For 2.4Ghz, download rate: 90, 70, 35
3. For 5Ghz, upload rate: 500, 200, 100
4. For 5Ghz, download rate: 500, 200, 100

1. For 2.4Ghz, the range is 30%
2. For 5Ghz, the range is 40%

6.3.2 Spatial Consistency Test Results

6.3.2 Spatial Consistency Test Details