Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USB Compliance Verification Test Suite (USBCV) #1059

Open
hathach opened this issue Aug 30, 2021 · 5 comments
Open

USB Compliance Verification Test Suite (USBCV) #1059

hathach opened this issue Aug 30, 2021 · 5 comments

Comments

@hathach
Copy link
Owner

hathach commented Aug 30, 2021

What is this issue for

The test suite include many corner cases which require both update of usbd and dcd to pass. This issue is used to keep track of current status of compliance test of tinyusb device stack on each DCD ports. Once it is all passed, we could close this.

What is USB Compliance Verification (USBCV)

USBCV is test suite tool used by usb-if to verify all usb hardware/software/controller which is required to passed in order to have USB logo on the product. https://www.usb.org/compliancetools . It is a good stress test with unusual case such as switching configuration 0 -> 1 -> 0, suspend/resume/remote-wakeup. Enumeration 150 times in a row as well as weird case like MSC READ10 with direction bit set to output etc ...

How to run USBCV

usbcv tool only runs on windows , depending on usb controller type on your PC you will need to either download the USB3CV (xHCI) or USB2CV (EHCI) from https://www.usb.org/compliancetools . Note: even your mcu is only full speed, but testing on xHCI host, you would need to run USB3CV.

Which tests suite to run

Within the scope of tinyusb device stack, we only need to run follow test suite

  • Chapter 9 Test [USB 2 devices]
  • HID Tests
  • MSC Tests

usbcv

Compliance Status

Example/Application can have large impact on the test result, the follow result is tested mostly with cdc_msc and hid_composite

MCU \ Test Suite Chapter9 HID MSC
nRF52840 ✔️ ✔️ ✔️
SAMD 21/51 ✔️ ✔️ ✔️
RP2040 ✔️ ✔️ ✔️
STM32F4 ✔️ ✔️ ✔️
Renesas RX ✔️ ✔️ ✔️
iMXRT ✔️ ✔️ ✔️
ESP32-S2/3 ✔️ ✔️ ✔️
Kinetic KL25/32 ✔️ ✔️ ✔️
DA146x ✔️ ✔️ ✔️
NUC 12x ✔️
@HiFiPhile
Copy link
Collaborator

HiFiPhile commented Aug 30, 2021

I'm trying to test my project based on STM32L071.

SetConfiguration passed even without #1058

TD 9.13 - SetConfiguration Test (Configuration Index 0)Passed
INFO
Start time: Aug 30, 2021 - 14:23:20
INFO
SetConfiguration with configuration value : 0x1
INFO
Unconfigured the device
INFO
SetConfiguration with configuration value : 0x1
INFO

Stop time: Aug 30, 2021 - 14:23:21
INFO
Duration:  1 second.
INFO
Stopping Test [ TD 9.13 - SetConfiguration Test (Configuration Index 0):
     Number of: Fails (0); Aborts (0); Warnings (0) ]

TD 9.30 Configuration Summary Descriptor Test (Configuration Index 0x00) - Device State ConfiguredPassed
INFO
Start time: Aug 30, 2021 - 14:23:21
INFO
Configuration Index 0x00 has a Configuration Value of 1.
INFO
Initialized the device to the Configured state with Configuration Value 1.
INFO
Retrieved Device Descriptor.
INFO
Device Under Test supports USB Version 0x2.0x10.
INFO
Retrieved BOS Descriptors.
INFO
No Configuration Summary Descriptors found.
INFO

Stop time: Aug 30, 2021 - 14:23:22
INFO
Duration:  1 second.
INFO
Stopping Test [ TD 9.30 Configuration Summary Descriptor Test (Configuration Index 0x00) - Device State Configured:
     Number of: Fails (0); Aborts (0); Warnings (0) ]

TD 9.14 - Suspend/Resume Test (Configuration Index 0)Passed
INFO
Start time: Aug 30, 2021 - 14:23:22
INFO
Suspended the parent port of the Device Under Test.
INFO
Resumed the parent port of the Device Under Test.
INFO
Got device descriptor of Device Under Test
INFO

Stop time: Aug 30, 2021 - 14:23:23
INFO
Duration:  1 second.
INFO
Stopping Test [ TD 9.14 - Suspend/Resume Test (Configuration Index 0):
     Number of: Fails (0); Aborts (0); Warnings (0) ]

Then it fails at L1Suspend/Resume Test (Configuration Index 0)

INFO
Start time: Aug 30, 2021 - 14:23:25
INFO
Checking Device Under Test for LPM L1 Compatibility...
INFO
USB version of device is 2.10.
INFO
DUT IS compatible with LPM.
INFO
LPM IS required for DUT
FAIL
(USB: 9.6.2.1.9) A USB device with a USB version greater than or equal to 0x210 must include a USB 2.0 Extension descriptor.
INFO

Stop time: Aug 30, 2021 - 14:23:26
INFO
Duration:  1 second.
INFO
Stopping Test [ L1Suspend/Resume Test (Configuration Index 0):
     Number of: Fails (1); Aborts (0); Warnings (0) ]

I set bcdUSB to 0x0210 to be driver-free like WebUSB. However I looked USB 2.0 and 3.2 specifications but can't find this 9.6.2.1.9 chapter. Do you have idea where it is ?


TUD_HID_REPORT_DESC_GENERIC_INOUT fails HID Class Report Descriptor Test :

Report descriptor length: 32
Item count in report descriptor: 16
External report descriptor parser encountered 2 errors

(HID: 3.2.61) The report descriptor returned in response to a GetDescriptor(Report) must be compliant with the HID specification.
Byte Number:   11h (  17d)
  Data Field: 81 02
  Mnemonic:  Input
  Value: (Variable)
  Errors: Error:   Logical Minimum MUST be less than Logical Maximum

Byte Number:   1Dh (  29d)
  Data Field: 91 02
  Mnemonic:  Output
  Value: (Variable)
  Errors: Error:   Logical Minimum MUST be less than Logical Maximum

@hathach
Copy link
Owner Author

hathach commented Aug 30, 2021

@HiFiPhile USB 2.0 extension section 9.6.2.1 is part of USB 3.2 revision 1.0 specs

SetConfiguration passed even without #1058

it passes but fake switching configuration to 0, and your device didn't implement the multiple configuration. Along the MSC test, it does assume switching configuration from 1 to 0 (unconfig) then back to 1 will reset all the driver state including stalled endpoint etc ... which could cause an issue. dcd_edpt_close_all() is added to resolve this correctly.

@HiFiPhile
Copy link
Collaborator

NUC126 passed chapter 9 tests with custom vendor class.
Chapter 9 Tests - USB 2 - Passed - 2021-09-09 20-35-38.zip

@hathach
Copy link
Owner Author

hathach commented Sep 10, 2021

NUC126 passed chapter 9 tests with custom vendor class.
Chapter 9 Tests - USB 2 - Passed - 2021-09-09 20-35-38.zip

Thank for the testing, though, did you test with the hid composite, one of the chapter9 is remote wakeup which is only tested with hid example due to TUSB_DESC_CONFIG_ATT_REMOTE_WAKEUP

@HiFiPhile
Copy link
Collaborator

HiFiPhile commented Sep 10, 2021

Thank for the testing, though, did you test with the hid composite, one of the chapter9 is remote wakeup which is only tested with hid example due to TUSB_DESC_CONFIG_ATT_REMOTE_WAKEUP

It didn't work.
There is a typo of |= but still something more.

USBD->ATTR = USBD_ATTR_RWAKEUP_Msk;

HID test passed.
USB3CV.zip


Something is wrong with suspend detection.
_usbd_dev.remote_wakeup_en is 1 but _usbd_dev.suspended is 0

USBD Setup Received 00 03 01 00 00 00 00 00
  Set Feature
    Enable Remote Wakeup
  Queue EP 80 with 0 bytes ...
USBD Xfer Complete on EP 80 with 0 bytes

USBD Setup Received 80 00 00 00 00 00 02 00
  Get Status
  Queue EP 80 with 2 bytes ...
USBD Xfer Complete on EP 80 with 2 bytes
  Queue EP 00 with 0 bytes ...
USBD Xfer Complete on EP 00 with 0 bytes

USBD Setup Received 80 00 00 00 00 00 02 00
  Get Status
  Queue EP 80 with 2 bytes ...
USBD Xfer Complete on EP 80 with 2 bytes
  Queue EP 00 with 0 bytes ...
USBD Xfer Complete on EP 00 with 0 bytes
USBD Suspend : Remote Wakeup = 1
USBD Resume
USBD Resume

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🆕 New
Development

No branches or pull requests

2 participants