Ubuntu – USB ports keep resetting on Linux

usb

Problem

USB Ports (2.0) keep resetting causing connected peripherals to stop working after some time.

Hardware

Motherboard: ASUS J1800I-C
CPU: Intel(R) Celeron(R) CPU J1800 @ 2.41GHz
RAM: 4GB

Number of units: > 100 devices working as payment kiosks, all showing same issue.

Connected USB devices (lsusb)

Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 013: ID 0dd4:015d Custom Engineering SPA
Bus 001 Device 012: ID 12d1:1506 Huawei Technologies Co., Ltd. Modem/Networkcard
Bus 001 Device 011: ID 23d8:0285 
Bus 001 Device 010: ID 1ff7:0013 CVT Electronics.Co.,Ltd CVTouch Screen (HID)
Bus 001 Device 009: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 002: ID 0e8f:0022 GreenAsia Inc. multimedia keyboard controller
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

OSs reproducing the problem:
Ubuntu 14.04-server (64bit)
Ubuntu 16.04-server (64bit)
Ubuntu 18.04-server (64bit)
Ubuntu 14.04 (server&desktop) (32bit)
Ubuntu 16.04-server (32bit)
Tried the above OSs with both UEFI & BIOS boot.

OSs NOT producing any problem with the same hardware:

Windows 7

Symptoms:
On startup, all goes well and USB devices are working for 15~60 minutes.
After this time, one or more ports will start resetting with errors coming in dmesg:

    [ 1484.129248] perf interrupt took too long (2520 > 2500), lowering kernel.perf_event_max_sample_rate to 50000
    [ 2645.972003] usb 1-4-port1: cannot reset (err = -71)
    [ 2646.236044] usb 1-4.1: reset full-speed USB device number 4 using xhci_hcd
    [ 2646.324439] usb 1-4.1: ep 0x1 - rounding interval to 64 microframes, ep desc says 80 microframes
    [ 2647.558156] usb 1-4.1: reset full-speed USB device number 4 using xhci_hcd
    [ 2647.646546] usb 1-4.1: ep 0x1 - rounding interval to 64 microframes, ep desc says 80 microframes
    [ 3942.619669] usb 1-4-port1: cannot reset (err = -71)
    [ 3942.619742] usb 1-4-port1: cannot reset (err = -71)
    [ 3942.619821] usb 1-4-port1: cannot reset (err = -71)
    [ 3942.619861] usb 1-4-port1: cannot reset (err = -71)
    [ 3942.619901] usb 1-4-port1: cannot reset (err = -71)
    [ 3942.619905] usb 1-4-port1: Cannot enable. Maybe the USB cable is bad?
    [ 3942.690031] usb 1-4.1: reset full-speed USB device number 4 using xhci_hcd
    [ 3942.690038] usb 1-4.1: hub failed to enable device, error -22
    [ 3942.762176] usb 1-4.1: reset full-speed USB device number 4 using xhci_hcd
    [ 3943.170800] usb 1-4.1: device not accepting address 4, error -22
    [ 3943.242967] usb 1-4.1: reset full-speed USB device number 4 using xhci_hcd
    [ 3943.651734] usb 1-4.1: device not accepting address 4, error -22
    [ 3943.653063] usb 1-4.1: USB disconnect, device number 4
    [ 3943.751603] usb 1-4.1: new full-speed USB device number 8 using xhci_hcd
    [ 3943.840892] usb 1-4.1: New USB device found, idVendor=1ff7, idProduct=0013
    [ 3943.840899] usb 1-4.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
    [ 3943.841068] usb 1-4.1: ep 0x1 - rounding interval to 64 microframes, ep desc says 80 microframes
    [ 3943.850194] input: HID 1ff7:0013 as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.1/1-4.1:1.0/0003:1FF7:0013.0005/input/input11
    [ 3943.850467] hid-multitouch 0003:1FF7:0013.0005: input,hiddev0,hidraw3: USB HID v1.11 Mouse [HID 1ff7:0013] on usb-0000:00:14.0-4.1/input0
    [ 4215.269989] usb 1-4-port1: cannot reset (err = -71)
    [ 4215.270192] usb 1-4-port1: cannot reset (err = -71)
    [ 4215.270264] usb 1-4-port1: cannot reset (err = -71)
    [ 4215.270343] usb 1-4-port1: cannot reset (err = -71)
    [ 4215.270383] usb 1-4-port1: cannot reset (err = -71)
    [ 4215.270386] usb 1-4-port1: Cannot enable. Maybe the USB cable is bad?
    [ 4215.342214] usb 1-4.1: reset full-speed USB device number 8 using xhci_hcd
    [ 4215.342221] usb 1-4.1: hub failed to enable device, error -22
    [ 4215.358192] hub 1-4:1.0: hub_ext_port_status failed (err = -71)
    [ 4215.358263] usb 1-4-port1: cannot reset (err = -71)
    [ 4215.622566] usb 1-4.1: reset full-speed USB device number 8 using xhci_hcd
    [ 4215.755250] usb usb1-port4: disabled by hub (EMI?), re-enabling...
    [ 4215.755273] usb 1-4: USB disconnect, device number 3
    [ 4215.755277] usb 1-4.1: USB disconnect, device number 0
    [ 4216.031181] usb 1-4.1: device not accepting address 8, error -22
    [ 4216.064588] usb 1-4.2: USB disconnect, device number 5
    [ 4216.065324] usb 1-4.3: USB disconnect, device number 6
    [ 4216.065632] option1 ttyUSB0: GSM modem (1-port) converter now disconnected from ttyUSB0
    [ 4216.065647] option 1-4.3:1.0: device disconnected
    [ 4216.065760] huawei_cdc_ncm 1-4.3:1.1 wwx001e101f0000: unregister 'huawei_cdc_ncm' usb-0000:00:14.0-4.3, Huawei CDC NCM device
    [ 4216.079590] option1 ttyUSB1: GSM modem (1-port) converter now disconnected from ttyUSB1
    [ 4216.079611] option 1-4.3:1.2: device disconnected
    [ 4216.079752] option1 ttyUSB2: GSM modem (1-port) converter now disconnected from ttyUSB2
    [ 4216.079768] option 1-4.3:1.3: device disconnected
    [ 4216.148026] usb 1-4.4: USB disconnect, device number 7
    [ 4216.148183] usblp2: removed
    [ 4216.387741] usb 1-4: new high-speed USB device number 9 using xhci_hcd
    [ 4216.517658] usb 1-4: New USB device found, idVendor=05e3, idProduct=0610
    [ 4216.517666] usb 1-4: New USB device strings: Mfr=0, Product=1, SerialNumber=0
    [ 4216.517670] usb 1-4: Product: USB2.0 Hub
    [ 4216.518341] hub 1-4:1.0: USB hub found
    [ 4216.518643] hub 1-4:1.0: 4 ports detected
    [ 4216.792385] usb 1-4.1: new full-speed USB device number 10 using xhci_hcd
    [ 4216.885849] usb 1-4.1: New USB device found, idVendor=1ff7, idProduct=0013
    [ 4216.885855] usb 1-4.1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
    [ 4216.886042] usb 1-4.1: ep 0x1 - rounding interval to 64 microframes, ep desc says 80 microframes
    [ 4216.895096] input: HID 1ff7:0013 as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4.1/1-4.1:1.0/0003:1FF7:0013.0006/input/input13
    [ 4216.951230] hid-multitouch 0003:1FF7:0013.0006: input,hiddev0,hidraw2: USB HID v1.11 Mouse [HID 1ff7:0013] on usb-0000:00:14.0-4.1/input0

Things I tried and did NOT solve the problem:
I tried to disable autosuspend for USB ports by adding linux kernel cmdline: usbcore.autosuspend=-1 among many other options and they all failed.
I also tried to change multiple BIOS options related to USB, nothing helped.

Useful Facts:
– All USB 2.0 4-ports are causing the problem.
– The board has single USB 3.0 port which seems unaffected by the problem.
– Using powered USB hub connected to the USB3 port, then connecting all USB devices to it seems to solve the problem.
– Using 32bit version of the same OS seems to solve the USB problem. Tested some kiosks with 32bit installation for 3 days uptime and there were no single dmesg warning or error.

Question:

How can I solve this problem on my 64bit Ubuntu 16.04 ?

Converting to 32bit is an unrealistic solution for me as it requires the reinstallation of over 100 devices. Besides, some of my equipment comes with 64bit proprietary SDK.

Update 1:
The problem seems to be unrelated to the CPU architecture and it happened after a while on Ubuntu 14.04-32bit and 16.04-32bit

Best Answer

A possible solution - Blacklisting USB related kernel modules in a trial and error procedure revealed that the ehci-pci kernel module was the cause of the problem.

For a temporary solution, simply remove the kernel module:

sudo rmmod ehci_pci

For a permanent solution, either blacklist the module via kernel boot arguments, e.g.

modprobe.blacklist=ehci_pci

nb. Blacklisting ehci_pci will also prevent kernel module ehci-hcd from getting loaded.

The ehci* kernel modules are the kernel’s USB 2 drivers.

Luckily, the USB 3 kernel modules xhci* support USB 2 devices as well and you will experience no performance loss with these