April 26, 2023
This blog is in progress. I will update it if I can find a way to overcome these kernel and userspace issues.
TL;DR:
I wanted to capture 4K video from HDMI by using my Ubuntu Desktop PC.
This blog explains my adventures on V4L2 Kernel Driver and FFmpeg Userspace issues.
Summary of the problems
- Kernels 6.2 and 6.3 prints crash messages with 4K Capture device
- Can not capture from 4K HDMI device consecutively with ffmpeg ( it just works wfine with 1080p devices )
The CreHivi 4K Capture Device
As an ex-canonical employee, I still love to use Ubuntu. And these days, due to my work, I frequently need to deal with multimedia devices ( STB, TV) so I wanted to be able to capture 4K HDMI signals to my Linux Desktop PC.
Of course I knew that there are much more professional HDMI capture devices from various brands that might work well with Linux, but I wanted something cheap from China. And I was looking at ITE chipsets (ITE9325 specifically), so I found one below:
Here is the link to the product: http://hi-video.com.cn/Crehivi/pro/Video_Capture/2022/0223/164.html
And here are some photos:
Linux Kernel dmesg
The device has an interesting USB behavior. I think it has a USB HUB chip (From GenesysLogic) inside.
When I insert the device to a any USB2.0 port on my desktop, here is the log:
[ 542.670825] usb 1-2: USB disconnect, device number 5
[ 542.962824] usb 2-2: USB disconnect, device number 3
[ 547.378720] usb 1-6: new high-speed USB device number 6 using xhci_hcd
[ 547.637380] usb 1-6: New USB device found, idVendor=05e3, idProduct=0610, bcdDevice=62.04
[ 547.637386] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 547.637389] usb 1-6: Product: USB2.1 Hub
[ 547.637391] usb 1-6: Manufacturer: GenesysLogic
[ 547.645436] hub 1-6:1.0: USB hub found
[ 547.650375] hub 1-6:1.0: 4 ports detected
[ 550.454662] usb 1-6.1: new high-speed USB device number 7 using xhci_hcd
[ 550.610117] usb 1-6.1: config 1 has an invalid interface number: 7 but max is 4
[ 550.610122] usb 1-6.1: config 1 has no interface number 4
[ 550.622111] usb 1-6.1: New USB device found, idVendor=048d, idProduct=9325, bcdDevice= 1.00
[ 550.622116] usb 1-6.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 550.622118] usb 1-6.1: Product: 4K USB3.0 Capture Pro
[ 550.622120] usb 1-6.1: Manufacturer: USB
[ 550.622122] usb 1-6.1: SerialNumber: 00000001
[ 550.647125] usb 1-6.1: Found UVC 1.10 device 4K USB3.0 Capture Pro (048d:9325)
[ 550.661336] input: 4K USB3.0 Capture Pro: 4K USB3. as /devices/pci0000:00/0000:00:02.1/0000:01:00.0/usb1/1-6/1-6.1/1-6.1:1.0/input/input33
[ 550.695388] hid-generic 0003:048D:9325.000A: hiddev0,hidraw0: USB HID v1.11 Device [USB 4K USB3.0 Capture Pro] on usb-0000:01:00.0-6.1/input7
And this creates following device nodes under /dev :
stulluk ~ $ ls /dev/video*
/dev/video0 /dev/video1
stulluk ~ $
Notes:
- config 1 has an invalid interface number: 7 but max is 4 !!!
- config 1 has no interface number 4
- Why there are two video%d nodes?
- But I find /dev/video0 to be the working one, keep reading…
But if I insert the device to a USB3.x port, here is what happens:
[ 1795.764274] usb 1-2: new high-speed USB device number 8 using xhci_hcd
[ 1796.022898] usb 1-2: New USB device found, idVendor=05e3, idProduct=0610, bcdDevice=62.04
[ 1796.022905] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 1796.022908] usb 1-2: Product: USB2.1 Hub
[ 1796.022910] usb 1-2: Manufacturer: GenesysLogic
[ 1796.029970] hub 1-2:1.0: USB hub found
[ 1796.034884] hub 1-2:1.0: 4 ports detected
[ 1797.813043] usb 2-2: new SuperSpeed USB device number 4 using xhci_hcd
[ 1797.834734] usb 2-2: LPM exit latency is zeroed, disabling LPM.
[ 1797.841230] usb 2-2: config 1 has an invalid interface number: 7 but max is 4
[ 1797.841234] usb 2-2: config 1 has no interface number 4
[ 1797.843233] usb 2-2: New USB device found, idVendor=048d, idProduct=9325, bcdDevice= 1.00
[ 1797.843236] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1797.843238] usb 2-2: Product: 4K USB3.0 Capture Pro
[ 1797.843240] usb 2-2: Manufacturer: USB
[ 1797.843242] usb 2-2: SerialNumber: 00000001
[ 1797.856925] uvcvideo 2-2:1.1: Unknown video format 30313050-0000-0010-8000-00aa00389b71
[ 1797.856931] uvcvideo 2-2:1.1: Unknown video format e436eb7e-524f-11ce-9f53-0020af0ba770
[ 1797.856937] usb 2-2: Found UVC 1.10 device 4K USB3.0 Capture Pro (048d:9325)
[ 1797.860772] input: 4K USB3.0 Capture Pro: 4K USB3. as /devices/pci0000:00/0000:00:02.1/0000:01:00.0/usb2/2-2/2-2:1.0/input/input37
[ 1797.866602] hid-generic 0003:048D:9325.0010: hiddev0,hidraw0: USB HID v1.11 Device [USB 4K USB3.0 Capture Pro] on usb-0000:01:00.0-2/input7
So, did you notice this part:
[ 1797.856925] uvcvideo 2-2:1.1: Unknown video format 30313050-0000-0010-8000-00aa00389b71
[ 1797.856931] uvcvideo 2-2:1.1: Unknown video format e436eb7e-524f-11ce-9f53-0020af0ba770
So my journey for investigation began.
ffmpeg and v4l2-ctl
When device is inserted to USB2.0 port, here are the supported formats:
stulluk ~ $ ffmpeg -hide_banner -f video4linux2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x55d1c0460680] Raw : yuyv422 : YUYV 4:2:2 : 1920x1080 1280x720 720x576 720x480 640x480
[video4linux2,v4l2 @ 0x55d1c0460680] Raw : nv12 : Y/CbCr 4:2:0 : 1920x1080 1280x720 720x576 720x480 640x480
/dev/video0: Immediate exit requested
stulluk ~ $
The same capabilities can be verified with v4l2-ctl:
stulluk ~ $ v4l2-ctl --list-formats-ext /dev/video0
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 1920x1080
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.020s (50.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.017s (60.000 fps)
[1]: 'NV12' (Y/CbCr 4:2:0)
Size: Discrete 1920x1080
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 720x576
Interval: Discrete 0.020s (50.000 fps)
Size: Discrete 720x480
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.017s (60.000 fps)
stulluk ~ $
However, if I insert the device to USB3.0 port:
stulluk ~ $ ffmpeg -hide_banner -f video4linux2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x55cf58a99680] Raw : nv12 : Y/CbCr 4:2:0 : 3840x2160 2560x1440 1920x1080 1360x768 1280x1024 1280x720 1024x768 800x600 720x576 640x480
[video4linux2,v4l2 @ 0x55cf58a99680] Raw : yuyv422 : YUYV 4:2:2 : 2560x1440 1920x1080 1360x768 1280x1024 1280x720 1024x768 800x600 720x576 720x480 640x480
[video4linux2,v4l2 @ 0x55cf58a99680] Raw : Unsupported : 30313050-0000-0010-8000-00aa003 : 1920x1080 1280x720
[video4linux2,v4l2 @ 0x55cf58a99680] Raw : Unsupported : e436eb7e-524f-11ce-9f53-0020af0 : 1920x1080 1280x720
/dev/video0: Immediate exit requested
stulluk ~ $
So, what are these unknown / unsupported video formats ?
Here is the answer: https://lore.kernel.org/linux-media/2295088.CIBgFgJDf4@avalon/T/
So, when looking at this table here: https://gix.github.io/media-types/
Search for: 30313050-0000-0010-8000-00aa00389b71
Search for: e436eb7e-524f-11ce-9f53-0020af0
So, it seems, for the current kernel ( 5.15.0-67-generic ) installed on my desktop, it seems these formats are not yet in the V4L driver.
Lets switch to latest mainline kernel ( using Ubuntu Mainline Kernel installer ) 6.3.0, and lets see if that makes any difference:
Linux Kernel 6.3
After booting 6.3.0, when inserting device to USB3.0 port:
[ 36.236693] usb 1-2: new high-speed USB device number 2 using xhci_hcd
[ 36.495324] usb 1-2: New USB device found, idVendor=05e3, idProduct=0610, bcdDevice=62.04
[ 36.495332] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 36.495336] usb 1-2: Product: USB2.1 Hub
[ 36.495338] usb 1-2: Manufacturer: GenesysLogic
[ 36.502390] hub 1-2:1.0: USB hub found
[ 36.507310] hub 1-2:1.0: 4 ports detected
[ 38.293472] usb 2-2: new SuperSpeed USB device number 2 using xhci_hcd
[ 38.315158] usb 2-2: LPM exit latency is zeroed, disabling LPM.
[ 38.321661] usb 2-2: config 1 has an invalid interface number: 7 but max is 4
[ 38.321666] usb 2-2: config 1 has no interface number 4
[ 38.323657] usb 2-2: New USB device found, idVendor=048d, idProduct=9325, bcdDevice= 1.00
[ 38.323661] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 38.323663] usb 2-2: Product: 4K USB3.0 Capture Pro
[ 38.323665] usb 2-2: Manufacturer: USB
[ 38.323667] usb 2-2: SerialNumber: 00000001
[ 38.364212] hid-generic 0003:048D:9325.0006: hiddev2,hidraw5: USB HID v1.11 Device [USB 4K USB3.0 Capture Pro] on usb-0000:01:00.0-2/input7
[ 38.390697] videodev: Linux video capture interface: v2.00
[ 38.395607] uvcvideo 2-2:1.1: Unknown video format 30313050-0000-0010-8000-00aa00389b71
[ 38.395612] usb 2-2: Found UVC 1.10 device 4K USB3.0 Capture Pro (048d:9325)
[ 38.399327] usbcore: registered new interface driver uvcvideo
[ 38.430687] ------------[ cut here ]------------
[ 38.430690] Unknown pixelformat 0x00000000
[ 38.430700] WARNING: CPU: 14 PID: 1348 at drivers/media/v4l2-core/v4l2-ioctl.c:1507 v4l_fill_fmtdesc+0x1b61/0x24c0 [videodev]
[ 38.430717] Modules linked in: uvcvideo videobuf2_vmalloc uvc videobuf2_memops videobuf2_v4l2 videodev videobuf2_common xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack_netlink nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype nft_compat nf_tables libcrc32c nfnetlink br_netfilter bridge stp llc xfrm_user xfrm_algo l2tp_ppp l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel pppox nvme_fabrics overlay zram amdgpu snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_usb_audio intel_rapl_msr snd_hda_codec intel_rapl_common iommu_v2 snd_usbmidi_lib snd_hda_core drm_buddy binfmt_misc edac_mce_amd mc snd_hwdep gpu_sched drm_ttm_helper ttm snd_pcm kvm_amd drm_display_helper snd_seq_midi nls_iso8859_1 snd_seq_midi_event kvm snd_rawmidi cec irqbypass crct10dif_pclmul polyval_clmulni polyval_generic ghash_clmulni_intel joydev snd_seq sha512_ssse3 rc_core snd_seq_device input_leds aesni_intel snd_timer drm_kms_helper crypto_simd snd
[ 38.430752] i2c_algo_bit cryptd syscopyarea sysfillrect rapl wmi_bmof soundcore k10temp ccp sysimgblt mac_hid sch_fq_codel zstd msr parport_pc ppdev lp parport drm ramoops reed_solomon pstore_blk pstore_zone efi_pstore ip_tables x_tables autofs4 hid_logitech_hidpp hid_logitech_dj hid_jabra hid_generic nvme usbhid hid crc32_pclmul nvme_core r8169 ahci realtek i2c_piix4 xhci_pci libahci nvme_common xhci_pci_renesas video wmi gpio_amdpt
[ 38.430772] CPU: 14 PID: 1348 Comm: pipewire Not tainted 6.3.0-060300-generic #202304232030
[ 38.430774] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./B550M-HDV, BIOS P1.90 08/09/2021
[ 38.430775] RIP: 0010:v4l_fill_fmtdesc+0x1b61/0x24c0 [videodev]
[ 38.430787] Code: 74 67 0f 86 8f f7 ff ff 81 fe 51 30 38 43 0f 84 70 f5 ff ff 80 7b 0c 00 0f 85 90 e5 ff ff 48 c7 c7 ba 6b ea c0 e8 ef 8b 46 e0 <0f> 0b be 20 00 00 00 48 8d 4b 2c 48 8d 7b 0c 45 31 e4 48 c7 c2 d6
[ 38.430789] RSP: 0018:ffffaa6dc41d7bf0 EFLAGS: 00010246
[ 38.430790] RAX: 0000000000000000 RBX: ffffaa6dc41d7d50 RCX: 0000000000000000
[ 38.430791] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[ 38.430792] RBP: ffffaa6dc41d7c18 R08: 0000000000000000 R09: 0000000000000000
[ 38.430792] R10: 0000000000000000 R11: 0000000000000000 R12: ffff920b93878800
[ 38.430793] R13: ffffffffc0c4cde0 R14: ffff920b0157e3c0 R15: ffff920b0421c018
[ 38.430794] FS: 00007fc4d9ede740(0000) GS:ffff9219de580000(0000) knlGS:0000000000000000
[ 38.430795] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 38.430796] CR2: 0000555839b640d8 CR3: 0000000138f38000 CR4: 0000000000750ee0
[ 38.430797] PKRU: 55555554
[ 38.430797] Call Trace:
[ 38.430798] <TASK>
[ 38.430801] v4l_enum_fmt+0x1a8/0x1e0 [videodev]
[ 38.430813] __video_do_ioctl+0x1ea/0x5f0 [videodev]
[ 38.430825] ? __check_object_size.part.0+0x3a/0x150
[ 38.430829] video_usercopy+0x154/0x790 [videodev]
[ 38.430841] ? __video_do_ioctl+0x1ea/0x5f0 [videodev]
[ 38.430853] ? __pfx___video_do_ioctl+0x10/0x10 [videodev]
[ 38.430866] video_ioctl2+0x15/0x30 [videodev]
[ 38.430877] v4l2_ioctl+0x6c/0xb0 [videodev]
[ 38.430889] __x64_sys_ioctl+0xa0/0xe0
[ 38.430892] do_syscall_64+0x5b/0x90
[ 38.430894] ? video_ioctl2+0x15/0x30 [videodev]
[ 38.430906] ? v4l2_ioctl+0x6c/0xb0 [videodev]
[ 38.430918] ? exit_to_user_mode_prepare+0x30/0xb0
[ 38.430920] ? syscall_exit_to_user_mode+0x29/0x50
[ 38.430922] ? do_syscall_64+0x67/0x90
[ 38.430923] entry_SYSCALL_64_after_hwframe+0x72/0xdc
[ 38.430925] RIP: 0033:0x7fc4d9d1aaff
[ 38.430926] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <41> 89 c0 3d 00 f0 ff ff 77 1f 48 8b 44 24 18 64 48 2b 04 25 28 00
[ 38.430927] RSP: 002b:00007fff9fafdae0 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[ 38.430928] RAX: ffffffffffffffda RBX: 0000000000000020 RCX: 00007fc4d9d1aaff
[ 38.430929] RDX: 0000555839b5ee0c RSI: ffffffffc0405602 RDI: 0000000000000020
[ 38.430930] RBP: 0000555839b5ee0c R08: 00000000ffffffff R09: 0000000000000002
[ 38.430931] R10: 0000000000000017 R11: 0000000000000246 R12: 00007fff9fafdbb0
[ 38.430931] R13: 0000000000000000 R14: 0000000040000114 R15: 0000555839b5ec38
[ 38.430933] </TASK>
[ 38.430934] ---[ end trace 0000000000000000 ]---
So, I think this is a kernel bug, and needs to be fixed in V4L2 driver.
However, this time the output of FFmpeg is different:
stulluk ~ $ ls /dev/video*
/dev/video0 /dev/video1
stulluk ~ $ ffmpeg -hide_banner -f video4linux2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x5613e01d6680] Raw : nv12 : Y/UV 4:2:0 : 3840x2160 2560x1440 1920x1080 1360x768 1280x1024 1280x720 1024x768 800x600 720x576 640x480
[video4linux2,v4l2 @ 0x5613e01d6680] Raw : yuyv422 : YUYV 4:2:2 : 2560x1440 1920x1080 1360x768 1280x1024 1280x720 1024x768 800x600 720x576 720x480 640x480
[video4linux2,v4l2 @ 0x5613e01d6680] Raw : Unsupported : .... little-endian (0x00000000) : 1920x1080 1280x720
[video4linux2,v4l2 @ 0x5613e01d6680] Raw : bgr0 : 32-bit BGRX 8-8-8-8 : 1920x1080 1360x768 1280x1024 1280x720 1024x768 800x600 720x576 720x480 640x480
/dev/video0: Immediate exit requested
stulluk ~ $
And of course, when inserting to USB2.0 port, there is no such issue:
[ 191.688733] usb 1-6: new high-speed USB device number 3 using xhci_hcd
[ 191.947359] usb 1-6: New USB device found, idVendor=05e3, idProduct=0610, bcdDevice=62.04
[ 191.947366] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 191.947370] usb 1-6: Product: USB2.1 Hub
[ 191.947372] usb 1-6: Manufacturer: GenesysLogic
[ 191.954414] hub 1-6:1.0: USB hub found
[ 191.959344] hub 1-6:1.0: 4 ports detected
[ 194.748743] usb 1-6.1: new high-speed USB device number 4 using xhci_hcd
[ 194.908086] usb 1-6.1: config 1 has an invalid interface number: 7 but max is 4
[ 194.908092] usb 1-6.1: config 1 has no interface number 4
[ 194.920081] usb 1-6.1: New USB device found, idVendor=048d, idProduct=9325, bcdDevice= 1.00
[ 194.920086] usb 1-6.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 194.920089] usb 1-6.1: Product: 4K USB3.0 Capture Pro
[ 194.920091] usb 1-6.1: Manufacturer: USB
[ 194.920093] usb 1-6.1: SerialNumber: 00000001
[ 194.942095] usb 1-6.1: Found UVC 1.10 device 4K USB3.0 Capture Pro (048d:9325)
[ 195.012656] hid-generic 0003:048D:9325.0007: hiddev2,hidraw5: USB HID v1.11 Device [USB 4K USB3.0 Capture Pro] on usb-0000:01:00.0-6.1/input7
However, if we look at the supported formats now:
stulluk ~ $ ffmpeg -hide_banner -f video4linux2 -list_formats all -i /dev/video0
[video4linux2,v4l2 @ 0x55c475eaf680] Raw : yuyv422 : YUYV 4:2:2 : 1920x1080 1280x720 720x576 720x480 640x480
[video4linux2,v4l2 @ 0x55c475eaf680] Raw : nv12 : Y/UV 4:2:0 : 1920x1080 1280x720 720x576 720x480 640x480
/dev/video0: Immediate exit requested
stulluk ~ $
So, there is no 4K capture possiblitiy now ( Which makes sense since USB2.0 bandwidth may not be enough for 4K streams )
4K Capture
For Capturing the 4K video, there are various tools to achieve that. I preferred ffmpeg:
stulluk ~ $ ffmpeg -hide_banner -f v4l2 -framerate 30 -video_size 3840x2160 -input_format nv12 -i /dev/video0 -preset faster -pix_fmt yuv420p out.mkv
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 651.009171, bitrate: 2985984 kb/s
Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 3840x2160, 2985984 kb/s, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x562a7cd45940] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x562a7cd45940] profile High, level 5.1, 4:2:0, 8-bit
[libx264 @ 0x562a7cd45940] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=4 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=24 lookahead_threads=6 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=20 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'out.mkv':
Metadata:
encoder : Lavf58.76.100
Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p(tv, progressive), 3840x2160, q=2-31, 30 fps, 1k tbn
Metadata:
encoder : Lavc58.134.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 198 fps= 28 q=-1.0 Lsize= 437kB time=00:00:06.73 bitrate= 531.5kbits/s speed=0.966x
video:435kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.466897%
[libx264 @ 0x562a7cd45940] frame I:2 Avg QP:10.43 size:190427
[libx264 @ 0x562a7cd45940] frame P:50 Avg QP:11.65 size: 578
[libx264 @ 0x562a7cd45940] frame B:146 Avg QP:15.94 size: 239
[libx264 @ 0x562a7cd45940] consecutive B-frames: 1.5% 0.0% 1.5% 97.0%
[libx264 @ 0x562a7cd45940] mb I I16..4: 80.4% 12.3% 7.3%
[libx264 @ 0x562a7cd45940] mb P I16..4: 0.1% 0.0% 0.0% P16..4: 0.3% 0.1% 0.0% 0.0% 0.0% skip:99.5%
[libx264 @ 0x562a7cd45940] mb B I16..4: 0.0% 0.0% 0.0% B16..8: 0.0% 0.0% 0.0% direct: 0.0% skip:100.0% L0: 5.0% L1:95.0% BI: 0.0%
[libx264 @ 0x562a7cd45940] 8x8 transform intra:12.0% inter:88.6%
[libx264 @ 0x562a7cd45940] coded y,uvDC,uvAC intra: 30.0% 30.4% 25.8% inter: 0.0% 0.1% 0.0%
[libx264 @ 0x562a7cd45940] i16 v,h,dc,p: 87% 4% 7% 2%
[libx264 @ 0x562a7cd45940] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 14% 38% 5% 5% 4% 4% 5% 6%
[libx264 @ 0x562a7cd45940] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 22% 16% 6% 6% 6% 5% 5% 4%
[libx264 @ 0x562a7cd45940] i8c dc,h,v,p: 76% 8% 12% 4%
[libx264 @ 0x562a7cd45940] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x562a7cd45940] ref P L0: 99.0% 1.0%
[libx264 @ 0x562a7cd45940] ref B L0: 50.0% 50.0%
[libx264 @ 0x562a7cd45940] ref B L1: 97.4% 2.6%
[libx264 @ 0x562a7cd45940] kb/s:520.50
Exiting normally, received signal 2.
stulluk ~ $ mediainfo out.mkv
General
Unique ID : 47512344754812862591665713139628711149 (0x23BE8B74ADE4BC3432DDAF69D47D1CED)
Complete name : out.mkv
Format : Matroska
Format version : Version 4
File size : 437 KiB
Duration : 6 s 833 ms
Overall bit rate : 524 kb/s
Writing application : Lavf58.76.100
Writing library : Lavf58.76.100
ErrorDetectionType : Per level 1
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 6 s 833 ms
Bit rate : 513 kb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 30.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.002
Stream size : 428 KiB (98%)
Writing library : x264 core 163 r3060 5db6aa6
Encoding settings : cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=4 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=24 / lookahead_threads=6 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=20 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default : Yes
Forced : No
Color range : Limited
stulluk ~ $ mpv out.mkv
(+) Video --vid=1 (*) (h264 3840x2160 30.000fps)
Cannot load libcuda.so.1
Using hardware decoding (vaapi).
VO: [gpu] 3840x2160 vaapi[nv12]
V: 00:00:06 / 00:00:06 (99%)
Exiting... (Quit)
stulluk ~ $
However, I faced another problem at this mode. If I re-run above ffmpeg capture command again, this time it can’t get any frames, and stuck:
stulluk ~ $ ffmpeg -f v4l2 -framerate 30 -video_size 3840x2160 -input_format nv12 -i /dev/video0 -preset faster -pix_fmt yuv420p out.mkv
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, bitrate: 2985984 kb/s
Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 3840x2160, 2985984 kb/s, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
File 'out.mkv' already exists. Overwrite? [y/N] N
Not overwriting - exiting
Exiting normally, received signal 2.
stulluk ~ $
Also checked this with kernel 5.15.x and situation same there as well. Same problem: Once you capture 4K, you can’t capture anymore. I am still wondering if I am making a mistake or is there a bug in the device…
I tried this with my existing MS2109 based ( 1080p only) HDMI capture devices, and can consecutively capture without any problem.
So there is something weird with this 4K HDMI Capture device or there is a bug in the software.
Live 4K Playback
I prefer ffplay for the playback since I find it to be the lowest delay:
ffplay -fflags nobuffer -f video4linux2 -framerate 30 -video_size 3840x2160 -input_format nv12 /dev/video0
This results following to be displayed on your desktop:
Final words
Since I use same commands for my old 1080p capture devices I suspect there is something different for this 4K device.
Here are my old 1080p capture devices (based on Macrosilicon MS2109 chip):
I am using these two capture cards for about 2-3 years and they are extremely reliable under Ubuntu.
I think we still have some way to achieve perfect / simple 4K capture under Linux.
However, I think I achieved basic functionality with this and look for kernel fixes on V4L2 Driver and corresponding userspace apps.