Skip to content

driver: fix capability mask used for TX direction detection - #34

Open
bborosc2 wants to merge 1 commit into
chromeos:mainfrom
bborosc2:fix-vfl-dir-output-mplane
Open

bborosc2 wants to merge 1 commit into
chromeos:mainfrom
bborosc2:fix-vfl-dir-output-mplane

Conversation

@bborosc2

@bborosc2 bborosc2 commented Aug 2, 2026

Copy link
Copy Markdown

V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE is a v4l2_buf_type enum value (10), not a
capability bit, so OR-ing it into the device_caps mask yielded the wrong value:

  • before: V4L2_CAP_VIDEO_OUTPUT (0x2) | 10 = 0x0000000a
  • after: V4L2_CAP_VIDEO_OUTPUT (0x2) | V4L2_CAP_VIDEO_OUTPUT_MPLANE (0x2000) = 0x00002002

A device advertising only V4L2_CAP_VIDEO_OUTPUT_MPLANE therefore fell through to
VFL_DIR_RX instead of VFL_DIR_TX. The stray 0x8 bit that the old mask picked up
is not part of the V4L2_CAP_* space at all.

Impact

vfl_dir feeds determine_valid_ioctls() in the V4L2 core, where
is_rx = vfl_dir != VFL_DIR_TX and is_tx = vfl_dir != VFL_DIR_RX. With the direction
misdetected, an mplane-only output device:

  • gets -ENOTTY for VIDIOC_ENUMOUTPUT, VIDIOC_G_OUTPUT, VIDIOC_S_OUTPUT and the
    VIDIOC_*AUDOUT ioctls, even though the driver implements all of them
  • instead has the capture-only ioctls wrongly enabled: VIDIOC_ENUMINPUT,
    VIDIOC_G_INPUT, VIDIOC_S_INPUT, VIDIOC_QUERYSTD, VIDIOC_G_TUNER and friends

Format negotiation keeps working either way, because the driver registers both the
_vid_cap and _vid_out variants of ENUM_FMT/G_FMT/S_FMT/TRY_FMT. That is
likely why this went unnoticed; v4l2-compliance would flag the direction-exclusive
ioctls.

Notes

The line dates back to the initial commit. f365fc5 ("driver: properly set device
direction") touched these exact lines to change vfl_dir |= to =, but the typo one
line below was missed.

V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE is a v4l2_buf_type enum value (10),
not a capability bit, so OR-ing it into the device_caps mask yielded
0x0000000a instead of 0x00002002. A device advertising only
V4L2_CAP_VIDEO_OUTPUT_MPLANE therefore fell through to VFL_DIR_RX
instead of VFL_DIR_TX, and the stray 0x8 bit is not part of the
V4L2_CAP_* space at all.

With the wrong direction, determine_valid_ioctls() in the V4L2 core
rejects VIDIOC_ENUMOUTPUT, VIDIOC_G_OUTPUT, VIDIOC_S_OUTPUT and the
VIDIOC_*AUDOUT ioctls with -ENOTTY even though the driver implements
them, and instead enables capture-only ioctls such as
VIDIOC_ENUMINPUT, VIDIOC_G_INPUT, VIDIOC_S_INPUT and VIDIOC_QUERYSTD.
Format negotiation keeps working because the driver registers both the
_vid_cap and _vid_out variants of ENUM_FMT/G_FMT/S_FMT/TRY_FMT, which
is why the bug went unnoticed.

Fixes: f364e65 ("Initial commit")
@google-cla

google-cla Bot commented Aug 2, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant