Media summit 25-10-2018

Edinburgh
https://www.spinics.net/lists/linux-media/msg141095.html

Attendees


Topics

CEC (Hans Verkuil)

1 pin bus, cec-gpio error injection support
Tda998x (including BeagleBoard Bone support)
ChromeOS CEC support
DisplayPort CEC Tunneling over AUX for i915, nouveau, amdgpu
Still lots of work happening around CEC utilities, the compliance test is in continuous use at cisco.
URL to CEC status: https://hverkuil.home.xs4all.nl/cec-status.txt

RC-Core progress report (Sean Young)


Linux supports more IR hardware than any other operating system.

Future/TODO

Support with gpio and simple ir led is available

Persistent storage of controls (Ricardo Ribalda - Qtec)

Sensors usually come with some calibration data from the manufacturer about dead pixels etc.
How to get this calibration data into the sensor.

proposal: 
    - new vid interface /dev/videoX /dev/v4l-subX
    - (reuse v4l2 ioctls for save/restore/get value(s))
    - implementation in two parts: main driver intf, storage

why not set sensor vals from userspace? - user might impl wrong

no get_min/max coz storage might not have space to save all ctrls

NVMEM might be good for storage rw
 - Is it a filesystem (or is that more overhead)
 - controls validate the data in the same way as they are validated by the device sensor.


Maintenance process tools - DIM (Laurent Pinchart - Ideas on Board)

DRM have switched to co-maintainer model
 - Burden is shared
 - Increased commit rights to the mainline (multi-maintaner model)
 - no enforeced maintainer process (each their own) - multi-comitter means each has to do checks - need to standardize tools -> DIM
 - DRM inglorious maintenance tool : https://01.org/linuxgraphics/gfx-docs/maintainer-tools/dim.html 
 - conflict resolution by commiter, not maintainer
 - DIM uses conflict resoultion system in git to pre-prepare the merge conflicts in a shared cache.
 - tool enforces quality checks

drawbacks
- pushing = testing = rebuilding :/ -> becomes a burden for the committers

- Patches are integrated from a patchwork instance, and the commits then reference the history of the patch upstream
- Adds an extra tool that must be used, and then an extra delay.

easyish to get commit rights
 - Responsibility to perform correctly
 - spreads the review workload (creates review-economics)

- "please submit patches" "also please merge conflict resolution and rebuild"
- If automated testing can validate the process, it can simplify commit process for multi-commiters.

had this model in media for a few years
- somebody merged 9k lines and lost a handful of devs
- probably exception, model fine
- many comitters but few maintainers - problem could be technically resolved

too few ppl reviewing patches - wait time measured in months :)
- multi-commiter model try to solve this problem

incentive to get more maintainers
- commit rights
- responsilibity - get attached to code :)


Automated testing (Ezequiel Garcia, Gustavo Padovan, Sakari Ailus)

Testing only one config, one version, one patch application, one hardware (one test to rule them all). Sometimes testing is omitted altogether by a developer, or it has been done before making changes to a patch. A lot of problems will only be found in the additional validation steps Mauro now performs, or when the patches already have reached the master branch. We could do better.

Ideal CI: patch submission -> auto test pass -> review +1, +2 -> merged! :)

Isn't the core question "what level of quality standards to we want to enforce" ? The maintainance process should be modelled around the answer to that question, not the other way around. Automated testing can be part of the quality standard enforcement procedure.

Three steps problem:

  1. Define the quality standards
  2. Define how to quantify them
  3. Define how to enforce them.

Automated testing is part of the enforcement.

Auto testing status
https://linuxtv.org/wiki/index.php/Media_Open_Source_Projects:_Looking_for_Volunteers
v4l2-compliance

The good:

* complete in terms of API testing
* quick and easy to run
* nice human readable machine parsable output
* new drivers are required to pass the test

The bad:

* no stateful/statless codec support
* no sdr and touch support
frequently updated
* only one contributor

Lack of a DVB: virtual driver and dvb-compliance. Added the need of write a virtual driver and a DVB compliance tool in order to check Digital TV core to the list of things to be done at https://linuxtv.org/wiki/index.php/Media_Open_Source_Projects:_Looking_for_Volunteers. 

* Test format output consolidation (Test summit are working on this)

Ezequiel reported that some people are complaining that v4l2-compliance is updated too frequently, but this is unavoidable according to Hans. Mauro proposed moving v4l2-compliance to the kernel source tree but Hans prefers keeping it separate as it makes it easier to develop it.

v4l2-compliance has a single contributor. We should push developers submitting new APIs to implement the corresponding tests. To achieve this we first need to clean up the code base to make contributions easier. There is currently no internal API documentation, and source filies are quite big. Splitting code into separate unit tests could be useful.

There are still lots of compliance tests to be developped (MC-compliance for instance). Should we have them all in a single tool or in separate tools ?

Other test options


Most of our tests are low-level, do we want high-level testing ?

e.g. gst-validate-1.0 --set-scenario pause_resume videotestsrc ! v4l2fwhtenc ! v4l2fwhtdec ! fakevideosink

Testing of real hardware

KernelCI

- kernel CI joining LinuxFoundation recently: extend test use cases and LTS
- UVC target of build integration testing (report on linux-media mailing list: RFC)
- V4L testing first test beyond boot testing

https://www.mail-archive.com/linux-media@vger.kernel.org/msg135787.html

- MIssing components to identify driver under test
- test case in v4l2-compliance output parse
- Need to collaborate with kernelCI to decide test report frequency and triggering: eg. weekly report
- To discuss if report regressions only or new driver/features testing
- Tests could be run on any branch (but only triggered by changes: eg new commits)
- Test same branch with different configurations (eg. plain V4L2, V4L2+MC, DVB with no MC etc)
- Number of possible tests only depends on how many lab instances are available
- 32bit compliance test utility against a 64bit kernel. 
- Might require simplying build of test tools (for different configurations, eg 32 vs 64 bits)
- Virtual machines used to run tests are LAVA slaves (created by LAVA itself)
- "Labs" instances are distributed and federated to KernelCI, each lab tests specific boards
- DRM uses custom FPGA (-not sure I got this right-) to compare actual output wiht the expected one

Post-presentation talk on testing


Eveyrone: reply Ezequiel's mail on the list!

Stateless codecs

Support for stateless codecs will be merged for v4.20 as a staging driver being used by AllWinner
how to support in userspace?

bit parsing is application-specific - so there's no point in providing it

should there be simple parser for compliance-testing?

All main applications support libva, developed originally by Intel to provide access to their accelerators.
libva backend was written for stateless cedrus decoder

Mainly allwinner, but some work on Rockchip stateless codec

Requirements for removal from staging is at least one other decoder and encoder using the API.
API is not stable atm so should expect it to change.



V4L2 Create better ioctls

VIDIOC_G/S_PARM - really only used for get/set frameinterval

why separate API to set ctrl?

VIDIOC_ENUM_FRAMEINTERVALS

struct v4l2_frmivalenum

ns vs fracs 

Combine ENUM_FMT, ENUM_FRAMESIZES and ENUM_FRAMEINTERVALS

make ioctl to combine all three that returns all possiblities/combinations wihout having to enumerate everything
is it worth the effort? - might be big eg. vivid 1710 combinations

- Memory requirements of the argument are not an issue as such: video buffers will require more memory anyway.
- Don't change now. But revisit, if any of the following happens

struct v4l2_buffer

problems:
- timestamp is not 2038-safe
- planes - single vs multi plane handing is really different and a mess

proposal: new struct v4l2_buffer
- __u64 timestamp
- struct v4l2_ext_plane planes[VIDEO_MAX_PLANES] - put planes arraywithint struct v4l2_buffer
ideas:
    - improve plane description: offset at start of plane, padding at end of plane, fd refer to one or all planes (-1 for remaining if allowed by SoC, eg exynos4)

struct v4l2_create_buffers

only format.pix.sizeimage is used
want to simplify - coz struct v4l2_format is kinda big
- Create buffers one at a time

proposal: more generic stuff:
    - VIDIOC_CREATE_BUF - with just size per plane instead of format (if 0 then use current format's size like REQBUFS)
    - VIDIOC_DELETE_BUF <- currently canoot be done at all, might need it in a few years anyway
    - VIDIOC_DELETE_ALL_BUFS

struct v4l2_format and VIDIOC_G/S/TRY_FMT

v4l2_pix_format vs v4l2_pix_format-mplane - painful

...that's it?


Fault Tolerant V4L2

problem: 8 cameras, one dies, they all cannot be used :/

V4L2 designed to work only when all devs working - but still need fault tolerance :/

- Media events to tell when new devices appear
- A change in the graph increments the media graph version number (G_TOPOLOGY IOCTL)
- Way to tell the user whether the registration is complete or not


Complex Cameras

how to make generic apps work with complex hw and mc -> libcamera

we gonna have more complex cameras in notebooks coz easier/cheaper to make
- so we need to do this or else we're going to lose laptop webcams

dev process

complaints?
- ppl want recognition for being involved - have intermediate multicommit tree?
- commit-to-master rights vs merge rights

Linuxtv.org

- Hosted in a virtual machine somewhere in a German university
- The long term need is to move the virtual machine out