15:02 -!- pinchartl [~pinchartl@perceval.ideasonboard.com] has joined #v4l-meeting
15:02 < pinchartl> hi
15:02 <@sailus> pinchartl: Bonjour!
15:02 <@sailus> I wonder whether I should expect some others to join.
15:03 <@sailus> Sylwester had comments on my latest mail.
15:04 < pinchartl> ping him ?
15:05 -!- snawrocki [~snawrocki@217-67-201-162.itsa.net.pl] has joined #v4l-meeting
15:05 <@sailus> snawrocki: Hi!
15:05 <@sailus> I think we now have everyone.
15:05 < pinchartl> hi Sylwester
15:05 < snawrocki> sailus: Hi! this time I'm not late ;]
15:05 <@sailus> :-)
15:06 <@sailus> I assume we're all here since we wanted to say something about my latest suggestion to change how composition works on subdevs.
15:06 <@sailus> snawrocki: I did read your mail.
15:06 < snawrocki> Hi Laurent, hope to have interesting conversation today, hopefully shorter than last time ;)
15:08 < pinchartl> :-)
15:08 <@sailus> snawrocki: What you referred to looks like a valid use case indeed.
15:08 <@sailus> Some of that is already supported since the user may set bytesperline field at buffer allocation time.
15:09 < snawrocki> sailus: do you mean the selection thread ? or interleaved image data ?
15:09 <@sailus> That achieves the same with the exception that the buffer start address isn't changeable --- think of dma_buf buffers rather than user pointer buffers only.
15:09 <@sailus> snawrocki: In selection thread.
15:10 <@sailus> Perhaps we should go one step back before considering this.
15:10 < pinchartl> sailus: in your diagram, the ping rectangle is essentially the selection bounds rectangle, right ?
15:10 < snawrocki> ok, AFAIR we agred that more than just bytesperline is needed
15:10 <@sailus> pinchartl: ping rectangle?
15:11 < pinchartl> pink
15:11 < pinchartl> sorry
15:11 <@sailus> I can make it properly pink. Now it's just "salmon", I think.
15:11 < pinchartl> salmon is fine :-)
15:12 <@sailus> So let's go back to the salmon rectangle.
15:12 < pinchartl> (lohi ?)
15:12 <@sailus> That's right.
15:12 <@sailus> We need a point where the source pad rectangles can refer to single sink pad coordinate system.
15:13 <@sailus> That, in my proposal, is the salmon composition rectangle.
15:13 < pinchartl> there are two things that bother me here
15:14 < snawrocki> sailus: I wonder whether we need compose bounds at all
15:14 < pinchartl> the first one is that the subdev performs composition, but there's no compose rectangle
15:15 < pinchartl> or at least no sink pad compose rectangle
15:15 < pinchartl> even though every sink stream is composed onto the final stream
15:15 <@sailus> pinchartl: What is the sink pad compose rectangle needed for?
15:16 < pinchartl> the second thing that bothers me is that there's now a rectangle not bound to any pad, but that's less of an issue
15:16 <@sailus> We have the scaling rectangles and those essentially are what used to do the job of the old compose rectangles.
15:16 < pinchartl> sailus: in your diagram the scaled image is composed onto the final image, right ?
15:17 <@sailus> Uh, depends on what you mean by final image.
15:17 <@sailus> The scaling rectangle will compose the image to the compose rectangle.
15:17 <@sailus> On source pads.
15:18 < pinchartl> that's the only compose operation, right ?
15:18 <@sailus> Cropping can be done to that compose rectangle --- the first rectangle on source pads.
15:18 <@sailus> pinchartl: That's right.
15:18 < pinchartl> so the sink streams are composed, but the composition is controlled by the scaling rectangle
15:18 <@sailus> But as Sylwester pointed out, we might need another one.
15:18 < pinchartl> that sounds a bit counter-intuitive to me
15:18 < tstanisl> sorry.. Ias I see scaling target is used for composing, sdo why not keep calling it compose target?
15:18 < pinchartl> on the other hand, you can argue that controlling scaling through a compose rectangle would be counter-intuitive as well :-)
15:19 < pinchartl> tstanisl: my point exactly
15:19 <@sailus> I think scaling is far more common than composition.
15:19 <@sailus> That's why I called the scaling rectangles such.
15:19 <@sailus> So if the hardware doesn't do composition, as most do not, there's just the scaling rectangle.
15:20 <@sailus> --- Assuming it will do scaling.
15:20 <@sailus> We needed a name for the internal coordinate system the scaled images are composed onto.
15:20 < pinchartl> do I understand it right that what you call the sink scaling rectangle controls both scaling and composition ?
15:20 <@sailus> So I called that the composition window instead.
15:20 < tstanisl>  why not use: #define V4L2_SEL_TGT_SCALE_ACTIVE V4L2_SEL_TGT_COMPOSE_ACTIVE
15:21 <@sailus> pinchartl: It controls scaling, and if the hardware supports composition, composition as well.
15:21 <@sailus> tstanisl: Because the two are different.
15:21 < pinchartl> sailus: and if the hardware supports composition and not scaling, it will control composition only :-)
15:21 <@sailus> pinchartl: True.
15:22 < pinchartl> so this is just a matter of names, right ? we have two actions that are controlled by a single rectangle, and we don't know how to call it
15:22 < pinchartl> is that correct ?
15:22 <@sailus> I'd like not to call it TGT_SCALING_OR_IF_NOT_THEN_PROBABLY_AT_LEAST_COMPOSITION_ACTIVE.
15:22 < tstanisl> ok.. but what is the difference between scaling 100x100 to 200x200 and composing 100x100 into 200x200@0,0 ?
15:24 <@sailus> tstanisl: If the hardware doesn't support composition, there is no compose rectangle (referring to the diagram).
15:24 <@sailus> For configuring the scaling / composition interface-wise for that exact rectangle, nothing.
15:24 < pinchartl> could we first discuss what rectangles we need, and then device on names ?
15:25 < pinchartl> s/device/decide/
15:25 <@sailus> pinchartl: Agreed. First things first.
15:25 < tstanisl> ok.. if device does not support composing then the driver always keeps {left,top} at 0,0
15:25 < pinchartl> let's go from left to right then
15:25 < tstanisl> for compose rectangle
15:26 < tstanisl> ok
15:26 < snawrocki> that would be good, TBH the pink rectangle is making things hard to follow (at least for me)
15:26 < pinchartl> sink format and sink crop are pretty much straightforward. any comment on them ?
15:26 < tstanisl> order of configuration?
15:27 <@sailus> Order of configuration is always from sink to source.
15:27 < snawrocki> format/crop ?
15:27 < tstanisl> fmt influences the crop, doesn't it?
15:27 <@sailus> tstanisl: It does.
15:27 < tstanisl> ok
15:28 < pinchartl> sink format must be set first, then sink crop
15:28 < pinchartl> no more comment about those ?
15:28 <@sailus> The sink scaling rectangle defines scaling related to the sink crop rectangle.
15:28 < tstanisl> what is crop_bounds?
15:28 <@sailus> tstanisl: What crop_bounds?
15:29 < tstanisl> there is V4L2_SEL_TGT_CROP_BOUNDS target
15:29 < snawrocki> Doesn't just format limit the crop rectangle ?
15:29 < tstanisl> how it it defined?
15:29 < tstanisl> does't it include any blanking lines?
15:30 < pinchartl> on the sink pad, I believe that V4L2_SEL_TGT_CROP_BOUNDS will be equal to the sink format in most cases
15:30 <@sailus> My thinking is that the active rectangle must fit inside the bounds rectangle, whether or not it is crop.
15:30 < pinchartl> however, in some cases, the hardware might need to crop a minimum number of lines and columns
15:30 <@sailus> The maximum size of the active rectangle may be smaller than bounds rectangle.
15:30 < pinchartl> so V4L2_SEL_TGT_CROP_BOUNDS could be smaller than the sink format
15:30 < tstanisl> what do you think about following definition 'crop bounds is the smallest rectagnle that contains all valid crop rectangles'
15:31 < pinchartl> sounds very good to me
15:31 <@sailus> Ack.
15:31 < pinchartl> so sink format >= V4L2_SEL_TGT_CROP_BOUNDS >= V4L2_SEL_TGT_CROP_ACTIVE
15:31 < tstanisl> notice that crop bounds may not be equal to sink format, is it ok?
15:31 < pinchartl> in most case sink format == V4L2_SEL_TGT_CROP_BOUNDS
15:31 < snawrocki> tstanisl: yes, makes sense
15:31 < pinchartl> tes
15:31 < pinchartl> yes
15:31 < tstanisl> ok.. agreed
15:32 <@sailus> Are we ready for scaling now, or are there further questions? :-)
15:32 < tstanisl> let's continue
15:32 <@sailus> The sink scaling rectangle defines scaling related to the sink crop rectangle.
15:32 <@sailus> (I don't want to go composition yet.)
15:33 <@sailus> Any questions on that?
15:33 < tstanisl> is scaling origin always at {0,0}?
15:33 <@sailus> Yes.
15:33 < tstanisl> what are scaling bounds?
15:34 <@sailus> We just cropped, so if cropping is desired then that must be specified in sink crop rectangle.
15:34 < pinchartl> scaling origin is {0,0}/{sink crop width x sink crop height}
15:34 < pinchartl> scaling target is {left,top}/{sink scaling width x sink scaling height}
15:34 < pinchartl> (left and top will be defined in a moment when we'll discuss composition)
15:34 < pinchartl> is that correct ?
15:35 <@sailus> pinchartl: My thinking as well.
15:35 < tstanisl> by origin I meant {left, top}
15:35 <@sailus> Are we ready for composition now?
15:35 < tstanisl> do you allow scaling {top,left} be different than {0,0}
15:35 < tstanisl> ?
15:36 < pinchartl> sailus: you should probably add points to your diagram to define the various origins, and name them with a letter, to make it easier to refer to them in the documentation
15:36 < pinchartl> tstanisl: are you talking abuot the sink scaling target ?
15:36 <@sailus> pinchartl: Good point.
15:36 < tstanisl> pinchartl: yes
15:36 <@sailus> tstanisl: In composition, yes, which we're about to discuss soon. :-)
15:37 <@sailus> If the hardware only does scaling, left / top must be zero always.
15:37 < pinchartl> the sink scaling rectangle is a hybrid rectangle
15:37 < pinchartl> its *size* defines scaling
15:37 < pinchartl> its *origin* defines composing
15:38 < snawrocki> right
15:38 < tstanisl> no
15:38 <@sailus> tstanisl: No?
15:38 < tstanisl> ratio crop size / scaling size  defines scaling
15:39 < tstanisl> in RFC for selection it was told that scaling is controlled by crop size / compose size
15:39 <@sailus> tstanisl: I think we're talking about the same but are using a bit different expressions.
15:39 < pinchartl> tstanisl: the green rectangle, called sink scalling rectangle on the diagram, is the sink compose rectangle
15:39 < pinchartl> it's just two different names for the same rectangle
15:39 < tstanisl> ok
15:39 < pinchartl> we'll talk about names a bit later, I want to discuss features first
15:40 < tstanisl> ok
15:40 < tstanisl> let's continue
15:40 <@sailus> In composition the subdev has multiple sink pads.
15:40 <@sailus> Several streams enter the subdev in this case, if I'm not severely mistaken. :-)
15:40 <@sailus> Each stream has its own format, crop and scaling configuration.
15:41 <@sailus> The scaled streams end up to the compose rectangle which sometimes has also been referred as "subdev's internal coordinate system".
15:42 <@sailus> (How it's exactly determined which stream will be top of which is to be determined separately.)
15:43 <@sailus> (As well as what will the area with no stream contain.)
15:43 < pinchartl> right
15:43 <@sailus> The compose rectangle is specific to subdev.
15:43 <@sailus> Not bound to any pad.
15:44 < tstanisl> ?
15:44 < tstanisl> why not to any pad?
15:45 < pinchartl> sailus: are you talking about the pink/salmon rectangle ?
15:45 < tstanisl> yes, what is it?
15:45 < pinchartl> that's the rectangle onto which the input streams are composed
15:45 < tstanisl> I mean: what does it mean in HW context?
15:46 < snawrocki> how about associating it with source pad, since everything ends up an the source pad ? 
15:46 < tstanisl> so it is compose bounds
15:46 < snawrocki> s/an/at
15:46 <@sailus> pinchartl: Yes.
15:46 < tstanisl> is the pink rectangle read only?
15:46 <@sailus> If we associate the compose rectangle to a source pad, we only can have one source pad.
15:47 <@sailus> I could imagine there may be several.
15:47 < pinchartl> sailus: I think we can say that it's a compose bounds rectangle, right ?
15:47 <@sailus> It used to be called like that.
15:47 <@sailus> We could do the same.
15:47 < pinchartl> in a single-source subdev, the compose bounds rectangle can be associated with the source pad
15:48 < pinchartl> but if the subdev has multiple source pads, that becomes problematic
15:48 <@sailus> But the difference to V4L2 is that the compose bounds rectangle here is changeable, not static.
15:48 < tstanisl> BTW: is there any need to read/set this pink rectangle?
15:48 <@sailus> stanisl: Certainly there is a need to get / set it.
15:49 < tstanisl> sailus: when?
15:49 <@sailus> Well, you might want to know what's the size of the image you can expect to get from the source pad of the subdev.
15:49 < snawrocki> tstanisl: applications may want to find out what the valid rectangles can be
15:49 <@sailus> And how to configure your scaling rectangles.
15:49 < tstanisl> use crop bounds on source pad to get it
15:50 < pinchartl> tstanisl: what if there are multiple source pads ?
15:51 < tstanisl> pinchartl: what is the purpose of pink rectangle if there are multiple sinks/sources?
15:51 < snawrocki> salmon/pink rectangle might depend somehow on all sink formats, IIUC
15:52 < pinchartl> multiple sink pads isn't an issue
15:52 < pinchartl> the problem comes from multiple source pads
15:52 < pinchartl> let me give you an example
15:52 < pinchartl> let's say we have a subdev with 4 sink pads
15:52 < tstanisl> pinchartl: ok
15:52 < pinchartl> it composes the 4 video streams onto an image
15:53 < pinchartl> let's further assume that the subdev has two source pads
15:53 < pinchartl> they both output with same composed stream
15:53 < pinchartl> with optional cropping
15:53 < tstanisl> pinchartl: images are composed using scaling target, right?
15:53 < pinchartl> tstanisl: yes
15:53 < tstanisl> ok
15:53 < pinchartl> in this case, how do you set the size of the image onto which the 4 streams are composed ?
15:54 < tstanisl> by setting their sink scaling tagets?
15:55 < snawrocki> we're talking about source pads
15:55 < tstanisl> pinchartl: the image described by pink rectangle may only be a virtual entity
15:55 <@sailus> tstanisl: That configures the scaling for a single stream originating from one pad, but does not configure what is the size of the salmon rectangle.
15:55 < pinchartl> tstanisl: the pink rectangle is a virtual rectangle, yes. but it's still needed
15:56 < tstanisl> ok.. but I still do not see any purpose of pink rectangle, the sum of source crop bounds can be used for that
15:56 <@sailus> tstanisl: Not quite.
15:56 <@sailus> tstanisl: What rectangle would you refer to when configuring source crop?
15:57 < tstanisl> if the pink rectangle is a virtual entity the it probably has no boundaries. It is only some virtual, internal coordinate system hidden inside the subdev
15:57 < tstanisl> dashed violet
15:57 < pinchartl> tstanisl: the pink rectangle is a boundary
15:58 <@sailus> tstanisl: It still may have boundaries. Depends on hardware, I presume.
15:58 < pinchartl> and it needs to be configurable
15:58 < pinchartl> let's go back to my example, I'll make it more precise
15:58 < pinchartl> the 4 video streams are all 200x200
15:59 < pinchartl> I want to compose them on a 1000x1000 image, at (0,0), (500,0), (0, 500) and (500,500)
15:59 < pinchartl> and then output on source 0 the 1000x1000 image cropped to (100,100)/800x800
15:59 < tstanisl> ok
16:00 < pinchartl> and on source 0 the 1000x1000 image cropped to (200,200)/600x600
16:00 < pinchartl> what rectangles do you set, and in which order ?
16:00 < tstanisl> I assume that source crop bounds are 1000x1000@0,0 for both sources.
16:02 < tstanisl> IMO, the order is not important. Am I wrong?
16:02 <@sailus> tstansl: The order matters.
16:02 <@sailus> stanisl: The subdev drivers by default propagate any changes towards the source pads.
16:02 < pinchartl> wait
16:03 < pinchartl> tstanisl: the 1000x1000 size is configurable
16:03 < pinchartl> tstanisl: do you *set* the source crop bounds to 1000x1000 ?
16:03 < tstanisl> pinchartl: why should it be?
16:03 < pinchartl> tstanisl: because the hardware allows it to be configurable
16:04 < tstanisl> no.. the source c rop bounds is configured by the driver while setting sink compose (aka scaling) rectangles
16:04 < tstanisl> according to pipeline configuration rules
16:04 < tstanisl> by FIXED I mean that you cannot use VIDIOC_S_SELECTION to change the bounds target
16:05 < pinchartl> tstanisl: is the source crop bounds rectangle you refer to the pink rectangle on the diagram ?
16:05 < tstanisl> but it may be changed by properies higher in the pipeline
16:05 < tstanisl> pinchartl: yes, if all sources have the same cropping capabilities
16:06 < pinchartl> you can VIDIOC_G_SELECTION() on it, right ?
16:06 < tstanisl> pinchartl: I am afraid that it may be even higher if HW allows to crop undefined pixels:)
16:06 < tstanisl> subdev equivalent of VIDIOC_G_SELECTION
16:07 < tstanisl> but driver must support it 
16:07 < pinchartl> let's assume it works that way
16:07 < pinchartl> I will set the 4 compose/scaling green rectangles
16:08 < pinchartl> to (0,0)/200x200, (500,0)/200x200, (0,500)/200x200 and (500,500)/200x200,
16:08 < pinchartl> what will the driver return if I VIDIOC_G_SELECTION(source crop bounds) then ?
16:09 < tstanisl> it depends on HW capabilities
16:09 < tstanisl> if HW cannot consist only of udefined pixels then:
16:10 < tstanisl> the crop bounds should be 700x700@0,0
16:11 < tstanisl> hmm.. is information about pink rectangle useful for the driver?
16:11 < pinchartl> I get your point now
16:11 < pinchartl> sailus: we probably don't need the pink rectangle
16:12 <@sailus> pinchartl: I'm beginning to think the same way.
16:12 <@sailus> It's just that we need to be able to specify a few things.
16:12 < pinchartl> it can just be thought as having the biggest size the hardware supports
16:12 <@sailus> - What are the limits for scaling rectangles (bounds rectangle, perhaps)
16:13 <@sailus> - What do source crop rectangles refer to?
16:13 <@sailus> I think that's what I was thinking of when I introduced the compose rectangle. :-)
16:14 <@sailus> (And renamed the old compose rectangle as scaling rectangle.)
16:14 <@sailus> So, in a typical use case the bound rectangle for sink compose and source crop would be the same, right?
16:15 < pinchartl> yes
16:15 < pinchartl> well
16:15 < tstanisl> sailus: may not
16:15 < pinchartl> hmmmm...
16:15 <@sailus> And they  are static, dependent on hardware?
16:15 < pinchartl> no :-)
16:15 <@sailus> No? :-)
16:15 < pinchartl> there are several reasons why they can be different
16:15 < tstanisl> the are dependant on HW and configuration of blocks higher in the pipeline
16:16 < tstanisl> so the are not static
16:16 < pinchartl> first, the hardware could compose on a larger image than the scaler maximum output size
16:16 <@sailus> pinchartl: That's right.
16:16 < pinchartl> then, depending on the hardware, you could compose your images partly outside of the crop bounds. the hardware will then clip the result
16:17 <@sailus> pinchartl: But it's not to say that the compose bounds rectangle wouldn't be even larger than this larger image you're talking about.
16:17 < tstanisl> notice that source crop bounds are not fixed until source compose configuration is finished 
16:17 < pinchartl> tstanisl: source compose ?
16:18 <@sailus> pinchartl: It's true that the compose bounds may be larger than crop bounds.
16:18 < tstanisl> s/source/sink
16:18 < tstanisl> sorry
16:18 < tstanisl> s/source compose/sink compose 
16:18 < tstanisl> :)
16:19 < pinchartl> tstanisl: why aren't the source crop bounds fixed before the sink compose rectangles are configured ?
16:19 < tstanisl> due to pipeline configuration rules
16:19 < tstanisl> consiguration goes from sink to source
16:19 <@sailus> tstanisl: That's a property of hardware, isn't it?
16:20 <@sailus> It's not configurable. That was my understanding of the discussion.
16:20 < tstanisl> no.. IMO opinion it should be properdy of Media API
16:21 < pinchartl> hmmm...
16:22 < pinchartl> I'm not sure to agree here
16:22 < tstanisl> it was stated that changes should propagate down the pipeline
16:22 < pinchartl> if the hardware can compose onto an image of at most 2000x2000 pixels, why would the source crop bound be reported with a different value ?
16:23 < pinchartl> i.e. in which use case would we need to report source crop bounds that depend on other rectangles, instead of a hardcoded value based on the hardware capabilities ?
16:23 < tstanisl> hmm.. some hardware not not allow to clip images that were already composed 
16:23 <@sailus> pinchartl: the hardware might not allow drawing to the whole area that comes out of the source pad. Unlikely, but possible.
16:24 < tstanisl> it would be better to say... bounds may not be fixed
16:24 <@sailus> tstanisl: In what kind of a case the bounds could then change?
16:25 < pinchartl> or the size could depend on the input format... (I'll just crop my fingers to never run into subdevs where the source crop bounds depends on the source format)
16:25 < tstanisl> sailus: ok .. the simplest case
16:25 < tstanisl> setting format on sink changes sink crop bounds
16:26 <@sailus> tstanisl: That's right.
16:26 <@sailus> tstanisl: But how about compose bounds then?
16:26 < pinchartl> sailus: what compose bounds ?
16:26 < tstanisl> sailus: this is the simple case of propagation that changes bounds rectangles
16:27 <@sailus> pinchartl: Assume the original proposal --- bounds the hardware places for composition.
16:27 < tstanisl> if the HW does not support scaling then sink's compose bounds is always equal to crop_size@ 0,0
16:28 <@sailus> Uh, does the hardware need to support scaling to support composition? I don't think so?
16:28 <@sailus> Part of the image may also be undefined, or filled with a background colour.
16:28 < tstanisl> sailus: no
16:29 < tstanisl> subdev may be able to compose images from multiple sinks, without saling support
16:29 < tstanisl> scaling
16:29 < pinchartl> tstanisl: isn't the sink compose bound the smallest rectangle that includes all the valid sink compose rectangles ?
16:30 < tstanisl> pinchartl: it should be. But there might be some issues with that :)
16:30 < pinchartl> thinking about it, there are issues
16:31 < tstanisl> pinchartl: explain
16:31 < tstanisl> please 
16:31 < pinchartl> let's say the pink rectangle is 2000x2000
16:31 < pinchartl> but the scalers can only upscale by up to 4
16:31 < pinchartl> if the sink crop is 200x200
16:32 < pinchartl> the green rectangle, after scaling, will be 800x800 at most
16:32 < pinchartl> but can be placed anywhere in the 2000x2000 pink rectangle when composing
16:32 < pinchartl> so what should the sink scaling/composing bounds be ? 800x800 or 2000x2000 ?
16:32 <@sailus> pinchartl: 2000x2000.
16:33 <@sailus> That's the bounds rectangle where the images can be composed.
16:33 <@sailus> It's true that it doesn't explicitly tell the maximum scaling ratio.
16:33 <@sailus> But is that needed?
16:33 < tstanisl> pinchartl: I agree 2000x2000 contains all valid sink's compose recatngles
16:34 <@sailus> The user may, if (s)he chooses, use the try target to see what is the actual size (s)he's get.
16:34 < tstanisl> sailus: you can use VIDIOC_TRY_SELECTION to get limits for scaling rations
16:34 < pinchartl> sailus: agreed, the try target can be used for that
16:35 <@sailus> tstanisl: SUBDEV_S_SELECTION with try target; it keeps the other try rectangles specific to the file handle so the user can try the whole configuration, not just a single rectangle.
16:36 < tstanisl> sailus: ok
16:37 < tstanisl> sailus: my intuition says that there might be still issues with SUBDEV_S_SELECTION and try target, but let's better skip this for now
16:37 <@sailus> Are there further questions? :-)
16:37 < tstanisl> let's continue
16:38 <@sailus> It looks like we have rough consensus here unless someone brings up something that explodes this again. ;)
16:38 < pinchartl> it looks good to me
16:38 < pinchartl> but we should really try detailed use cases when writing the proposal
16:39 <@sailus> So, I think the conclusion is that the proposal before my last one is good recarding this part, at least.
16:39 < tstanisl> I expect that we skip source compose targets for now?
16:39 <@sailus> Yeah, more documentation is needed.
16:39 <@sailus> We haven't gotten that far yet.
16:39 <@sailus> So we have no selection rectangles, and compose rectangles are back.
16:40 <@sailus> Or, rather, sink compose rectangle.
16:40 < tstanisl> sorry I missed somethink. Do we need to keep pink rectangle?
16:40 < pinchartl> sailus: no selection rectangle ?
16:40 <@sailus> tstanisl: The pink rectangle is the compose bounds rectangle now.
16:40 < snawrocki> sailus: what do you mean by not having selection rectangles ?
16:40 <@sailus> s/selection/scaling/ :-)
16:41 < snawrocki> don't scratch everything, yet ;)
16:41 <@sailus> At least you're awake. ;-)
16:41 < pinchartl> sailus: we haven't decided on whether to call the green rectangle compose or scaling yet
16:41 < snawrocki> yes, I was going to propose 5 min break..
16:42 <@sailus> 10 minute break?
16:42 < tstanisl> ok .. 10 minutes :)
16:42 < pinchartl> ok
16:42 < snawrocki> Sounds good, thanks.
16:53 <@sailus> Ping?
16:54 < snawrocki> I'm back
16:54 < snawrocki> Tomasz is contemplating some drawings at a whiteboard.. :D
16:56 < tstanisl> pong
16:57 < tstanisl> I noticed a flaw in selection API for subdevs. It is great for complex and sofisticated device. But very problematic for simple devices :)
16:57 <@sailus> pinchartl: ping?
16:57 <@sailus> tstanisl: How so?
16:57 < tstanisl> the same functionality can be implemented in many was by selection API 
16:57 < tstanisl> s/was/ways
16:58 <@sailus> For example?
16:58 < tstanisl> a simple scaler
16:58 < tstanisl> scaling can be controlled by sink crop / sink compose ratio
16:59 < tstanisl> or source crop / source (compose or format)
16:59 <@sailus> We've agreed on scaling (or compose) rectangle on sink pads so far.
16:59 <@sailus> So there's only a single place for scaling configuration.
16:59 < pinchartl> pong
16:59 < pinchartl> sorry for the delay
16:59 < tstanisl> ok.. so it must explicitly stated in the spec
16:59 <@sailus> Welcome back, all!
17:00 < tstanisl> hmm it indicates that source crop == crop format in all cases
17:00 < pinchartl> tstanisl: yes, it needs to be told explicitly in the spec
17:00 < pinchartl> tstanisl: yes, they're always ==
17:01 <@sailus> I think we can now say we dived to source pad configuration now. :-)
17:02 < tstanisl> hopefully there will never be hardware that can use different scalings for different SOURCE pads :)
17:03 <@sailus> Would it make sense to state that sink composition rectangle should be used to configure scaling if both sink compose and source compose rectangles could be used for the same?
17:04 < pinchartl> sailus: are there source compose rectangles ?
17:04 < tstanisl> as I remeber we should avoid 'source compose'
17:04 <@sailus> pinchartl: In the previous proposal, yes.
17:05 <@sailus> That's something we should likely discuss now.
17:05 <@sailus> But perhaps source crop first.
17:05 <@sailus> We did partially discuss it some time ago; does anyone have something to add to that?
17:05 < snawrocki> hmm, since we can configure composition on just sink pad
17:05 < pinchartl> what are they used for ?
17:06 <@sailus> I'm probably not the best to comment this since I have no such hardware.
17:07 <@sailus> Except --- I think Sylwester had commented something by e-mail previously.
17:07 <@sailus> snawrocki: Would you like to elaborate?
17:08 < tstanisl> pinchartl: source crop could be used for a subdev that split input into to TV signal, one should right side of image, the second left side
17:08 <@sailus> I think it was related to composing into a video buffer.
17:08 < tstanisl> s/to/two
17:08 < pinchartl> tstanisl: sure, source crop is useful. I was asking what source compose was used for
17:10 < tstanisl> pinchartl: if the cropped area would not have to fill the whole display
17:10 <@sailus> tstanisl: I think Sylwester had a use case related to composition into video buffers.
17:10 < tstanisl> but I think that this is a very rare case
17:10 <@sailus> That might not be so rare.
17:10 < snawrocki> I have just one use case for composition
17:11 <@sailus> snawrocki: Go ahead! :-)
17:11 < snawrocki> where the capture frame is composed onto an output buffer
17:11 < snawrocki> there is following set of operations
17:12 < snawrocki> at the subdev which is connected to sensor
17:12 < snawrocki> crop -> scaling -> compose
17:13 < snawrocki> then the question is whre do we configure the rectangle
17:14 < snawrocki> that the frame is compose onto
17:14 < pinchartl> isn't compose handled using the sink compose rectangles then ?
17:14 <@sailus> snawrocki: I guess that is the source pad format.
17:15 < snawrocki> pinchartl: I would think so, after our discussion
17:15 <@sailus> When setting the active compose rectangle, the user would have to provide V4L2_SUBDEV_SEL_FLAG_KEEP_CONFIG to prevent propagating the change to the source format.
17:17 < tstanisl> sailus: this flag indicate that the call is not allowed any other blocks in the pipeline?
17:17 < snawrocki> sailus: do you mean not to set source compose rectangle from sink composition rect. ?
17:17 <@sailus> tstanisl: This flag tells that it must not propagate any changes inside the subdev.
17:18 <@sailus> snawrocki: There's source crop rectangle in between the two.
17:18 < snawrocki> ah, ok
17:18 < pinchartl> is there a source compose rectangle ?
17:18 < snawrocki> I'm just trying to clarify what the propagation means here
17:18 <@sailus> pinchartl: Think of a case where the user wants to compose the image into a video buffer which is larger than the image.
17:19 < tstanisl> could we use some example with some numbers?
17:19 <@sailus> Yes.
17:19 < pinchartl> sailus: I still don't get it
17:20 < snawrocki> sailus: not sure if we can assume that on source pad
17:20 <@sailus> snawrocki: Assume what?
17:20 < snawrocki> that source compose rectangle can be greater than source format ?
17:21 <@sailus> snawrocki: It can't be.
17:21 <@sailus> The rectangle you'd be composing onto, is the rectangle defined by the source pad format.
17:21 < snawrocki> ok, now I see, sorry
17:21 <@sailus> I'm trying to present a practical example.
17:22 <@sailus> The user has configured the active source crop rectangle as 0,0/200x200.
17:23 <@sailus> The size of the source format, the pad of which is connected to a video node, is 500x500.
17:23 <@sailus> What the user wants to do, is to write the image at location 100,100 on the video buffer.
17:24 <@sailus> With upscaling to size 300x300.
17:24 <@sailus> To do that, (s)he would set the source active compose rectangle as 100,100/300x300.
17:25 <@sailus> snawrocki: Does this make sense to you?
17:25 < pinchartl> sailus: not to me :-)
17:25 < pinchartl> why don't we do that on the sink pad instead ?
17:25 <@sailus> The rest of the pixels in the video buffer would not change.
17:26 < pinchartl> sink crop 0,0/200x200, sink scaling/compose 100,100/300x300, source crop 0,0/500x500, source format 500x500
17:26 <@sailus> pinchartl: Good point.
17:27 < snawrocki> sailus: Laurent's version is much more straightforward
17:27 <@sailus> Right.
17:27 <@sailus> So, should I understand the conclusion is that composition is not needed on source pads at all?
17:28 <@sailus> Or at least in this case.
17:28 < pinchartl> I think so
17:28 < pinchartl> at least I don't see a need for that, as we can do it on the sink pad anyway
17:28 <@sailus> It might be another matter if there were two sink pads which are both connected to video nodes, but that could be represented as two subdevs as well.
17:28 < pinchartl> regarding composing onto a memory buffer, it might even make sense to configure composition on the video node
17:29 <@sailus> pinchartl: I haven't thought how that'd work exactly.
17:29 <@sailus> But I think it could be done using the subdev selection API as well.
17:29 < snawrocki> pinchartl: we talked about this with Tomasz
17:29 <@sailus> Ok.
17:30 < snawrocki> i.e. using VIDIOC_?_SELECTION on video node instead
17:30 <@sailus> But I think this topic is a little bit out of scope for now, which way to configure it.
17:30 < pinchartl> snawrocki: yes
17:31 <@sailus> As far as I understand, we now agree that compose rectangle is not needed on source pads.
17:31 < pinchartl> composing in that case will likely be done by the DMA engine. at first sight (but I haven't thought about it much), it looks to me like the selection API on the video node would be a good target for this
17:31 < snawrocki> sailus: yes, I think so
17:31 < pinchartl> but I would need to think about it a bit more
17:31 < pinchartl> sailus: agreed
17:32 <@sailus> As the API spec will say, implementing any non-existent rectangle will be skipped by the drivers, so we could add it later on if it proves useful for something we can't think of now.
17:32 <@sailus> But that's for later in any ase.
17:32 <@sailus> s/ase/case/
17:32 < snawrocki> we would have to define interaction between both subdev and video node Selections
17:33 <@sailus> snawrocki: Could we try to do that another time?
17:33 <@sailus> We still have one topic left, which is naming.
17:33 < snawrocki> sailus: sure, I need to think about it more myself..
17:34 < tstanisl> sailus: I have a simple rule for pad vs video node iteractions
17:34 <@sailus> snawrocki: I agree it should be defined. The general rule is that the subdev interface must implement what is necessary to provide V4L2 interface using a user space library, i.e. libv4l2.
17:35 <@sailus> So... on naming.
17:35 < tstanisl> for capture devices sink crop = selection crop, sink compose = selection compose
17:35 <@sailus> I'm actually proposing to keep the names unchanged from the proposal I had before the last one, where I introduced the scaling rectangle.
17:36 < snawrocki> I's probably best to comment on names directly on RFC/patches, let me check you proposal again
17:37 <@sailus> http://www.retiisi.org.uk/v4l2/tmp/format.eps
17:37 < snawrocki> thx
17:37 <@sailus> http://www.spinics.net/lists/linux-media/msg43014.html
17:38 <@sailus> http://www.spinics.net/lists/linux-media/msg43012.html
17:39 < snawrocki> What do we need V4L2_SUBDEV_SEL_TGT_CROP/COMPOSE_DEFAULT for ?
17:39 <@sailus> Those are the default values for the active rectangles.
17:40 <@sailus> They may depend on the configuration of the other rectangles in the subdev.
17:40 <@sailus> E.g. you might expect that the default for sink crop rectangle is the size of the sink format.
17:41 <@sailus> And that's what you could actually get from default sink crop rectangle.
17:41 <@sailus> What I have just been doing is to mirror functionality of V4L2 selection API here. :-)
17:42 < tstanisl> I think that for now we should skip ddefault targets. The are useful for TV signals where there are blanking lines etc
17:43 <@sailus> I'm fine with skipping the default targets for now.
17:43 <@sailus> Any objections to that?
17:43 < pinchartl> sailus: fine with me
17:43 <@sailus> We can always extend things later on but it's difficult to change once they're in mainline.
17:43 <@sailus> And someone might be using them...
17:43 < tstanisl> I think that for now we sshould skip PADDED targets
17:44 < snawrocki> sailus: yes, I'm for skipping it for the initial version
17:44 <@sailus> tstanisl: I didn't have them in the latest patchset either. :-)
17:44 <@sailus> So we'll just have ACTIVE and BOUNDS targets then.
17:44 < tstanisl> padded target is too enigmatic feature :)
17:44 <@sailus> :-)
17:45 < snawrocki> I can't find them in Sakari's patches anyway :)
17:45 < snawrocki> but I agree, they're probably good only for video nodes
17:46 <@sailus> Any questions on naming, then? :-)
17:47 < tstanisl> no
17:48 < snawrocki> not from me
17:48 <@sailus> pinchartl: ?
17:48 < pinchartl> sailus: nope
17:48 < snawrocki> I guess camera exposure targets are a topic for another meeting ?
17:49 <@sailus> snawrocki: I think so.
17:49 <@sailus> I have to say I'm quite happy with the results of this meeting.
17:49 <@sailus> We succeeded in everything else except keeping the meeting shorther than the last one. :-)
17:49 < pinchartl> lol
17:49 < snawrocki> hy
17:50 < snawrocki> I had a feeling it was shorter anyway :)
17:50 <@sailus> I'll write notes on this and post them to the list.
17:50 <@sailus> Plus change my current patches.
17:51 <@sailus> I want to thank everyone for participating to this meeting and ensuring we'll have a future-proof selection API.
17:52 < snawrocki> sailus: thanks