Valkka  0.17.0
OpenSource Video Management
Code walkthrough: rendering

From RTSP stream to X-window.

Let's recall a typical filtergraph from Library architecture

(1.LiveThread) --> {2.FifoFrameFilter} --> [3.FrameFifo] -->> (4.AVThread) --> {5.FifoFrameFilter} --> [6.OpenGLFrameFifo] -->> (7.OpenGLThread)

In detail, frames are transported from (1) to (7) like this:

1 LiveThread

4 AVThread

6 OpenGLFrameFifo

7 OpenGLThread

OpenGLThread uses the following classes:

Internal data structures of OpenGLThread:

OpenGLThread::slots_ == A vector. A SlotContext for each slot. Warehouses SlotContext instances

OpenGLThread::render_groups == Mapping. Window_id => RenderGroup mapping. Warehouses the RenderGroup instances (each RenderGroup instance warehouses RenderContext instances)

OpenGLThread::render_lists == A vector. There is one element for each slot. Each element is a list of references to RenderGroup instances. Only RenderGroup(s) that have some RenderContext(es) should be in this list.

So, OpenGLThread::run reads a new frame from OpenGLThread::infifo. It is passed to the presentation queue, i.e. to OpenGLThread::presfifo. OpenGLThread::handleFifo is eventually called.

OpenGLThread::handleFifo inspects Frame::n_slot => takes a SlotContext from OpenGLThread::slots_. Preload the textures to SlotContext::yuvtex. So, textures are now loaded and ready to go. Now we have to find all render contexes that use these textures (there might be many of them).

Using Frame::n_slot, pick a list from OpenGLThread::render_lists (there is one list for each slot)

Run through that list: each element is a RenderGroup (remember that each RenderGroup has a list of RenderContexes)

RenderGroup has method RenderGroup::render. That method fixes the current X window for manipulation. So, X window has been chosen. Let's draw into it.

Then RenderGroup runs through all its RenderContex(s). For each RenderContext, RenderContext::render is called

RenderContext has vertex array, vertex buffer objects, transformation matrix, etc., everything needed for rendering a YUV image in any way you wish (straight rectangle, twisted, whatever).

RenderContext also has a reference to a valid SlotContext: RenderContext::slot_context. From there we get the Shader and YUVTEX (pre-loaded textures) which are now used together with the vertex array, vertex buffer objects, transformation matrix, etc.

Finally, render the texture with glDrawElements in RenderContext::bindVertexArray.

Recycle frame back to OpenGLFrameFifo