[Libav-user] Handling a large number of rtsp cameras ( > 100) - image processing

[Strahinja Radman]

On Fri, Feb 4, 2022 at 10:05 AM Francesco Taioli <
taioli.francesco98 at gmail.com> wrote:

> Hi all,
> I have a performance problem regarding image acquisition from a
> large number of cameras (they are greater than 100) in a *python* program.
> Currently, I use cv2 with a ffmpeg pipeline.
>
> *What I want to do*
> I want to do some image processing on those cameras multiple times in a
> minute.
>
> *Current solution - pseudocode*
>
> while True:
>     for every rtsp_cameras_connection_params:
>         *open_connection*
>         read_current_frame # the real time frame
>         process_frame
>         close connection
>
> This solution works, but opening the connection takes a lot of time.
>
> *Why do you open the connection inside the loop, you may ask?*
> As far as I know and I have tried, this is the only solution that allows
> me
> to fetch the LAST frame from a camera (the real time frame, no buffer)
> without
> giving  performance issues for the powerful high-end server that this code
> runs on.
>
> *What do I want?*
> Ideally, this
>
> open_all_the_cameras
>
> while True:
>     for every rtsp_camera_already_opened:
>         read_current_frame # the real time frame
>         process_frame
>
> As you can see, the ideal solution is to open *once* all the cameras
> and save them into an array. Then, at every loop, read the last frame.
>
> *What have I tried?*
>
> Basically every solution. *Multithreading*, *multiprocessing,* and every
> stackoverflow answers. These solutions work only for a limited amount of
> cameras (10-15),
> then the CPU will be 100% on usage (because the threads, in the
> background, are
> always reading the last frame and discarding it if I am not asking the
> last).
>
> There is a way to open the connection, and read the last frame (the real
> time one),
> not the one that I was before (the buffered one)?
>
> Thanks to all. Hope it's clear.
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/libav-user
>
> To unsubscribe, visit link above, or email
> libav-user-request at ffmpeg.org with subject "unsubscribe".
>

Hello Francesco,

If you are using av_read_frame to retrieve a AVPacket* structure it is
nearly cost free to do av_packet_unref() if you don't need that packet.
Decoding is what takes the majority of the time, ie from AVPacket ->
AVFrame. Decoding 100 streams is costly and can be done on a GPU if you can
spare one. You would also need to make sure that on a set interval you are
decoding full GOPs as one packet may not contain full information to decode
a frame that you need. Maybe you can set buffer size in the input
parameters while opening the connection, here is the example for UDP

udp://localhost:8888?overrun_nonfatal=1&buffer_size=524288000?fifo_size=2788766

Regards
Strahinja Radman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20220204/eba49a32/attachment.htm>