[FFmpeg-devel] [PATCH v2 2/3] avformat/udp: Fix IP_MULTICAST_TTL for BSD compatibility

[Chad Fraleigh]

Since any [valid] value over 255 is impossible in the IPv4 protocol (the TTL field is only 8-bits), it should always be capped at 255 (for consistency) or return an invalid value error (the one I would suggest).

Despite VLC's reversed comment, using an int seems to be the exception to the rule (i.e. only linux and windows seem to use it [as-documented]), perhaps doing the unsigned char first and using the int as the fallback would be better? It's not really just a BSD thing, unless you also count LWIP and Solaris as BSD. Unless VLC's code history shows them doing it this way at one time and swapping it (but forgetting the comment) to fix a known bug?


On 2/4/2022 9:28 PM, lance.lmwang at gmail.com wrote:
> From: Limin Wang <lance.lmwang at gmail.com>
> 
> Suggested by zhilizhao, vlc project has solved the compatibility by
> the same way, so I borrowed the comments from vlc project.
> 
> Fix #ticket9449
> 
> Signed-off-by: Limin Wang <lance.lmwang at gmail.com>
> ---
>  libavformat/udp.c | 15 +++++++++++++--
>  1 file changed, 13 insertions(+), 2 deletions(-)
> 
> diff --git a/libavformat/udp.c b/libavformat/udp.c
> index 3dc79eb..34488d6 100644
> --- a/libavformat/udp.c
> +++ b/libavformat/udp.c
> @@ -164,6 +164,10 @@ static int udp_set_multicast_ttl(int sockfd, int mcastTTL,
>  {
>      int protocol, cmd;
>  
> +    /* There is some confusion in the world whether IP_MULTICAST_TTL
> +     * takes a byte or an int as an argument.
> +     * BSD seems to indicate byte so we are going with that and use
> +     * int as a fallback to be safe */
>      switch (addr->sa_family) {
>  #ifdef IP_MULTICAST_TTL
>          case AF_INET:
> @@ -183,8 +187,15 @@ static int udp_set_multicast_ttl(int sockfd, int mcastTTL,
>      }
>  
>      if (setsockopt(sockfd, protocol, cmd, &mcastTTL, sizeof(mcastTTL)) < 0) {
> -        ff_log_net_error(logctx, AV_LOG_ERROR, "setsockopt");
> -        return ff_neterrno();
> +        /* BSD compatibility */
> +        unsigned char ttl;
> +
> +        ff_log_net_error(logctx, AV_LOG_DEBUG, "setsockopt");
> +        ttl = (unsigned char)(( mcastTTL > 255 ) ? 255 : mcastTTL);
> +        if (setsockopt(sockfd, protocol, cmd, &ttl, sizeof(ttl)) < 0) {
> +            ff_log_net_error(logctx, AV_LOG_ERROR, "setsockopt");
> +            return ff_neterrno();
> +        }
>      }
>  
>      return 0;