[FFmpeg-devel] [PATCH v2 1/1] avformat: Add IPFS protocol support.

[Michael Niedermayer]

On Tue, Feb 01, 2022 at 10:58:30PM +0100, Mark Gaiser wrote:
> This patch adds support for:
> - ffplay ipfs://<cid>
> - ffplay ipns://<cid>
> 
> IPFS data can be played from so called "ipfs gateways".
> A gateway is essentially a webserver that gives access to the
> distributed IPFS network.
> 
> This protocol support (ipfs and ipns) therefore translates
> ipfs:// and ipns:// to a http:// url. This resulting url is
> then handled by the http protocol. It could also be https
> depending on the gateway provided.
> 
> To use this protocol, a gateway must be provided.
> If you do nothing it will try to find it in your
> $HOME/.ipfs/gateway file. The ways to set it manually are:
> 1. Define a -gateway <url> to the gateway.
> 2. Define $IPFS_GATEWAY with the full http link to the gateway.
> 3. Define $IPFS_PATH and point it to the IPFS data path.
> 4. Have IPFS running in your local user folder (under $HOME/.ipfs).
> 
> Signed-off-by: Mark Gaiser <markg85 at gmail.com>
> ---
>  configure                 |   2 +
>  doc/protocols.texi        |  30 +++++
>  libavformat/Makefile      |   2 +
>  libavformat/ipfsgateway.c | 267 ++++++++++++++++++++++++++++++++++++++
>  libavformat/protocols.c   |   2 +
>  5 files changed, 303 insertions(+)
>  create mode 100644 libavformat/ipfsgateway.c
> 
> diff --git a/configure b/configure
> index 5b19a35f59..6ff09e7974 100755
> --- a/configure
> +++ b/configure
> @@ -3585,6 +3585,8 @@ udp_protocol_select="network"
>  udplite_protocol_select="network"
>  unix_protocol_deps="sys_un_h"
>  unix_protocol_select="network"
> +ipfs_protocol_select="https_protocol"
> +ipns_protocol_select="https_protocol"
>  
>  # external library protocols
>  libamqp_protocol_deps="librabbitmq"
> diff --git a/doc/protocols.texi b/doc/protocols.texi
> index d207df0b52..7c9c0a4808 100644
> --- a/doc/protocols.texi
> +++ b/doc/protocols.texi
> @@ -2025,5 +2025,35 @@ decoding errors.
>  
>  @end table
>  
> + at section ipfs
> +
> +InterPlanetary File System (IPFS) protocol support. One can access files stored 
> +on the IPFS network through so called gateways. Those are http(s) endpoints.
> +This protocol wraps the IPFS native protocols (ipfs:// and ipns://) to be send 
> +to such a gateway. Users can (and should) host their own node which means this 
> +protocol will use your local machine gateway to access files on the IPFS network.
> +
> +If a user doesn't have a node of their own then the public gateway dweb.link is 
> +used by default.
> +
> +You can use this protocol in 2 ways. Using IPFS:
> + at example
> +ffplay ipfs://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> + at end example
> +
> +Or the IPNS protocol (IPNS is mutable IPFS):
> + at example
> +ffplay ipns://QmbGtJg23skhvFmu9mJiePVByhfzu5rwo74MEkVDYAmF5T
> + at end example
> +
> +You can also change the gateway to be used:
> +
> + at table @option
> +
> + at item gateway
> +Defines the gateway to use. When nothing is provided the protocol will first try 
> +your local gateway. If that fails dweb.link will be used.
> +
> + at end table
>  
>  @c man end PROTOCOLS
> diff --git a/libavformat/Makefile b/libavformat/Makefile
> index 3dc6a479cc..4edce8420f 100644
> --- a/libavformat/Makefile
> +++ b/libavformat/Makefile
> @@ -656,6 +656,8 @@ OBJS-$(CONFIG_SRTP_PROTOCOL)             += srtpproto.o srtp.o
>  OBJS-$(CONFIG_SUBFILE_PROTOCOL)          += subfile.o
>  OBJS-$(CONFIG_TEE_PROTOCOL)              += teeproto.o tee_common.o
>  OBJS-$(CONFIG_TCP_PROTOCOL)              += tcp.o
> +OBJS-$(CONFIG_IPFS_PROTOCOL)             += ipfsgateway.o
> +OBJS-$(CONFIG_IPNS_PROTOCOL)             += ipfsgateway.o
>  TLS-OBJS-$(CONFIG_GNUTLS)                += tls_gnutls.o
>  TLS-OBJS-$(CONFIG_LIBTLS)                += tls_libtls.o
>  TLS-OBJS-$(CONFIG_MBEDTLS)               += tls_mbedtls.o
> diff --git a/libavformat/ipfsgateway.c b/libavformat/ipfsgateway.c
> new file mode 100644
> index 0000000000..2b52fc9392
> --- /dev/null
> +++ b/libavformat/ipfsgateway.c
> @@ -0,0 +1,267 @@
> +/*
> + * IPFS and IPNS protocol support through IPFS Gateway.
> + * Copyright (c) 2022 Mark Gaiser
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "libavutil/avassert.h"
> +#include "libavutil/avstring.h"
> +#include "libavutil/internal.h"
> +#include "libavutil/opt.h"
> +#include "libavutil/tree.h"
> +#include "avformat.h"
> +#include <fcntl.h>
> +#if HAVE_IO_H
> +#include <io.h>
> +#endif
> +#if HAVE_UNISTD_H
> +#include <unistd.h>
> +#endif
> +#include <sys/stat.h>
> +#include <stdlib.h>
> +#include "os_support.h"
> +#include "url.h"
> +

> +typedef struct Context {

i also agree that the name is a bit too unspecific


> +    AVClass *class;
> +    URLContext *inner;
> +    char *gateway;
> +} Context;
> +
> +// A best-effort way to find the IPFS gateway.
> +// Only the most appropiate gateway is set. It's not actually requested (http call) to prevent
> +// a potential slowdown in startup. A potential timeout is handled by the HTTP protocol.
> +//
> +// When done and the return value is 1, a potential IPFS Gateway is set in the gateway
> +// variable in the inner URLContext.

> +static int ff_populate_ipfs_gateway(URLContext *h)

ff_ prefix seems not needed


[...]

> +    FILE *gateway_file;
> +    char gateway_file_data[1000];

[...]

> +    if (!gateway_file) {
> +        av_log(h, AV_LOG_ERROR, "Unable to open the IPFS gateway file (full uri: %s).\n", ipfs_gateway_file);

> +        return -1;

more specific error codes would be better and more consistent to existing code


> +    }
> +

> +    fscanf(gateway_file, "%[^\n]", gateway_file_data);

how large is the array ?
how large is the data written into the array ?
what is the relation between the 2 ?

[...]
> +    int ret;
> +
> +    ret = ffurl_read(c->inner, buf, size);
> +
> +    return ret;

if thats all that is needed then 

return ffurl_read(c->inner, buf, size);

is simpler


thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If you drop bombs on a foreign country and kill a hundred thousand
innocent people, expect your government to call the consequence
"unprovoked inhuman terrorist attacks" and use it to justify dropping
more bombs and killing more people. The technology changed, the idea is old.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20220202/dc85087f/attachment.sig>