[FFmpeg-devel] core infrastructure badge for FFmpeg

Michael Niedermayer michael at niedermayer.cc
Wed Jul 6 00:26:59 EEST 2016

On Mon, Jul 04, 2016 at 09:15:27PM -0400, Ganesh Ajjanagadde wrote:
> 04.07.2016, 15:55, "Ronald S. Bultje" <rsbultje at gmail.com>:
> >  Hi,
> >
> >  On Mon, Jul 4, 2016 at 3:44 PM, Ganesh Ajjanagadde <gajjanag at mit.edu> wrote:
> >
> >>   04.07.2016, 15:36, "Ronald S. Bultje" <rsbultje at gmail.com>:
> >>   > Hi,
> >>   >
> >>   > On Mon, Jul 4, 2016 at 3:29 PM, Ganesh Ajjanagadde <gajjanag at mit.edu>
> >>   wrote:
> >>   >
> >>   >> 04.07.2016, 10:33, "Clément Bœsch" <u at pkh.me>:
> >>   >> > On Mon, 4 Jul 2016 at 13:41 Ganesh Ajjanagadde <gajjanag at mit.edu>
> >>   >> wrote:
> >>   >> >> Hi,
> >>   >> >>
> >>   >> >> https://bestpractices.coreinfrastructure.org/.
> >>   >> >>
> >>   >> >> Thoughts on getting this done for FFmpeg?
> >>   >> >
> >>   >> > Any thing we need to adjust in the project? I don't see much things
> >>   to
> >>   >> > change by looking at
> >>   >> https://bestpractices.coreinfrastructure.org/projects/1
> >>   >>
> >>   >> I could not either.
> >>   >> It was something I noticed a week back, judged low-hanging and of some
> >>   >> benefit to the project.
> >>   >> I thus am willing to do it, provided there are no objections.
> >>   >
> >>   > I think if any changes are necessary to the website or documentation or
> >>   > code, these would simply go through the relevant review process, right?
> >>   Or
> >>   > am I missing something?
> >>
> >>   True. At the moment, it seems like the relevant forms can be filled in
> >>   directly from existing information,
> >>   and no changes are necessary.
> >>
> >>   I can send a screenshot of the filled out form before submitting if people
> >>   want.
> >
> >  Oh I see, I misunderstood. I personally don't have objections to that.
> Turns out that the first few pages are easy to fill and we already achieve them.
> The last few are much harder, and FFmpeg does not currently meet all of them.
> Here is the progress so far: https://bestpractices.coreinfrastructure.org/projects/235.
> I have filled them to the best of my ability.
> Here is a summary of where we fail to meet the criteria,
> and some suggestions for what we could do.
> The "MUST" constraints are the ones where FFmpeg needs to improve,
> assuming the project wishes to get to 100%.

> 1. The project MUST have evidence that such tests are being added in the most recent major changes to the project.
> Suggestion: enforce tests for all new filters, muxers, etc. At least 6 months back, this was not enforced for lavfi.

The details for this reads "Major functionality would typically be mentioned in the ChangeLog. (Perfection is not required, merely evidence that tests are typically being added in practice.) "

diffstat between 3.0 and 3.1 of make fatelist shows
fatelist-3.1 |  281 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 275 insertions(+), 6 deletions(-)

so there are 269 more tests in 3.1 than 3.0

limiting this to filters shows
grep filter fatelist-3.0> fatelist-3.0-filter
grep filter fatelist-3.1> fatelist-3.1-filter
diff -u fatelist-3.0-filter fatelist-3.1-filter | diffstat
 fatelist-3.1-filter |   25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

we have 15 filters in the 3.0->3.1 changelog if i didnt miscount

so i would say we are maybe a bit behind with adding tests but they
do get typically added eventually even for libavfilter.
No question, it would be better if tests would be added quicker ...


> 5. The project security mechanisms MUST use default keylengths that at least meet the NIST minimum requirements through the year 2030 (as stated in 2012).
> Suggestion: add --enable-unsafe-crypt to configure, and by default not enable it.
> Change API's and functions accordingly, document it.
> Note: strictly speaking, per the sentence above, it is ok as we do not really have "default" keylengths.
> But the extended rationale shows why we fail.

"For example, FFmpeg's DES API accepts 64 bit keys, and there is no configuration option for disabling this."

The low level DES API may be part of a security mechanism, for example
3DES used for encrypting something which is having sufficiently sized
keys. Also the low level APIs are used for format interoperability
outside any "security mechanisms"

> 6. The default project security mechanisms MUST NOT depend on cryptographic algorithms that are broken
> (e.g., MD4, MD5, single DES, RC4, or Dual_EC_DRBG).
> Suggestion: same as 5 above.

"For example, FFmpeg provides a DES API, and there is no configuration option for disabling this."

the point above speaks about the "default", i would argue a user using
the DES API explicitly has made a choice and is not using a default

> 7. The project security mechanisms SHOULD NOT by default depend on cryptographic algorithms with known serious weaknesses (e.g., SHA-1).
> Suggestion: same as 5 above.

> 8. At least one static code analysis tool MUST be applied to any proposed major production release of the software before its release,
> if there is at least one FLOSS tool that implements this criterion in the selected language.
> Suggestion: force the use of coverity before release. AFAIK, we do not clean this up fully before releases.

we do run coverity around the release time generally, the point does
not seem to require strictly fixing every warning it finds, we fix most

more regular testing and testing with more tools would be a very good
Some tools though are very verbose and the warnings produced are of
rather low quality, such tools should be avoided as they would
misdirect manhours ...

> 9. All medium and high severity exploitable vulnerabilities discovered with static code analysis MUST be fixed in a timely way after they are confirmed.
> Suggestion: this is related to 8, same idea.

which "exploitable vulnerabilities discovered" have not been fixed ?

> 10. It is SUGGESTED that static source code analysis occur on every commit or at least daily.
> Suggestion: ignore, or if we have the resources and it is deemed worthy, get more regular coverity reports.

> 11. It is SUGGESTED that at least one dynamic analysis tool be applied to any proposed major production release of the software before its release.
> Suggestion: agree upon some fuzzer, and run it on a variety of inputs before release. Can be a fate-fuzz.

There are some researchers which regularly check FFmpeg with fuzzers
this does not match our release shedule though

> 12. It is SUGGESTED that if the software is application-level software written using a memory-unsafe language (e.g., C or C++)
> then at least one dynamic tool (e.g., a fuzzer or web application scanner)
> be routinely used with a mechanism to detect memory safety problems such as buffer overwrites.
> Suggestion: AFAIK this is not regular, simply get someone to regularly run a fuzzer, especially before release.
> 13. All medium and high severity exploitable vulnerabilities discovered with dynamic code analysis MUST be fixed in a timely way after they are confirmed.
> Suggestion: Force the fixing of fuzzed problems before release, make this part of release checklist.
> No idea if we have the resources to enforce this without delaying a release too much.
> There are some others where I lack knowledge, and either have not filled yet or am not satisfied with my response.
> Help would be greatly appreciated.
> 14. The project MUST have at least one primary developer who knows how to design secure software.
> I marked this as yes, and gave Nicholas and Michael as examples.
> I apologize to any who are offended by it.
> Alternative wording would be nice.
> 15. The project SHOULD implement perfect forward secrecy for key agreement protocols
> so a session key derived from a set of long-term keys cannot be compromised if one of the long-term keys is compromised in the future.
> No idea

> 16. If passwords are stored for authentication of external users, the project MUST store them as iterated hashes
> with a per-user salt by using a key stretching (iterated) algorithm (e.g., PBKDF2, Bcrypt or Scrypt).
> No idea

I dont think we store any "passwords for authentication of external users"

> 17. There MUST be no unpatched vulnerabilities of medium or high severity that have been publicly known for more than 60 days.
> Do we guarantee this?

I think we fix public vulnerabilities much quicker than 60 days in

> 20. (Future criterion) It is SUGGESTED that the project have a reproducible build. 
> AFAIK we can do this easily by fixing on a particular toolchain,
> unless I am missing something.

I think andreas worked on reproducible builds as required by debian
for something, thus i would assume that does work
this should be in the discussions around the src/ directory IIRC

Thanks alot for your work!


Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Breaking DRM is a little like attempting to break through a door even
though the window is wide open and the only thing in the house is a bunch
of things you dont want and which you would get tomorrow for free anyway
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160705/229fe5ba/attachment.sig>

More information about the ffmpeg-devel mailing list