[FFmpeg-devel] core infrastructure badge for FFmpeg

[Ganesh Ajjanagadde]

04.07.2016, 15:55, "Ronald S. Bultje" <rsbultje at gmail.com>:
>  Hi,
>
>  On Mon, Jul 4, 2016 at 3:44 PM, Ganesh Ajjanagadde <gajjanag at mit.edu> wrote:
>
>>   04.07.2016, 15:36, "Ronald S. Bultje" <rsbultje at gmail.com>:
>>   > Hi,
>>   >
>>   > On Mon, Jul 4, 2016 at 3:29 PM, Ganesh Ajjanagadde <gajjanag at mit.edu>
>>   wrote:
>>   >
>>   >> 04.07.2016, 10:33, "Clément Bœsch" <u at pkh.me>:
>>   >> > On Mon, 4 Jul 2016 at 13:41 Ganesh Ajjanagadde <gajjanag at mit.edu>
>>   >> wrote:
>>   >> >> Hi,
>>   >> >>
>>   >> >> https://bestpractices.coreinfrastructure.org/.
>>   >> >>
>>   >> >> Thoughts on getting this done for FFmpeg?
>>   >> >
>>   >> > Any thing we need to adjust in the project? I don't see much things
>>   to
>>   >> > change by looking at
>>   >> https://bestpractices.coreinfrastructure.org/projects/1
>>   >>
>>   >> I could not either.
>>   >> It was something I noticed a week back, judged low-hanging and of some
>>   >> benefit to the project.
>>   >> I thus am willing to do it, provided there are no objections.
>>   >
>>   > I think if any changes are necessary to the website or documentation or
>>   > code, these would simply go through the relevant review process, right?
>>   Or
>>   > am I missing something?
>>
>>   True. At the moment, it seems like the relevant forms can be filled in
>>   directly from existing information,
>>   and no changes are necessary.
>>
>>   I can send a screenshot of the filled out form before submitting if people
>>   want.
>
>  Oh I see, I misunderstood. I personally don't have objections to that.


Turns out that the first few pages are easy to fill and we already achieve them.
The last few are much harder, and FFmpeg does not currently meet all of them.

Here is the progress so far: https://bestpractices.coreinfrastructure.org/projects/235.

I have filled them to the best of my ability.
Here is a summary of where we fail to meet the criteria,
and some suggestions for what we could do.

The "MUST" constraints are the ones where FFmpeg needs to improve,
assuming the project wishes to get to 100%.

1. The project MUST have evidence that such tests are being added in the most recent major changes to the project.
Suggestion: enforce tests for all new filters, muxers, etc. At least 6 months back, this was not enforced for lavfi.

2. It is SUGGESTED that this policy on adding tests be documented in the instructions for change proposals.
Suggestion: Fix 1 above, and then make it unambiguous in the relevant docs.

3. It is SUGGESTED that projects be maximally strict with warnings, but this is not always practical.
Suggestion: don't bother, I have outlined the rationale on the page.

4. If the project software is an application or library, and its primary purpose is not to implement cryptography,
then it SHOULD only call on software specifically designed to implement cryptographic functions;
it SHOULD NOT re-implement its own. 
Note: This is one where I am personally interested as to why we fail; is it for performance reasons that we reimplement crypto?
Suggestion: No idea until I understand the above.

5. The project security mechanisms MUST use default keylengths that at least meet the NIST minimum requirements through the year 2030 (as stated in 2012).
Suggestion: add --enable-unsafe-crypt to configure, and by default not enable it.
Change API's and functions accordingly, document it.
Note: strictly speaking, per the sentence above, it is ok as we do not really have "default" keylengths.
But the extended rationale shows why we fail.

6. The default project security mechanisms MUST NOT depend on cryptographic algorithms that are broken
(e.g., MD4, MD5, single DES, RC4, or Dual_EC_DRBG).
Suggestion: same as 5 above.

7. The project security mechanisms SHOULD NOT by default depend on cryptographic algorithms with known serious weaknesses (e.g., SHA-1).
Suggestion: same as 5 above.

8. At least one static code analysis tool MUST be applied to any proposed major production release of the software before its release,
if there is at least one FLOSS tool that implements this criterion in the selected language.
Suggestion: force the use of coverity before release. AFAIK, we do not clean this up fully before releases.

9. All medium and high severity exploitable vulnerabilities discovered with static code analysis MUST be fixed in a timely way after they are confirmed.
Suggestion: this is related to 8, same idea.

10. It is SUGGESTED that static source code analysis occur on every commit or at least daily.
Suggestion: ignore, or if we have the resources and it is deemed worthy, get more regular coverity reports.

11. It is SUGGESTED that at least one dynamic analysis tool be applied to any proposed major production release of the software before its release.
Suggestion: agree upon some fuzzer, and run it on a variety of inputs before release. Can be a fate-fuzz.

12. It is SUGGESTED that if the software is application-level software written using a memory-unsafe language (e.g., C or C++)
then at least one dynamic tool (e.g., a fuzzer or web application scanner)
be routinely used with a mechanism to detect memory safety problems such as buffer overwrites.
Suggestion: AFAIK this is not regular, simply get someone to regularly run a fuzzer, especially before release.

13. All medium and high severity exploitable vulnerabilities discovered with dynamic code analysis MUST be fixed in a timely way after they are confirmed.
Suggestion: Force the fixing of fuzzed problems before release, make this part of release checklist.
No idea if we have the resources to enforce this without delaying a release too much.

There are some others where I lack knowledge, and either have not filled yet or am not satisfied with my response.
Help would be greatly appreciated.

14. The project MUST have at least one primary developer who knows how to design secure software.
I marked this as yes, and gave Nicholas and Michael as examples.
I apologize to any who are offended by it.
Alternative wording would be nice.

15. The project SHOULD implement perfect forward secrecy for key agreement protocols
so a session key derived from a set of long-term keys cannot be compromised if one of the long-term keys is compromised in the future.
No idea

16. If passwords are stored for authentication of external users, the project MUST store them as iterated hashes
with a per-user salt by using a key stretching (iterated) algorithm (e.g., PBKDF2, Bcrypt or Scrypt).
No idea

17. There MUST be no unpatched vulnerabilities of medium or high severity that have been publicly known for more than 60 days.
Do we guarantee this?

18. Projects SHOULD fix all critical vulnerabilities rapidly after they are reported. 
As worded, extremely vague as the time frame is unspecified, "rapidly" is vague.
Also no idea why this is separate from the above, they are very related.
Marked it as yes.

19. The public repositories MUST NOT leak a valid private credential
(e.g., a working password or private key) that is intended to limit public access.
I believe the answer is yes, and accordingly marked it. No idea what to put to justify it.

20. (Future criterion) It is SUGGESTED that the project have a reproducible build. 
AFAIK we can do this easily by fixing on a particular toolchain,
unless I am missing something.

21. (Future criterion) The project SHOULD NOT use unencrypted network communication protocols
(such as HTTP and telnet) if there an encrypted equivalent (e.g., HTTPS/TLS and SSH), unless the user specifically requests or configures it. 
No idea about FFmpeg's network code.

22. (Future criterion) The project SHOULD, if it supports TLS, support at least TLS version 1.2. 
Note that the predecessor of TLS was called SSL.
Same as above.

23. (Future criterion) The project MUST, if it supports TLS,
perform TLS certificate verification by default when using TLS, including on subresources.
Same as above.

24. (Future criterion) The project SHOULD, if it supports TLS, perform certificate verification
before sending HTTP headers with private information (such as secure cookies).
Same as above.

25. (Future criterion) It is SUGGESTED that the project website, repository (if accessible via the web),
and download site (if separate) include key hardening headers with nonpermissive values. 
No idea

26. (Future criterion) It is SUGGESTED that hardening mechanisms be used so software defects are less likely to result in security vulnerabilities.
No idea, if/when FFmpeg writes something in e.g Rust, that is easy to include here.
Otherwise, no idea as we want portability - can't simply use seccomp, Capsicum, pledge, etc.
Don't really know if any of this is worth it.

[...]