[FFmpeg-devel] [PATCH 1/2] test: hlsenc: Don't generate test data with -re

[Michael Niedermayer]

On Thu, Aug 06, 2020 at 10:08:33PM +0300, Martin Storsjö wrote:
> On Mon, 3 Aug 2020, Martin Storsjö wrote:
> 
> > On Mon, 3 Aug 2020, Michael Niedermayer wrote:
> > 
> > > is this removing the only test which tests the "-re" flag ?
> > > if so that would reduce test coverage
> > 
> > Well, tehcnically yes, but this test doesn't actually test whether the
> > "-re" flag does what it's supposed to or anything like that - it only
> > tests that adding the flag doesn't error out and doesn't alter the
> > output. 

yes


> > And it looks very much unintentional here.

maybe


> 
> Do you want to follow up the discussion here? I'd like at least some sort of
> acknowledgement back on the discussion before going forward with it in any
> direction.

sorry for the lack of reply, theres always more to reply to and do then i seem
to have time.

About it not fully testing things, that is of course not great but testing
things halfway is still better than not testing at all.

Also it may make sense given the large number of features we have to test
multiple per test.
For example with 200 features, it would take 200 individual tests to test each
in its own test, but only 10 tests if 4 of these would always test the same 
feature. this of course assumes all kinds of things, its more meant to show
that a bit of "chaos" in what each test tests could actually improve the
amount of information we could obtain from the pattern of failing tests

Thanks

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

During times of universal deceit, telling the truth becomes a
revolutionary act. -- George Orwell
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200807/683f1bf1/attachment.sig>