[FFmpeg-devel] tkhd transformation matrix in mov is ignored except for width/height and scaling

[Jarred Nicholls]

Hey Sam,

That sounds good, I'd like to see the source.  In the meantime I've just
written a script to detect the rotation of the original MOV and to put the
frames through a rotation filter when encoding to its output.  I'm never
outputting to Quicktime format, so preserving the original headers are of no
concern to me - I only wished for the output to be encoded identical to how
Quicktime would decode the movie during playback.

Thanks,
Jarred

On Thu, Nov 12, 2009 at 3:31 PM, Samuel Gendler
<sgendler at ideasculptor.com>wrote:

> I haven't had time to work on a more 'correct' solution, but I'm happy to
> share my kludge, which works quite nicely.  Here's a description:
>
> In the quicktime decoder, I shove the matrix of any track with both height
> and width into a global array (which is otherwise initialized with the
> identity matrix).  In the encoder, I grab the original matrix from the
> global array, parse it to determine if there is any rotation, and then
> insert an identity x rotation matrix.  I don't just copy the matrix across,
> since that might also copy scale information which would not be relevant to
> the new video track. It works nicely on iphone videos rotated in any way,
> and whether they are in the original size or the resized form that is used
> when you mail a video from the iphone.  It would break on a quicktime file
> with multiple video tracks that have different matrices, but I am not aware
> of any way to create such a thing without building a custom tool.
>
> I don't have my modified source handy, but if you'd like a look at it, I'll
> be near a computer that has access to it later this evening.  Let me know
> and I'll forward it along.  I'd rather not just post it straight to the
> list, so that I have some sense of how many people are using my hack job.
>
> --sam
>
>
> On Tue, Nov 10, 2009 at 12:52 PM, Jarred Nicholls <
> jarred.nicholls at gmail.com
> > wrote:
>
> > I would like to continue a previous thread (I was not a part of this list
> until today).  I am experiencing the same problem as Sam mentions below in
> his original post.  I would also like to patch this issue in the "correct"
> and "cleanest" way possible - that is, read the rotation matrix from the
> original .mov and either 1) apply the same matrix metadata into a resulting
> .mov/.mp4 container, OR 2) if the output container doesn't hold a rotation
> matrix metadata, transform the pixels accordingly (perform the rotation)
> during encoding.
> >
> > Any news on this issue?
> >
> > Thanks much,
> > Jarred
> >
> >
> --------------------------------------------------------------------------
> >
> >
> > I've been trying for months to get transcoding an mov via ffmpeg to
> > correctly rotate a video that includes a transformation matrix in the
> tkhd
> > node (iphone video shot in portrait, basically).  I could see that
> various
> > patches about the matrix were discussed on this list, and I couldn't
> figure
> > out why nothing seemed to be functioning even after said patches were
> > applied. The main patch is here:
> > http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-July/049941.html
> >
> > and it was corrected by this patch:
> > http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-June/071622.html
> >
> > In desperation, I finally dug into the source and tried to figure out
> what i
> > was doing wrong.  What was almost immediately apparent is that those
> patches
> > deal only with width and height of the track as well as scaling the width
> > and height.  In fact, the values in the matrix are only stored locally in
> > variables on the stack in the static function that reads the tkhd node.
> The
> > width and height are computed from the matrix and stored in a location
> > accessible from outside the mov_read_tkhd() function, but the rest of the
> > matrix is completely inaccessible once the function returns.
> >
> > I'll admit to being fairly surprised that this is the case, since the
> iphone
> > is an increasingly popular platform for recording video, and the default
> > orientation for the phone in a users hand (vertical - portrait mode) will
> > result in a video that appears rotated counterclockwise by 90 degrees
> when
> > played via libavformat.  More importantly, any video transcoded via
> > libavformat will result in a video that is rotated AND which no longer
> has
> > the transformation matrix values to compensate stored in the tkhd, so
> that
> > even quicktime and the iphone play the video sideways.  This is such a
> > strange state of affairs so long after the release of the iphone, that I
> > have to wonder if I'm reading things incorrectly.
> >
> > Incidentally, a description of the matrix transformation process is here:
> >
> http://developer.apple.com/mac/library/documentation/QuickTime/RM/MovieBasics/MTEditing/K-Chapter/11MatrixFunctions.html
> >
> > I'm an experienced developer, but I know next to nothing about av codecs
> and
> > such.  I could attempt to hack in a fix, but I don't even know where to
> > begin.  Storing the matrix somewhere that is accessible outside
> > mov_read_tkhd() is simple enough.  Running each pixel through the
> transform
> > is also pretty easy.  But I've got no clue where to place that code.
> Could
> > someone maybe point me in the correct direction for where to find code
> that
> > places each pixel in the outbound stream?
> >
> > My alternative is a lame hack of the code in libavformat/movenc.c which
> > writes a default identity matrix into the matrix.  I can copy a rotation
> > matrix into there instead based on a commandline param, but I am loathe
> to
> > do that.
> >
> > Thanks in advance for your help.
> >
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at mplayerhq.hu
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel
>