In a previous post, I asked for input on what sort of video metadata FITS should produce, and I’ve gotten a number of responses. Here I’d like to dig in some more and hopefully get more feedback.
We’re talking here primarily about technical metadata. Following the Harvard Library definition, “Technical metadata focuses on how a digital object was created, its format, format-specific technical characteristics, storage and location, etc. Accurate technical metadata helps a repository deliver digital content appropriately to users and to manage digital objects over time and keep them usable.” Other kinds of metadata are important, but the job of FITS is primarily to report what kind of file a digital object is: its format, size, compression method, encoding, etc. Some other kinds of metadata, such as copyright information (rights metadata), are reported, but technical metadata is primary.
I’m interested in a model, not a specific schema or representation. FITS pulls information from many different tools; the question I’m looking into is what properties to report, and with what vocabulary.
For still image metadata, we can go to Exif as a model. For audio, we have the AES standards. In the video realm, things are less settled so far. MPEG-7 offers a standard, but it focuses on the characteristics of the sounds and images, not of the files. It’s technical metadata, but at a level of abstraction which is less relevant to file characterization software.
XMP Dynamic Media looks more useful. It covers low-level technical properties such as frame rate, pixel depth, and compression. The high-level properties for music (no flat keys and only a limited choice of time signatures) are poorly thought out, but they aren’t relevant here.
Some people have recommended MediaInfo‘s output as a model to follow. I’m really not very impressed. Its output isn’t particularly consistent. Archivematica’s list of significant characteristics of video files is more useful, offering some properties which XMP doesn’t cover.
I’ve put together a spreadsheet of properties which represent my current thinking. Suggestions and comments are welcome.
Mad File Format Science 
Hi Gary,
MediaInfo has a few different reporting methods which are more or less suitable to your effort. Try these options:. I usually parse from mediainfo with -f and –language=raw and use –output=XML if needed:
mediainfo -f file.mov
mediainfo -f –lanaguage=raw file.mov
And you can get specific demuxer info like this:
mediainfo –inform=’Details;1′ file.mov
Dave Rice
(side note: I am not english native, and I am the main developper of MediaInfo, so this comment is not unbiasied)
> Archivematica’s list of significant characteristics of video files is more useful
All properties listed on Archivematica’s page (and in you spreadsheet) are available in MediaInfo… Output from MediaInfo is maybe not a model to follow, but the question is more about how to get theses properties (does other tools support as many file formats as MediaInfo? Do you really need to use several tools for one goal?), and MediaInfo can be easily adapted to output exactly as you wish.
> (MediaInfo) output isn’t particularly consistent
Each line in your spreadsheet has a corresponding line in MediaInfo, difficult to do more consistent.
The major problem is to map a container metadata to a line (you write “creator PBCore: pbcoreCreator”, ok, but what is PBCore: pbcoreCreator if you have only a MOV file? MediaInfo analyzes metadata in all containers and tries to do the right mapping between information from any file and your export format)
A quick and dirty MediaInfo mapping:
coverage To be defined, depends of the container and your exact need
creator MediaInfo: WrittenBy or EncodedBy, depending of your need
delay MediaInfo: Delay
description MediaInfo: Description
frameCount MediaInfo: Frame_Count
genre MediaInfo: Genre
interlaceMode MediaInfo: ScanOrder
numAudioChannels MediaInfo: AudioCount
numVideoChannels MediaInfo: VideoCount
pixelAspectRatio MediaInfo: PixelAspectRatio
publisher MediaInfo: Publisher
rights MediaInfo: Copyright
rotation MediaInfo: Rotation
scanType MediaInfo: ScanType
subject MediaInfo: Subject
title MediaInfo: Title
videoCompressor MediaInfo: Encoded_Library
And there are a lot more interesting information from MediaInfo, depending of your needs… (technical e.g. compression method, profile, level, chroma subsampling, time code… or other metadata e.g. author, writing date, service provider, Original source form, Lightness, Producer…)
The basic design of FITS is to use several tools; whether that’s a good idea or not for any particular purpose is outside the scope of what I’m being paid to do. But does MediaInfo have an API which would allow FITS to add it to its repertoire of tools? Even that is beyond what I’m doing on this project, but it might be something to consider as a future enhancement to FITS.
Thanks for the mapping. That’s helpful.
> But does MediaInfo have an API which would allow FITS to add it to its repertoire of tools?
You can access MediaInfo output by command line (for computers instead of human beings: “mediainfo -f –Language=raw” for text output or “mediainfo -f –Language=raw –Output=XML” for basic XML output) or by .so/.dylib/.dll API (C, C++, C#, Python 2, Python 3 and Java bindings are available: http://sourceforge.net/p/mediainfo/code/HEAD/tree/MediaInfoLib/trunk/Source/Example/ ).
For a list of useful video metadata fields, techincal and otherwise I recommend looking at SMPTE RP 210.
First, to second Dave Rice, I also use mediainfo -f –lanaguage=raw file.mov for the richest mediainfo output. Then map the fields I want. The nice thing about mediainfo is that it separates the output neatly into container and track information. This is actually very important for video. It’s not as nice to have it all mixed up. Video files are complex objects, each track is sort of it’s own animal with it’s own characteristics.
Exiftool only seems to output with track groupings with exiftool -X file.mov (maybe there’s another option that will produce container and track info, but I haven’t found it yet. not that i’ve looked extensively). Then you get RDF/XML, which may or may not be what you want, but it works. But in a comparison between exiftool and mediainfo output for the same dataset recently, I found that mediainfo was actually much more consistent than exiftool for the same data points (exiftool wasn’t outputting duration in constant format, for example). But for the most part exiftool -X file.mov and mediainfo [-f –language=raw] file.mov both produce the type of output I’d want to see. Plain exiftool output does not — there is too much noise and the formatting doesn’t work as well. ffprobe works too.
As an example of what subset of fields you would want to output from any tool — mediainfo, exiftool, ffprobe — I recommend looking at the pbcore instantiation and essenceTrack fields: http://www.pbcore.org/elements/. But that’s definitely not perfect or everything. Another place to look is videoMD and audioMD: http://www.loc.gov/standards/amdvmd/audiovideoMDschemas.html
Here is quick wishlist:
General —
file name
file size
format
format profile
mime type
duration
date
bit rate
bit rate mode
language
timecode
byte order
color
writing library
Track (repeat for all) —
track type (all tracks)
encoding (all tracks)
width (video tracks)
height (audio tracks)
sampling rate (audio tracks)
chroma subsampling (video tracks)
bit depth
frame rate (video tracks)
broadcast standard (video tracks)
color space (video tracks)
scan type (video tracks)
track data rate (all tracks)
language (audio and text tracks)
track duration (all tracks)
channels (audio tracks)
aspect ratio (video tracks)
writing library (all tracks)
> I recommend looking at the pbcore instantiation and essenceTrack fields
FYI, MediaInfo has a basic PBCore output “mediainfo –Output=PBCore_1.2”
Issue with such output format is that they are definitely not exhaustive enough (I did not see how to put all information you would like inside a PBCore output, and there is no list of known formats e.g. how do we fill “H.264/AVC” in essenceTrackEncoding field, but I did not work yet on PBCore 2.0).
Totally in agreement. There are a lot of improvements after pbcore 1.3, but again, I stress that pbcore is not perfect. I can’t point to any one thing that is. I provide it as a reference that gives an idea of what people want to see for video technical metadata. None of AES, XMP, or MPEG-7 come close. PBCore and videoMD are probably the closest. Mediainfo, in my opinion, provides the best output to get the full set of technical metadata that I want, in the best format. In exiftool land, exiftool -X option comes close, but not as good.
Hi,
Nice and interesting post and comments, listing a lot of standards to choose from ;) – so here’s one more I’d like to add:
From a preservation perspective, i think it’s essential to keep the schema you choose in line with the PREMIS data model (http://www.loc.gov/standards/premis)