SemanticMetadata.net

Converting video for flash video players to H.264/AAC

Have you ever tried to put a video online? Well actually it is quite easy if you user YouTube. No matter what codec you use you have a good chance to get a decent result. If you want to host the video yourself you basically need a flash video player (assuming that flash is the most widely spread tool on multiple platforms) like the JW FLVPlayer. Finally you’ll need to get your video file to a format flash can play using progressive download (which means you can watch it while downloading, just like on YouTube).

Since Adobe Flash Player 9 Update 3Flash can play back MP4 files with H.264 video and AAC audio streams [see here], so we can just focus on this one. First step is to get a ffmpeg version compiled with libx264 and libfaac. You might check this on the command line, just execute ffmpeg without parameters:

FFmpeg version SVN-r16573, Copyright (c) 2000-2009 Fabrice Bellard, et al.
configuration: [...] –enable-libfaac –enable-libgsm –enable-libx264 [...]

The bold ones should be there to support the needed codecs. I used FFmpeg Revision 16537 from this page, which works fine.

If the libraries are there you can proceed to the next step:

ffmpeg -i <inputfile> -b 1024k -vcodec libx264 \\
-acodec libfaac -ab 160k <output.mp4>

This converts your input file to the needed mp4 file. You can also change the size of the file with the switch “-s”, like for instance “-s 320×240″. Take a close look on the switches “-b” and “-ab” which define video and audio bitrate. If the sum of both bitrates is too high for the network the user will not be able to watch the video smoothly.

One might think s/he’s finished, but no … unfortunately progressive download doesn’t work with too many mp4 files. The file index (an atom == “mp4 metadata unit”) containing the file index (== the description where the video and the audio stream are located in the file and how they are stored) is at the end of the MP4 file. So the flash player has to download the whole file before starting the playback, ka-ching!

Fortunately there is an ffmpeg tool called qt-faststart (linux users will find it in the tools folder of ffmpeg) moving the index from end to start. For windows user a precompiled binary can be found here. Use this to move the metadata:

qt-faststart <infile.mp4> <outfile.mp4>

Now you are done with the file. Use for instance the JW FLVPlayer setup wizard to create an HTML snippet. Note that in height you have to add 19 pixels to your video dimensions, as this is the height of the control bar of the player

Music portal reduced to max: Grooveshark

Lately it was getting more and more challenging to hear a song you want online. YouTube sorts out based on Geo-IP, samples in online stores get shorter if even there. But I was pointed to a straightforward portal: Grooveshark. You just searcfh for the song you’d like to hear, press play and there you are. If you want to listen to multiple songs, there’s a queue usable without registration. Nice!

Links

• Grooveshark — Listen to Free Music.

Contribution @ I-KNOW 09 accepted!

The contribution of Christoph Kofler and me with the title “An exploratory study on the explicitness of user intentions in digital photo retrieval” has been accepted for publication and presentation at the I-Know ‘09. Here is the abstract (the full paper will follow as soon as we have prepared the camera ready version):

Search queries are typically interpreted as specification of information need of a user. Typically the search query is either interpreted as is or based on the context of a user, being for instance a user profile, his/her previously undertaken searches or any other background information. The actual intent of the user – the goal s/he wants to achieve with information retrieval – is an important part of a user’s context. In this paper we present the results of an exploratory study on the interplay between the goals of users and their search behavior in multimedia retrieval.

This work has been supported by the SOMA project.

Links

• I-KNOW 09
• SOMA project

Self Organizing Multimedia Architecture (SOMA) – The Project

Currently we are working a lot on a research project funded by a Carinthian agency. The project goes on for 3 years (2.5 years left) and has enough funds to pay 4 PhDs and 1 post doc researcher. Here is the description and the link to the blog:

The project Self-organizing Multimedia Architecture (SOMA) aims to capture the whole life-cycle of multimedia content in a single architecture for large distributed multimedia information systems. In SOMA we focus on scenarios where events, which we understand as “limited time periods of special importance”, are a central concept. Examples for such scenarios are sports events stretching over time, where start, finish or critical parts of a race are possible events, or traffic monitoring, where events like traffic jams or accidents have to be reported and investigated.

via Project Description | Self Organizing Multimedia Architecture.

Video Summaries for YouTube Videos?

Applying old things to new platforms has become common in recent times, here’s my contribution. I recently developed a video summary tool based on FFMPEG and Lire for a friend … just to test if common approaches are usable in a specific domain. Video summarization – especially of small videos – is a rather easy thing. You just need to find a number of frames with maximized pairwise difference, to cover a maximized visual range of the video. I applied my tool on YouTube and got the following summaires for the “hippo bathing” video:

Based on the CEDD descriptor the most important keyframe is really chosen well – just watch the video to know what I mean

With the auto color correlogram feature the dog is not explicitely part of the picture. However the first frame chosen (the big one) gives a good impression on the “bathing” part.

With the Gabor texture feature the dog gets prominent in the first place. Noite that the result is quite the same as the result kwith the Tamura texture feature not shown here.

With the most simple feature (RGB color histogram with L2 distance) the summary also looks appealing. There is a frame featuring the dog, one showing the whole scene and one for the hippo.

All in all I think the results are quite appealing. The runtime of my implementation is a fraction of actual video play time. Perhaps I’ll find some time to present the whole thing tomorrow at the barcamp

Report from the Open Source Track @ ACM MM

Within the open source track of the ACM Multimedia conference the SIGMM  (Special Interest Group on Multimedia of the ACM) wants to provide a platform on discussing tools and code free to use in research as well as industry project. As I already blogged I had a presentation on Lire there. However all other contributors were nearly as interesting, if not even more

Marco Lohse presented a Network-Integrated Multimedia Middleware (NMM), which was developed at Saarland University and is now available with dual licensing. He is also CEO of the company selling the non GPL license. The software is impressive and his demo was great. No surprise he won the prize of 700 $. (http://www.motama.com/nmm.html)

Yannick Alluse presented GpuCV, an extension of OpenCV towards CUDA and shader based computing. Important thing is that the library decides whether CPU, Shader or the GPU will be used for processing. (https://picoforge.int-evry.fr/cgi-bin/twiki/view/Gpucv/Web/)

Andreas Berger showed a framework for DVB-* transmission. It allows for inexpensive setup of for instance a DVB-T or DVB-H broadcasting service in labs or research projects. As its very generic one can compose the DVB stream fully free, so MHP, IP Data, EPG and so on are possible. (http://sourceforge.net/projects/fatcaps/)

Jose San Pedro Wandelmer from the University of Sheffield presented his project FOBS, which is actually a wrapper for ffmpeg for C++ and JMF. I already knew FOBS as I have used it several times, but meeting Jose and talking to him was great. (http://fobs.sourceforge.net/)

Lire accepted for the Open Source Contest @ ACM MM

Although its quite some time ago that I got the acceptance mail I forgot to blog the good news: Lire (Lucene Image Retrieval) has been acccepted to be presented at the ACM Multimedia within the Open Source Contest track. As it is a contest I assume we have chances to win something?

Lire development: a big next step ..

While it has been quiet for some time around Lire, recently development has been pushed forward. I switched to SVN for development and integrated simple RGB color histograms as a feature for comparison with the MPEG-7 features. Savvas Chatzichristofis (or on facebook, his image search engine) contributed the CEDD feature, which works great! Marko Keuschnig and Christian Penz contributed implementations for the Gabor texture feature and the Tamura texture features, where the latter is already in the SVN. I also integrated the new features in LireDemo. A new version – already compiled – can be downloaded here: liredemo-svn-2008-05-29-jdk16.tar.bz2 Note that Java 1.6 is required.

WIAMIS 2008 starts today …

The 9th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) takes place for the next three days in Klagenfurt, Austria. As we (the Multimedia Metadata Community) organized a special session there we will gather on Thursday latest and discuss current and upcoming research topics.

While I still have some things to prepare for tomorrow I will visit Horst Bischof’s keynote talk on “Robust Person Detection for Surveillance using Online Learning” and I hope I will find some time to blog about the conference.

Related Links

• WIAMIS Program

Deadline Extension: I-Media Special Session on Multimedia Metadata

The paper submission deadline has been extended to April 28, 2008.

Original CfP: 

Studies show that sales of digital capture devices like video camera, digital photo cameras, or mobile phones with digital cameras are still rising. Therefore it can be expected that the number of created digital multimedia content will rise dramatically in future. Multimedia metadata is currently the only way to cope with problems like semantics based retrieval or organization of content and provides means to specify adaptation and delivery constraints and rules. Within this special session the importance of metadata for media technologies is discussed. We encourage the submission of high quality scientific work as well as application papers. Topics include but are not limited to:

• Multimedia technologies and metadata in the social web
• Multimedia semantics
• Annotation of multimedia content
• Metadata in pervasive multimedia computing
• Metadata for new media
• Studies and surveys in context of multimedia metadata and new media

We encourage the submission of high quality scientific work as well as application papers.

More Information …