Caliph & Emir are MPEG-7 based Java prototypes for digital photo and image annotation and retrieval supporting graph like annotation for semantic metadata and content based image retrieval using MPEG-7 descriptors.
About Citation of Caliph & Emir
We kindly ask you to refer the following paper in any publication mentioning Caliph & Emir:
Lux, Mathias. “Caliph & Emir: MPEG-7 photo annotation and retrieval.” Proceedings of the 17th ACM international conference on Multimedia. ACM, 2009.
Features of Caliph
- Extraction of IPTC and EXIF metadata and conversion to MPEG-7
- Extraction of following content based image retrieval descriptors:
- ColorLayout (measures color distribution in an image)
- ScalableColor (basically a color histogram)
- EdgeHistogram (measures edginess)
- DominantColor (measure dominant colors)
- Annotation of images with
- Free text metadata
- Quality rating
- Structured text metadata
- Meta^2 data (metadata about metadata)
- Semantic Annotations
- Based on a editable catalogue for reusable semantic objects (like specific persons, places, …)
- Fully visual & graphical user interface
- Automatic placement and layout of visual descriptions
- Export of semantic descriptions to Structured Text Descriptions
- Annotation of shapes
- Load and save annotations as MPEG-7 XML file
Features of Emir
- Retrieval of annotations using the file system
- with XPath queries
- with keyword queries
- with quality constraints
- with example images for content based image retrieval
- Retrieval of Annotations using a Lucene index
- with inbuilt index creation
- using different fields for structured retrieval
- Retrieval of semantic graphs based on a Lucene powered index
- supporting wildcards for relations and nodes
- based on query expansion
- graphical query editor
- 2D data repository visualization based on FastMap and FDP algorithms
- for image content descriptors ColorLayout, ScalableColor and EdgeHistogram
- for graphs based on the minimum common sub graph metric
Besides reading existing information, which already exists in digital photographs, and transformation of this information to MPEG-7, Caliph supports the creation of new metadata. Semantic information about the image is presented as directed graph, where the nodes reflect semantic objects, locations, agents, states, times or concepts and the edges define the relations between these semantic entities. To enhance retrieval efficiency content-based metadata is extracted and new instances of the image for faster visualization, like thumbnails, are created. The MPEG-7 description consists of the following parts: metadata description, creation information, media information, textual annotation, semantics, visual descriptors.
The experimental metadata based image retrieval tool Emir supports retrieval in file system based photo repositories created with Caliph. Different types of retrieval mechanisms are supported:
- Content based image retrieval using the MPEG-7 descriptors ColorLayout, EdgeHistogram and ScalableColor.
- Keyword based searching using the Lucene search engine.
- Graph based retrieval supporting wildcards for semantic relations and semantic objects.
- 2D data repository and result set visualization based on FastMap & FDP algorithms.
Furthermore clustering abilities are built in but not acitvated yet.
For questions regarding the usage or for submitting bugs please use the sourceforge project forum of Caliph & Emir. Your help request will normally be answered within one workdays as long as I (Mathias Lux) am online.
Additionally there are two Flash demos, which show the basic usage of Caliph & Emir, available:
This screenshot shows the startup screen of Caliph. Here you can annotate the photos with free text and you can rate the quality. Pre existing metadata like EXIF or IPTC tags inside images is loaded and converted to MPEG-7.
Lucene based keyword search of Emir, search in different fields is supported.
2D visualization of the data repository using the FastMap algorithm and force directed placement for multidimensional scaling based on the semantic descriptions based on the maximum common subgraph metric
 Mathias Lux, Jutta Becker and Harald Krottmaier, “Caliph & Emir: Semantic Annotation and Retrieval in Personal Digital Photo Libraries”, Proceedings of CAiSE03 Forum at 15th Conference on Advanced Information Systems Engineering, p. 85-89, June 16th-20th 2003, Velden, Austria.
 Mathias Lux, Werner Klieber and Michael Granitzer, “Caliph & Emir: Semantics in Multimedia Retrieval and Annotation“, 19th International CODATA Conference 2004: The Information Society: New Horizons for Science, November 7th-10th 2004, Berlin, Germany.
[Download paper] [Download Slides]
 Mathias Lux and Michael Granitzer , “Retrieval of MPEG-7 based Semantic Descriptions“, BTW-Workshop “WebDB Meets IR” in context of the “11. GI-Fachtagung für Datenbanksysteme in Business, Technologie und Web”, March 1st 2005, University of Karlsruhe, Germany.
[Download paper ] [Download slides]
 Mathias Lux and Michael Granitzer, “A Fast and Simple Path Index Based Retrieval Approach for Graph Based Semantic Descriptions“, in Proceedings Workshop on Text-Based Information Retrieval TIR 05, in context of the 28th German Conference on Artificial Intelligence, Koblenz, Germany, September 2005.
 Mathias Lux, “Retrieval und Annotation von digitalen Photos mit MPEG-7”, MPEG-7 Workshop, March 10th-11th 2005, University of Klagenfurt, Austria.
[Download slides in german]
The development started by Mathias Lux in a lecture of the IICM, an institute of the University of Technology Graz, Austria. Further development was done by adapting it to a project of the Know-Center, a competence center for knowledge based applications and systems in Graz, Austria, by Mathias Lux and Werner Klieber. Quite some time ago it was decided to license the sources, which were partly under copyright of the Know-Center, under the GPL. All private copyrighted sources from the above mentioned lecture were also opend. After registering the project at sourceforge.net the project administration and development was carried on by Mathias Lux. Later additions include code from the VizIR project, which was lead by Horst Eidenberger. Furthermore, Werner Klieber added parts of the results from his master thesis to Caliph to allow the definition of shapes in images. Margit Lang ( MrsMojo) added a search routine for Emir combining different visual descriptors.
Jutta Becker, Harald Krottmaier, Michael Granitzer, Wolfgang Kienreich, Helmut Neuschmied, Werner Klieber, Vedran Sabol, Christian Gütl, the IICM and the TU Graz in general for supporting me by being my scientific environment. Further thanks go to Harald Kosch from the University of Klagenfurt for all the MPEG-7 support and to Horst Eidenberger from Vienna Technical University and his VizIR project team, for contributing edge histogram and other descriptors.