I just uploaded version 0.9.2 of Lire and LireDemo to Google Code. Yes, Google Code! I also migrated (more or less in a under cover action some month ago) the SVN trunk to Google Code and will move on with development there. Main reasons were that ads were getting more and more aggressive over at sf.net and the interface of a Google Code project is so much cleaner and easier to handle from a project manager point of view.
Lire 0.9.2 fixes two bugs in KMeans and GenericImageSearcher. Both were critical. The KMeans fix allows now for the use of the bag of visual words approach. The GenericImageSearcher fix makes search much faster.
Due to numerous requests I prepared a package showing off a simple indexer and a simple search. Inside there are two classes: Indexer and Searcher. Each of them does what their name suggests.
Indexer takes the first command line argument, interprets it as directory, gets all images from this directory and indexes and stores them in a newly created directory called “index”. Searcher searches in excactly this image index for the query image specified with the first argument.
The sample application employs CEDD and provides an ANT build file. IDEs like NetBeans, Eclipse or IntelliJ IDEA should have no problems importing the sources and using the build.xml file for compiling and running. Arguments can be changed in the build.xml file.
Recently I posted binaries and packaged libraries for face detection based on OpenCV an OpenIMAJ here and here. Basically both employ similar algorithms to detect faces in photos. As this is based on supervised classification not only the algorithm but also the employed training set has strong influence on the actual precision (and recall) of results. So out of interest I took a look on how well the results of both libraries are correlated:
imaj_20 1.000 0.933 0.695
imaj_40 0.933 1.000 0.706
opencv_ 0.695 0.706 1.000
Above table shows the Pearson correlation of the face detection algorithm with the default models of OpenIMAJ (with a minimum face size of 20 and 40 pixels) and OpenCV. As can be seen the results correlate, but are not the same. Conclusion is: make sure that you check which one to use for your aplication and eventually train one yourself (as actually recommended by the documentation of both libraries).
This experiment has been done on just 171 images, but experiments with larger data sets have shown similar results.
Obviously the release cycle of lire is quite a long one. Therefore I just added a cronjob for a nightly build on one of our institute’s servers. The current SVN version of Lire will be downloaded, compiled, packaged and put online at 0:01 am CET everyday, 7 days a week. While there won’t be too much change on an everyday scale, you still can obtain a freshly compiled lire.jar for use in your project.
You’ll find the link in the right column of the page or at the end of this post. Please let me know if there are any errors etc.
Apache Commons has a nice sub project called Sanselan. It’s a pure Java image library for reading and writing images from and to PNG, PSD (partially), GIF, BMP, ICO, TGA, JPEG and TIFF. It also supports EXIF, IPTC and XMP metadata formats, read for all, write for some. Examples for reading and writing images, EXIF, guessing image formats etc. are provided in the source package. Currently Sanselan is available in version 0.9.7 and the release date of this version seems to be in 2009. I’m not sure if this counts as abandoned project, but it definitely doesn’t count as alive 🙂
Face detection is basically a common tasks in image retrieval and management. However, finding a stable, well maintained and free-to-use Java library for face detection may prove hard. The OpenIMAJ project contains a common approach and yields rather fine results. However, the packaged version of all the JARs used in OpenIMAJ is quite bunch of classes making up a 30 MB jar file.
For those of you just interested in face detection I compiled and packaged the classes needed for this tasks in a ~5MB file. Finding the faces then with this library is actually a 3 lines of code task:
MBFImage image = ImageUtilities.readMBF(new FileInputStream(“image.jpg”));
FaceDetector<DetectedFace,FImage> fd = new HaarCascadeDetector(80);
List<DetectedFace> faces = fd. detectFaces (Transforms.calculateIntensity(image));
All the imports needed along with their dependencies are packaged in the facedetect-openimaj.jar file (see archive below).
Frequently asked question in the mailing list is: Lire cannot handle my images, what can I do? In most cases it turns out that Java can’t read those images and therefore the indexing routine can’t create a pixel array from the file. Java is unfortunately limited in it’s ability to handle images. But there are two basic workarounds.
(1) You can convert all images to a format that Java can handle. Just use ImageMagick or some other great tool to batch process yout images and convert them all to RGB color JPEGs. This is a non Java approach and definitely the faster one.
(2) You can circumvent the ImageIO.read(..) method by using ImageJ. In ImageJ you’ve got the ImagePlus class, which supports loading and decoding of various formats and is much more error resilient than the pure Java SDK method. Speed, however, is not increased by this approach. It’s more the other way round.
Find some code example on how to do this in the wiki.
LireDemo 0.9.1 was released earlier today. Changes include several bug fixes, whereas the most critical was the one that prevented the indexing process from indexing a whole directory. Also the an SVN version of Lire has been compiled and added.
Having received several complaints about the slowness of Lire when searching in 100k+ documents I took my time to write a small how to to explain approaches for search in big (relatively) data sets.
Lire has the ability to create indexes with lots of different features (descriptors, like RGB color histograms or CEDD). While this opens the opportunity to flexibility at search time as we can select the feature at the time we create a query, the index tends to get bigger and bigger and searcher take longer and longer.
With a data set of 121,379 images the index created with the features selected for default in Lire Demo has a size of 14,3 GB on the disk. In contrast to that an index just storing the CEDD feature along with the image identifier has a size of 29 MB.
Due to the size of the index also linear search tends to get slower. While for the index stripped down to the CEDD feature and the identifier searching takes (on a AMD Quad-Core computer with 4GB RAM and Java 1.7) roughly 0.33 seconds, searching the big index takes 7 minutes and 3 seconds.
So if you want to index and search big data sets (> 100.000 images for instance) I recommend to
select which features you need,
create the index with a minimum set of features, and
eventually split the index per feature and select the index on the fly instead of the feature
also you can load the index into RAM
For more on loading the index to RAM and the option to use local features read on in the developer wiki.
As to be found in this month’s SIGMM record, which is the electronic SIGMM newsletter, a publication about Lire is in the top 10 downloads of the ACM special interest group on multimedia for September 2011.
I co-authored the paper with Savvas Chatzichristofis:
Mathias Lux, Savvas A. Chatzichristofis. Lire: lucene image retrieval: an extensible java CBIR library. In ACM Multimedia 2008
It’s also the paper I recommend to include in references if Lire is used within a scientific publication, so my thanks also go to the authors citing and therefore pointing to our work!