Due to numerous requests I prepared a package showing off a simple indexer and a simple search. Inside there are two classes: Indexer and Searcher. Each of them does what their name suggests.
Indexer takes the first command line argument, interprets it as directory, gets all images from this directory and indexes and stores them in a newly created directory called “index”. Searcher searches in excactly this image index for the query image specified with the first argument.
The sample application employs CEDD and provides an ANT build file. IDEs like NetBeans, Eclipse or IntelliJ IDEA should have no problems importing the sources and using the build.xml file for compiling and running. Arguments can be changed in the build.xml file.
Apache Commons has a nice sub project called Sanselan. It’s a pure Java image library for reading and writing images from and to PNG, PSD (partially), GIF, BMP, ICO, TGA, JPEG and TIFF. It also supports EXIF, IPTC and XMP metadata formats, read for all, write for some. Examples for reading and writing images, EXIF, guessing image formats etc. are provided in the source package. Currently Sanselan is available in version 0.9.7 and the release date of this version seems to be in 2009. I’m not sure if this counts as abandoned project, but it definitely doesn’t count as alive 🙂
Face detection is basically a common tasks in image retrieval and management. However, finding a stable, well maintained and free-to-use Java library for face detection may prove hard. The OpenIMAJ project contains a common approach and yields rather fine results. However, the packaged version of all the JARs used in OpenIMAJ is quite bunch of classes making up a 30 MB jar file.
For those of you just interested in face detection I compiled and packaged the classes needed for this tasks in a ~5MB file. Finding the faces then with this library is actually a 3 lines of code task:
Frequently asked question in the mailing list is: Lire cannot handle my images, what can I do? In most cases it turns out that Java can’t read those images and therefore the indexing routine can’t create a pixel array from the file. Java is unfortunately limited in it’s ability to handle images. But there are two basic workarounds.
(1) You can convert all images to a format that Java can handle. Just use ImageMagick or some other great tool to batch process yout images and convert them all to RGB color JPEGs. This is a non Java approach and definitely the faster one.
(2) You can circumvent the ImageIO.read(..) method by using ImageJ. In ImageJ you’ve got the ImagePlus class, which supports loading and decoding of various formats and is much more error resilient than the pure Java SDK method. Speed, however, is not increased by this approach. It’s more the other way round.
Find some code example on how to do this in the wiki.
Having received several complaints about the slowness of Lire when searching in 100k+ documents I took my time to write a small how to to explain approaches for search in big (relatively) data sets.
Lire has the ability to create indexes with lots of different features (descriptors, like RGB color histograms or CEDD). While this opens the opportunity to flexibility at search time as we can select the feature at the time we create a query, the index tends to get bigger and bigger and searcher take longer and longer.
With a data set of 121,379 images the index created with the features selected for default in Lire Demo has a size of 14,3 GB on the disk. In contrast to that an index just storing the CEDD feature along with the image identifier has a size of 29 MB.
Due to the size of the index also linear search tends to get slower. While for the index stripped down to the CEDD feature and the identifier searching takes (on a AMD Quad-Core computer with 4GB RAM and Java 1.7) roughly 0.33 seconds, searching the big index takes 7 minutes and 3 seconds.
So if you want to index and search big data sets (> 100.000 images for instance) I recommend to
select which features you need,
create the index with a minimum set of features, and
eventually split the index per feature and select the index on the fly instead of the feature
also you can load the index into RAM
For more on loading the index to RAM and the option to use local features read on in the developer wiki.
I just released Lire and Lire Demo in version 0.9 on sourceforge.net. Basically it’s the alpha version with additional speed and stability enhancements for bag of visual words (BoVW) indexing. While this has already been possible in earlier versions I re-furbished vocabulary creation (k-means clustering) and indexing to support up to 4 CPU cores. I also integrated a function to add documents to BoVW indexes incrementally. So a list of major changes since Lire 0.8 includes
Major speed-up due to change and re-write of indexing strategies for local features
Auto color correlation and color histogram features improved
Re-ranking filter based on global features and LSA
Parallel bag of visual words indexing and search supporting SURF and SIFT including incremental index updates (see also in the wiki)
Added functionality to Lire Demo including support for new Lire features and a new result list view
I just released LIRe v0.8. LIRe – Lucene Image Retrieval – is a Java library for easy content based image retrieval. Based on Lucene it doesn’t need a database and works reliable and rather fast. Major change in this version is the support of Lucene 3.0.1, which has a changed API and better performance on some OS. A critical bug was fixed in the Tamura feature implementation. It now definitely performs better 🙂 Hidden in the depths of the code there is an implementation of the approximate fast indexing approach of G. Amato. It copes with the problem of linear search and provides a method for fast approximate retrieval for huge repositories (millions?). Unfortunately I haven’t tested with millions, just with tens thousands, which proves that it works, but it doesn’t show how fast.
The NetBeans community acceptance survey has voted for thhe last NetBeans 6.7 RC to be stable enough to be shipped. While this sounds great there is one minor details I consider critical for the significance of the survey: Only 182 people responded. (re-engineneered from 144 people being 79%). If we go with common numbers in empirical research ~5 % of the population take part in survey like these and therefore I conclude that the size of the NetBeans community is around 3.600 people.
NetBeans is actually having quite a a hard time with Eclipse pressing from open source and Idea pressing from commercial alternatives. Also the free IDEs of Microsoft’s .NET family affect the scene. However, I still think that if NetBeans manages to advance from the “I can do all” principle to a small and lean application development environment featuring a fast and intelligent editor and a WYSIWYG gui builder there is definitely a chance.
Java 1.6 u7 was released recently by Sun. While not bringing major changes it brought along some bug fixes and solved some security issues. However there is one main addition: The VisualVM. This is a really great developer tool: It connects to running VMs and shows “some statistics” about them. Besides memory usage and threads information it also allows to do some basic profiling. In my opinion Sun did a good job on including VisualVM in the package! Not that this thing is build on the NetBeans Platform 😉
I recently found myself in a scenario, where I tried to figure out how implementation clusters have been implicitly created within a group of students. All of them were given a task (with 4 sub tasks) for a whole semester. Everyone was meant to do the task alone, but collaboration was allowed. However I needed to know who helped whom and – of course – who helped whom with source code.
A colleague had a similar problem and he pointed me to PMD CPD (= PMD Copy & Paste Detector) . This tool works lightning fast and has a GUI 🙂 Also its open source -> respect!