- Man has only one means of discovery and that is to find likenesses between things (Jacob Bronowski)
- The dermatologist is a visual libertine seduced daily by the beauty of a rash (Sam Shuster)
- The images should take the place of the patients and the text should take the place of the teacher (von Hebra)
- “Explain, explain,” grumbled Étienne. “If you people can't name something you're incapable of seeing it.”— (Cortázar, 1966, Hopscotch)**
Our central research (and teaching) problem is how to deal with images. Books have text indexes and, given certain basic knowledge, a novice can search for more information. But what do you do about images? Metatext will clearly not work for what we want because people (even experts) are unable to use words in a simple unambiguous way. And yet, as the quotes above hint, blind dermatology does not exist (yet). How can we proceed?
We are following two approaches. The first, called DERMOFIT, is to build a content based image retrieval (CBIR) system for skin lesions. We view DERMOFIT as a cognitive prosthesis, a tool that supplements human skills such that tasks that would have been impossible are now rendered feasible (much as in the same way that a set of log tables assists calculations, or a dictionary assists understanding). In pilot work (now published [JID 2009.pdf]) we showed that novices were able to use matching to assign index cases to the correct diagnostic group if they were given a structured image database to assist them. The definitions of likeness were not defined, rather novices were apparently able to build webs of likeness between different instances from different diagnostic classes. The problem now is whether this approach will scale, and to examine this we need to build an interface that allows us to record user interaction in a much bigger database. Currently we have about 3000 images, all quality controlled in terms of lighting and clinical data, and are testing early versions of the software on users.
If DERMOFIT is a ‘hybrid technology’ in which we are taking advantage of humans’ ability to categorize objects based on implicit rather than explicit rules, then our second approach is more ambitious, as we want to understand what it required for fully automated diagnosis of skin lesions. Our approach, based on Bob Fisher’s expertise, is to see what additional information is provided by 3D rather than 2D image capture. To do this we use a rig from dimensional imaging with whom we collaborate. Some example images are below.