Citation:
iCanDraw? - Using Sketch Recognition and Corrective Feedback to Assist a User in Drawing Human Faces
Daniel Dixon, Manoj Prasad, and Tracy Hammond
Summary:
This paper draws its motivation from the fact that no matter how imaginative and perceptive people are but they always find themselves reluctant and short on confidence when it comes to put pen on paper and draw. The paper puts together the novel techniques of Sketch Recognition and Face Recognition and comes up with an idea of Intelligent Tutoring System - iCanDraw? that assists users in drawing Human Faces with corrective feedback mechanism.
The application displays step-by-step instructions defining the task in detail. It simultaneously keeps track of the users drawing. If the user's drawing is not in conjunction with the reference image template, it provides corrective feedback in the form of text, gestures and visuals.
Here are the few implementation steps in brief:
- Image Pre-processing: It uses a Face Recognition Library (Luxand FaceSDK, version 1.81) for generating the templates for the image to be replicated,
- Re-sampling strokes: This is done so as to ensure that the points on the drawn sketch are at equal distant from each other (PaleoSketch: accurate primitive sketch recognition and beautification)
- Classifying Strokes: This is done to put the data points into the closest facial feature. It is accomplished using KNN with k = 3
- Scanning Windows: The paper employs the idea of Scanning Windows for Correction Mechanism (Figure 8). It is calculated based on the standard deviation of points on user's sketch and the points on actual image template. Once this values becomes less than a threshold, the system accepts the user's current sketch task.
Discussion:
Though a Proof-of-Concept application, but iCanDraw? still does a good job at providing corrective feedback to the user by comparing a user-drawn face and the respective reference image. It provides users an all together a different perspective for drawing faces. We, as human beings, based upon our perceptions, tend to draw eyes in upper part of the head which essentially lie in its vertical center (Figure 6).
With iCanDraw?, we can draw faces based on 'what Computers see' and not 'what we perceive and know', which according to me, is really fascinating. This is also analogous with the 'R-Mode' vs 'L-Mode' comparison in which the right-brain (good at processing visual data) scores well over left brain when it comes to drawing.
Future Work:
As mentioned above (in Discussion), iCanDraw? is pretty good despite being a PoC. Since, it uses a Face Recognition library to understand and develop the model of face features from any given reference image, so a part of its success relies upon how good the underlying library is. Extending it to provide feedback and assist user in drawing a General Scene would be very ambitious as the underlying Computer Vision libraries for Scene Understanding are themselves in a nascent state. So, a more realistic addition would be to assist users in drawing Faces of Cartoon Characters, Animals, Birds and other living beings.
I think - Assistive feedback mechanism can be made even better by simultaneously collecting the following statistics:
- Keep track of how many times a user made the mistake and explicitly asked for a feedback,
- If the feedback provided proved helpful in correcting the mistakes
- If s/he paid any heed to the feedback provided (it could be possible that feedback provided was not intuitive or helpful enough)
These statistics can be used to categorize users based upon the kind of assistance (detailed or brief) they need while drawing. It might also help the application figure out their proficiency level at drawing and suggest the corrections accordingly. In addition, it can give application an idea if particular type of face template images are difficult to draw and can accordingly alter the feedback mechanism for images belonging to that category.
Other Helpful Links:
