CSCE 624 Sketch Rec: Blog Comments

CSCE 624 Sketch Rec Blog Comments by

Anurag Garg (P) - 323007195

1. After working on some of the code, I completely agree with you on Messy Code thing..!!

http://ankurgupta-624-sketch.blogspot.com/2014/12/five-haves-and-have-nots-of-sketch.html?showComment=1418883710313

2. I think - it all depends upon the kind of application we are trying to apply any feature to. Something that works for one domain may not work for other. So, "one size fits all" doesn't really hold true here

http://ankurgupta-624-sketch.blogspot.com/2014/11/using-entropy-to-distinguish-shape-vs.html?showComment=1418883710313

3. It would be nice to find a solution that is of Polynomial Order. It would be a great improvement over the worst case scenario (Exponential).

http://csce624-polsley.blogspot.com/2014/12/recognizing-interspersed-sketches.html?showComment=1418884060127#c5130556786738772391

4. I probably feel that Entropy is kind of feature that might work really well for some domains and might just not perform for others. Its probably because this feature is very specific.

http://csce624-polsley.blogspot.com/2014/11/using-entropy-to-distinguish-between.html?showComment=1418884478821#c5014599864200030562

5. I agree with you on your research ideas. I am not sure but I think - they can also be used in domains that require strategy planning like - Soccer, Hockey or may be Military purposes.

http://csce624-polsley.blogspot.com/2014/11/tahuti-sketch-recognition-on-uml.html?showComment=1418884674912#c242145079027011355

6. I second you on trying this with devices like Google Glass. I strongly feel that with Human Wearable Computing gaining relevance day-by-day, it can prove to be an interesting experiment.

http://csce-624-jaideep-ray.blogspot.com/2014/12/world-of-workout-contextual-mobile-role.html?showComment=1418884874871

7. Since its easy to code, fast and requires only a single template, this can be easily plugged into any HCI app..!!

http://csce-624-jaideep-ray.blogspot.com/2014/11/1-recognizer.html?showComment=1418885155324

8. That's few valid questions that you have raised. I also feel these kinds of thing also depend a lot on what kind of mood person is in and this might have impact on the expected results

http://csce-624-jaideep-ray.blogspot.com/2014/09/context-free-user-differentiation.html?showComment=1418885409560

9. One of the other reasons could be - Since it was published way back in 1991, not many ML algorithms could have been in heavy use or explored up to an extent to be tried !! Linear Regression would have been among the well known algos..

http://ankurgupta-624-sketch.blogspot.com/2014/09/specifying-gestures-by-example.html?showComment=1418886213006

10. These are some useful conclusions that you have drawn from the brushing exercise. Never really thought that brushing can be that random. Just a thought - not sure if this is in the right direction

- we know the length of the brushing pattern along the x-axis in both the directions/orientations would almost be similar. Can't we use this factor to align all the brushing patterns..

http://jcherian42.blogspot.com/2014/12/paper-19-world-of-workout-contextual.html?showComment=1418887509760

11. Agree with you on your conclusion. The algorithm for ShortStraw matches more with our intuitions rather than complex Mathematical equations.

http://jcherian42.blogspot.com/2014/10/paper-10-shortstraw-simple-and.html?showComment=1418887993717

12. Agree with you on your Have and Haven'ts. Sketch Rec system should not also be aggressive in giving feedback as this may distract the users and affect their normal sketching skills.

http://isketchrec.blogspot.com/2014/12/5-dos-and-donts-for-sketchrec-system.html?showComment=1418888268405#c9046385789527863057

13. I agree with your point on system without complex domain restrictions. Users always run away from system with steep learning curve.

http://csce624ashwini.blogspot.com/2014/12/sketch-system-principles.html?showComment=1418891675613

14. If you look at the timings, bulk of them is taken by SBFS. Almost 60-70%. SBFS is a greedy algorithm. So, it doesn't yield the optimal solution as well. Biggest improvement can be made if we can find a replacement for that or find a way to optimize it. A trained classifier approach can be one such solution.

http://sketch-recognizer.blogspot.com/2014/12/combining-corners-from-multiple.html?showComment=1418892138100

15. I agree with you on Polar Co-ordinate transformations. It really speeds-up the whole algorithm.

http://sketch-recognizer.blogspot.com/2014/12/trainable-symbol-recognizer.html?showComment=1418892287535

16. An interesting view of Fitt's Law. The figure explains it all. Extending Fitt's law into space where movement takes place in 3D would be one interesting thing to look at.

http://camitsketchrecognition624.blogspot.com/2014/11/fitts-law-as-research-and-design-tool.html?showComment=1418892512811#c3156934544843027396

17. I agree with you on that. I was also unsure about how effective the system can be until I watched few videos related to that.Few more videos can be found here:

https://www.youtube.com/watch?v=epFcYYtOVNQ

https://www.youtube.com/watch?v=LIt7gEftAXM

https://www.youtube.com/watch?v=qhmPd3caMXA

https://www.youtube.com/watch?v=HgqDK19XDnA

http://camitsketchrecognition624.blogspot.com/2014/09/k-sketch-kinetic-sketch-pad-for-novice.html?showComment=1418892670575#c4255324015293619359

18. Good explanation on how it helps in recognizing Interspersing Sketches faster.

http://yachiehhsu.blogspot.com/2014/12/recognizing-interspersed-sketches.html?showComment=1418892897153

19. Great Presentation. You covered all the important aspects of the paper particularly how Indexing Algo works and helps in speed-up.

http://csce624junginkoh.blogspot.com/2014/12/paper-review-20-recognizing.html?showComment=1418893241493#c8823828964818492420

20. I think - beautification has its use cases only in very specific domains. I completely agree with Raniero and Josh's idea of using it sparingly and only when its needed.

http://yujialiutamu.blogspot.com/2014/10/paper-8-paleosketch.html?showComment=1418893495173#c1007569389253212542

Sketch Recognition: 5 Have's and Have not's Revisited

Have's:

1. Simple Design and Good User Interface (Fitt's Law)
2. Fast and Responsive (Recognizing Interspersing Sketches Quickly)
3. Context-Aware Feedback/Assistive Mechanism (iCanDraw, Mechanix)
4. Minimum Constraints on users (Recognizing Interspersing Sketches Quickly; Paleo-Sketch, $1) like rotation invariance, scale etc
5. Good Disambiguation Methods (Paleo-Sketch, ShortStraw)
6. Extensible/Generic to other domains (LADDER)

Have not's:

1. Slow and Unresponsive
2. Over-correcting the users or Aggressive Feedback that hinders the natural drawing process
3. Unclear, Out-of-Context and Ambiguous Error Messages
4. Complex UI with barrage of features
5. From technical/developer point of view: code with high dependencies, high Coupling, old versions of libraries, Duplicate, High number of lines per function, not extensible

Recognizing Interspersed Sketches

Citation

T. A. Hammond and R. Davis, “Recognizing Interspersed Sketches Quickly,” pp. 157–166, 2009.

Link: http://dl.acm.org/citation.cfm?id=1555917

How often do we complete one sketch and then move on to the other while drawing complex shapes with multiple inter-related sketches? Natural sketching always imply completing some logical part of a sketch and then move to some other sketch that is related to the current sketch. So, there is lot of interspersing involved in a natural sketch.

In Sketch Recognition, if we enforce the user to draw a sketch completely before moving onto the next one, it takes away from the user the feel of natural drawing. This completely goes against the design principles of Sketch Recognition. Users must be allowed to draw sketches with minimum constraints. However, recognizing such sketches in Sketch Recognition often means considering all the possible combination of sub-shapes which is pretty time consuming(exponential) and might take forever thereby giving user a pretty bad experience (this, then, violates the other design principle of Sketch Recognition - Responsiveness). This paper came up with the idea that still runs in exponential time but is very responsive.

This paper solves the problem described above by introducing lot of new tweaks, and techniques enabling it to identify sub-shapes faster (in linear time - some cases; although worst case scenarios are still exponential) and with much less average space.

One of the methods by which it reduces the number of possible sub-shapes combination is by carrying out effective early sub-tree pruning. It also introduced the concept of Indexing that really helped in the further speed-up. It is done in essentially 3 steps:

1. Domain Independent Primitive Finding
2. Domain Independent Constraint Indexing
3. Domain Dependent Shape Formation

Thoughts and Future Work:

Though Indexing seemed to be a really good idea, but I didn't quite get hold of all the concepts explained in this paper. I would like to read more about this and look for possible enhancements and improvements. 

World of Workout: A Mobile RPG for encouraging healthy habits

Citation

J. Bartley, J. Forsyth, G. Brown, P. Hagseth, D. W. Goldberg, and T. Hammond, “World of Workout : A Contextual Mobile RPG to Encourage Long Term Fitness Categories and Subject Descriptors,” pp. 60–67, 2013.

Link: http://dl.acm.org/citation.cfm?id=2535718

With more and more people spending more time on internet, mobiles and their laptops, their lifestyle is becoming increasingly sedentary. With this, there's more need of healthy lifestyle by encouraging them to do exercises. However, we often tend not to do things for which we don't see the tangible rewards. The authors of this paper came up with the novel idea of addressing exactly the same problem. They developed a Mobile RPG (Role-playing game) where the character would get evolved with the exercises performed by the user in reality. To achieve the same, this paper employed the use of Activity Recognition. This paper mainly discusses the architecture and design principles for developing such a game.

Architecture

1. Client-Server Architecture: Data recorded using User's smartphone (Accelerometer data) and pushed to Server
2. HTTP being used to send the data
3. Database used as Information Repository

Design Principles

1. The game mainly recorded the data using GPS and Accelerometer present in the user's smartphone,
2. The activities were divided into two categories: Stationary and Moving
3. For every exercise performed, system would reward the user by giving more powers to the character,
4. The variety of real world exercises were combined into 3 game attributes:
1. Strength
2. Stamina
3. Speed
5. It also incorporated the Fatigue Factor. Its levels were quantified and was used as indicator to discourage user to over-exercising and thus, preventing them from hurting themselves.

Thoughts and Future Research Ideas:

This is a beautiful and innovative idea to encourage users to perform exercises and stay fit. Technically this is an application from the domain of Activity Recognition which also interests me. It finds the use of various Machine Learning Algorithms. With IoT (Internet of Things) and Human-Wearable Computing gaining more relevance day-by-day, its uses and applications can explored/exploited in those domains as well.

Judge the age and user attributes based on Sketching Skills

KimCHI: Sketch-Based Development Classifier for Children

Citation

H. Kim, P. Taele, S. Valentine, E. Mctigue, and T. Hammond, “KimCHI : A Sketch-Based Developmental Skill Classifier to Enhance Pen-Driven Educational Interfaces for Children,” pp. 33–42, 2013.

Link: http://dl.acm.org/citation.cfm?id=2487389

This paper gets its motivation from the fact that the sketching habits/skills can tell us a lot about the user/person. The author uses this concept to develop a Machine-Learning based Sketch interface that can predict the age, developmental levels in children, adults etc. This can also be further used to improve the Sketch designing interfaces. This is used for determining the developmental disorders among the people of various ages by using the standard sketch based tests.

The Technique/Approach

In order to achieve the objective, the children were asked to draw three different kinds of sketches:

1. Simple Shape Tracing without any external help/instructions.
2. Drawing a shape as quickly as possible.
3. Drawing a shape as slowly as possible.

As a part of this experiment, children of different ages were asked to draw sketches and 725 such sketches were collected from them. These sketches were carefully chosen so as to ensure that various geometric as well as gesture based features  get collected required for the correct prediction of children's age. The authors used 10-fold cross-validation to classify the children’s developmental progress and gender. Out of the 8 classifiers that were tried, the combination of Random forests and  Bagging performed the best.

Thoughts and Further research:

The idea is a very good one and can be further used to determine the user attributes like activeness/sleepiness, state of mind (attentive etc)

Fitt's Law: for designing Applications with better User Experience

Citation

S. MacKenzi, “Fitt’s Law as a Research and Design Tool in Human-Computer Interaction.” 1992.

Link: http://dl.acm.org/citation.cfm?id=1461857

With this world becoming more tech-savvy day-by-day, there is even more need of having applications/devices that caters well to a great user experience. No matter what how good or complex a problem an application can solve, it cannot succeed until and unless it has a captivating and intuitive user interface. So, if provided a metric that can judge how well an application scores on user interface, the UI can be improved by leaps and bounds. Fitt's Law is one such metric that is being widely used in the field of Human-Computer Interaction.

Fitt's Law is being used to model the movement time from a point to a target. It is given by:

T = a + b log2(2A/W)

where T = Movement time from a point to the target point

           A = Distance from that point to the target point

          W = Width/Area of the target

           a,b  are the model parameters

As is clear from the formula, the more is the distance, more is the movement time. More is the width of the target, lesser is the movement time.

Thoughts:

Though the idea presented by Fitt's is pretty old (1954) but it still holds relevance in today's world and is being widely used as a research and designing tool in the field of HCI.

Applications and Ideas:

This can be used to judge the age of a person as movement time is one of the major distinguishing factor for people with different ages.

Tahuti: Recognize UML Diagrams

Citation

T. Hammond and R. Davis, “Tahuti : A Sketch Recognition System for UML Class Diagrams,” pp. 1–2, 1998.

Link: http://rationale.csail.mit.edu/pubs/hammond/hammondsketchsymp2001.pdf

This paper presents a simple yet a very excellent idea of how to recognize the components of a UML diagram. This paper uses a set of intuitive Geometric rules to recognize things like Arrows (Simple and Diamond Arrow Heads). e.g. It performs recognition of a Diamond Arrow (shown in the diagram below) in the following manner:

Fig: Diamond Arrow (used in UML Diagrams)

(Source: https://docs.google.com/document/d/1wg0goMTn22zs82zkjo-khW0arM02X2FXp2hkfpw9HT0/edit)

1. It first looks for two furthest points in the diagram (A & B in this case) and label them as end-points of Arrow Shaft,
2. It, then, looks for two furthest points on the either side of the arrow (in this case C and D) originating from one of the points A or B,
3. It, then, finds a point of intersection for a line going from C to D on AB (let's say F),
4. If it finds any point E that is at a distance in the range equal to distance between F and B and is on the other side of F, it labels it the whole arrow as Diamond Arrow. Else it labels it as a simple arrow.

My thoughts:

As explained above, this idea is pretty intuitive and very simple to implement as well. Since this system is based on Geometric rules, it is free from the errors/complexities of a Gesture based recognition system. The components can be drawn in any manner.

Future Work:

Going on the lines of this idea, I would like to design such a recognition in domains that are based on Action-plans (e.g. Military attack planning, for Strategies in games like Hockey, Football).