apply shot detection on cooking

testing on 3/27

1. record cooking video in our smart kitchen
   (including putting things on, moving, cutting)
   
2. apply my shot detection program on it
3. result:
• although I haven't tuned the threshold and the rate of dimensions of color space yet, the result is not bad - important events can be detected
  
• can be real time
4. problems
   
• if using weight detection as basement, it senses changes and then check shot detection - what if no shot? e.g. a hand takes things and moves upward - so not shot, but actually there is weight decrease
  --> use local histogram to check ->
• weight event -> check shot -> shot event happens within time threshold -> real event
• (to be continued...)
  

project idea

I'm Peggy, from ubicomp lab, researching on HCI (human-computer interaction). I'd like to have the final project of this course be related to my research:

• Pei-yu Chi, Jen-hao Chen, Hao-hua Chu, Bing-Yu Chen. Enabling nutrition-aware cooking in a smart kitchen. in ACM SIGCHI 2007 extended abstract, April, 2007. (pdf)
• Jen-hao Chen, Keng-hao Chang, Pei-yu Chi, Hao-hua Chu. A smart kitchen to promote healthy cooking. Poster Paper & Adjunct Proceedings of the 8th International Conference on Ubiquitous Computing (ACM UBICOMP 2006), California, September, 2006. (pdf)

We have a "smart kitchen" in lab336/338 in CSIE building. What we are trying to do is to recognize cooking activities and then provide nutrition awareness to family cook, for promoting healhty cooking. Currently we only make use of weight transference detection to do activity recognition, which is obvious not enough for accuracy. Thus, in this course, I'd like to make use of video analysis and indexing techniques to improve activity recognition.
I've searched for related works about cooking video indexing (shown in the previous post,) and I'm still figuring out what can be done/improved/applied into my research.

cooking video indexing - related work

Ichiro Ide
Itiro Siio

• Reiko Hamada, Koichi Miura, Ichiro Ide, Shin'ichi Satoh, Shuichi Sakai, Hidehiko Tanaka:
  "Multimedia integration for cooking video indexing",
  Advances in Multimedia Information Processing - PCM2004 5th Pacific Rim Conf. on Multimedia, Tokyo, Japan, November/December 2004, Proceedings Part II, - K. Aizawa, Y. Nakamura, S. Satoh eds., Lecture Notes in Computer Science, Springer-Verlag, vol.3332, pp.657-664 (At: Tokyo Intl. Exchange Center, Daiba, Tokyo, Dec. 2004)
  Paper: Adobe Acrobat Format Presentation: Microsoft Power Point Format Abstract
• Koichi Miura, Reiko Hamada, Ichiro Ide, Shuichi Sakai, Hidehiko Tanaka:
  "Associating cooking video segments with preparation steps",
  Image and Video Retrieval -Second Intl. Conf. CIVR2003, Urbana-Champaign, IL, USA- E. M. Bakker, T. S. Huang, M. S. Lew, N. Sebe, X. (S.) Zhou eds., Lecture Notes in Computer Science, vol.2728, pp.174-183, Springer-Verlag (At: Univ. of Illinois at Urbana-Champaign, Urbana IL, USA, July 2003)
  Paper: Gzipped Adobe PostScript Format Presentation: Microsoft Power Point Format Boaster Slide Abstract
• Koichi Miura, Reiko Hamada, Ichiro Ide, Shuichi Sakai, Hidehiko Tanaka:
  "Motion based automatic abstraction of cooking videos (in Japanese)",
  IPS Japan Trans. Computer Vision and Image Media, vol.44, no.SIG9, pp.21-29 (July 2003)
  Paper: Gzipped Adobe PostScript Format
  
• Koichi Miura, Reiko Hamada, Ichiro Ide, Shuichi Sakai, Hidehiko Tanaka:
  "Motion based automatic abstraction of cooking videos",
  Proc. ACM Multimedia 2002 Workshop on Multimedia Information Retrieval (At: Hotel Ambassadeur, Juan-les-Pins, France, Dec. 2002)
  Paper: Gzipped Adobe PostScript Format
• Reiko Hamada, Koichi Miura, Ichiro Ide, Shin'ichi Satoh, Shuichi Sakai, Hidehiko Tanaka:
  "A motion based automatic abstraction of cooking videos (in Japanese)",
  Proc. Meeting on Image Recognition and Understanding (MIRU) 2002, vol.2, pp.203-208 (At: Nagoya Institute of Technology, Aug. 2002)
  Paper: Gzipped Adobe PostScript Format
  

cooking process (aids) with video:

• Reiko Hamada, Jun Okabe, Ichiro Ide, Shin'ichi Satoh, Shuichi Sakai, Hidehiko Tanaka:
  "Cooking Navi: Assistant for daily cooking in kitchen",
  Proc. 13th ACM Intl. Multimedia Conf. (ACM-MM2005), pp.371-374 (At: Singapore Hilton Hotel, Singapore, Nov 2005)
  Paper: Adobe Acrobat Format Presentation: Microsoft Power Point Format
• Reiko Hamada, Ichiro Ide, Shuichi Sakai, Hidehiko Tanaka:
  "Associating cooking video with related textbook",
  Proc. ACM Multimedia 2000 Workshops, pp.237-241 (At: Marina Beach Marriott, Marina del Rey CA, USA, Nov 2000)
  Paper: Gzipped Adobe PostScript Format Abstract
• Reiko Hamada, Ichiro Ide, Shuichi Sakai, Hidehiko Tanaka:
  "Structural analysis of cooking preparation steps in Japanese",
  Proc. Fifth Intl. Workshop on Information Retrieval with Asian Languages (IRAL2000), pp.157-164 (At: The Regent Hotel, Hong Kong SAR, Peoples Republic of China, Oct. 2000)
  Paper: Gzipped Adobe PostScript Format Abstract
• Cook's Collage: Deja Vu Display for a Home Kitchen, proceedings of HOIT: Home-Oriented Informatics and Telematics 2005
  http://www3.cc.gatech.edu/fce/ecl/projects/dejaVu/cc/index.html
• Itiro Siio, Noyuri Mima, Ian Frank, Tetsuo Ono, Hillel Weintraub, "Making Recipes in the Kitchen of the Future", Extended Abstracts, Conference on Human Factors in Computing Systems (ACM CHI 2004), page 1554, April 24-29 2004, Vienna, Austria.

2007 Spring - Multimedia Analysis and Indexing

This blog is for the course "Multimedia Analysis and Indexing" term project, 2007 Spring, National Taiwan University.
course website
Lecturer: Winston Hsu