EffecTV is a software system that adds visual effects to video camera footage in real-time (without delay), reflecting the motion within the video. Simply by moving your body in front of the video camera, you can experience various fun visual effects such as glowing, moving strangely, bursting into flames, or becoming invisible, all synchronized with your movements. EffecTV’s real-time video processing technology has been widely used for audience-participation attractions at music events, background visuals synchronized with band performances, and stage productions for theater and dance performances.
EffecTV was released as open-source software in 2001 and has continued to be developed and evolved under an open development framework. For information about the software content, all visual effects, and source code, please visit www.effectv.dev. Personal works using EffecTV are featured on Kentaro Fukuchi’s personal page.
EffecTV
EffecTV is built by combining optimization techniques for high-speed visual generation and motion detection techniques developed in the software field known as “demos (mega demos)”, along with some creative ideas. This achieves both “real-time performance (= visual effects without delay)” and “interesting visual effects (= not just simple effects like optical filters)”. Such a system has the following characteristics.
Interactivity
Participants can learn and enjoy by seeing themselves with visual effects reflects their motion in real-time.
Synchronization with external information
Visual effects that are nearly synchronized with the movements of performers and actors on stage can be obtained.
The video effects built into traditional video cameras and video editing equipment were limited to relatively simple effects such as color conversion and coordinate transformation. While non-linear editing systems provide diverse visual effects, they require time for video storage and effect generation, making it difficult to immediately check results or experiment with body movements through trial and error.
Basic System Configuration
EffecTV can be used in various ways, but here are two representative configuration examples. In the left figure below, users stand in front of the camera and can watch their effected image on the front display while moving their body. This configuration is used when setting up as an interactive toy on streets or in show windows. On the other hand, in the right figure, the camera captures performers on stage, the effect video is projected behind the performers, and the audience views both the performers and the display simultaneously. This configuration is most commonly used for theater and music performance stages.
Applications
EffecTV has been used in the following settings, taking advantage of its high interactivity:
- Interactive advertising
- Background visuals for dance performances
- Special visual effects for theater
- Background visual effect generation for band performances
- Music band promotional video creation
Because visual effects can be generated in real-time, especially for stage use, there is no need to match stage progression to the visuals; instead, visuals synchronized with the movements on stage can always be presented. This allows improvisational movements by performers to be incorporated into the production. Additionally, visual effects synchronized with movement naturally match the rhythm of the music in musical performances.
Visual Examples
Other Information
EffecTV has been featured in the following online media:
- 【LinuxWorld Expo/Tokyo 2002レポート】(その7) 展示会場レポートー.ORGパビリオン編
- ITmedia: SmartSkin 人体形状センサーをエンターテインメントに応用する