Aug.12.13 Front Pictures and the FULLDOME Experience
Front Pictures, a studio founded in Kiev, Ukraine in 2003 has been perfecting Fulldome environments with TouchDesigner to create some very convincing immersive experiences. The studio's work was recently brought to our attention by digital artist Andrey Yamkovoy who is also a TouchDesigner specialist at Front Pictures. The team of sixteen led by Yuri Kostenko (Head of Studio) and Vitaly Slyusarenko (Technical Director) are driven by a love for high-tech creativity which has led them to specialize in developing "next media solutions" which includes programming, interactive design, video production, and hardware engineering.
For recent projects Front Pictures produced a true 3D environment for fulldome projection using TouchDesigner. First the team developed their own GLSL shader generating an angular fisheye camera. This setup is used to make impressive real-time visualizations for shows, presentations and performances in planetariums around the world. TouchDesigner renders real-time content in 2.7 x 2.7K resolution that is captured by their Fulldome.pro media server that is in turn responsible for creating a seamless fulldome projection with multiple projectors.
Andrey: Fulldome technology which emerged about 20 years ago is an immersive dome-based video projection environment. It has evolved from simple "star sky" projection in planetariums to educational fulldome theatres (edutainment). We believe Fulldome technology has far greater potential and can be transformed into a next-generation interactive environment. We set up the Fulldome.pro company 3 years ago to develop and create affordable, highly flexible and easy-to-use projection systems. Today our automatic digital calibration system can "glue" dozens of projectors in minutes, giving the resolution up to 33 mega pixels with the help of a single media server. This technology opens the way for the integration of Fulldome environments with the most advanced and powerful media tools. TouchDesigner is definitely one of them!
Fisheye projection from Canon 5D mark2 onto the sphere
Astropilot360 Show Network
Going Back, a Short History
Andrey: I joined the company in 2005 as a digital artist and started with hi res 3D models for print materials (Front Pictures' chief area of specialization at that time).
After motion graphics came into place we found there was still something missing – interactivity. So we started to look for the applications that could give us interactive freedom!
Our Technical Director Vitaly Slyusarenko (Vitous) told me about TouchDesigner. I was shocked with its approach to construct and handle 3D data, its compositing abilities and later with CHOPs and DATs. After a few hours of playing with TouchDesigner I imported some geometry, set up rendering and understood how to output video on a fullscreen. It was so logical and easy to start doing things in TouchDesigner, plus the ability to interact with data. It was exactly what we needed! From that day I dived right into the world of realtime graphics, GLSL, OSC and Kinect.
After couple months we were confident that TouchDesigner could be successfully used for our commercial projects.
Interactive Stairs, Russian Railway Conference
This was Front Pictures' first project using TouchDesigner. The occasion was the 4th International Business Forum "Strategic Partnership 1520" held in Sochi and our client was Russian Railways, the leading transportation company in Russia. Representatives of 32 countries were there to discuss the idea of a single transport space in Eurasia. The meeting took place in the Radisson Lazurnaya Hotel - one of the most prestigious hotels on the Black Sea coast.
The stairs leading to the venue were mapped with exciting effects to intrigue and engage forum participants:
- virtual waterfalls made the way for walking up or down the stairs
- virtual carpet rolled down the stairs
- an interactive tableau showed the Sochi train schedule and its flip boards greeted the participants in six languages.
The installation consisted of 3 projectors with a total resolution of 2516x742 px and cameras for tracking people. TouchDesigner was used to control the effects together via a MIDI controller and Wacom tablet. One part of the content was generated in real-time and the other was pre-rendered. The effects were designed to change at the moment a person was about to step on the stairs to give the impression of interacting with a live object.
Andrey: For our next project with TouchDesigner we made a touch painting board for the Kiev Planetarium (Atmasfera 360) interactive zone where visitors could not only paint, but also send their creations via e-mail.
The painting board began as an experiment. I was looking at ways to use TouchDesigner to develop a creative application everybody could use and the idea of a touch painting board for children came to mind. I like to work closely with graphics and visual data with an idea of the future, so I was happy with the concept that children would be able to draw and express their emotions with the help of their fingers, like drawing on the sand or a frozen window but with more possibilities.
We used a 60" plasma display in a vertical orientation to maximize the height range of the user or 'painter'. It is comfortable both for a small child and a basketball player! All the paintings are stored on hard drive with the ability to send the work by email. The most challenging tasks for me were creating the UNDO command. I tried several approaches (one of the coolest features of TouchDesigner is that you can solve the task in many different ways) and at the end came up with the solution that worked best for me. This project taught me TouchDesigner's advantage for the input and output many types of data (visual, audio, text and other). The installation has been hugely successful with planetarium visitors!
Prototype of the OSC paint system
ISO surface and GLSL experiments
Back to Andrey's Experience
All my experience with digital art began when I saw Autodesk 3D Studio for DOS many years ago when I was a kid. Since then I was certain about what I wanted to do in the future.
The first conscious thing I did in TouchDesigner was a painting system that was driven with the iPhone's accelerometers, practicing with OSC. I also did an installation called "Interference" which is much better with headphones! Then Mir came into life, another interactive piece which modifies everything that a camera captures.
Andrey: Before I discovered TouchDesigner, my everyday software was 3DS Max, Modo, Lightwave and Photoshop. That's why the architecture and workflow of TouchDesigner seemed very different at first sight.
Here are the major reasons why it was easy to start learning, experimenting and working in TouchDesigner:
First of all - the NODE SYSTEM. With the usage of the nodes as visual data blocks it is much clearer to see "what is going on" in the scene generally and in each particular case. One glimpse is usually enough to get the initial understanding, then you start looking "closer" to get into details. It should be mentioned about color differentiation of node family types (COMPs, CHOPs, SOPs, TOPs and DATs). This feature also speeds up and clarifies the understanding of the data flow.
Second reason is the use of EXPRESSIONS. They open a whole new level of proceduralism. Expressions help to create precise animation in seconds. Another strong advantage of expressions - almost any data can be used as a source of animation.
And the third "pillar" is HELP. All of the information about the particular node can be found and seen in seconds. That helps moving through the unknown areas of TouchDesigner really fast! Pop-ups work very well too. There's a huge amount of information and techniques to be found in the forum where you can also meet friendly and creative people of the TouchDesigner community.
Once you start exploring, you can't stop! I believe TouchDesigner is the most flexible and powerful tool in the industry and can be recommended to anybody who is involved in interactive design and media.
Thank-you very much Andrey for taking us behind the scenes at Front Pictures and for sharing your experience with us.Inspiring work all-around and we look forward to seeing more from Kiev!