Lukasz Mirocha: The Synthetic Cameraman

Summary

The artwork explores the potential of creating hybrid and processual representations of dynamic virtual environments with procedurally-generated aesthetics and cinematography that transgresses the established conventions of visual media such as synthetic photorealism, offering a glimpse into the future media based on programmable and machine-generated visuals and media genres.

Abstract

The Synthetic Cameraman is an installation using a high-definition screen connected to a computer workstation. The setup is running a full-screen, real-time, 3D graphics simulation that critically challenges the notions of remediation, processuality, linearity, and creative agency in computer-generated virtual environments. The application is rendering a virtual scene depicting a volcanic mountain landscape. The central element of a fairly flat and barren caldera surface is a violently erupting volcanic cone with pyroclastic flow and rocks of different sizes being expelled as molten lava rivers are traveling down the slope forming a lava lake at the foot of the cone. The visual aspect of the phenomenon is enhanced by deep sounds of rumbling earth and rocks hitting the bottom of the caldera, falling down the slope, and the sounds of howling wind. The control over individual elements in the scene has been given over to the algorithms. The weather conditions, eruption, and the virtual camera settings – its dynamic framing and movement – are procedurally generated in real-time. Every few seconds, a new virtual camera is created with settings chosen from a wide variety of values. The range of possible values is beyond the capabilities of physical cameras, which makes every virtual camera a hypermediated representational apparatus, producing partially abstract and fluid visuals. The algorithms are also controlling the parameters of various post-processing effects that are procedurally applied to the camera feed. All of these processes are taking place in real-time, therefore every second of the experience is conceived through a unique entanglement of settings and parameters directing both the eruption and its representation. Each second of the simulation as perceived by the viewer is a one-time event, that constitutes this ever-lasting visual spectacle.

The main goal behind creating the artwork was to challenge the pervasiveness of carefully recreated (remediated) lens- based aesthetics and photorealism used as a conventional representational style used in several media types that use today’s computer graphics, e.g. video games, CG animations, etc. As a result, a deconstructive and speculative approach has been taken towards synthetic photorealism. The Synthetic Cameraman is expanding the representational spectrum of photorealistic, real-time computer graphics by generating a hypermediated and dynamic camera-based visual representation of the scene in real-time. The volcanic environment was chosen as a subject for the simulation in order to emphasize the structural and ontological unpredictability of both the phenomenon and the models of its representation as they unfold – an already dynamic and uncontrollable volcanic eruption is visualized by an equally dynamic and procedural camera system. The results of algorithmic agency in The Synthetic Cameraman reveal the creative potential of real-time computer graphics and today’s real-time content creation tools (e.g. general-purpose game engines), that allow us to design new types of visual content based on various aesthetic and formal styles and conventions. This trend will only accelerate thanks to advancements in machine learning techniques with systems like GANs and CANs that can produce images situated within a broad representational spectrum of computer graphics – from producing new types of image hybrids and aesthetics to mimicking historical artistic styles, generating perfect copies of paintings that have been fundamental for Western culture. Importantly, the artwork, by offering a unique visual experience virtually every second, illustrates how programmable real-time computer graphics bring us closer to processual and variable media culture based on untitled and particularly untied, ever-evolving media hybrids, bringing new visual experiences and allowing for new means of creative expression.

The core component of the installation is a full-screen Windows 10 software application. At a physical level, the artwork is composed of two main elements: a graphics workstation (e.g. a PC laptop/tower: 4-core CPU, 8GB GPU RAM, 16 GB RAM, 128 SSD) connected to a display device (e.g. a large FHD TV screen, 60” +) and the connecting cables, with the screen being the audience-facing device. The TV should be preferably mounted on a VESA or equivalent rolling stand, or be mounted on the wall, depending on the general setup/design of the exhibition venue. Preferred mounting height: 1.20- 1.50 m. The piece should be set up and tested in about 2-3 hours. The setup is mainly about connecting all the devices together, running the software application, and making sure that the performance and output meet the design goals. A minimum 3-gang extension power cable is required to power the installation, its length depends on the venue setup, but is estimated at minimum 3 m.

Although it is challenging to discuss the artwork in separation from my theoretical research, I’d like to emphasize that it demonstrates the creative potential of real-time 3D graphics in exploring new aesthetic conventions and media genres emerging out of the unique affordances of computer-designed visuals: programmability, processuality, hybridity. This potential has become more accessible to non-expert parties thanks to new software suites such as general-purpose game engines or web-based design frameworks. Additionally, The Synthetic Cameraman illustrates how established or media-specific aesthetic conventions such as photorealism or the languages of cinematography can be questioned, rethought and in combination with affordances of computer graphics, put to creative use across media to design new types of experiences.

I have studied and explored the artistic potential of real-time 3D graphics and general-purpose game engines by creating a positive feedback loop in which the theoretical study was constantly supplied with new findings originating from my hands-on interactions with the medium as its user. The Synthetic Cameraman is one of several artworks that emerged based on the same virtual volcanic environment. The other artworks include a VR experience and a real-time CG animation. The artworks were designed with the aim to explore the boundaries of a particular camera-based representation model that is unique for a specific media genre, which drastically changes how the virtual environment is depicted and experienced in each artwork. It is worth mentioning that I come from theory-oriented research communities and I don’t have a formal computer science/design education, therefore, the development of the artwork presented a steep learning curve. From one side, I had to learn how to communicate my ideas not only with natural language but also using artistic techniques and visual communication. From the other, l also had to figure out how to align my ideas and artistic goals with concrete affordances of computer systems at hand, solving scripting, 3D design as well as challenges and issues related to the development pipelines. Working on all the aspects of The Synthetic Cameraman has been a demanding process, however, it also provided a tremendous chance for self-development.

Inspiration Behind the Project

The project emerged as a combination of my research activities and personal experiences. In the first case, I see it as a result of my explorations and experimentations conducted as a practice-based component of my studies on representation models (e.g. virtual camera-related techniques and approaches) for multi-purpose, spatial, CGI-based, real-time environments that I have been engaged with during my soon-to-be finalized Ph.D. studies at the School of Creative Media, City University of Hong Kong. In the second case, the idea for using a volcanic-themed virtual environment as the basis of the artwork that would explore the creative boundaries of the medium came out of my recent personal experiences. About a year ago, I knew that I wanted to use 3D, real-time environment as a medium for the artwork, however, I was still looking for a theme that would allow me to maintain an equilibrium between the represented subject and an equally dynamic and processual representation process. In other words, the theme had to emphasize the structural and ontological unpredictability and processual nature of both the phenomenon and the models of its representation. Meanwhile, in late 2019, I traveled to one of the few publicly accessible islands with active volcanos. Unexpectedly, the volcano violently erupted with tragic consequences, including fatal casualties, only a week after I had visited it. It was a deeply moving experience that also pointed me to the fact that in many cultures volcanoes are regarded as untamed, dynamic and self-governing agents that we hardly have any control over. Eventually, I chose a volcanic landscape as a theme for the artwork that became The Synthetic Cameraman.

Key Takeaways for the Audience

The importance of interdisciplinarity and cross-disciplinary communication is well known to the SIGGRAPH Asia community. My first SIGGRAPH Asia (Bangkok 2017) helped me to clarify my research goals and interests thanks to the fact that I was inspired by individuals having different backgrounds – industry professionals, researchers, artists – who would apply their interrelated skillsets in different domains. All these domains share a strong conviction about an almost infinite potential of computer technologies. I believe that works like The Synthetic Cameraman, which is situated at the intersection of liberal arts, artistic practice and computer technologies, emerge largely due to constant dialogue between different disciplines and individuals representing these disciplines. I am sure that if the SIGGRAPH Asia participants take some time during the conference to step out of their main domains, they might find themselves inspired by a tech demonstration, working paper, presentation or artwork that is seemingly unrelated to their daily work.