Abstract: George Poonkhin Khut’s sensory artwork, Distillery: Waveforming 2012, was the winner of the 2012 National New Media Art Award. This immersive installation artwork is a biofeedback, controlled interactive that utilises the prototype iPad application ‘BrightHearts’. Khut has an interest in the continued development of the ‘BrightHearts’ app to the point of making it available as a download from iTunes App Store to be used in conjunction with specialised pulse-sensing hardware. The configuration of Distillery: Waveforming presented in 2012 at the Gallery of Modern Art, Brisbane, incorporated Apple iPad 4th generation devices running the ‘BrightHearts’ app supported by Mac mini computers that processed data and mapped sound and visuals that were fed back to users as animations on the iPads. At the conclusion of the exhibition the artwork was acquired into the Queensland Art Gallery collection. The Curator of Contemporary Australian Art requested that the acquisition ensure that the artwork was captured in perpetuity in its prototype state. The iPad devices underwent jailbreaks to safeguard their independent operation and management, and to allow for the permanent installation of non-expiring copies of the ‘BrightHearts’ app. Source code for the ‘BrightHearts’ app was also archived into the collection. This paper describes the development of the artwork and the issues that were addressed in the acquisition and archiving of an iPad artwork
Figure 1. George Poonkhin Khut, Australia b.1969, Distillery: Waveforming 2012, Custom software and custom heart rate monitor on iPad and Mac mini signal analysis software: Angelo Fraietta and Tuan M Vu; visual effects software: Jason McDermott, Greg Turner; electronics and design: Frank Maguire; video portraits: Julia Charles, Installed dimensions variable, The National New Media Art Award 2012. Purchased 2012 with funds from the Queensland Government. Image: Mark Sherwood
George Poonhkin Khut’s digital artwork Distillery: Waveforming is a body-focused, controlled, interactive experience. The artwork was acquired by the Queensland Art Gallery / Gallery of Modern Art (QAGOMA) in 2012 and has been the subject of an ongoing dialogue between the artist and the Gallery, through the Head of Conservation and Registration, regarding its long-term preservation. At the heart of the artwork is an individual, human experience with certain intrinsic elements combining to create this experience. In their endeavour to provide a sound future plan for Distillery: Waveforming they have questioned ‘the experience’ from their individual perspectives – that of the artist and the collecting institution.
Distillery: Waveforming is both an independent artwork and an affiliated outcome of Khut’s long running work with heart rate biofeedback. This unusual duality plays a significant role in the ways in which the artist and the institution perceive the artwork, its preservation and future installations. Since the artwork’s acquisition into the QAGOMA collection the artist has remained involved and interested in the Gallery’s management of Distillery: Waveforming. Khut’s progress in his work on the biofeedback project has seen him make significant advances in software development, allowing him to release the iTunes application BrightHearts that was in-development at the time that Distillery: Waveforming was created. These advances in the biofeedback project provide current context to the dialogue and continue to shape the opinions of both artist and institution. Through this collaborative process QAGOMA has been able to build an extensive resource for the long-term preservation of Distillery: Waveforming.
HISTORY AND BACKGROUND: BIOFEEDBACK IN ART AND MEDICINE
George Poonhkin Khut’s biofeedback artwork Distillery: Waveforming was the winning entry in the 2012 National New Media Award (NNMA) held at the Gallery of Modern Art (QAGOMA 2012). The artwork entered the Queensland Art Gallery / Gallery of Modern Art collection at the conclusion of the exhibition. The curator of Contemporary Australian Art requested that the artwork be acquired to accurately reflect its display in the NNMA exhibition – that is as a prototype.
In 2011 when Khut was invited to enter the NNMA he was working as the Artist in Residence at the Children’s Hospital Westmead. In this residency Khut and his research colleagues commenced the BrightHearts Project that aimed ‘to assess the potential of small, portable biofeedback-based interactive artworks to mediate the perception and performance of the body in paediatric care: as experienced by children undergoing painful recurrent procedures’ (Khut et.al 2011).
Apple iPads loaded with games were already in use for diversion and distraction purposes during painful procedures at the Children’s Hospital Westmead. Khut chose to adapt his work for iPad technology for the BrightHearts Project based on this ‘diversional’ precedent and the excellent optical qualities of the iPad display (Khut 2014). In realising Distillery: Waveforming Khut channelled years of artistic practice in biofeedback and body-focused interactivity in the development of a cross-disciplinary artwork at the core of which was the prototype BrightHearts application (app) for Apple iPad.
When Distillery: Waveforming was displayed in the NNMA exhibition, from August to November 2012, the BrightHearts app was still in-development under a short-term Apple Developer licence. At this point in the provisioning, the prototype app generated the visuals on the iPad in response to a multilayered array of messages transmitted from a laptop or desktop computer over a network connection. This approach enable Khut to quickly prototype a variety of visualisation ideas by adjusting parameters on the desktop computer, without needing to compile and install the app on to the iPad each time. More importantly, at the time of its development – this networked approach also enabled him to incorporate live heart rate sensor data in a way that was not supported by the Apple operating system (iOS) at the time (before the introduction of the Bluetooth 4.0 wireless standard), and to continue his work with complex signal analysis, mapping and sonification algorithms that have been central to his work with body-focussed interactions since 2003. Essentially Distillery: Waveforming and the trial therapeutic devices at the Children’s Hospital Westmead were operating as ensembles that included iPads loaded with the prototype BrightHearts app, data collection devices, and desktop/laptop computers and network routing systems.
DISTILLERY: WAVEFORMING
Acquiring Distillery: Waveforming to reflect its status as a prototype was a curatorial imperative. Khut describes his approach to the long-running biofeedback project as ‘iterative’ and in this regard the artwork is an incremental representation of Khut’s artistic practice and a model demonstration of the developmental BrightHearts app for touch screen devices (Khut and Muller 2005). In the future Distillery: Waveforming will become a legacy artwork intrinsically linked to past and future iterations in the biofeedback project.
Distillery: Waveforming derives from Khut’s earlier work on BrightHearts that commenced in 2011 and his Cardiomorphologies series from 2004-2007. The mandala-like visuals were initially developed for Cardiomorphologies v.1 by John Tonkin using Java programming which Khut controlled via Cycling’74’s Max (version 4.5) application, a popular visual programming language for Apple and Windows computers. In 2005 the original visualisation software was expanded upon by Greg Turner for Cardiomorphologies v.2 using visuals generated from within the Max application. Turner used the C++ programming language to develop ‘Fireball’ a specialised graphic module (known in the Max programing environment as an ‘object’) – that enabled Khut to control the visuals with messages to each layer, for example, drawing a red coloured ring, the width of the screen, with a thickness of 20 pixels, and a green circle with a gradient, with a diameter of 120 pixels (Khut 2014; Pagliarino 2015, pp. 68-69).
Then in 2011 Jason McDermott, a multi-disciplinary designer working in the area of information visualisation and architecture, was engaged to re-write Greg Turner’s ‘Fireball’ visualisation software to enable it to run on hand-held technologies with touch sensitive controls. Using the open source C++ library openFrameworks with Apple’s Xcode (version 4) McDermott redesigned and expanded the potential of the software, developing BrightHearts as an iOS 5 mobile operating system application (McDermott 2013).
Development of the app continued when Khut received the NNMA prize worth AUS$75,000 and in April 2014 the BrightHearts app was released into the iTunes store. Heart rate data acquisition and processing is now integrated into the application software and the only external device that is required in conjunction with the app is a Bluetooth 4.0 heart rate monitor that captures the real-time heart rate data. The app is categorised as a Health and Fitness product that can be used to assist with relaxation and body awareness (iTunes 2014).
This history of development, change, modification and repurposing creates a landscape in which Distillery: Waveforming is an important new media artwork. As a legacy artwork the Gallery aims to maintain the component parts and software in their original form and function for as long as possible. Technologies change at such a rapid rate that the artwork will date in the years to come to reflect, quite evidently, an artwork of 2012. Perhaps future users will consider what are at present beautifully rich and transcendent animations as rudimentary and the touch screen navigation amusing and unsophisticated. Perhaps future users will recapture the sense of appeal that early touch screen devices inspired in consumers. However, it is not the intention of the Gallery to create a sense of nostalgia but to offer insights into the balance between art, technology and science at this fixed point in time. As a legacy the artwork will be an authentic installation and will offer an unambiguous window into Khut’s interdisciplinary artistic practice.
In its presentation in the NNMA exhibition Distillery: Waveforming was configured of five iPad devices running the prototype BrightHearts app that were built into a long, shallow, tilled table at which participants sat on low stools to interact with the artwork. Specifications set by the artist allow the Gallery to modify the configuration for smaller displays of no fewer than three stations in future installations. However it is necessary that the ambiance of the installation space affect a sense of calm and contemplation by utilising low light levels, soft dark colours and discrete use of technology. In the original installation the spatial arrangement situated participants in front of three video portraits of the artwork in use (Fig 1). Distillery: Waveforming is a composite artwork incorporating the iPad devices loaded with the prototype BrightHearts app, external data collection and processing equipment and video portraits displayed on monitors. The combined hardware and software systems include:
- Five Apple iPads (3rd generation) operating on the iOS 5.1.1 operating system, with retina display high resolution (2,048 x 1,536 pixels at 264 ppi) and dual-core Apple A5X chip
Loaded with:
- BrightHearts app (in-development)
- Cydia – a software application that enables the user to search for and install applications on jailbroken iOS devices
- Activator app – a jailbreak application launcher for mobile devices
- IncarcerApp – an application that disables the home button and effectively locks on the BrightHearts app when in use, preventing the user from inadvertently exiting the app
- Five heart rate sensors incorporating Nonin PureSat Pulse Oximeters (ear clip type) sensors, a Nonin OEM III Pulse Oximetry circuits and Aduino Pro Mini 328 microcomputers, running specially written code (OemPulseFrank.pde) to receive the pulse data from the pulse oximeter sensors – and relay this to the MacMini’s via a USB-serial connection.
- Five Mac minis 5.2, 2.5 GHz dual-core Intel Core i5 processor, 4GB RAM, 10.7.5 (OSX Lion) operating system
Running:
- Max6 application (Cycling74, 2012)
- Custom written scripts running from OSX ‘Terminal’ utility, that receive pulse data from the sensors via USB port and pass this along to Max6
- One 5.0 GHz network router that transmits control data from the Max6 software on the Mac minis to the corresponding iPads.
- Three digital portraits displayed on 40” LCD / LED monitors hung in portrait orientation
- Video portrait files include MPEG-4, QuickTime ProRes and AVC file formats
- Five headband-style stereo headsets
The prototype BrightHearts app for Distillery: Waveforming was written for Apple iPad (3rd gen) models running iOS 5.1.1. It was written under a short-term Apple Developer licence that allowed for provisioning and testing of the app on multiple devices. The licencing arrangement for the app expired in July 2013, nine months after the artwork was acquired by into the collection. A key aspect of archiving this artwork was the need to gain control of the app and in advance of the expiration the Gallery implemented an archiving strategy that was developed through consultation between the conservator, curator and the artist who was in contact with the software developer.
The most challenging aspect of the acquisition for the collecting institution was the long-term management of the proprietary technology and software. At the time of acquisition the prototype BrightHearts app was capable of performing a function with external support but did not have status as an independent Apple-approved application. In fact, its completion and approval was still one and half years away. It was also important for the iPad operating system to be locked down to iOS 5.1.1 as the prototype BrightHearts app for Distillery: Waveforming will only launch in this version. Through a consultative process it was agreed that to administer the artwork as an authentic prototype it was necessary to increase the end user control of the technology and software. This was achieved through jailbreaking the iPad devices and loading a non-expiring copy of the prototype BrightHearts app on the iPads (Pagliarino 2015).
MAINTAINING AN AUTHENTIC EXPERIENCE
Distillery: Waveforming has been acquired with the intention of maintaining authenticity and as such the Gallery has archived a full complement of digital files for the artwork. Included in this is source code for compiling the BrightHearts application with Xcode and source code for compiling the pulse-sensing Arduino microcontrollers for which Khut owns both copyrights (Khut 2012).
In conventional object-oriented programming source code, a programming sequence in readable text, outlines the steps that are necessary to compile software and make it function as intended, for example an app for an iPad. The source code has to be interpreted or compiled by a programmer in order to create the necessary machine code, for example Xcode if the work is developed for Apple OSX. Acquiring source code is thought to be a means of future-proofing digital artworks (Collin and Perrin 2013, p.52). This is undeniable as without the source code there is very little that can be used as a structural guide. However, Laforet et al. (2010, p.27) questions whether source code can really act as a safety net for software artworks in an age where there is a strong commercial imperative driving the development of digital technologies at the expense of the conservation of data. The success of source code to future-proof artworks relies on accurate interpretation and, in the context of an authentic experience, a complete lack of bias towards alternate or more efficient ways of programming the software to run an artwork as it was intended.
In cases where an artwork was developed using a suite of applications and programming languages, documenting source code becomes a complicated task in comparison to artworks where the source code is contained entirely within a single object-oriented programming environment. The programing for Distillery: Waveforming is distributed across three operating systems and four programming languages: Arduino for the sensor hardware; Objective C and iOS5 (via Xcode 4) for the BrightHearts app; OSX for the desktop computer that operates as a terminal emulator, running sensor data routing and analysis processes; and most significantly Max, the visual programing application that is used to perform the core analysis, mapping and sonification processes between the incoming heart rate data and the outgoing messages controlling the appearance of the various layers of the iPad visuals and sounds. Laforet et al. confirms that the difficulties faced with software artworks created by individual programmers are that:
These projects are relatively small efforts, putting the work created with it in a very fragile position. Unlike more popular software and languages, they are not backed up by an industry or a community demanding stability. The software only works under very specific conditions at a very specific time. Migrating such a work is a tremendous task, likely to involve the porting of a jungle of obscure libraries and frameworks. (Laforet et al. 2010, p. 29)
The complexity of combining multiple source codes from various programming platforms to work within one artwork significantly increases the risk of error in interpretation. In the case of Distillery: Waveforming it seems highly unlikely that source code alone would be sufficient to recreate the artwork in future. Khut has recognised this and has considered alternate bespoke and existing documentation systems for both Distillery: Waveforming and BrighHearts for the purpose of preservation and representation.
Visually the prototype BrightHearts app consists of 22 individually controlled graphic layers. Each layer is comprised of a single polygon that can be drawn as a solid shape or a ring, the edges of the shape can be blurred and the colour can be varied according to hue, saturation, alpha & value (brightness). The layers are then blended using an ‘additive’ compositing process, so that the layers interact with one another, for example a combination of overlapping red, green and blue shapes would produce white. This additive blending is a crucial aspect of the work’s visual aesthetic.
While the visuals are rendered on the iPad by the app developed by Jason McDermott, using Xcode and the openFrameworks libraries, the actual moment-by-moment instructions regarding what shapes are to be drawn, colour, size brightness etc. are all sent from the Max document.
The Max document, the top level ‘patch’ as it is referred to in the Max programming environment, is the heart of the work: the primary mediating layer between the sensor and display hardware that determines how changes in heart rate will control the appearance and sound of the work. It consists of an input section that receives sensor data, an analysis section that generates statistics from the heart rate measurements, and mapping layers that map these statistics to the various audio and graphic variables of colour, shape, volume etc.
The modular design methods used in the Max programming environment allow for the creation of modular units of code referred to as ‘abstractions’ and ‘bpatchers’, that can be re-used in multiple instances to process many variables using a very simple set of instructions. The programming for Distillery: Waveforming makes extensive use of these modules, which are stored as discreet ‘.maxpat’ files within the Max folder on the Mac mini computer. These modules are used for many of the repetitive statistical processes used to analyse the participant’s heart rate, as well as the mappings used to create the highly layered visuals and sounds that are central to the aesthetic of Distillery: Waveforming.
In the analysis section of the programming, changes in average heart rate are calculated over different time frames: the average rate of the last four heart beats, the average rate of the last sixteen heart beats, then thirty-two heart beats and so on, as well as information about the direction of these changes, enabling the work to track when the participant’s heart rate is starting to increase or decrease.
Within Max, the twenty-two graphic layers of visuals used in the prototype BrightHearts app, are each controlled by a corresponding ‘bpatcher’ layer-control module. Each of these ‘bpatcher’ modules contain 107 variables that determine how the parameters of all the layers are controlled. That is what aspect of the participant’s heart rate patterning it responds to and how these changes are mapped to variables such as diameter and colour of the layer in question.
Each layer-control module is comprised of sixteen sub-modules responsible for specific aspects of each layer’s appearance such as diameter, hue, saturation, and shape-type. In the programming of the layer-control modules the boxes of numbers visible in each module describe how incoming data relating to heart rate is mapped to the behaviour of the layer, in this case its diameter, and what statistical information it will respond to such as a running average of the last thirty-two heart beats, a normalised and interpolated waveform representing breath-related variations in heart rate, or the pulse of each heartbeat (Fig 2).
Figure 2: Four of the twenty-two layer-control mapping modules in Max – used to control the shapes drawn on the iPad by the BrightHearts (prototype) app.
All of these variables, controlling the appearance of each layer, are stored and recalled using a table of preset values describing which statistics each layer and variable responds to and how it interprets this input. These numbers are adjusted by the artist to produce the desired mapping and dynamic range and then stored in the .json file and recalled as presets. The information contained in this table is stored as a ‘preset file’ in a .json xml format file that is read when the Max document is launched. These preset files document the precise mapping and scaling settings that determine the appearance and behaviour of each layer of the artwork. Together these layered behaviours and the preset values that describe them produce the final interactive visual aesthetic of the artwork.
Figure 3: Example of one section of the .json ‘preset’ file containing preset data that is read by each of the graphics mapping modules – in this example showing all the parameters used to control the behaviour of the diameter for Layer 15.
For the artist, these preset tables are of central importance for documenting the appearance and interactive behaviour of the artwork for future interpretations, since it is these values that determine how the work responds to changes in the participant’s heart rate.
Strategies for hardware independent migration and reinterpretation
Khut has begun the process of documenting and describing the interactive principles and behaviour of the artwork independent from current technologies to enable the work to be recreated in the future, based on the Variable Media Network approaches set out by Ippolito (2003a).
For creators working in ephemeral formats who want posterity to experience their work more directly than through second-hand documentation or anecdote, the variable media paradigm encourages creators to define their work independently from medium so that the work can be translated once its current medium is obsolete.
This requires creators to envision acceptable forms their work might take in new mediums, and to pass on guidelines for recasting work in a new form once the original has expired.Variable Media Network, Definition – Ippolito, 2003b
For Khut, the essence of the artwork that would need to be preserved and recreated, independent of the specific technologies currently used, is the experience of having one’s breathing, nervous system, pulse and heart rate patterning represented in real time in an interactive audio visual experience, and the various optical and kinaesthetic sensations and correlations that are experienced during this interaction.
Taking an experience-centred approach it is not the source code as much as the experience of the visuals and sounds changing in response to the live heart rate data that is most essential to recreating the artwork. The aesthetic experience of interacting with the artwork, and the maner in which it responds to changes in heart rate initiated through slow breathing and relaxation is crucial to its authenticity.
The schematic approach: an open ‘score’ for reinterpretation
The simplest approach to documentation for future reinterpretation is the use of a very flexible set of instructions outlining the core interactive form and behaviour of the artwork. This approach leaves many aspects of the artwork’s appearance open to interpretation. Essentially what is preserved is the basic nature of the transformation – from breath, pulse and nervous system to colour, diameter, shape and sound. Such an approach would comprise the following instructions:
The visuals and sounds have been designed to respond to two forms of interaction:
1) gradual decreases in heart rate caused by a general increase in the participants ‘parasympathetic’ nervous system activity that can be initiated through conscious relaxation of muscles in the face, neck, shoulders and arms,2) breath-related variations in heart rate known as ‘respiratory sinus arrhythmia’ whereby slow inhalation causes an increase in heart rate, and slow exhalation causes a decrease in heart rate.
The result being a wave-like (sine) oscillation in heart rate to which the work owes its name (wave forming).
| Features extracted from Participant’s pulse and heart rate | Name of modulation source (controling the sounds/visuals) | Visual representation on tablet surface | Sonic representation as heard through headphones |
| Pulsing heart beat | /beat/bang | Gently throbbing circular shapes that either contract subtly with each pulse, or darken slightly with each pulse – to create a visual effect of subtle pulsing. | A deep and soft throbbing noise that gets louder and brighter as heart rate increases, and softer as heart rate decreases. |
| Breath-related variations in heart rate – normalised and rescaled to emphasise slow, wave-like oscillations in heart rate that can be induced through recurrent slow breathing at around 6 breaths per minute. | /IBI/dev-mean/4/normalised | Ring-shaped layers that expand when heart rate is increasing, and contract when heart rate is decreasing. | Synthesized drone sound, modulated with a ‘phasor’ effect controlled by breath-related changes in heart rate. |
| Gradual changes in average heart rate (average of last 32 beats) mediated by changes in autonomic nervous system (stress/relaxation), neck, shoulder arm muscle relaxation etc. | /IBI/how-slow/32 | Colour of background gradient – red for fastest heart rates recorded since start of session, green for medium, and blue for slowest average heart rate recorded since start of session. | Pitch of synthesized drone sound – crossfades through overlapping notes in C Melodic Minor scale – from B6 to A2 |
| threshold points triggered by decreases in heart rate (/IBI/how-slow/32) musical notes and burst of colour. | Circular, expanding bursts of colour from centre – fading out when they reach the edge of the frame. | Highly reverberant electric piano sounds triggered when threshold crossed – synchronised with burst of colour. Pitch descends in C Melodic Minor scale according to decrease in heart rate | |
| When participants sustain a slow relaxed breath pattern at around 6 breaths per minute, Frequency-domain analysis of heart rate variability will report the appearance of a ‘resonant peak’ around 0.1Hz (6 breaths per minute). There are six thresholds: 25, 30, 35, 40, 45, 50. Each time one of these thresholds is crossed – a message is generated that is used to control an audio and visual event | /spectrum/resonant-peak-resonance | A large, soft-edged blue ring expands slowly out beyond the edges of the frame and then slowly fades away.
Threshold 25 = yellow Threshold 30 = yellow-green Threshold 35 = green-yellow Threshold 40 = green Threshold 45 = cyan Threshold 50 = indigo
|
Very soft, muted and heavily reverberated piano note, with slow decay
Threshold 25 = D#3 Threshold 30 = A#3 Threshold 35 = D#4 Threshold 40 = F4 Threshold 45 = G4 Threshold 50 = A#4 |
Table 1: showing relationship of key mappings in Distillery Waveforming heart rate controlled artwork. Table 1 lists the key heart rate variables and their mapping to the main visual and sonic representations. The most basic recreation of the work according to the scheme laid out in this table would still require instructions for obtaining and generating the modulation sources from the heart rate data: the algorithms that scale and interpolate the heart rate data and translate these beat-by-beat messages into smooth, continuous control signals.
The translation approach: calibration tools and resources
A second, more precise approach for reinterpretation provides a set of documents to help future developers interpret and translate the original code and .json preset data to provide an aesthetic experience more closely aligned to the artwork at the time it was acquired by QAGOMA (Fig 3). This information is contained in a set of calibration images and accompanying tables that provide a crucial link for reinterpretations of the artwork, allowing future programmers to determine how values stored in the original preset files relate to the appearance of each of the work’s 22 graphic layers. Many aspects of the prototype BrightHearts app’s interpretation of these messages are not linear in their response and it can be seen that the gradients for each shape blend differently according to hue (Fig 4). It is hoped that these calibration images will help future programmers to compare how their own code interprets the messages stored in the preset files, against the behaviour and appearance of original prototype BrightHearts iPad app.
Figure 4: Example of one of the calibration images and accompanying tables describing how the messages from the Max software are interpreted by the visualisation software of the BrightHearts (prototype) App on the iPad.
Summary of documentation strategy for future translation
| Documentation element | Description |
| Broad schematic mapping of real time heart rate statistics to sounds and visuals | Describes the basic interaction concept and interaction experience: images and sounds controlled by slow changes in heart rate that can be influenced through slow breathing and relaxation/excitement. |
| Experiential aims and conditions for interaction | Describes the environmental conditions proscribed by the artist – to ensure optimum conditions for interaction i.e. minimise audio-visual distractions. |
| Documentation of Max patch: | Annotations in each section of Max code: the subsections (‘subpatches’, ‘abstractions’, and ‘bpatchers’) of the main file – describe the flow of information, through each section.
Document each section as a numbered image file, accompanied by notes describing how information is being modified/transformed. |
| Heart Rate Analysis | |
| Sounds | |
| Visuals – misc. top-level controls i.e. manage storage and retrieval of preset data, transition to ‘live’ visuals, control overall size, hue, position etc. | |
| Annotated table of preset values describing the mapping of heart rate information to the behaviour of the visuals, extracted from .json presets | Describing how each layer responds to the various heart rate statistics, and the quality of response over time (i.e. ‘easing’, non-linear scaling etc.) |
| iPad visuals – Annotated Calibration Images and tables, | Indicating how the visuals should look given specific layer-control messages i.e. diameter, hue, alpha etc. describing the idiosyncrasies of the visualisation code. |
Table 2 – George Khut’s documentation strategy for “Distillery: Waveforming”
Conclusions
For Khut, Distillery: Waveforming is foremost an experiential artwork and therefore his ideas about documentation focus on capturing its functionality and the aesthetics of the interaction. Khut sees the fundamental element of Distillery: Waveforming to be something other than the source code and the technical hardware: namely the mappings between breath and relaxation-mediated changes in heart rate and the appearance of the sounds and visuals, and how these mappings give form to the subject’s experience of interactions between their breath, heart rate and autonomic nervous system.
The modular Max patch programming and the presets in the .json file form, for the artist, the compositional heart of Distillery: Waveforming. This programming draws the visuals in response to the real time heart rate data: the key to the artwork. By further documenting the interactive principles independent from the current technology, drawing on approaches proposed in the Variable Media Questionnaire, Khut has developed reference documents that allow for the translation of the original preset data and calibration for future interpretations of the visualisation software. In this way Khut can describe the artwork with greater clarity in a non-vernacular, opening up opportunities for the artwork to be recreated in alternate modes.
As an artwork in the QAGOMA collection, Distillery: Waveforming sets a precedent as the first prototype-artwork to be acquired. Technology-based digital artworks are prone to being superseded at a rapid pace and attempting to manage even the medium-term future for such artworks is perplexing. To gain the assistance of the artist at the time of acquisition is constructive and very beneficial, but to secure the commitment of the artist to engage in collaborative, long-term conservation strategies is extraordinary and this has resulted in the Gallery acquiring an unparalleled archival resource (Pagliarino 2015, p. 74). Although the Gallery maintains an interest and intention to preserve Distillery: Waveforming in its original developmental state, providing clear evidence of Khut’s ‘iterative’, evolving art practice, the archival resource provides scope to reinterpret the artwork at some point in the future when the original technology no longer functions as intended.
Through this process of defining the experience, the artist and the institution have collaboratively addressed their common and divergent interests in the future care of Distillery: Waveforming. These differing views have created an opportunity to better understand the artwork and its position as an asset within a state collection and a physical, historical link to an ongoing, evolving artistic practice. Khut’s continued interest in the preservation of Distillery: Waveforming and his participation in dialogues about this artwork and other iterations of the biofeedback project have provided the Gallery with an extraordinary reference and flexibility to manage and display the artwork long into the future.
Works Cited
Collin, JD and Perrin, V 2013, ‘Collecting Digital Art: Reflections and Paradoxes – Ten years’ experience at the Espace multimedia gantner’ in Serexhe, Berhnard(Ed.), Digital Art Conservation – Preservation of digital art theory and practice, Germany, ZKM Centre for Art and Media Karlsruhe.
Cycling74, 2012, Max (version 6.0.8) computer software, Walnut, California, Accessed 22 September 2014.
Ippolito, Jon. 2003a, ‘Accomodating the Unpredictable’ in The Variable Media Approach: Permanence through change, Guggenheim Museum Publications, New York, and The Daniel Langlois Foundation for Art, Science & Technology, Montreal, pp. 46-53. Accessed 22 September 2014.
Ippoliti, Jon. 2003b, ‘Variable Media Network, Definition’ Accessed 22 September 2014.
iTunes 2014, ‘BrightHearts by Sensorium Health’ viewed 29 August 2014
Khut, George Poonkhin 2014, ‘On Distillery: Waveforming (2012)’ Born Digital and Cultural Heritage conference, Melbourne, Australia, 19-20 June 2014, viewed 29 August 2014
Khut, George Poonkhin 2014, personal communication, interview 5th September 2014.
Khut, George Poonkhin 2012, Distillery: Waveforming 2012 user’s manual (draft), in the possession of the Queensland Art Gallery, Brisbane.
Khut, George Poonkhin, Morrow, A & Watanabe, MY 2011, ‘The BrightHearts Project: A new approach to the management of procedure-related paediatric anxiety’, Preceding of OzCHI 2011: The Body InDesign. Design, Culture & Interaction, The Australasian Computer Human Interaction conference, Canberra, Australia, 28-29 November 2011, pp. 17-21.
Khut, George Poonkhin & Muller, L 2005, ‘Evolving creative practice: a reflection on working with audience experience in Cardiomorphologies’, in Lyndal Jones, Pauline Anastasious, , Rhonda Smithies, Karen Trist, (Eds.), Vital Signs: creative practice and new media now, Australian Centre for the Moving Image, Melbourne, Australia, RMIT Publishing.
Anne Laforet, Aymeric Mansoux and Marloes de Valk, 2010, ‘Rock, paper, scissors and floppy disks’, in Annet Dekker (ed.), Archive 2020: Sustainable archiving of born digital cultural content, Virtueel Platform, viewed 4 February 2014, pp.25-36.
McDermott, J 2013, ‘Bright Hearts (2011)’, jmcd, viewed 4 September 2013, <http://www.jasonmcdermott.net/portfolio/bright-hearts>
Pagliarino, Amanda 2015, ‘Life beyond legacy: George Poonhkin Khut’s Distillery: Waveforming’, AICCM Bulletin, vol. 36, no. 1, pp. 67-75.
Queensland Art Gallery / Gallery of Modern Art, 2012, National New Media Award 2012 – George Poonkhin Khut 2012 NMA winner, QAGOMA, viewed 15 October 2014.
Bios
Amanda PAGLIARINO is Head of Conservation at the Queensland Art Gallery / Gallery of Modern Art, Brisbane. Since 2003 she has worked on the conservation of audiovisual and electronic artworks in the Gallery’s collection. Amanda received a Bachelor of Visual Arts from the Queensland University of Technology in 1991 and a Bachelor of Applied Science, Conservation of Cultural Material from the University of Canberra in 1995.
George Poonkhin Khut is an artist and interaction-designer working across the fields of electronic art, interaction design and arts-in-health. He lectures in art and interaction design at UNSW Art & Design (University of New South Wales, Faculty of Art & Design) in Sydney, Australia. Khut’s body-focussed interactive and participatory artworks use bio-sensing technologies to re-frame experiences of embodiment, health and subjectivity. In addition to presenting his works in galleries and museums, George has been developing interactive and participatory art with exhibitions and research projects in hospitals, starting with “The Heart Library Project” at St. Vincent’s Public Hospital in 2009, and more recently with the “BrightHearts” research project – a collaboration with Dr Angie Morrow, Staff Specialist in Brain Injury at The Children’s Hospital at Westmead, Kids Rehab, that is evaluating the efficacy of his interactive artworks as tools for helping to reduce the pain and anxiety experienced by children during painful and anxiety-provoking procedures.