Skill/Ability: Digital Print Production

Possessions of a Randomly Acquired Mind is an animated video piece that matches closeups of biological and mechanical processes to outline drawings of men in motion. The video, which also yielded a series of prints, illustrates what the artist calls the multiple axes of sensation, emotion and cognition. Movement is easier to depict than thought, of course, but Smallwood’s handsome renderings neatly juxtapose micro and macro.

Octava, an app designed to augment live performance and arts events with contextual information in real-time, has been in development for 5 years, as a collaboration between myself, and composer Linda Dusman.

The UX/UI design challenges were very unique to the development of Octava. A blend of institutional memory, tradition, dedication to loyal audiences and a general resistance technological adoption has made many orchestra organizations resistant to the changing perceptions and tastes of the contemporary public.

Essentially, our goal was to augment the traditional concert experience with supplemental information that is queued to precise moments in the musical score, providing supplemental context that enriches the over all experience. It was hoped that this would elicit more of an active listening experience as opposed to passively allowing the music to wash over the audience.

Having spent a decade working on multi-disciplinary teams of scientists, artists, earth scientists, and satirists (to name a few), I very much knew the role data could play in broadcasting any information to the public. With this in mind, from the beginning of the Octava project I knew we desperately needed a way to gauge user impressions.

The challenge here is that aesthetic impressions are qualitative, not quantitative, so we needed a metric that could allow users to quickly assess their experience in an unguided way. Collaborating with Human Centered Computing at UMBC in 2011, I developed a hybrid qualitative reaction survey that fueled 3 full revisions of the Octava app over the next 5 years.

While not empirically conclusive, the data yielded from these surveys gave clear impressions of our user test audiences that guided my UX/UI design decisions, allowing us to completely revert many of the initial critiques.

Our first iteration of Octava (then called Symphony Interactive), featured a scrolling score that followed the orchestra through the use of a flight simulator throttle control. As laid bare by our user responses, this approach pressurized the experience as user felt compelled to watch the screen every moment. This was diametrically opposed to our goals so a major overhaul of the app was necessary.

The second iteration removed the scrolling score from the interface, and replaced it with a rotating “pin wheel,” where each spoke of the wheel was a single annotation. This design allowed for users to access the information as they saw fit, even if they wished to return to a previous annotation no longer relevant to the current moment of the music. This design completely turned our user responses around, letting the use Octava fold more seamlessly into the concert experience.

Building on the success found earlier, this version of Octava advanced the user experience further by providing additional context and information. While the pin wheel design was a step forward from the scrolling score interface, it too had limitations. While users felt the interface gave them direct access to the content, there was a still an element of confusion as to the progression through a given symphony. Addressing this feedback, the interface was once again completely redesigned, replacing the pin wheel interface with an interactive timeline. Each dot represent a single annotation, and the timeline progresses automatically during a performance. Users now had complete access to all information as well as a road map charting their course through lengthy compositions. Once again, our user reactions continue the positive trend.

Octava, an app designed to augment live performance and arts events with contextual information in real-time, has been in development for 5 years, as a collaboration between myself, and composer Linda Dusman.The art production design process for Octava posed a unique challenge. Everything we learned from our user testing has shown us that users are highly receptive to supplemental materials, but the chosen delivery method for that information was key in minimizing distraction. If the visual experience of Octava was going to enrich the experience of live performance, the aesthetic of the app needed to match the elegance of the music, environment, and texts being perceived.

The core challenge visually was to provide a visual context for the experience of the live performance that enabled an ease of reception with regard to the annotation texts, all while minimizing distraction. Here is an example annotation from an Octava performance:

Method

As a designer/animator, motion needed play a key role in uniting the musical performance and the app into a single experience, so I designed the UI to accommodate animated frame sequences that would progress in concert with the release of each annotation. A breakdown of the how the Octava app produces the above image reveals the relationship between text, interface elements and animations composited with Octava:

As the orchestra plays, annotations are released to coincide with particular moments in the performance. Simultaneously with the release of each annotation, the visual themes shift as well, providing a sense of progression, especially through longer, more dense compositions.

While ultimately interperative, this design work was not arbitrary, and based on researching musicological, historical, and cultural resources to produce colors and forms that complimented each symphony. I personally created or art directed/edited every animated animated sequence for Octava, acting both as the conceptual and technical director, as well as lead animator.Two methods were developed during production to facilitate varying level of animation.

1) Looping Method: 3-5 frame animated sequences were designed to loop endlessly within specific movements of a symphony. This method allowed for frame by frame deformation, as well as pan and scan methods. The image here showcases this approach as used for accompanying Dvorak’s “New World Symphony.”

2) Animation Scroll Method: As an alternative, the scroll method focuses on developing a single image across which a virtual camera will pan. With this method, the illusion of motion is created without the need for frame by frame animation. Below is an image of the animation scroll produced for Mahler’s 5th Symphony.

During the height of production for our Phase I Maryland Innovation Initiative grant, I oversaw a team of 7 interdisciplinary artists working around the country. The result was a series of animated sequences for the 30 most performed works nationwide. Below you will find a .gif sampling of visual materials from some of the symphonies within our catalogue. These changes would occur almost subliminally over time, but have been sped up hear to reveal the motion design within.

A Short Ride In A Fast Machine is the working title of a new project by UMBC visual arts professor and experimental animator Eric Dyer that is currently in development with the National Film Board of Canada. I was fortunate to have the opportunity to produce a style frame to be included in his initial application. The process developed here utilized renders of 3D models built in Dyer’s unique “cinetrope” style as background templates upon which I illustrated the foreground characters and planes shown above.