Skill/Ability: UX/UI

Octava, an app designed to augment live performance and arts events with contextual information in real-time, has been in development for 5 years, as a collaboration between myself, and composer Linda Dusman.

The UX/UI design challenges were very unique to the development of Octava. A blend of institutional memory, tradition, dedication to loyal audiences and a general resistance technological adoption has made many orchestra organizations resistant to the changing perceptions and tastes of the contemporary public.

Essentially, our goal was to augment the traditional concert experience with supplemental information that is queued to precise moments in the musical score, providing supplemental context that enriches the over all experience. It was hoped that this would elicit more of an active listening experience as opposed to passively allowing the music to wash over the audience.

Having spent a decade working on multi-disciplinary teams of scientists, artists, earth scientists, and satirists (to name a few), I very much knew the role data could play in broadcasting any information to the public. With this in mind, from the beginning of the Octava project I knew we desperately needed a way to gauge user impressions.

The challenge here is that aesthetic impressions are qualitative, not quantitative, so we needed a metric that could allow users to quickly assess their experience in an unguided way. Collaborating with Human Centered Computing at UMBC in 2011, I developed a hybrid qualitative reaction survey that fueled 3 full revisions of the Octava app over the next 5 years.

While not empirically conclusive, the data yielded from these surveys gave clear impressions of our user test audiences that guided my UX/UI design decisions, allowing us to completely revert many of the initial critiques.

Our first iteration of Octava (then called Symphony Interactive), featured a scrolling score that followed the orchestra through the use of a flight simulator throttle control. As laid bare by our user responses, this approach pressurized the experience as user felt compelled to watch the screen every moment. This was diametrically opposed to our goals so a major overhaul of the app was necessary.

The second iteration removed the scrolling score from the interface, and replaced it with a rotating “pin wheel,” where each spoke of the wheel was a single annotation. This design allowed for users to access the information as they saw fit, even if they wished to return to a previous annotation no longer relevant to the current moment of the music. This design completely turned our user responses around, letting the use Octava fold more seamlessly into the concert experience.

Building on the success found earlier, this version of Octava advanced the user experience further by providing additional context and information. While the pin wheel design was a step forward from the scrolling score interface, it too had limitations. While users felt the interface gave them direct access to the content, there was a still an element of confusion as to the progression through a given symphony. Addressing this feedback, the interface was once again completely redesigned, replacing the pin wheel interface with an interactive timeline. Each dot represent a single annotation, and the timeline progresses automatically during a performance. Users now had complete access to all information as well as a road map charting their course through lengthy compositions. Once again, our user reactions continue the positive trend.

During my time as the Technical Director of the Imaging Research Center @ UMBC, we piloted a app that attempted to visualize set theory mathematics. UMBC President, Freeman Habrowski often utilized a elementary set theory problem during live events to showcase the role of creativity in mathematics. The problem is as follows:

29 children are in a classroom. 20 have dogs, and 15 have cats. How many students have a dog and a cat?

Due to nature of set theory, the answer to this problem is variable depending on how you assign cats and dogs to the children in the classroom. The app needed to allow users to easily assign the pets to characters on screen, and see how their choices affects the total number of students with dogs and cats. I lead the development team and was responsible for the UI design. The video below showcases this functionality.

Despite being a problem at the 7th grade level, rigid perceptions of mathematics prevent many adults from answering the problem. Can you!?

This project has been a collaboration with Linda Dusman and Michael Richards (of the Department of Music) and the Imaging Research Center. The goal of the Symphony Interactive project is to leverage the ubiquity of portable wifi enabled smart devices to contextualize and further enrich the experience of live orchestral concerts.

Traditional arts institutions such as symphony orchestras are at a crossroads. With attendance down, such organizations are developing new models of sustainable growth that will resonate with contemporary audiences, who increasingly expect multi-faceted, interactive, and user-defined experiences. This is reflected in the myriad of mobile “smart” devices, saturating everyday experience. Such devices provide an opportunity to enhance the experience of attending orchestra concerts by creating multiple streams of information and media through which a performance may be experienced and contextualized. By utilizing these technologies, such institutions can reengage the wider public through contemporary forms.

The research will develop software for mobile platforms that will allow users to view scrolling musical scores in realtime, synchronized with a live orchestra by a human time keeper sending pulses via wifi. Utilizing and iPad, users will be able to view portions of the score yet to be played or revisit prior moments of particular interest, as well as access detailed annotations with compositional, historical, and aesthetic information. The Symphony Interactive prototype will provide a unique permutation of the traditional concert experience, leveraging mobile technologies in a way that heightens engagement through active participation.

The Imaging Research Center participated in the visualization of the UMBC Campus Master Plan Update process and presentation. Using an extensive 3D model of campus which I help manufacture several years before, we created three visualizations: a highly rendered animation showing future buildings; video motion tracking showing future buildings in live video settings, and a cutting-edge augmented reality display of the campus to illustrate over-arching campus planning themes.

This project was a collaboration between the Imaging Research Center @ UMBC and the Baltimore Museum of Art as part of a traveling show of the sculptural works of Henri Matisse. Both pieces, a short film about the sculptural process as well as an interactive kiosk were designed to better inform the public about how to understand, approach and discuss sculpture in general.
Several sculptures by Matisse were digitally scanned in order to obtain high-resolution polygonal models that were processed and then animated as either part of the film or the interactive kiosk. Further information can be found here.

Noetic was a pilot initiative launched by the Imaging Research Center in 2007. Developed around Howard Gardener’s notion of “multiple intelligences”, users were required to function in teams towards a given solution, relying on game play mechanics designed to test and respond to the real-world strength of users. More information can be found here.

During my tenure at the Imaging Research Center, I have spent significant time devoted to the development of digital puppetry. This was initiated by a collaboration between the IRC and the political cartoonist Kevin “Kal” Kallagher in 2005 to re-envision illustrated satire for an increasingly interconnected and mobile world. A team of four, including myself as lead animator, brought Kal’s illustrations of former President Bush to life as a 3D caricature, translating his signature style by digitally scanning and animating a bust made by Kal himself. After personally working with Kal to ensure all additional shapes and expressions modeled met his personal vision, I animated the two shots below as a proof of concept. Once we were sure Kal’s sensibility could be maintained, this data set was nested within a programming environment that allowed users to control the model as a puppet through the use of flight simulator joy sticks. Bush v1.0 was debuted at the Walters Art Museum as part of the opening night celebration of a retrospective show of Kal’s work entitled Mightier Than the Sword: The Satirical Pen of Kal. Much more about the project can be found here.

Years later this same technology and process was utilized when working with the Center for Women in Information Technology @ UMBC to develop avatars for their yearly festival Computer Mania Day. I worked directly with Collette Searles of UMBC Theater, who expertise in puppetry elevated the performative aspect of our system immensely by enlisting undergraduate theater majors to perform as a developed character named Jennifer Webb. More about the project can be found here.