Introducing the Visual Narrative Faculty Cluster

It’s only a year old, but already the Visual Narrative faculty cluster has come together as a team to find new ways of telling stories—and to offer their resources to community members. Its members are physically spread across campus, hailing from four departments. Of the 20 clusters hired so far by the Chancellor’s Faculty Excellence Program, it is perhaps the most diverse.

The cluster includes Todd Berreth, Assistant Professor of Art + Design; Frederico Freitas, Assistant Professor of History; Tianfu Wu, Assistant Professor of Electrical and Computer Engineering; and is coordinated by Matthew Booker, Associate Professor of History; Helen Burgess, Associate Professor of English; and Arnav Jhala, Associate Professor of Computer Science.

Jhala explains that the cluster concept was developed to give faculty members a structure that would support interdisciplinary research in teams that could encourage and support each member’s ideas. By spanning several colleges, a cluster offers each member access to far more knowledge and approaches than if they were working alone. Berreth notes, “We each have our projects and our lines of research, including some problems we think are intractable. Someone from another discipline might say, ‘I have this technique that might apply to that to solve your problem.’ You know your own domain, but someone else’s domain might have an application that might help yours.”

“You know your own domain, but someone else’s domain might have an application that might help yours.”

“We all were committed to that kind of research already in the institutions we were at,” he adds, “but here we want to take it to the next level. Rather than saying, ‘Here’s my project; I want someone to consult me on technology,’ we are building a project that has a technological research component at the same time as history, research, and art and design components.”

And indeed, when they’re all together, ideas and suggestions fly back and forth. The group meets weekly in the Games and Visual Narrative Suite to discuss their progress and brainstorm. Their ideas are varied, and multiple projects are being pursued at once. But what drives them as a unit is the concept of visual narrative: “Our society has increasingly become visual; the data that we get is exceedingly visual. We make sense of the world through narratives, and most of our communication is essentially media direct to technology,” says Jhala. The visual narrative group examines: “the ways in which we can develop new technologies that enable better communication through visual narratives, make technologies that actually understand at a deeper level the narratives that make sense to us and the meaning they provide to us, as well as understanding us through analysis of narratives that we generate.”

One completed project is housed in the James B. Hunt Jr. Library. It’s an activity called (Not So) Silent Movie that was developed last year by the cluster’s artist-in-residence, Hope Hutman. Accessed in the iPearl Immersion Theater, which includes wraparound screens, it offers passersby the opportunity to create short silent films using their choice of movie clips from old black and white Westerns and quotes drawn from Jane Austen’s Sense and Sensibility. Jhala intends to collect data from the installation that considers human-computer interaction.

The group’s largest undertaking so far is the Urban Panorama Project. It’s in the planning and research stages, but the group is pairing with the NCSU libraries and archives in order to develop a tool to geolocate historical photographs of Raleigh. Using images from the archive’s collection (and eventually, they hope, from the State Archives), technologies like Google Streetview, and computer vision and machine learning techniques, cluster members are developing an algorithm that will enable a computer to identify where old images have been taken. Everyone’s skills come into play—Berreth’s architectural knowledge and Freitas’ historical expertise can pinpoint features on buildings that give them a sense of when an image was taken; Wu and Jhala will engineer the algorithms that recognize physical traits of the buildings and landscape.

Booker, who helped make hiring decisions about the cluster, says: “When I look at this cluster, what I think about is how unlikely it is that we were able to do this; it’s not happening anywhere else. It’s a tribute to the Visual Narrative hiring committee that we exist, and frankly to [Professor of English/Women’s Studies] Laura Severin, who gave us practical advice throughout the entire process. It’s a huge, cool opportunity, and I think it’s already paying off because of this Urban Panorama Project, which did not exist in anybody’s mind or heart before the four hires came to campus. I’m genuinely impressed by their collaboration and desire to make things.”

With the help of Mike Nutt, Director of Visualization Services for the NCSU libraries, they are pursuing funding from the National Endowment for the Humanities for this project. Nutt also helped coordinate the installation of (Not So) Silent Movie. “The cluster is special to the libraries for several reasons,” he says. “It’s one of the only clusters that has a humanities component, and just because of what we do and the collections we provide, there’s a natural fit there in trying to uncover narrative-driven research. The other reason it’s important to us is the Hunt Library is a storytelling building; these spaces are made for faculty and students to tell the stories of their research and learning that’s happening on campus. In some ways, they’re a perfect fit for the technology we provide in this building and in D.H. Hill as well, so we recognize that their presence and their founding was a good opportunity for us.”

“These spaces are made for faculty and students to tell the stories of their research and learning that’s happening on campus.”

Jhala, Berreth, and Freitas also received a DELTA Grant for their proposal: “Visual Narrative Multi-disciplinary Project Studio.” The grant “allows us to explore different platforms so we can effectively engage students from all these different fields,” says Jhala. He explains that the tools used in each field, as well as the teaching methods, vary greatly. The grant gives way to opportunities to cultivate what most interests students. Berreth explains: “We could say, ‘bring me your most interesting storytelling idea using these technologies,’ and pitch it to our students, and then the students would pick projects they could get really excited about.” He envisions the outcome of these projects leading to internships, exhibitions, and future career opportunities.

Their goal is to hold classes across the disciplines of the humanities, design, and engineering at the same time slot during a given semester so students in those classes can benefit from cross-disciplinary exposure and collaboration.

Like cluster faculty, students will be learning to “respect other disciplines’ methods and outcomes,” says Jhala. “Through direct interactions with practical projects, they will develop the skills as they go. What I would want students to get out of this is not only the ability to be successful interdisciplinary researchers but also to understand the workflows and work ethics and values of the other disciplines. And more practically speaking, I want them to come out with portfolio projects so they are immediately attractive to industry, foundations, or grad schools, etc.”

In addition to collaborating with the library, the Visual Narrative group may pair with other clusters. Overall, Berreth is enthusiastic about developing solutions that can be shared not just across the university but with anyone: “There are a lot of people that are huge champions of the open source movement, the ideas of open design, certain realms in which you build capabilities and tools and things that you share with the community freely hoping that other people will do the same and everyone’s work can be helped.” Indeed, he’s already undertaken several projects with partners at Duke University, including a “smart trowel” and an interactive table that responds to 3-D objects.

Overwhelmingly, the members of the Visual Narrative cluster and their collaborators are excited for what the future holds. “We will come up with very interesting projects for sure,” says Jhala. “Whatever project we’ll take on, we’ll do very well because of the dynamics we’ve generated in the last year.”