A comic character’s next line appears only when the reader looks directly at them, as three-dimensional figures, speech bubbles and sound effects appear nearby.
Researchers at the Korea Advanced Institute of Science and Technology have developed an extended reality platform that turns comics and webtoons into spatial experiences shaped by a reader’s gaze, movement and physical surroundings.
KAIST said Tuesday that its ComiXR platform was designed to explore how digital comics could move beyond flat smartphone screens.
The research team also proposed a set of design principles for future extended reality comics based on a study involving human-computer interaction experts, webtoon creators and readers.
XR is a broad term covering virtual, augmented and mixed reality technologies.
Built for the Meta Quest Pro headset, ComiXR allows users to place three-dimensional characters, backgrounds, speech bubbles and written sound effects throughout a scene.
Participants in the study used the platform to adapt existing print comics into XR content and arrange each scene according to their preferred level of immersion.
Researchers found that users responded more positively to layouts that took advantage of spatial depth than to those that simply displayed flat comic panels in a virtual environment.
The platform also uses eye tracking to control the order in which dialogue appears. A character’s next line, for example, could remain hidden until the reader looks directly at the character, reducing the likelihood of seeing dialogue or plot developments too early.
Other features include visual effects triggered by a reader’s facial expressions and haptic feedback designed to convey sensations such as a character’s heartbeat.
Based on the user study, the team outlined four potential formats for XR comics.
The “Panel Gallery” format uses the walls of a room to display comic panels, while “Pop-up” places comic scenes above desks, walls or other surfaces in a manner similar to a pop-up book.
“Around Comic” incorporates three-dimensional characters and comic elements into outdoor spaces, while “Inclusive ComiX” focuses on making XR comics more accessible to readers, including those with low vision.
The team said XR comics were more likely to complement, rather than replace, smartphone-based webtoons.
“This research goes beyond simply turning comics into three-dimensional content,” said Ian Oakley, a professor at KAIST’s School of Electrical Engineering who led the research. “It presents ways to design future comics by taking into account the reader’s physical space, gaze and movement.”
