Harmonie






PaperMythXR
Role:
The design of animation


PaperMythXR is a mixed-reality(MR) Chinese paper-cutting with mythological storytelling through XR technologies. Users interact with a physical book to trigger 3D animations and progress through the story by completing paper-cutting tasks, transitioning from passive observers to active creators. The system introduces a "Task-Creation-Narrative" cycle, dynamically integrating handcrafted paper-cuts into the evolving virtual narrative. Leveraging Pan Chang patterns to seamlessly connect physical and virtual spaces, PaperMythXR showcases the potential of XR in cultural preservation. 


The system comprises three interconnected modules that work in concert to deliver a cohesive experience.

The Physical Book Tracking Module serves as the foundation for physical-virtual interaction, implementing image tracking through image matching algorithms that recognize seven normal images and one incomplete image in the picture book. This module incorporates a triggering system for tracked image update events and calling animation functions, enabling seamless transitions between physical book browsing and virtual content display. 

The Paper-Cutting Task Module facilitates hands-on creative interaction through spatial UI buttons and interactive guidance for folding and cutting operations. It integrates physical tools including papers and scissors with virtual tutorials, creating a practice-based learning environment that bridges physical creation and digital guidance. 

The Dynamic XR Animation Module controls visual content presentation utilizing Vertex Animation Texture (VAT) shader technology, accompanied by precise time and position control systems.

      
                     Dual-layer Interaction Framework of PaperMythXR

To enhance narrative immersion, we establish a cohesive visual language that bridges physical and digital realms through traditional Pan Chang patterns. These patterns serve as visual anchors, maintaining continuity between the physical book's artwork and digital animations.



 Blender animation testing


 Export animation


The system is developed based on Unity, ARFoundation, and Polyspatial. Custom Unity scripts are used to control the timing of subtitles and animations, ensuring a well-designed delay between text and animations to effectively prevent information overload while gradually guiding users into the narrative. Image Tracking is utilized to recognize content on physical book pages and trigger tasks, while the paper-cutting tutorial module dynamically guides users through tasks, supporting both gesture and eye-tracking interactions.  

©harmonieyilinzhang0121@gmail.com