Scene-Aware Behavior Synthesis for Virtual Pets in Mixed Reality

Wei Liang¹    Xinzhe Yu¹    Rawan Alghofaili²    Yining Lang³    Lap-Fai Yu²

¹Beijing Institute of Technology    ²George Mason University    ³Alibaba Group   

Abstract

Virtual pets are an alternative to real pets, providing a substitute for people with allergies or preparing people for adopting a real pet. Recent advancements in mixed reality pave the way for virtual pets to provide a more natural and seamless experience for users. However, one key challenge is embedding environmental awareness into the virtual pet (e.g., identifying the food bowl’s location) so that they can behave naturally in the real world.

We propose a novel approach to synthesize virtual pet behaviors by considering scene semantics, enabling a virtual pet to behave naturally in mixed reality. Given a scene captured from the real world, our approach synthesizes a sequence of pet behaviors (e.g., resting after eating). Then, we assign each behavior in the sequence to a location in the real scene. We conducted user studies to evaluate our approach, which showed the efficacy of our approach in synthesizing natural virtual pet behaviors.

Keywords

Virtual Pets, Behavior Synthesis, Scene Semantics.

Publication

Scene-Aware Behavior Synthesis for Virtual Pets in Mixed Reality
Wei Liang, Xinzhe Yu, Rawan Alghofaili, Yining Lang, Lap-Fai Yu
ACM Conference on Human Factors in Computing Systems (CHI 2021)
Paper, Supplementary, Video

BibTex

@article{vrpet2021,
    title= {Scene-Aware Behavior Synthesis for Virtual Pets in Mixed Reality},
    author = {Liang, Wei and Yu, Xinzhe and Alghofaili, Rawan and Lang, Yining and Yu, Lap-Fai},
    journal={ACM Conference on Human Factors in Computing Systems (CHI 2021)},
    series = {CHI '21},
    year = {2021},
    publisher = {ACM},
    location = {Yokohama, Japan},
    keywords = {Virtual Pets, Behavior Synthesis, Scene Semantics}
}

• 媒体计算与智能系统实验室


• Media Computing and Intelligent Systems Lab

Beijing Institute of Technology Copyright Address: 5 South Zhongguancun

Street, Haidian District, Beijing Postcode: 100081