Social Butterfly Experiment
Producer & Set Fabricator
Asana, Google Drive, Discord, UNITY/C#, GIT/GITHUB, QUALYSIS MOTION CAPTURE, ADOBE PHOTOSHOP, PROCREATE, pen/pencil on paper & other physical materials
RELIVE THE EXPERIENCE ON the team project site here!
~ Winner of the Immersive Media award at the 2024 AET Showcase Inside Out ~
Welcome to Professor M’s Lab for Investigating Butterflies (LIM)!
The Social Butterfly Experiment is a collaborative 1-4 player immersive, narrative-driven game experience. Players are onboarded through themed physical elements and gamemaster instruction to then guide monarch butterflies on their migration journey from the Great Lakes to Mexico in a game projected on a large screen with real-time motion capture, 360 spatial audio, and physical lighting elements. The players wear motion capture headtrackers to control drones that guide the butterflies in-game. This project is inspired by real-world research on monarch butterflies and was crafted with thorough creative and technical documentation.
Front-to-back video of full experience put together by Phoebe Lin
PROJECT CONTEXT:
The Social Butterfly Experiment was developed in 8 weeks from February 2024 - April 2024 in the Spring 2024 AET Immersive Environments Senior Design Project with Professor Kyle Evans and Lab Tech Brandon LeJeune and was hosted in the Lab for Immersive Media (LIM) on the UT Austin campus.
Our team was the first class to use this space since the LIM opened in December 2023, with many unknowns integrating the LIM tech with each other and different technologies on both the student and faculty sides.
GOALS:
Team
Design a site-specific immersive experience with game elements and a social impact-driven narrative for the Spring 2024 AET Showcase Inside Out.
Utilize and connect the Qualysis motion capture system, physical lighting set up, and 360 ten channel spatial audio in the LIM.
Individual
Design a clear infrastructure for and manage a team of 9 (the largest team I’ve managed so far!) through a manageable, well-documented, and quality project that reflects industry standards and pipelines.
Balance team dynamics, ideas, and skillsets to consider each teammate’s desires for the project and reach compromises as a team through discussion and playtesting.
Improve my production skills in project management, cross-team communication, and upkeep of documentation.
Utilize and adapt my existing game production documents (game design document, asset trackers) and research knowledge (citing works) to the needs of the team and project.
Pitch in technically and creatively where I can.
THE TEAM:
Eriane Austria, Producer & Set Fabricator
Phoebe Lin, Creative Director
Kelly Zhang, Technical Artist
Jeremy Suh, Gameplay Designer
Tim Lin, Gameplay Engineer
Julio Martinez, 3D Modeler & VFX Artist
Juan Lopez, 3D Animator & Lighting Designer
Rayna Sevilla, Audio Designer
Sharon Chang, 2D & Texture Artist
Set Fabricator
set dressing
experience design
asset fabrication
sketching
resourcefulness
MY ROLES & Skills:
Producer
leadership
project & team management
documentation
technical direction
programming, design
My involvement: February 2024 - Present
IMPACT:
Team
We received great feedback on all elements of our project at the Spring 2024 AET Showcase Inside Out, including compliments on the physicality of the gameplay experience, the themed communication of information through the onboarding table, and the meaningfulness of the narrative. Several visitors suggested that we connect with monarch butterfly organizations about our project, and many learned more about monarchs, their journey, and how we humans can help, which communicated the strength of our project’s social impact. We were also voted as the winners of the Immersive Media award by showcase visitors!
After reflecting on our work and constructive feedback from visitors, we wish we would have done more playtesting throughout the development to have had more balanced gameplay and overall polish to the experience. Ultimately, our team was happy with the work we produced, especially having taken on the challenges of working on a large team in a new space with new tech!
Individual
Based on feedback from my team and an overall smooth development, I was able to successfully design for and manage a large team of 9 as the Producer through expanding upon my existing game production documents, maintaining thorough upkeep of task management and process documentation, and filling in needs in multiple departments to meet deadlines and develop a quality, well-documented project within our scope. Moreover, I felt the most successful in my roles, especially as Producer, the week of AET Showcase because I got to see my teammates glow upon seeing their work come together and be appreciated by so many people — we truly felt like a team as we presented our work to the world! I also felt very touched and validated by my teammates upon hearing their positive feedback on my skills and efforts as their Producer. As the Set Fabricator, my teammates and showcase visitors complimented the detail in set fabrication and how the onboarding table contextualized the experience. Translating my team’s vision into physical elements was deemed a success.
Taking care of such a large team as developers and people was daunting at first, but I’m glad that we were up for and met the challenge. I’m very grateful for my teammates who trusted my leadership, delivered amazing work, and were great company throughout this project.
THE JOURNEY
My Responsibilities & deliverables
PRODUCER
Responsibilities
As the Producer, I:
developed materials for and managed overall project management, scope, pipelines, and documentation using and checking Google Suite, Asana, Discord, Unity, and Github
set up infrastructure that reflects industry practices and aligns roles with teammates' strengths and interests
provided guidance and feedback on all elements of project to ensure consistency, quality, and working within scope
coordinated work and playtesting sessions in the LIM and documented findings into action items
led technical direction of full project implementation in the LIM with guidance from LIM staff which connected QTM, Unity, Ableton, and Resolume
assisted with programming, asset implementation, and design tasks where needed
led communication with LIM staff on scheduling lab time, progress updates, and resolving development and logistical questions
DELIVERABLES: organizational documents
I set up the following materials except for Playtesting Documentation in Pre-Production and updated them through the completion of this project. I used Google Drive (Docs, Sheets, Slides) and guided the creation of the Art Direction Guide on FigJam for the following:
Management, pipelines, & timeline overview spreadsheet
I developed this Google Sheets spreadsheet detailing management, pipelines, and overall project timeline which includes roles, responsibilities, and notes for each team member; organization of technical and creative mini-teams; and defining tools, use of tools, and safeguards to keep in mind when using the tools for the development of this project. I drew on build knowledge and background research to develop a team infrastructure that reflected industry practices and kept manageable for me as Producer.
This was one of the first documents I created in Pre-Production, which I updated throughout the full development of the project. Having kept mental notes on my teammates’ individual introduction presentations early in the semester, I prioritized each person’s strengths and project interests in defining everyone’s roles and responsibilities and checking in on these assignments before we dove into Production.
Designing and communicating clearly defined roles, workflows, and weekly deliverables was essential to establish early on for a team this large so as to keep everyone on the same page for what they’re responsible for and who to talk to for what topics. Showing the bigger picture and its moving parts in this spreadsheet also allowed for the team to work more autonomously with an understanding of boundaries, while also keeping feedback on project direction open for all to pitch in.
PRE-PRODUCTION PLANNING
UPDATED POST-PRODUCTION
game design document (gdd) template
I adapted my existing Game Design Document (GDD) template to this team and project by removing sections documented elsewhere and creating a narrative section. The GDD primarily served as a single source of truth to reference for the team to ensure clear, consistent, accessible communication about the project throughout development. The GDD also helped me with assigning initial tasks early in Production as I could easily identify obvious assets needed, as well as gaps within the game design and creative direction to fill. Furthermore, the team could look to the GDD to evaluate consistency between all project elements.
This template was then fleshed out with game design by Jeremy and with narrative by Phoebe.
GDD TEMPLATE
FINAL GDD USED BY JEREMY AND PHOEBE
ART ASSET TRACKER SPREADSHEET
I expanded upon my existing Art Asset Tracker spreadsheet by adding dropdowns and more fields that describe more about the asset, creator, and status. Because we had many creatives on the team with many assets I anticipated to document, I added such fields like task assignment, asset requests, whether the asset was self-made or outsourced, and whether the asset was implemented into Unity to make asset use and credits easier in Post-Production.
Julio, Kelly, and I were the main users of this spreadsheet, logging and updating throughout development. My edits to this Art Asset Tracker helped tremendously in filling out the Outsourced Asset Credits spreadsheet.
SNAPSHOT OF CREATION IN PRE-PRODUCTION
OUTSOURCED ASSET CREDITS SPREADSHEET
This was a new spreadsheet I created that draws on the Art Asset Tracker, Rayna’s Audio Library, the Git/Github repo with our Unity project, the FigJam Art Direction Guide, and Phoebe’s Research document to fill its contents. My teammates and I documented outsourced assets in the aforementioned tools/documents, which helped me document these assets and credit their creators in more detail, such as according to the license stated on the asset webpages. Kelly helped me document changes made to art assets, as well as confirmed which models were used in the final work. I then formatted the information logged here onto our team website to formally and publicly credit.
SNAPSHOT OF ART ASSETS TO CREDIT IN POST-PRODUCTION
“where is ___?” document
This document was created for easy access to frequently visited links and documents.
PLAYTESTING DOCUMENTATION & scoping
I created the outlines for playtesting documentation for our 4/9 Full MVP Open Playtest, 4/16 Polish Playtest, and 4/18 Showcase Debrief. Phoebe, Kelly, and I mainly filled out the observations and feedback. After each playtesting session, I read the findings, prioritized which changes to make based on importance and scope, and then translated those into action items via Discord communication and task assignment in Asana.
DELIVERABLES: PROJECT MANAGEMENT & DOCUMENTATION
I set up a team Discord server before Pre-Production during very early brainstorming sessions. I set up and organized the 9-person team Discord server, Google Drive, and Asana in Pre-Production, which I continued to manage and update through the completion of the project.
SNAPSHOTS
TEAM DISCORD SERVER ORGANIZATION & ANNOUNCEMENTS
I invited my teammates to and set up the Discord server according to my Management, Pipelines, & Timeline Overview spreadsheet, which included discipline-specific channels organized by mini-teams, as well as an announcements and important links channel. At the start of every week, I sent out an announcement describing the week’s deliverables and plans during in-class worktime to keep everyone on the same page. I sent additional announcements as needed, for example when pivots in production schedule arose. I frequently kept up with the Discord server to stay on top of team discussions, understand and resolve where conflicts or confusion existed, and update production documents and tools promptly.
Giving feedback & staying within scope after 4/9 playtest
ASANA SET UP & UPKEEP OF PROJECTS, TASKS, & CHECKING PROGRESS
I invited my teammates to Asana and used the Pitch Deck as a trial run for Asana as our primary project management tool because only a handful of teammates had used this before. After seeing that the team had some intuition for using Asana and briefly covering this tool in Pre-Production, I then set up Asana Projects by development phase and disciplines and maintained regular upkeep of task management. I created, assigned, organized, and updated tasks for my teammates based on the needs and timeline of the project.
I checked the Git/Github repo housing our Unity Project, Art Asset Tracker, Audio Library, Google Drive asset folders, and FigJam before messaging my teammates on Discord for updates on progress to reduce redundancy on my part and maintain Asana as the leading project management tool.
GOOGLE DRIVE SET UP & FILE ORGANIZATION
Google Drive housed our asset uploads, production documents, photo and video documentation, and other relevant files. I set up and maintained the file structure to keep the space organized and free of clutter.
PROJECT DOCUMENTATION VIA PHOTOS & VIDEOS
I took videos and pictures on my phone for documentation of project progress. During AET Showcase and Week 7, I took videos and pictures with my camera for better quality documentation of the final work for my team to use on the team site and our individual sites.
DOCUMENTING AND RELAYING INFO FOR TEAMMATES: Physical lighting & audio
The team did not have experience in Resolume, which was the suggested lighting software, and developments in physical lighting did not start until Week 3 due to our team’s lack of knowledge and waiting on the project to be further along to design the lighting with the gameplay and aesthetics.
I screen recorded videos scrolling through the game environments at different points in development, sketched potential lighting designs, and communicated information about the physical lighting set up and creative direction from the FigJam Art DIrection guide to Juan and the audio timing to Rayna through instruction and videos of the space and level design timing.
DOCUMENTING AND RELAYING INFO FOR TEAMMATES: Technical direction
In directing the full technical implementation of our project in the LIM, I set up the start of the block diagram for Kelly to visualize the connections between all the technologies used and also wrote the flow of OSC messaging across the different technologies in Asana. The technical direction was implemented by Tim, Jeremy, Brandon, and Kelly using Juan’s Resolume and Rayna’s Ableton files.
DELIVERABLES: ASSISTANCE WITH PROGRAMMING, ASSET IMPLEMENTATION, DESIGN
I assisted with programming, asset implementation, and design tasks where needed to keep us on track.
INTRO CUTSCENE IMPLEMENTATION
I implemented Sharon’s 2D animations in Unity and communicated timing to Tim to start gameplay and send OSC messages.
FINAL CODE
I was responsible for the EndStats2 and EndStatsText scripts and code working with cutscene game objects in Manager script
END CUTSCENE IMPLEMENTATION
I implemented and tweaked Sharon’s 2D animations in Unity and Unity Aimator; wrote scripts that reference Tim’s scripts to display game stats and added code within Tim’s Manager script to display certain assets depending on gameplay progression in Unity/C# in the Unity Canvas; and iterated on font, text, and layout with Sharon and Phoebe. These efforts culminated in end cutscenes responsive to gameplay conditions, in which gameplay stats are shown in the text, multiple animated elements are timed together, and animated elements are shown based on players’ gameplay progression.
PROGRESS @ MVP & OPEN PLAYTESTS
game music implementation in unity
I uploaded and implemented Rayna’s main composition, intro cutscene, success end cutscene, and fail end cutscene music in Unity/C#.
FINAL WITH ALL END CONDITIONS
PROJECT MONEYSHOT DESIGN COLLABORATION
I collaborated with Kelly and Phoebe on designing the “moneyshot” promo image of our project that was used in the AET Showcase voting form. We decided to merge their designs and the team’s game assets into the final moneyshot, which I compiled working with Julio and Sharon’s game assets, replicating LIM lighting in Unity and editing elements together in Adobe Photoshop and Procreate. Unsplash images were incorporated and the background photo was taken by Phoebe. I was also able to identify bugs and progress on game assets through this process.
SNAPSHOT OF FINAL IN POST-PRODUCTION
ASSET PREP FOR POSTER DESIGNS
I prepared assets for use in Phoebe’s poster designs by taking screenshots of game assets in Unity provided by Julio and Sharon and pictures of the headtrackers, cleaning them up in Adobe Photoshop and Procreate, and sending them to her to implement in Canva.
DELIVERABLES: team project site
BRINGING PRODUCTION DELIVERABLES ONTO TEAM SITE
Selected project progress and showcase documentation helped fill the overall site with content. I used my Milestones, Pipelines, & Timeline Overview spreadsheet; weekly Discord announcements; Asana; and Outsourced Asset Credits spreadsheet to write copy about the tools used and production timeline; format Phoebe’s links to research into APA citations; and translate outsourced asset info more formally into outsourced asset credits. Notably, I filled the “Our Journey” page with copy and media using the webpage design Phoebe had set up for me.
Set Fabricator
Responsibilities
As the Fabricator, I:
collected, created, and set up physical fabricated elements for the onboarding table according to creative direction and playtesting feedback for cohesive physical and digital experience in the LIM
collaborated on poster designs with Phoebe by translating game assets provided by Julio and Sharon into 2D poster assets
assisted with setting up informative signage (warnings about lighting, video/photos; giving feedback) designed by Kelly
DELIVERABLES: Process to final works
My Fabrication Deliverables included:
a fulfilled physical asset list for the onboarding table which included found objects, purchased objects, handmade assets, and printed designs
the physical set up of the onboarding table, headtracker labeling, and informative signage around the LIM
renderings of FINAL FABRICATION
The onboarding table was directed by Phoebe’s creative vision which conveys the narrative and Jeremy’s game design. I printed and placed Phoebe’s poster designs, Sharon’s map, and Kelly’s informational posters. I collected and/or created the remaining onboarding table objects myself, which includes found objects, bought objects, sketches based on Unsplash images and Phoebe’s research on monarchs, and self-constructed assets. Because playtesters previously had trouble identifying which player they were on screen, Brandon provided colored tape, and Kyle purchased colored LED lights which I used to color-code the head trackers according to their designated player particles in-game with Tim’s help.
I took the following videos and pictures.
IDEATION
When we decided to include fabrication in our project, Phoebe and Sharon suggested a large physical letter and envelope to onboard players into the experience. I decided to take on fabrication and sketched this early plan based on their suggestion.
PROGRESS AT OPEN & POLISH PLAYTESTs
This was the MVP for set fabrication where we decided on an onboarding table according to Phoebe’s creative direction. We experimented with different placements of the onboarding table to get players to spend time and interact with the physical assets in a seamless, immersive way.
REFLECTION
The Social Butterfly Experiment was a huge learning experience that helped instill confidence in me and my teammates in our ability to deliver quality, meaningful work in our respective disciplines. Because of the diversity of skills, experiences, and interests within our team, we were able to create a comprehensive experience with physical, digital, and never-been-done-before elements in the LIM. Furthermore, feedback received during AET showcase communicated the social impact of our project due to the cohesive physical and digital experience we achieved. The Social Butterfly Experiment expressed the plight of monarch butterflies and ways we humans can help them, with potential opportunities brought up by folks that could further bring this project into real-world research on monarch butterflies.
Lastly, big thanks to our playtesters, AET showcase visitors, Kyle, and Brandon for their support throughout this development of this project!