Importing Rokoko character animations into Xcode using Blender
Jun 15, 2024
Rokoko provides an at home motion capture solution. The following articles explains how to import these animations across to a 3D character in Xcode, using a combination of Blender and a custom made cleanup tool called xcode-collada.
This article extends upon the previous blog article "Importing Mixamo characters into XCode". It assumes you already have a character in Xcode and now wish to add custom animations outside of those provided by Mixamo. In particular, I wanted to explore how animations could be taken from the Rokoko motion capture solution. Rokoko provide both a suit option, as well as a free camera based solution - give-or-take a licence fee for the edition that provides dual camera recordings.
As before, before jumping into some of the hurdles, let's cover off the proposed workflow.
Workflow
To make your animation compatible with the character previously rigged using Mixamo, we need to bring both the Mixamo character and the Rokoko animation into Blender and then "retarget" the animation onto the character. The following describes the steps to retarget the animation.
One-time set up
Export your Mixamo rigged character in a t-pose
Set your character into Blender (one-time step). An official guide can be found here, featuring this video tutorial
For each animation
Use the Rokoko Vision tool to capture an animation
Export as an
FBX
file.Import your
FBX
animation into Blender alongside your character and retarget to your characterExport animation from Blender as
DAE
fileClean up
DAE
files for use in Xcode (using the xcode-collada tool, discussed below)Import into Xcode
As a lot of above is already documented online, the rest of this article focuses on the solutions I devised to make this workflow successful.
Issue Summary
As with using the Mixamo animation, a number of issues arose for me, including:
Camera capture failed to work with Safari and required some juggling to work with Chrome
Two of the Blender plugins failed to install properly and one subsequently errored when trying perform a function
The exported DAE file required clean up before Xcode would recognise it
The rest of the article discusses the solutions used to overcome these issues.
Issue: Finicky Camera Capture
When using the camera-based capture method with Rokoko, it utilises the MacOS continuity feature to activate both the built in webcam and a connected iPhone. Impressively, with Safari this worked without any configuration. Unfortunately, with this browser the video is strangely cropped as pictured below.
Using Chrome fixes this issue, however getting both camera's to work at the same time is a pain. Some guidance is provided on the Rokoko Vision site. It recommends opening Photo Booth, switching to the iPhone camera and then refreshing the Rokoko Vision page. In addition to this, you should ensure Chrome has permission to access the camera by checking the permissions:
You should see both your laptop and phone cameras listed here. If not, I found I had to keep juggling the between Quicktime, Safari and Chrome.
Issue: Blender Plugins
As described in Rokoko's own Blender retargeting guide there are a number of plugins required to complete this process, including:
CATS (official plugin broken, see below)
The latter two both failed to install for me when using Blender 4.1.
Fixing Mixamo Plugin
Someone has kindly already resolved the issue. As noted on this blog post you need to unzip the plugin and make the following changes:
Update the top of
version.py
to the following:
Update
armature.py
to the following:
Credit to the author who discusses these in this video. You will need to rezip the files and reinstall in Blender.
Fixing the CATS plugin
Similar to the previous plugin, luckily someone has already done the work. There's no need to edit any code, you can download their version of the plugin on the unofficialcats GitHub page. Like before, ensure you have the files zipped up and reinstall them in Blender.
Issue: Xcode fails to load the Collada (.dae) file (again)
As mentioned in the workout, you will need to export the retargeted animation from Blender as a DAE
file. Unfortunately, similar to issues discussed in the previous article, Xcode is not able to load this file without cleaning it up. Some of the issues that it will complain about:
Multiple
<animation>
tags instead of just oneNames and IDs can be prefixed depending on how it was import/exported from Blender.
Some extra
nodes
may exist.
I wrote a tool called xcode-collada to automatically resolve a lot of these issues. You can install and use this by running:
You can then import the file into Xcode and load them following the instructions in the Mixamo animations article.