Importing Rokoko character animations into Xcode using Blender

This article extends upon the previous blog article "Importing Mixamo characters into XCode". It assumes you already have a character in Xcode and now wish to add custom animations outside of those provided by Mixamo. In particular, I wanted to explore how animations could be taken from the Rokoko motion capture solution. Rokoko provide both a suit option, as well as a free camera based solution - give-or-take a licence fee for the edition that provides dual camera recordings.

As before, before jumping into some of the hurdles, let's cover off the proposed workflow.

Workflow

To make your animation compatible with the character previously rigged using Mixamo, we need to bring both the Mixamo character and the Rokoko animation into Blender and then "retarget" the animation onto the character. The following describes the steps to retarget the animation.

One-time set up

• Export your Mixamo rigged character in a t-pose

• Set your character into Blender (one-time step). An official guide can be found here, featuring this video tutorial

For each animation

• Use the Rokoko Vision tool to capture an animation

• Export as an FBX file.

• Import your FBX animation into Blender alongside your character and retarget to your character

• Export animation from Blender as DAE file

• Clean up DAE files for use in Xcode (using the xcode-collada tool, discussed below)

• Import into Xcode

As a lot of above is already documented online, the rest of this article focuses on the solutions I devised to make this workflow successful.

Issue Summary

As with using the Mixamo animation, a number of issues arose for me, including:

• Camera capture failed to work with Safari and required some juggling to work with Chrome

• Two of the Blender plugins failed to install properly and one subsequently errored when trying perform a function

• The exported DAE file required clean up before Xcode would recognise it

The rest of the article discusses the solutions used to overcome these issues.

Issue: Finicky Camera Capture

When using the camera-based capture method with Rokoko, it utilises the MacOS continuity feature to activate both the built in webcam and a connected iPhone. Impressively, with Safari this worked without any configuration. Unfortunately, with this browser the video is strangely cropped as pictured below.

Using Chrome fixes this issue, however getting both camera's to work at the same time is a pain. Some guidance is provided on the Rokoko Vision site. It recommends opening Photo Booth, switching to the iPhone camera and then refreshing the Rokoko Vision page. In addition to this, you should ensure Chrome has permission to access the camera by checking the permissions:

You should see both your laptop and phone cameras listed here. If not, I found I had to keep juggling the between Quicktime, Safari and Chrome.

Issue: Blender Plugins

As described in Rokoko's own Blender retargeting guide there are a number of plugins required to complete this process, including:

• Rokoko's Blender plugin

• Adobe's Mixamo plugin

• CATS (official plugin broken, see below)

The latter two both failed to install for me when using Blender 4.1.

Fixing Mixamo Plugin

Someone has kindly already resolved the issue. As noted on this blog post you need to unzip the plugin and make the following changes:

• Update the top of version.py to the following:

import bpy

class ARP_blender_version:
    _string = bpy.app.version_string
    blender_v = bpy.app.version
    _float = blender_v[0]*100+blender_v[1]+blender_v[2]*0.01
    #_char = bpy.app.version_char

blender_version = ARP_blender_version()

• Update armature.py to the following:

import bpy

def restore_armature_layers(layers_select):
    armature - bpy.context.active_object.data

    #Check if the active object is an Armature
    if not isinstance(armature, bpy.types.Armature):
        return
    
    #Iterate over bones and restare their original hide status
    for bone, hide_status in zip(armature.bones, Layers_select):
        bone.hide = hide_status


def enable_all_armature_layers():
    armature - bpy.context.active_object.data

    # Ensure the active object is an Armature
    if not isinstance(armature, bpy.types.Armature):
        return []
    
    layers_select - []

    #Iterate over all bones
    for bone in armature.bones:
        #Store the current hide status
        layers_select.append(bone.hide)

        #Make the bone visible
        bone.hide = False

    return layers_select

Credit to the author who discusses these in this video. You will need to rezip the files and reinstall in Blender.

Fixing the CATS plugin

Similar to the previous plugin, luckily someone has already done the work. There's no need to edit any code, you can download their version of the plugin on the unofficialcats GitHub page. Like before, ensure you have the files zipped up and reinstall them in Blender.

Issue: Xcode fails to load the Collada (.dae) file (again)

As mentioned in the workout, you will need to export the retargeted animation from Blender as a DAE file. Unfortunately, similar to issues discussed in the previous article, Xcode is not able to load this file without cleaning it up. Some of the issues that it will complain about:

• Multiple <animation> tags instead of just one

• Names and IDs can be prefixed depending on how it was import/exported from Blender.

• Some extra nodes may exist.

I wrote a tool called xcode-collada to automatically resolve a lot of these issues. You can install and use this by running:

yarn add xcode-collada
yarn run xcode-collada ./original.dae ./clean.dae

You can then import the file into Xcode and load them following the instructions in the Mixamo animations article.