Ziva Real Time Trainer

Overview

The Ziva Real Time Trainer (ZRTT) is a standalone application for Windows that can train a ZivaRT character for real time applications.

ZRTT Overview

Installation

  1. Unzip the provided “ZivaRT.zip” file to a place on disk. For the purposes of this document, this folder path will be “C:\ziva”. Inside the unzipped folder structure will be subfolders for each of the products provided in this beta release. Open “C:\ziva\ZivaRT\ZRTTrainer”

  2. Run the “Configure.bat” script to set up environment variables. This script adds environment variables necessary for third party libraries (such as the FBX SDK, Qt, etc).

  3. Run the ZRT Trainer executable.

Basic Workflow

The following steps are the basic workflow for loading in training examples and training a ZivaRT character with them. ZivaRT Trainer refers to a trained ZivaRT character model as a “Rig”.

  • Go to “Mesh”->”Load Mesh (OBJ)” to load a rest-pose mesh of your character from an .obj file.

  • Go to “Mesh”->”Load Animation (ABC)” to load mesh animations for your character from an alembic bake (.abc). The mesh animation sequence is loaded to the selected row in the multi-asset view at the bottom of the window.

  • Go to “Motion”->”Load Animation of Skeleton / Extra Parameters (FBX)” to load a skeleton animation for your character. This skeleton animation must correspond to the mesh animation you provided in the previous step. Note that the mesh and skeleton animations are paired together in a row of the asset-management window.

  • Repeat the above process as many times as needed to load multiple FBX/ABC training pairs, each one into a separate row of the multi-asset view.

  • Go to “Rig”->”Train”. A dialog box will appear with a range of parameters. Click “Train” in this dialog box to train the ZivaRT rig. A dialog will present once the training has finished.

  • Go to “Rig”->”Save” to export the trained ZivaRT character rig to disk as a .zrt file, to be used by one of the player applications (Maya or UE4).

View Options

ZRTT View Options

The view options bar allows a user to customize the view of the model. In order from left to right, these options are:

  1. View world axes toggle. This button will toggle visualizing the XYZ axes of the world coordinate system. Note that our coordinate system is identical to that used in Maya.

  2. Hide mesh.

  3. Show only mesh wireframe.

  4. Show only mesh geometry.

  5. Show both mesh geometry and wireframe.

  6. Show only mesh geometry in X-ray mode.

  7. Show both mesh geometry and wireframe in X-ray mode.

  8. Hide skeletons.

  9. Show skeletons with joints.

  10. Show skeletons with joints and axes.

  11. The Orbit checkbox when checked will spin the camera around the character at a constant speed.

Playback Controls

ZRTT View Options

ZRTT provides VCR-style playback controls that allows a user to play through the frames of animation.

The timeline allows a user to scrub through the frames of the animation.

Rewind

Resets the timeline to frame 0. Frame 0 is the “neutral frame”, showing the neutral shape and the neutral skeleton of the character. It is not a part of any animation loaded into the multi-asset table.

Play

Plays the animations at the chosen frame rate.

Framerate

Choose the frame rate using the drop-down menu. Options include “Play every frame” which is as fast as the device can run, or fixed framerates of “12 fps”, “24 fps”, or “48 fps”.

FPS

Shows the current refresh rate of the viewport.

Left Arrow

Step back one frame.

Right Arrow

Step forward one frame.

Total Frames

Shows total number of frames across animations. For the example shown the total is 42 frames.

Frame Number

Shows current frame. Can be set to any number between 0 and total number of frames.

Log

Displays logging information such as indicating that a trainer completed an action, or loaded a file.

Multi Asset Table

ZRTT Multi-Asset Table

The multi-asset table allows for easy configuration of multiple assets to be used for training or playing of a trained model. Each column of the table contains useful information for managing training assets. We will discuss each column from left to right.

Train

Checkbox, when enabled, ensures this mesh animation/skeleton animation pair will be used as input to the training algorithm. Both the mesh animation and the skeleton must be available for this pair to be able to participate in the training.

Mesh Animation [Alembic]

The name of the alembic file that stored the mesh animation sequence and was imported into this row. If no mesh animation has been imported, this field will be “<none>”. To load an mesh animation into a row in the table, select the row with your mouse, then go to “Mesh” -> “Load Animation (ABC)”.

Show

Checkbox, when enabled, shows the imported mesh animation for this row in the table.

# Frames

Number that indicates the total number of frames present in the mesh animation that was loaded from an alembic bake. If this is less than the number of Frames in the FBX (below), then the minimum of the two will be used for training.

Skeleton / Extra Parameters Animation [FBX]

The name of the fbx file that stored the skeleton or extra parameter animation sequence and was imported into this row. If no skeleton / extra parameter animation has been imported, this field will be “<none>”. To load a skeleton / extra parameter animation into a row in the table, select the row with your mouse, then go to “Motion” -> “Load Animation of Skeleton / Extra Parameters (FBX)”.

Show

Checkbox, when enabled, shows the skeleton animation for this row in the table.

Play

Checkbox, when enabled after a ZivaRT character has been trained, will show (in the viewport) the result of “playing” the ZivaRT character using this row’s skeleton / extra parameter animation. This option can be enabled simultaneously with the “Show” option of the mesh animation, and will render its results on top of the mesh animation. This enables one to inspect the training error. To see either just the “playing” of a ZivaRT character OR the “showing” of the mesh animation, check only the option you would wish to see.

# Frames

Number that indicates the total number of frames present in the skeleton / extra parameters animation. If this is less than the number of Frames in the ABC (above), then the minimum of the two will be used for training.