ZivaRT Training - Python Scripting

Overview

The ZivaRT Trainer can be controlled by Python scripts. These scripts can be run from inside the Trainer’s Script Editor Window, or via the command line when the Trainer is running in ‘No-GUI’ (i.e., headless) mode.

Script Editor Window

The Script Editor Window can be opened from the Trainer’s menu bar. Python script can be executed in the Script Editor by inputting it into the lower tab, and pressing “Execute” (or Ctrl-Enter). If you select a part of the script in the Text Editor and press “Execute” (or Ctrl-Enter), only the selected text will be executed.

ZRTT Scripting Window

Controlling the Trainer with Python

The ZRT Trainer exposes a Python module which contains functions that be can be used to control the trainer. For example, the following script trains a rig. A more complete script zrttTrainRig.py is available in the “scripts” folder of the distribution.

import zrtt

# Load data to use for training.
path = 'C:/root'
zrtt.loadMesh(f'{path}/mesh.obj')
zrtt.loadMeshAnimation(f'{path}/animation.abc', 1) # Load into row 1
zrtt.loadSkeletonAndExtraParametersAnimation(f'{path}/skeletonAnimation.fbx', 1) # Load into row 1
zrtt.loadMeshAnimation(f'{path}/anotherAnimation.abc', 2) # Load into row 2
zrtt.loadSkeletonAndExtraParameters(f'{path}/anotherSkeletonAnimation.fbx', 2) # Load into row 2

# By default, all motions participate in the training.
# Just for the sake of illustration, disable motion 1 for training.
zrtt.enableTrainingParticipation(1, false)

# Put the trainer into the mode where the patches are calculated from the patch map.
# This option is the default; so technically speaking, we don't need this command.
zrtt.usePatchesFromPatchMap()

# Load a patch map.
zrtt.loadPatchMap(f'{path}/myPatchMap.zrtPatchMap')

# Set the patch map to be automatically geodesic split and despeckled before training.
zrtt.autoGeodesicallySplitPatchMapBeforeTraining(True)
zrtt.autoDespecklePatchMapBeforeTraining(True)

# Set some training settings that override defaults.
settings = {}
settings['shapeDropTolerance'] = 0.05

# Train.
zrtt.trainRig(settings)
# Alternatively, you can train with defaults as follows:
# zrtt.trainRig()

# Save the trained intermediate rig.
zrtt.saveIntermediateRig('{path}/test.zif')

# Save the trained production rig.
zrtt.saveProductionRig('{path}/test.zrt')

# Profile the production rig.
profileResult = zrtt.profileRig()
print('Results of profiling:')
print('=====================')
print('Profiling succeeded: ' + str(profileResult[0]))
print('Num used frames: ' + str(profileResult[1]))
print('Execution time per frame [msec]: ' + str(profileResult[2]))
print('Training error per vertex per frame [scene units]: ' + str(profileResult[3]))

Running from the command line

Python scripts can also be executed from the command line. In this mode, no GUI is displayed.

zrtTrainer.exe --no-gui script.py

Python Methods

The ZRTT Python module supports the following functions:

zrtt.newProject()

Creates a new ZRT Trainer project.

zrtt.openProject(filename)

[Only available in Python in the ZRTT GUI; not in the standalone Python module.]

Opens a project from a file. If there are any soft-linked Alembic mesh animations that cannot be found on disk, they are simply not loaded.

zrtt.saveProject(filename)

[Only available in Python in the ZRTT GUI; not in the standalone Python module.]

Saves the current project to a file. In order to keep the file small, the Alembic mesh animations are not saved, but are instead soft-linked.

zrtt.loadMesh(filename)

Loads an OBJ (.obj) file as a mesh into the ZRT Trainer.

zrtt.loadMeshAnimation(filename, row)

Loads a mesh animation from an ABC (.abc) file into the specified motion row. For consistency with the GUI, rows are numbered starting from 1. Empty rows are added as needed if the requested row index (“row”) is larger than that of the largest currently non-empty row.

zrtt.loadSkeletonAndExtraParametersAnimation(filename, row)

Loads a skeleton (and/or extra parameters) from an FBX (.fbx) file into given row of the multi-motion table. For consistency with the GUI, rows are numbered starting from 1. Empty rows are added as needed if the requested row index (“row”) is larger than that of the largest currently non-empty row.

zrtt.enableTrainingParticipation(row, enable)

Enables or disables the training participation of the given multi-motion table row. For consistency with the GUI, rows are numbered starting from 1.

zrtt.bakeToMeshAnimation(row, filename)

[Only available in Python in the ZRTT GUI; not in the standalone Python module.]

Bakes the animation in the given row to a mesh, using the trained ZivaRT rig. For consistency with the GUI, rows are numbered from 1.

zrtt.transferSkinningWeights(selectedVertices)

Transfer skinning weights from unselected mesh vertices to selected vertices.

zrtt.usePatchesFromJointImpactMaps()

Instructs the trainer to use the joint impact maps to generate the patches. This option is mutually exclusive with “usePatchesFromPatchMap”.

zrtt.usePatchesFromPatchMap()

Instructs the trainer to use the patch map to generate the patches. This option is mutually exclusive with “usePatchesFromJointImpactMaps”. This is the default option.

zrtt.loadJointImpactMaps(filename)

Loads joint impact maps from a file.

zrtt.saveJointImpactMaps(filename)

Saves joint impact maps to a file.

zrtt.generateJointImpactMaps(row, relativeSmallDistance)

Generates joint impact maps from training data.

zrtt.loadExtraParameterImpactMaps(filename)

Loads extra parameter impact maps from a file.

zrtt.saveExtraParameterImpactMaps(filename)

Saves extra parameter impact maps to a file.

zrtt.generateExtraParameterImpactMaps(row, relativeSmallDistance)

Generates extra parameter impact maps from training data.

zrtt.loadCorrectivesMask(filename)

Loads a correctives mask from a file.

zrtt.saveCorrectivesMask(filename)

Saves the correctives mask to a file.

zrtt.loadPatchMap(filename)

Loads a patch map from a file. Only applicable when “usePatchesFromPatchMap”.

zrtt.savePatchMap(filename)

Saves the patch map to a file. Only applicable when “usePatchesFromPatchMap”.

zrtt.generatePatchMap(numPatches, positionWeight, impactMapWeight, seed)

Generates a patch map from the geometry and impact maps. Only applicable when “usePatchesFromPatchMap”. The position and impact weights control the importance of geometry (vertex positions) and the impact weights. Seed controls the initialization of the internal random number generator; to obtain repeatable results, pass the same seed value.

zrtt.geodesicallySplitPatchMap()

Modifies the patch map by splitting patches into smaller patches, based on a geodesic distance relative to edge lengths around a vertex. Only applicable when “usePatchesFromPatchMap”. Use this function if you want to break patches that consist of multiple connected components into separate patches. However, patches that are sufficiently close in Euclidean distance are kept together. You typically run this command after “generatePatchMap”. This command is also run automatically before the training.

zrtt.autoGeodesicallySplitPatchMapBeforeTraining(enable)

Toggles on/off the automatic geodesic splitting of the patch map before training. Default is on.

zrtt.setDespecklePatchMapSettings(numIterations, minPatchSize)

Sets the settings for patch map despeckling. They will be used in all subsequent calls to “despecklePatchMap”. If you never call this function, the default values are maxIterations=3 and minPatchSize=20.

zrtt.despecklePatchMap()

Modifies the patch map using despeckling, so that no patch is smaller than minPatchSize and long thin patches (“hair”) are removed. Only applicable when “usePatchesFromPatchMap”. Uses the parameters previously set by “setDespecklePatchMapSettings”. The parameter numIterations controls how many iterations of alternating small patch and hair removal are performed. Typically, a small number such as 3 is sufficient.

zrtt.autoDespecklePatchMapBeforeTraining(enable)

Toggles on/off the automatic despeckling of the patch map before training. Default is off.

zrtt.geodesicSplittingAndDespecklingRespectSmoothingGroupBoundaries(enable)

When enabled, geodesic splitting will split the patch map at smoothing group boundaries, and despeckling will process each group separately.

zrtt.loadPatchSmoothingIterations(filename)

Load smoothing iterations vertex map to control the automatic patch blend weight generation process. Only applicable when “usePatchesFromPatchMap”.

zrtt.savePatchSmoothingIterations(filename)

Saves smoothing iterations vertex map. Only applicable when “usePatchesFromPatchMap”.

zrtt.setPatchSmoothingIterations(numSmoothingIterations)

Set a constant number of smoothing iterations at each vertex. A good default is 3. Only applicable when “usePatchesFromPatchMap”.

zrtt.loadPatchSmoothingGroups(filename)

Load smoothing groups vertex map to control to control the automatic patch blend weight generation process. Only applicable when “usePatchesFromPatchMap”. Each smoothing group forms a smoothing island; namely, smoothing does not proceed across a group boundary.

zrtt.savePatchSmoothingGroups(filename)

Saves smoothing groups vertex map. Only applicable when “usePatchesFromPatchMap”.

zrtt.useSinglePatchSmoothingGroup()

Put all vertices into a single smoothing group. This is the default behavior. Only applicable when “usePatchesFromPatchMap”.

zrtt.loadIntermediateRig(filename)

Loads an intermediate rig from a file. Also automatically creates the production rig from the loaded intermediate rig.

zrtt.saveIntermediateRig(filename)

Saves the intermediate rig to a file. You can use this, for example, after training.

zrtt.loadProductionRig(filename)

Loads a production rig from a file.

zrtt.saveProductionRig(filename)

Saves the production rig to a file. You can use this, for example, after training.

zrtt.selectIntermediateRig()

Selects the intermediate rig as the rig that will be used for subsequent operations (rig profiling).

zrtt.selectProductionRig()

Selects the production rig as the rig that will be used for subsequent operations (rig profiling).

zrtt.trainRig(settings)

Trains a rig using the given training settings, where settings is a Python dictionary of values. Creates both the intermediate and production rig. If settings are not specified, default settings are used. See the Training Settings section below.

zrtt.trainSkinningOnlyRig()

Trains a rig that only performs skinning (no correctives). Creates both the intermediate and production rig.

zrtt.profileRig(filename)

Measures the performance and quality of the production rig, based on motions currently enabled for “training participation”. You can do this, for example, after the training. Computes average solver execution time, and training error. Returns a 4-tuple, whereby the tuple entries are as follows. 0: True/False, whether profiling succeeded. 1: total number of frames used in profiling. 2: average solver per-frame execution time in milliseconds. 3: per-frame per-vertex training error in scene units. The error is calculated by squaring the distance between each vertex in the mesh animation and solver output, summing across all vertices on all frames, dividing by the number of vertices times the number of frames, and taking a square root. All motions currently enabled for “training participation” participate in this calculation.

Training Settings

The list of settings can be found in Training Parameters. The Python type of each parameter are as follows.

Training data pre-processing

minPoseDistance float

maxShapeToPoseSlope float

poseDistanceApproximationTolerance float

ignoreJointTranslationsAndScales bool

Kernels

kernelSelectionTolerance float

Shape representation

maxShapes int

shapeDropTolerance float

Python Version

The ZRT Trainer comes with its own copy of Python 3.7, but can use a local installation of Python if the environment variable ZIVA_USE_SYSTEM_PYTHON is set to point to a local installation of Python 3.7.

Using the Standalone Python Module

Python module only works in Python 3.7. In order to use the zrtt Python module, make sure that zma.cp37-win_amd64.pyd is discoverable by Python, such as by setting the PYTHONPATH environment variable. Next, launch your Python interpreter. You can then use the Python module in the usual way:

import zrtt

# Load data to use for training.
path = 'C:/root'
zrtt.loadMesh(f'{path}/mesh.obj')
...

Python Helper Scripts

The following scripts are meant to serve as a detailed template for both generating maps to get started with, as well as train your character. Each of these python files are provided in the zrtTrainer/scripts directory.

zrttWorkspaceFilenames.json

A file to specify the names of each of the files to use for maps generation and training. Usage of this file is optional as you could certainly rewrite the below scripts to avoid using it, but we find it easiest to place all the filenames into a json, so that they are all listed just once in one place.

zrttGenerateMaps.py

A script to automatically generate impact maps and patch map from a wiggle ROM animation. These maps can alternatively be painted, or generated then modified via painting, but the intention of this script is to give an automatic “first draft” for the paintable maps ZivaRT needs.

zrttTrainRig.py

A script to load in previously generated maps, and train a ZivaRT character.

importExportMaps.py

A script that includes functions to import/export all ZivaRT maps to and from Maya, to enable a workflow where maps are painted in Maya.

setupMayaPaintNodes.py

A script to create skinCluster or zrtIntegerPaint nodes with the optimal settings for painting ZivaRT maps. You need to call functions from this script before using the import/export functions from importExportMaps.py.