Part 1: Blender Simulation Environment
Our first step to create a new simulation is to generate the scene in Blender. We did this by first organizing all scene objects, then applying Blender's built-in rigidbody property. Below is an example forpachinkotestwithtriangles.
Part 2: Blender Export Data Structure
After fully rendering the scene in Blender, we needed to import the position and rotation data for each object at every timestep into our ViCMA Simulator. We were ultimately able to do this using Blender's scripting API bpy.
The script generates two important resources, a custom scene.json and a set of anim_files for each object.
scene.json
The scene.json contains initialization parameters for both static and dynamic objects in the scene and records their starting location, color settings, and render parameters for non-spherical meshes. Below is an example of a random entry in scene.json.
"Sphere.001": {
"start_color":
[0.75,0.26,1,0.9],
"end_color":
[0.3,1,0,0.9],
"origin":
[-8.361442565917969,26.15984535217285,29.98017120361328],
"radius": 1
}
anim_files
anim_files is a folder containing files labeled scene_name/object_name.txt which records position data of a dynamic object at each frame of the simulation.
In each file, every three entries cooresponds to one (x,y,z) position.
Interesting Debugging:
While implementing the export script for dynamic objects, we had an issue where the item.location paramater
for each object would not change. Upon debugging, we realized this was due to Blender using a similar object-world
space as described in the translations lecture. Thus, we were exporting out the positions in object space, which didn't
change throughout the animation. To resolve this, we first found the object2world matrix and multiplied it with the object
space vector to get the final world space vector.
v = item.data.vertices[vertex_ind].co
mat = item.matrix_world
v = mat @ v
Part 3: ViCMA Simulator Loading Pipeline
AnimationObject struct
To load the exported scene.json and anim_files into the simulator, we first created a new set set of AnimationObject structures that could load in and process the extra data.
Two structs, Spheres and Polygons, were implemented to handle this task. Below are some notable object parameters and functions that were implemented for each of these classes in the simulator.
int curr_frame;
int start_transition_frame;
vector<Vector3D> positions;
vector<Vector3D> velocities;
nanogui::Color start_color;
nanogui::Color curr_color;
nanogui::Color end_color;
shape objShape;
string name;
void render(GLShader &shader);
void simulate();
void reset();
The process of simulating AnimationObjects was simple, and consisted of updating the curr_frame parameter. The render() function would then use curr_frame to update the position of the rendered object based on positions.
Object Parser
The parsing of json files and anim_files was handled by modifications to loadObjectsFromFile. Basic key-switch operations were added to include the new polygons, and we also implemented the readFileAndGetPosVec function to convert each anim_file into vector<Vector3D> positions.
Example of Simplicity in Pipeline
Below, we provide an example demonstrating the simplicity of using our pipeline.
(2) Modifications to Cloth Simulation Rendering Pipeline
To setup the codebase for ViCMA implementation, we made the following modifications to the original cloth simulation pipeline:
- Built new rendering function for polygons; see below for our implementation:
void AnimationObject::renderPoly(GLShader &shader) {
Matrix4f model;
model << 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1;
shader.setUniform("u_model", model);
Vector3D centroid = Vector3D(0, 0, 0);
vector scaled_vertices;
for (auto &vertex : vertices) {
centroid += vertex;
}
centroid /= vertices.size();
for (auto &vertex : vertices) {
scaled_vertices.push_back((vertex - centroid) * 1 + centroid);
}
for (auto &face : faces) {
MatrixXf positions(3, face.size());
MatrixXf normals(3, face.size());
for (int i = 0; i < face.size(); i++) {
positions.col(i) << scaled_vertices[face[i]].x,
scaled_vertices[face[i]].y, scaled_vertices[face[i]].z;
normals.col(i) << vertex_normals[face[i]].x, vertex_normals[face[i]].y,
-vertex_normals[face[i]].z;
}
if (shader.uniform("u_color", false) != -1) {
shader.setUniform("u_color", this->curr_color);
}
shader.uploadAttrib("in_position", positions);
if (shader.attrib("in_normal", false) != -1) {
shader.uploadAttrib("in_normal", normals);
}
shader.drawArray(GL_TRIANGLE_FAN, 0, face.size());
}
}
- Incorporated spatial hashing for the ViCMA spatial neighborhood computation.
- Implemented per-object texturing by adding separate shaders for each object in
clothSimulator.cpp, mainly writing logic in ClothSimulator::drawContents().
- Built an pipeline for light scene creation; this was mainly used to experiment with the various shading implemented in the original Cloth Simulator.
(3) ViCMA Theory and Implementation
Overview
ViCMA is a visual control method for passive multibody systems, exploiting visual perception and attention for purely appearance-based control (as opposed to physically-based). In particular, given a set of keyframes for a multibody animation with a start/end appearance for all objects, ViCMA computes the optimal appearance transition keyframe for every object.
To do this, ViCMA computes a heuristic transition cost \(\phi_i^{vicma}\) for each object \(i\) in every frame, and valid appearance transition keyframes for each object are those with minimal cost.
anim_filesAnimationObject struct
(2) Modifications to Cloth Simulation Rendering Pipeline
void AnimationObject::renderPoly(GLShader &shader) {
Matrix4f model;
model << 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1;
shader.setUniform("u_model", model);
Vector3D centroid = Vector3D(0, 0, 0);
vector scaled_vertices;
for (auto &vertex : vertices) {
centroid += vertex;
}
centroid /= vertices.size();
for (auto &vertex : vertices) {
scaled_vertices.push_back((vertex - centroid) * 1 + centroid);
}
for (auto &face : faces) {
MatrixXf positions(3, face.size());
MatrixXf normals(3, face.size());
for (int i = 0; i < face.size(); i++) {
positions.col(i) << scaled_vertices[face[i]].x,
scaled_vertices[face[i]].y, scaled_vertices[face[i]].z;
normals.col(i) << vertex_normals[face[i]].x, vertex_normals[face[i]].y,
-vertex_normals[face[i]].z;
}
if (shader.uniform("u_color", false) != -1) {
shader.setUniform("u_color", this->curr_color);
}
shader.uploadAttrib("in_position", positions);
if (shader.attrib("in_normal", false) != -1) {
shader.uploadAttrib("in_normal", normals);
}
shader.drawArray(GL_TRIANGLE_FAN, 0, face.size());
}
}
clothSimulator.cpp, mainly writing logic in ClothSimulator::drawContents().
