Reconstructing Transforms from Baked FBX Vertices

There is a class of problem in the asset pipeline that looks trivial on paper and turns out to be genuinely interesting once you sit down with it. This one started with a request that sounded mundane: take an FBX exported from Maya containing a handful of cubes, and replace each one in Unity with a primitive cube at the same position, rotation, and scale. The use case was collider authoring. Artists would model the desired collider shapes in Maya as cubes, export them, and have an editor script swap them out for Unity primitives that could carry actual BoxCollider components.

The first attempt was three lines long. The final solution involved dot product orthogonality testing and rotation matrix construction from basis vectors. Here is how a five-minute task became an afternoon.

The setup that breaks the obvious approach

The straightforward version of this script writes itself. Walk the children of the imported FBX with GetComponentsInChildren<MeshFilter>, and for each one create a primitive cube whose transform matches the source. Three lines, ten if you count the boilerplate. I wrote it. I tested it. Every cube spawned at the origin with identity rotation and unit scale, stacked on top of each other.

The cause was Maya's Freeze Transformations. The artist had frozen the transforms before export, which meant every child GameObject in the imported FBX had a transform of (0,0,0), rotation (0,0,0), scale (1,1,1). The actual position, rotation, and scale data had been baked into the vertex positions of each mesh. Unity was reading exactly what was in the FBX, which was exactly nothing useful at the transform level.

There are two ways to handle this. The first is to ask the artist to stop freezing transforms. That is the correct answer in most studios, but in this case the export pipeline froze transforms for reasons that mattered downstream, and changing that would break things further along. The second way is to accept the baked vertices as the source of truth and reconstruct the transform from the mesh data. That is the path this took.

Position is free

Position falls out of the geometry immediately. A cube has eight unique vertices regardless of triangulation. The centroid of those eight vertices is the centre of the cube, which is the position you want.

Vector3 position = Vector3.zero;
foreach (Vector3 v in uniqueVertices)
    position += v;
position /= uniqueVertices.Count;

The only wrinkle is the word "unique". Unity's mesh vertex array splits vertices by UV seam and normal, so a cube can easily have twenty-four entries in the vertex array even though geometrically it has eight. Deduplication is straightforward: round each vertex to some small epsilon and use it as a hash key. Once that is done, position is solved.

Scale is almost free

Scale is the next easiest. A Unity primitive cube has a default size of one unit along each axis. If you can find the length of three perpendicular edges of the cube, those lengths are the scale.

The catch is "three perpendicular edges". From any corner vertex of the cube, three edges leave it. They are perpendicular to each other and they connect to three adjacent corners. The other four vertices are at face diagonals and the space diagonal of the cube. Distance from the corner gives a hint about which is which, but only sometimes.

For a uniform cube of any size, the three edge neighbours are the three closest vertices to the corner. Face diagonals are further away by a factor of square root of two. The space diagonal is further still. A naive script that picks the three nearest vertices and calls their distances the scale works perfectly, which is exactly the version I wrote next.

It worked on every test cube I had. Then I ran it on a cube the artist had stretched to a 1:1:10 proportion, and it produced a cube with scale (1, 1.4, 10). The Y axis was wrong by a factor of square root of two. That number is not subtle. It is the unmistakable signature of a face diagonal being mistaken for an edge.

The trap with elongated cubes

Here is the geometric fact that distance-based selection misses. On a cube with sides of length one, the edges are one unit long and the face diagonals are square root of two units long. Edges are always shorter. On a cube stretched to dimensions one by one by ten, the long edges are ten units, the short edges are one unit, and the face diagonals between long and short axes are square root of one hundred and one, which is about ten point zero five. The short edges are one. The face diagonals on the small faces are square root of two, which is about one point four. From any corner, the three closest vertices are not the three edge neighbours. They are the two short edges and one face diagonal, because the face diagonal on a one-by-one face is shorter than the long edge.

Picking by distance breaks. The fix is to pick by perpendicularity instead.

Dot products as the disambiguator

The dot product of two perpendicular vectors is zero. The dot product of two parallel vectors is the product of their lengths. The dot product of two vectors on a face diagonal pair from a shared corner is not zero, because those vectors lie in the same face plane and have a measurable angle between them.

This gives a robust test. Start from a corner. Look at every other vertex of the cube. For each candidate pair of vectors from the corner, compute the dot product of the normalised pair. The three edge directions are the only ones whose dot products with each other are at or very near zero. Face diagonals share a face with one edge, which means they share a plane with that edge, which means the dot product of a face diagonal with the edge it shares a plane with is not zero.

The algorithm becomes: from a starting corner, find any vertex whose vector from the corner is perpendicular to a second vertex's vector and a third vertex's vector, where the second and third are also mutually perpendicular. That triple is the three edges. Their magnitudes give the scale.

Vector3[] FindThreeEdges(Vector3 corner, List<Vector3> others)
{
    foreach (var combo in Combinations(others, 3))
    {
        Vector3 a = combo[0] - corner;
        Vector3 b = combo[1] - corner;
        Vector3 c = combo[2] - corner;
 
        const float eps = 0.001f;
        if (Mathf.Abs(Vector3.Dot(a.normalized, b.normalized)) < eps &&
            Mathf.Abs(Vector3.Dot(b.normalized, c.normalized)) < eps &&
            Mathf.Abs(Vector3.Dot(a.normalized, c.normalized)) < eps)
        {
            return new[] { a, b, c };
        }
    }
    return null;
}

The combinatorial search is cheap. Seven non-corner vertices give thirty-five three-element combinations to check, and the dot products short-circuit on the first failure.

Rotation from basis vectors

Once you have three orthogonal edge vectors, rotation falls out of them. Normalise the edges to get three orthonormal basis vectors. Those three basis vectors are the columns of the rotation matrix that takes a unit-aligned cube to the orientation of this cube in world space.

Unity does not expose a "build me a quaternion from three basis vectors" call directly, but Quaternion.LookRotation takes a forward and an up vector and constructs the rotation. Pick any two of the three orthonormal edges as forward and up. The third is implied. There is an ambiguity about handedness which can flip the rotation if you pick the wrong pairing, but for an authored cube whose vertex winding is consistent, this is stable.

Vector3 right = edgeA.normalized;
Vector3 up = edgeB.normalized;
Vector3 forward = Vector3.Cross(right, up);
Quaternion rotation = Quaternion.LookRotation(forward, up);

The cross product enforces a right-handed coordinate system regardless of the order in which the edges were discovered, which removes one source of bugs.

Putting it together

The full transform extraction is now: deduplicate vertices to get eight unique corners, take the centroid as position, pick any corner and find its three orthogonal edges by dot product, take the magnitudes as scale and the normalised vectors as basis for rotation. Apply that transform to a freshly created primitive cube, copy the source material over, strip the MeshRenderer and MeshFilter if you only want collider geometry, and you are done.

The tool exposes a small EditorWindow with options for whether to remove the visual components and whether to copy materials. Each generated cube is registered with Undo, so the entire conversion can be reversed with a single Ctrl+Z. The cleanup is the same pattern as everything else: name the generated objects with a recognisable prefix and provide a button to strip them out wholesale.

What this generalises to

The geometric trick is not specific to cubes. Any axis-aligned box has the same property that three perpendicular edges leave each corner, and orthogonality testing recovers them regardless of how the box is proportioned. The same approach works for rectangular prisms, which is what you get when an artist scales a cube non-uniformly before freezing the transforms.

It does not work for non-rectangular geometry. Rotated boxes give you the right answer because the edges remain orthogonal, but a sheared cube does not, and a cylinder approximated as a cube does not. For the use case of authoring colliders from primitive cubes, this is not a problem. For more exotic inputs, you would want to fit an oriented bounding box using principal component analysis on the vertex set, which is a different and more involved exercise.

The lesson I took from this is that geometric robustness is almost never about picking the right distance threshold. It is about picking the right invariant. Distance is the obvious one for finding neighbours, and distance is the one that breaks the first time the shape stops being uniform. Orthogonality is the invariant that actually holds, and once you switch to it, the elongated case stops being a special case.