Skip to main content
How can we help?

Assembly AI Action

Posted

Overview

AssemblyAI is a position-independent part placement verification system. It uses AI-powered object detection to verify that parts are placed in the correct positions during assembly — regardless of where the assembly sits on the workbench or how it is oriented.

AssemblyAI works by using anchor parts as spatial reference points. During training, a supervisor defines an anchor (a fixed, recognizable feature on the assembly) and one or more parts that must be placed relative to that anchor. During live operation, the system detects the anchor, calculates the spatial transformation, and verifies that each part falls within its defined tolerance area.

Key Concepts

  • Anchor: A fixed reference point on the assembly that the system uses to track position and rotation. The anchor allows AssemblyAI to verify part placement regardless of where the assembly sits on the workbench.
  • Part: A component that must be placed in a specific position relative to the anchor.
  • Tolerance Area: A bounding region around each part’s expected position. If the detected part falls within this area, it is considered correctly placed.
  • Variant: A different version of a part or anchor that is interchangeable. For example, parts from different manufacturers may have different colors or shapes but are otherwise interchangeable. Variants are trained on separate image samples to improve detection reliability.

Part Detection Status

During live operation, each part reports one of three statuses:

NOT DETECTED | The part has not been detected in the camera view

DETECTED NOT IN POSITION | The part is detected but is not within its tolerance area

DETECTED IN POSITION | The part is detected and is within its tolerance area

A step passes only when all parts are DETECTED IN POSITION. The system will wait indefinitely on the step until this condition is met — the operator must correct any misplaced parts.

Training Workflow (Supervisor)

Training an AssemblyAI step involves capturing a reference image, identifying the anchor and parts and defining tolerance areas.

Step-by-Step

1. Capture a reference image — this becomes the base image sample for the step

2. Identify the anchor

  • Choose a selection mode: Multi-click (click on the anchor to refine the mask) or bounding box (draw a rectangle around the anchor).
  • The colored mask over the object indicates what the AI model will be trained on, so ensure this completely covers the object of interest.

3. Add anchor variants (optional)

If the anchor may appear differently (e.g. different manufacturer), capture a new image sample and identify the variant

4. Identify one or more parts

The captured image is displayed with the anchor overlay for reference. Identify the part using multi-click or bounding box selection mode.

5. Set the tolerance margin

  • A slider (0–100%, default 25%) controls how much the tolerance area expands beyond the part’s bounding box
  • The tolerance area is displayed on the image for visual confirmation
  • The tolerance box can also be manually resized or moved by clicking and dragging

6. Add part variants (optional)

Same workflow as adding anchor variants — capture a new image and identify the part variant

7. Repeat steps 4–6 for additional parts on the same step

8. Finish — the system trains a per-step AI model using the identified anchors and parts

Tips for Training

  • Choose a distinctive anchor: Pick a part or feature that is always present, visually distinct, and unlikely to be confused with other components.
  • Set appropriate tolerance: A tolerance of 25% is a good starting point. A setting too tight is likely to cause false failures; one too loose may allow misplacements to go undetected.
  • Multiple parts per step: A single AssemblyAI step can verify multiple parts against a single anchor.
  • Variants matter: If parts come from different suppliers with visual differences, add variants to ensure reliable detection across all versions.

Editing an AssemblyAI Step

AssemblyAI steps can be edited like any other step type. From the assembly edit page, use the step card’s Edit button to:

  • Re-identify the anchor or parts
  • Adjust the tolerance margin
  • Add or remove variants
  • Add additional parts

Live Operation (Operator)

During a live assembly run, the operator sees a reference image (captured during initial assembly training) and the live camera feed from their workspace.

The operator’s task is to match the reference image with the live assembly by placing parts in their correct positions.

How It Works

1. The system waits until the anchor is detected in the live camera feed

2. Once the anchor is found, the system indicates the area in which the parts for this step must be installed

3. Thumbnails for each expected part are displayed next to a status report (DETECTED IN POSITION, DETECTED NOT IN POSITION, NOT DETECTED)

4. The step passes when all parts are DETECTED IN POSITION

5. The assembly advances to the next step

AAI Capture Review

A global setting called AAI Capture Review allows the operator to review the captured image that will represent the step in the QA report. When enabled:

  • After a step passes, a review modal appears showing the automatically captured image
  • If the image is not ideal (e.g. the operator’s hand was in the scene), the operator can recapture the image
  • This ensures the QA report contains clean, representative images of each completed step