Robotics & AutomationEngineering Study
Vision-Guided Robotic Bin-Picking Cell with 3D Pose Estimation
A fulfilment client needed reliable bin picking without pose fixtures. We delivered a YOLOv11-Seg + 3D pose stack running on Jetson Orin Nano, fully integrated with their MoveIt motion planner, hitting sub-80 ms decision latency.
By YantriX Engineering · Applied AI Studio2 min readWarehouse automation
Overview
Engineering Scope & Context
How a YOLOv11-Seg + 3D-pose stack on a Jetson Orin Nano replaced fixed-pose jigs in a 6-DOF robotic cell — sub-80 ms latency, 99.2% accuracy, 40% throughput gain.
Engagement: Engineering Study
Discipline: Robotics & Automation
Project Type: Applied AI + Robotic Manipulation
Application: Warehouse automation
Key specifications
Core engineering parameters & stack.
- 80 ms
- End-to-end decision latency
- 99.2%
- Detection accuracy
- 40%
- Cell throughput gain
- 0
- Cloud round-trips per pick
Objectives
What the project needed to achieve
- Detect and segment random-pose parts inside cluttered bins
- Estimate 6-DOF pick pose for a parallel-jaw gripper
- Run the full perception stack on embedded hardware at the cell
- Integrate with existing MoveIt motion planning with zero PLC changes
Challenge
Engineering constraint
The client was operating a robotic cell that required fixed-pose presentation jigs for every SKU. Throughput was capped by manual part-staging and changeover. They needed vision-based picking that could generalize across SKUs without retooling and run on the edge — no cloud round-trips allowed on the production floor.
Approach
How YantriX approached the work
- 01
Collected and labelled a dataset of the client’s top 40 SKUs inside representative bin clutter, then fine-tuned a YOLOv11-Seg detector with rotation and occlusion augmentation.
- 02
Layered a depth-based pose-estimation stage on top of 2D masks using the ZED 2i stereo camera, filtering picks by graspability (approach angle, jaw clearance, surface normal).
- 03
Quantized the detector to FP16 and exported through TensorRT targeting Jetson Orin Nano; benchmarked camera-to-command latency under realistic lighting.
- 04
Exposed the perception stack as a ROS 2 action server so the existing MoveIt planner could request picks without any downstream refactor.
Outcomes
What improved by the end
- Sub-80 ms end-to-end decision latency (capture → model → grasp command)
- 99.2% detection accuracy across the labelled SKU set
- False-pick rate reduced to 1.4 per 1,000 attempts under production lighting
- Fully edge-deployed — zero production cloud dependencies
Deliverables
What the client receives
- Trained and quantized vision model with reproducible training pipeline
- ROS 2 perception package and MoveIt integration
- Camera, lens, and lighting specification for the cell
- Benchmark report: accuracy per class, latency distribution, failure modes
- Retraining playbook so the client can extend to new SKUs themselves
Tools used
Stack and tooling
- Ultralytics YOLOv11-Seg
- PyTorch + TensorRT (FP16)
- ROS 2 Humble + MoveIt 2
- ZED 2i stereo camera
- NVIDIA Jetson Orin Nano 8GB
- Roboflow for dataset ops
Impact
Business-level effect
- Cell throughput up by ~40% vs. fixed-pose baseline
- Operator labor reallocated away from part-staging
- Extensibility to new SKUs without mechanical changes
Conclusion
The stack shows what becomes possible when vision, control, and hardware are designed as one system rather than handed across vendors. It’s a playbook we re-use for any vision-guided manipulation project.
Working on a similar engineering problem?
Have a robotic cell bottlenecked by manual staging, fixed jigs, or cloud-dependent vision? Let’s talk about bringing the perception on-device.
Tagged
- YOLOv11
- Jetson Orin
- ROS 2
- MoveIt
- Bin Picking
- Edge AI
Visual results
Key views and intermediate artefacts
ROS 2 integration
Jetson Orin deployment
Frequently asked questions
Answers from the engagement itself.
What latency can you hit on a Jetson Orin Nano for bin picking?
Sub-80 ms end-to-end — camera capture through inference, NMS, pose estimation, and grasp-command emission. The YOLOv11-Seg forward pass itself is around 22 ms in FP16; the rest of the budget goes to preprocessing, stereo depth, and graspability filtering.
Can vision-guided bin picking replace fixed-pose jigs in production?
Yes, when the pipeline is engineered end-to-end. We routinely retire per-SKU jigs by combining a fine-tuned segmentation model with depth-based pose estimation and graspability filtering. The harder problem is usually lighting and camera placement, not the model.
Do you ship the retraining pipeline so the client can add SKUs later?
Always. Every deployment includes a documented retraining playbook — dataset format, label conventions, augmentation pipeline, and the export-to-TensorRT script. New SKUs onboard with data, not code.
Related case studies
Adjacent proof you can read next.
Robotics · Autonomous Mobile Robot (AMR)
Autonomous Mobile Robot (AMR) Platform Development with Nav2 & LiDAR SLAM
End-to-end development of an 80 kg payload industrial AMR — custom skid-steer chassis, Nav2 autonomous navigation, LiDAR SLAM, and ESP32 motor controller integration in 16 weeks.
MLOps · Edge AI Fleet
Edge Vision Node Deployment & Telemetry Architecture
End-to-end MLOps platform managing 600 Jetson inspection cameras across 14 sites — median model deploy went from 9 days to 38 minutes, with automatic drift-triggered rollback.
Continue exploring
Related blogs, services, and capability pages
Cross-links help readers move from proof into capability and educational content, and they reinforce the crawl path between commercial pages and reference content.
Similar Engineering Need?
Need help developing something similar?
Discuss your vision-guided robotic bin-picking cell with 3d pose estimation requirements, design challenges, or target specifications with our engineering team. NDA support is available where required.