VERAGMIL

VERAGMIL

Learning robotic granular food manipulation from virtual reality demonstrations

An imitation-learning framework for robot-assisted feeding

Amanuel Ergogo, Diego Dall'Alba, and Przemyslaw Korzeniowski

2025 IEEE/RSJ International Conference on Intelligent Robots and Systems

Paper IEEE arXiv Code Video
Simulated scooping and transport across granular foods with different physical properties.

Overview

VERAGMIL is a simulation and data-collection framework for teaching robots to scoop and transport granular foods. It combines GPU-accelerated physics with a virtual reality interface so a person can demonstrate the task directly in 3D. The resulting data supports imitation learning for robot-assisted feeding scenarios where spillage and variation in food properties matter.

The study compares policies trained from VR and 3D space-mouse demonstrations using Behavioral Cloning (BC), recurrent Behavioral Cloning (BC-RNN), and Batch-Constrained Q-learning (BCQ). Performance is evaluated through task success, spillage, completion time, and generalization to unseen food items.

How it works

VERAGMIL system diagram connecting the granular-food simulation, VR interface, and imitation-learning pipeline
The framework links a robot-assisted feeding simulation, demonstration interfaces, and policy learning.
  1. Simulate. Granular foods with varied shapes and physical properties are modeled in NVIDIA Isaac Sim and Isaac Lab.
  2. Demonstrate. An operator records precise scooping and transport trajectories through VR; a 3D space mouse provides a comparison condition.
  3. Learn and evaluate. Imitation-learning policies are tested against a human-expert baseline and on food items not seen during training.

Evaluation design

The experiments separate two questions: how well different imitation-learning models perform, and how the demonstration interface affects the learned policy.

2. Evaluate. BC, BC-RNN, and BCQ are tested on diverse granular foods and benchmarked against a human baseline.
3. Compare. VR and 3D space-mouse demonstrations are compared for near-human performance and generalization.

Evaluation rollouts

Barley. Elongated grains slide, align, and settle around the spoon.
Macaroni. Large curved pieces produce pronounced inter-particle contact.
Buckwheat. Dense, angular particles flow into the spoon during acquisition.
Flakes. Thin, irregular pieces overlap and shift under utensil motion.
Green beans. Curved pieces roll and interlock as the spoon lifts.
Peas. Rounded particles roll and settle throughout the scoop.

Results

Policies learned from VR demonstrations consistently achieved higher success rates than policies trained from 3D space-mouse demonstrations. BCQ performed best overall and generalized most strongly to unseen foods.

Table II from the paper. Task success rate (%); VRI denotes virtual reality interface demonstrations and 3DM denotes 3D space-mouse demonstrations.
Material BC BC-RNN BCQ Human
VRI 3DM VRI 3DM VRI 3DM VRI 3DM
Training materials
Rice8265907593859791
Buckwheat8060887291809588
Barley7858857089789690
Pasta6545756080709085
Unseen materials
Peas554065507565——
Beans503860487060——
Flakes352050306040——
All materials (avg.)6447735880689589

Across all materials, BCQ reached 80% success with VR demonstrations versus 68% with 3D space-mouse demonstrations. Table III also reports lower BCQ spillage with VR data: 8 pieces on average compared with 15 for 3D space-mouse data.

Vectorized environments

VERAGMIL supports multiple simulated robot instances running concurrently, enabling vectorized environments for parallel policy training and evaluation.

Parallel simulation instances can collect experience and evaluate policies concurrently.

Paper

VERAGMIL: Virtual Environment for Scooping Granular Foods with Imitation Learning Models

Amanuel Ergogo, Diego Dall'Alba, and Przemyslaw Korzeniowski. IROS 2025, pages 13075–13082.

Abstract

Robot-Assisted Feeding (RAF) systems are essential for assisting individuals with disabilities or motor impairments in eating tasks. Manipulating granular food items, such as rice and beans, poses significant challenges due to their dynamic physical properties. Learning from human demonstrations offers a promising solution, but acquiring high-quality demonstrations is complex. VERAGMIL combines a high-fidelity simulator with an intuitive VR interface for recording demonstrations and supporting different imitation learning methods. Experiments compare BC, BC-RNN, and BCQ on scooping and transport tasks using VR and 3D space-mouse demonstrations. Results show that VR-based demonstrations lead to stronger policies, with BCQ performing best overall, especially in reducing spillage and approaching human performance.

Citation

@inproceedings{ergogo2025veragmil,
  author    = {Ergogo, Amanuel and Dall'Alba, Diego and Korzeniowski, Przemyslaw},
  title     = {VERAGMIL: Virtual Environment for Scooping Granular Foods with Imitation Learning Models},
  booktitle = {2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year      = {2025},
  pages     = {13075--13082},
  doi       = {10.1109/IROS60139.2025.11247362}
}