Lifelong Language-Conditioned Robotic Manipulation Learning

source code
Customization performance comparison

The proposed Lifelong Robotic Manipulation task: (a) SkillsCrafter is able to evolve and learn new skills based on learned skills. It maintains a skill knowledge base that stores learned knowledge to allow for efficient learning of new skills using shared knowledge of old skills, and for performing any unknown skill in the open world using previously learned knowledge. (b) Skills catastrophic forgetting under lifelong learning setting. (c) Our SkillsCrafter has a good anti-forgetting performance.

Abstract

Traditional language-conditioned manipulation agent adaptation to new skills leads to catastrophic forgetting of old skills, limiting more practical dynamic scene deployment. In this paper, we propose SkillsCrafter, a novel robotic manipulation model designed to continually learn multiple manipulation skills while reducing catastrophic forgetting of old skills. Specifically, to achieve lifelong learning of new skills, we propose a Manipulation Skills Adaptation to achieve retaining the old skills knowledge while inheriting the shared knowledge between new and old skills to facilitate learning of new skills. Meanwhile, we perform the singular value decomposition on the diverse manipulation skills instructions to obtain a common skill semantic subspace projection matrices to record skills essential semantic space. To achieve forget-less and generalization manipulation, we propose a Skills Specialization Aggregation to compute inter-skills similarity in skill semantic subspaces to achieve aggregation of the previously learned skill knowledge for any new or unknown skill manipulation. Extensive simulator experiments and real-world deployments demonstrate the effectiveness and superiority of the proposed SkillsCrafter.

Customization performance comparison

Illustration of the proposed SkillsCrafter pipeline. It includes (a) a Manipulation Skills Adaptation to achieve retaining the old skills knowledge while inheriting the shared knowledge between new and old skills to facilitate learning of new skills; (b) a Skills Specialization Aggregation to compute inter-skills similarity in skill-specific subspaces to achieve adaptive aggregation of skills knowledge; (c) a Skills Specified Inference to loads aggregated knowledge to achieve any skill manipulation inference.

Customization performance comparison

Illustration of the experiment robotic skill tasks setting. We establish a total of 12 robotic simulator skills and 6 real-environment robotic skills for incremental learning. The first 16 skills serve as training and testing tasks for continual learning, while the last two skills are not trained and serve as open-world generalization testing.

Photograph of the UR5 robotic system

(a): A photograph of the proposed UR5 robotic system.

Illustration of the UR5 robotic system

(b): Illustration of the proposed UR5 robotic system.

The proposed UR5 robotic system is shown in Figure (a). The system consists of a UR5 robotic arm, a RealSense SR300 camera, a PHLIPS P506 camera, a control cabinet, and a calculation platform, as shown in Figure (b).

Photograph of the UR5 robotic system
Illustration of the UR5 robotic system

For clarity, a summary of the proposed SkillsCrafter learning algorithm and the inference algorithm is provided above.

Real Robot Demonstrations

Task18: close the top drawer

Task18: open the top drawer

Task17: put the round block onto the square block

Task17: put the square block onto the round block

Task16: put the toy car into the box

Task16: put the steel ring into the box

Task15: grab the bottle with the white cap

Task15: grab the green bottle

Task14: push the blue button, then push the yellow button

Task14: push the green button, then push the red button

Task13: grab the yellow doll

Task13: grab the blue doll

Simulation Environment Demonstrations

Task12: take the steak off the grill

GIF 1

Task12: take the steak off the grill

GIF 2

Task12: take the steak off the grill

GIF 3

Task11: stack 2 rose blocks

GIF 3

Task11: stack 3 blue blocks

GIF 1

Task11: stack 4 navy blocks

GIF 2

Task10: stack the wine bottle to the left of the rack

GIF 1

Task10: stack the wine bottle to the middle of the rack

GIF 2

Task10: stack the wine bottle to the right of the rack

GIF 3

Task9: close the black jar

GIF 1

Task9: close the blue jar

GIF 2

Task9: close the blue jar

GIF 3

Task8: screw in the gray light bulb

GIF 1

Task8: screw in the blue light bulb

GIF 2

Task8: screw in the red light bulb

GIF 3

Task7: use the stick to drag the cube onto the gray target

GIF 1

Task7: use the stick to drag the cube onto the gray target

GIF 2

Task7: use the stick to drag the cube onto the gray target

GIF 3

Task6: turn left tap

GIF 1

Task6: turn right tap

GIF 2

Task6: turn left tap

GIF 3


Task5: push the maroon button

GIF 1


Task5: push the maroon button, then push the green button

GIF 2

Task5: push the maroon button, then push the green button, then push the orange button

GIF 3

Task4: sweep dirt to dustpan

GIF 1

Task4: sweep dirt to dustpan

GIF 2

Task4: sweep dirt to dustpan

GIF 3

Task3: put the money away in the safe on the top shelf

GIF 1

Task3: put the money away in the safe on the middle shelf

GIF 2

Task3: put the money away in the safe on the bottom shelf

GIF 3

Task2: slide the block to target

GIF 1

Task2: slide the block to target

GIF 2

Task2: slide the block to target

GIF 3

Task1: open the top drawer

GIF 1

Task1: open the middle drawer

GIF 2

Task1: open the bottom drawer

GIF 3