Semantically Grounded Skill Discovery via Vision-Language Models

Bachelor's Dissertation · University of Bath · 2026

PDF Code

Abstract

Unsupervised skill discovery methods enable agents to learn diverse behaviours without task-specific rewards, but existing approaches rely on task-trained visual encoders that lack semantic structure. We replace the standard CNN encoder in DIAYN and APS with a frozen CLIP ViT-B/32 encoder, and introduce textured MuJoCo environments designed to activate CLIP's visual priors. We evaluate across three locomotion domains — textured walker, quadruped, and cheetah — and show that CLIP representations improve both skill diversity during pretraining and downstream task performance during finetuning.

Citation

@thesis{mcdowell2026sgsd,
  title     = {Semantically Grounded Skill Discovery via Vision-Language Models},
  author    = {Ben McDowell},
  year      = {2026},
  school    = {University of Bath},
  type      = {Bachelor's Dissertation}
}