Thesis Work for Seeing Is Verifying: Simulation-in-the-Loop Development of AI-Generated Robot Programs with Local Vision-Language Models

Your role and responsibilities

Large language models can generate RAPID robot programs, and RobotStudio gives AI agents access to its station, RAPID editor, virtual controller and simulation through a Model Context Protocol (MCP) server. Text feedback can detect syntax and controller errors, but may miss programs that run and do the wrong thing. It focuses on small vision-language models running locally, so station data can remain on the customer's computer.

Details

• Period: January to June 2027
• Number of credits: 30 ECTS/högskolepoäng (hp)
• Number of students for this thesis work: 2
• Location: Partly on site at ABB Robotics, Göteborg, Sweden, and partly at the university, as agreed.

In collaboration with Ericsson Research, students will use automatic filter tuning as an end-to-end challenge: a robot adjusts a filter's tuning screws based on measurement data and filter-test outputs. The agent may generate the robot program from scratch or improve an existing one, using simulation feedback and test results to guide successive adjustments. This case evaluates the broader thesis approach; it does not replace its research focus on simulation-in-the-loop verification of AI-generated programs.

Students will review state-of-the-art methods and build a benchmark with a natural-language task, an initial program (empty or containing a planted error), and an automatic success check. The Ericsson filter-tuning case will serve as an end-to-end demonstrator.

They will develop verification signals (collision reports, tool paths, reachability, cycle time, object poses and rendered keyframes), implement text-, structured-data- and vision-based feedback, and compare local vision-language models with cloud models. Evaluation will measure task success, verifier precision and recall, iterations, latency and hardware needs; the filter case will also assess achieved tuning performance and test-and-adjust cycles. Deliverables include a demo, documented results and recommendations.

Qualifications for the role

• You are pursuing a Master's degree in Computer Science, Robotics, Artificial Intelligence/Machine Learning, Mechatronics, Electrical Engineering, Engineering Physics, or a related field.
• Strong programming skills in C# and.NET; experience integrating with SDKs or APIs is an advantage.
• Understanding of large language models and computer vision; experience with vision-language models, agent frameworks, or the Model Context Protocol is a plus.
• Experience running models locally, for example with Ollama, llama.cpp, ONNX Runtime, or Foundry Local, is an advantage.
• Interest in robotics simulation and offline programming; experience with RobotStudio, RAPID, or similar tools is a plus.
• Interest in measurement data, signal processing, optimization, or control is an advantage; experience with experimental or iterative tuning is a plus.
• Ability to design experiments, evaluate results, and communicate findings clearly in English; collaborative and able to work partly on site in Mölndal.

A Future Opportunity

Please note that this position is part of our talent pipeline and not an active job opening at this time. By applying, you express your interest in future career opportunities with ABB.

We value people from different backgrounds. Could this be your story?