AI roleplay for soft skills training
AI roleplay is practice of a work conversation by voice with an AI character that reacts to what the person says. In an assessment, the same kind of conversation gives a level for each competency, with evidence from the conversation.
In short
- Leaders trust realistic practice most. Simulation and role play was the method cited most in our 130+ interviews with HR and L&D leaders.
- Keep practice and assessment apart. Practice builds the skill. Only a new assessment without hints shows that the skill changed.
- Comparable feedback needs one situation, one scale and evidence. Without them, a rating tells more about the rater.
- ROI starts with a baseline. Assess before the training, again after 90 days on the same scale, and put the change next to one business metric.
AI roleplay is voice practice with a character that answers back
In a classic role play, a colleague or a trainer plays the other side. It works, but it needs a trainer’s time, and colleagues often go easy on each other. In e-learning, the person watches a video and picks an answer from a list. In AI roleplay, the person talks by voice, the AI character answers in real time, and the character pushes back when the answer is weak.
Two kinds of simulation do two different jobs. A development simulation is practice: each run ends with one next step. An assessment simulation has no hints and gives a level for each competency, with evidence from the conversation. How we put this together in a 90-day cycle: the methodology.
What 130+ HR and L&D leaders said about soft skills training
From October 2024 to March 2026, we ran more than 130 interviews with L&D, HR and business leaders in about 18 countries and published the result as an open-access report. Feedback that is too rare and too subjective came up in about 30% of the interviews. About 25% said that the ROI of training cannot be measured, and about 21% said that what people learn does not reach their work.
The method that leaders cited most, in about 30% of the interviews, was realistic case-based simulation and role play. The problem is the scale: the methods that leaders trust need expert time that teams do not have. The full findings: what 130+ HR and L&D leaders told us.
Simulation, coach or e-learning: each one has a job
A choice between them is a choice of what you want to change. Knowledge is cheap to deliver at scale. Behaviour under pressure needs practice with a counterpart and feedback on specific moments. A coach is the best counterpart and the most expensive one, so most teams use a coach for a few people and something else for the rest.
| Method | What it trains | How it measures | Scale | Best use |
|---|---|---|---|---|
| E-learning | Knowledge of a model or a framework | Quiz answers | High | The basics before practice |
| Classroom role play | Behaviour, with a colleague or a trainer | The opinion of the observer | Low | A team workshop |
| Coach | Behaviour and mindset, one to one | The opinion of the coach | Very low | Senior leaders, a few people |
| AI roleplay | Behaviour in a realistic conversation, by voice | A level for each competency, with evidence | High | Practice and assessment of a whole team |
| 360 feedback | Nothing: it is a measurement | Ratings of colleagues | Medium | A view of reputation, once a year |
Soft skills assessment: why ratings are subjective
Two managers watch the same conversation and give two different ratings. A study of two large samples of managers, each rated by bosses, peers and subordinates, found that the idiosyncratic rater effect explained 62% and 53% of the rating variance. The actual performance of the person explained less than half of that.
An assessment becomes comparable when everyone gets the same situation, the same scale and evidence from specific moments. A 360, an assessment centre and a simulation can all follow these rules, at a very different cost per person. Where the subjectivity comes from and how to remove most of it: why feedback on soft skills is subjective.
Practice and assessment are two different jobs
A person who repeats the same exercise learns the exercise. The third attempt is easier because the person remembers the character and the right words. The score goes up, but the skill at work can stay where it was. So we use one rule: only an assessment changes a level.
The ROI Methodology makes a similar split. Level 2 measures learning in the program, and level 3 measures application at work. An assessment without hints is the nearest proxy for level 3 that a team can run every quarter. The rule and what a manager sees after 90 days: practice does not move the level.
How to calculate the ROI of soft skills training
Most training reports count hours, completed courses and satisfaction. None of these shows whether a manager now leads a hard conversation better. The ROI Methodology splits outcome data into five levels: reaction, learning, application, business impact and ROI. Most reports stop at the first two.
Four steps make the benefit side measurable: a baseline for each competency, a link from each competency to one business goal, practice between the assessments, and a new assessment after 90 days on the same scale. Then count the full cost, including the time of the assessment. The steps and a one-page report: leadership development ROI. The formula, 15 input lines and a worked example: training ROI calculator.
A pilot for 10 people answers most questions in one cycle
A pilot shows two things: whether people take the simulations without pressure, and whether the assessment shows gaps that you did not see before. Choose one competency, one situation that happens every week and up to 10 people. Assess all of them before the practice, not only the people that you think are weak.
In one pilot, a support team of 11 agents was assessed in 10-minute conversations, and 89% of the agents had critical gaps that manual review did not show (case study). The pilot plan and the other plans are on the pricing page.
EU AI Act and GDPR: three questions for any vendor
The EU Artificial Intelligence Act and GDPR both apply when an AI tool evaluates people at work. Ask any vendor three questions. Where are the voice recordings and the transcripts stored, and for how long? Does the vendor train its models on your data? Can a person see the evidence behind each level?
SoftTrainer is GDPR compliant and aligned with the EU AI Act, and it does not train AI on customer data. Each level has evidence from the conversation, so a person can see why they got it.
All articles on this topic
5 articlesTraining ROI calculator: formula, inputs and a worked example
Training ROI is the monetary benefit that the training caused, minus the full cost, divided by the full cost. This calculator gives 15 lines to fill in, a worked example and the break-even point.
Practice does not move the level. A new assessment does
Keep training and measurement apart. Practice builds the skill, and only a new assessment without hints can show that the skill changed.
What 130+ HR and L&D leaders told us about soft skills
Leaders agree that soft skills are critical, but they cannot prove that training changes them. Feedback, ROI and assessment fail for the same reason: they need expert time that teams do not have.
Why soft skills feedback is subjective and how to make it comparable
A rating often tells you more about the rater than about the person. Feedback becomes fair when everyone gets the same situation, the same scale and evidence from specific moments.
Leadership development ROI: measure the behaviour, then the money
You can measure the return of leadership training if you assess the skill before and after it on one scale. Then you put the change next to one business metric and the full cost.
Questions and answers
What is AI roleplay training?
It is practice of a work conversation by voice with an AI character that reacts to what the person says. The person can repeat a hard talk, such as feedback or an angry customer, before it happens at work. An assessment version of the same conversation gives a level for each competency.
Can AI roleplay replace a coach?
Not for all cases. A coach works on mindset and context with a few people. AI roleplay gives practice and a comparable measurement to a whole team, so many companies use both.
How do you measure soft skills with AI?
Give everyone the same situation without hints and score the conversation on one scale for each competency. Keep the moments of the conversation as evidence for each level. Measure again after a practice period on the same scale.
How long does it take to see a change?
We use a 90-day cycle: an assessment, practice between assessments, then a new assessment on the same scale. One assessment conversation takes about ten minutes for each person.
Is AI roleplay GDPR compliant?
It depends on the vendor. Ask where the recordings are stored, whether the vendor trains models on your data and whether people can see the evidence behind their result. SoftTrainer is GDPR compliant and does not train AI on customer data.
Sources
- Kovalenko, R., & Rohovynska, K. (2026). The Soft Skills Measurement Gap: What 130+ L&D & HR Leaders Can’t Prove and Why. Zenodo.
- Scullen, Mount, Goff (2000). Understanding the latent structure of job performance ratings. Journal of Applied Psychology, 85(6).
- ROI Institute: ROI Methodology
- Regulation (EU) 2024/1689 (Artificial Intelligence Act), EUR-Lex