Keynote How AI Is Redefining HR Linking People Strategy To Business Impact

(0:08 – 1:39) And then Hugo, I’m just leaving this up and waiting for a couple of people to join. All right, hi everybody. And thank you for joining our 2025 People Leader Conference.

(1:40 – 1:49) We’re really excited to have you here. We’ve got a bunch of great sessions lined up. Just setting the table really quick, why we’re doing this.

(1:49 – 2:05) We’re in this crazy transition period around how AI is changing the nature of our work. Our ability to learn and adapt is gonna be crucial for our success. So with that in mind, I wanna introduce our keynote speaker, Hugo Serrazin.

(2:06 – 2:19) He’s the CEO of Udemy, an Emtrain client and now a partner as well. And they have a massive online learning platform designed to help upscale people and businesses in their learning and talent strategy. So without further ado, Hugo Serrazin.

(2:21 – 2:34) Thank you, I’m excited to be here. We share so much in common with Emtrain around our passion for learning. And what I’ll try to do is zip through very quickly some observation.

(2:35 – 3:00) I’ve had the pleasure in the last six months to speak to 400 CHRO People Leader, head of L&D around the world as I transitioned into this role. And there were some clear pattern that emerged and I wanna kind of share a few of those to stimulate conversations. I’ll leave a few minutes at the end for Q&A.

(3:00 – 3:23) I also wanna share a bit some of the things we’re doing internally and take a bit of the view of as a CEO, how am I trying to help transform the HR function and how they interact with the rest of the organization. So those will be the two lens. I’ll do it very quickly, but let’s buckle up and have fun.

(3:23 – 3:43) So first, let’s go to the next slide, please. First, I’ll state the obvious, HR has been in this mode for a very, very long time. By the way, before coming to Udemy, I was the president of UKG, a human capital management player.

(3:43 – 4:06) So I’ve been hanging up with CHROs for a long, long time. And this has been, this is not new news. CHRO, head of HR, have been wanting to do more strategic work, partner with the business, but they get pulled in all the time into a litany of minutiae, very tactical stuff that can be overwhelming.

(4:07 – 4:31) And it’s not getting any easier. And if I go to the next slide, AI is upon us and it’s putting a lot of pressure. And if I can pick up one of the themes from those 400 conversation is, people leader, head of L&D are feeling very anxious.

(4:32 – 4:52) There’s never been more that’s being demanded from them. They’re being expected to maximize reskilling ROI, but they have learning sprawl. They’re being asked to move from the traditional measures of outcome minutes and who’s engaged with what platform to more business aligned.

(4:53 – 5:10) Many of them are being put under a lot of pressure to reduce spend at a moment where they need to reskill their entire employee base. So a lot of extra pressure. Then everybody is experimenting with AI.

(5:10 – 5:49) There’s an enormous amount of proof of concept, but people are struggling to increase. Hi folks. Sorry, just had a little technical issue there. (5:50 – 6:36) Hugo should be back in just a sec. All right. Sorry for that little interruption.

(6:37 – 6:42) Hugo is back here. Thank you. Well, that was interesting.

(6:42 – 6:50) So I’m back. Yeah. The challenge of scaling AI are real.

(6:50 – 7:06) And we’ll talk a bit about some of the observations there. And in the third piece, we’re also noticing the skills that are required to do different types of roles. Their half-life is diminishing.

(7:06 – 7:32) So we’re constantly trying to reskill and adapt the talent of the organization. So if we move on to the next slide, and I’m gonna introduce a super basic framework and show it a bit in application in a second. As we think about AI, we’re seeing that there’s opportunity to automate existing work.

(7:32 – 7:47) So it’s basically, you’re not changing it. You’re just finding ways to use AI to take knowledge work and automate it in the same way that in the past, we were able to automate repetitive work. Now we’re able to automate non-repetitive work and unstructured data.

(7:48 – 8:13) The second is you can use AI to do analytics in a way that was counterintuitive. You can look at data and find patterns that were not easily identifiable. And obviously one of the use case that has been used often and is the identifying attrition risk, we were able to do that before by doing a multivariable analysis.

(8:14 – 8:45) Now you can do it with LLM and you can get a lot more insights. You can also use AI to create and make decisions in a way that were not possible in the past and in particular around workforce performance and workforce management. And then the last one, we’re gonna be dealing with a workforce that will be a mix of human and agent and finding ways to orchestrate that is a real, real opportunity and there are some learnings around that.

(8:45 – 9:15) All of that needs to be done in the context of trying to improve margin, which sometimes we lean in on revenue, sometimes we lean in on costs and there’s an enormous amount of value in being pretty broad and thinking about all of the above. So let me talk to you a bit about the Udemy journey, how we’ve done it on the next slide. And it’s something that will hopefully sound very, oh, sorry, before that, there is one more thing.

(9:16 – 9:57) There’s a lot of opportunity around agents as all of us read the news and it’s across all of HR. We can talk about revenue, resume screening, interviewing. I was speaking to one company now for entry level jobs, they reach out to employees and they give them the choice to be screened by an AI or book a meeting to have an individual person and they are getting more than 80% of the people wanting to be screened immediately by an AI so that they can move on and they can also do it off hours.

(9:57 – 10:14) So it creates the opportunity to not have to take some time off. So it’s kind of an interesting dynamic. There’s agents around employee lifecycle management that exists to facilitate and accelerate getting employees ramped.

(10:14 – 10:41) There’s agents in payroll management, benefit admin. I mean, in my previous role at UKG, I think there was like six agents that were introduced in the platform earlier this year, things that include monitoring payroll codes and how they’re changing. To how to write emails to employees around facilitating enrollment, et cetera.

(10:42 – 10:58) And then the last one, which obviously gets closer to where Udemy operates is performance and development. You can use AI to summarize performance reviews. You can use AI to create a learning path.

(10:58 – 11:22) You can use AI to do really, really interesting skills assessment and have a clear view of where your install, your workforce is today. You can create learning tutors and agents that are specialized by topic or areas. So with that in mind, let me kind of give you a bit of the Udemy adventure of the last 18 months or so on the next page.

(11:22 – 11:42) And you’re gonna see that, similar to many folks, we’ve tried many things. I’m not gonna pretend here that we had it right from the get-go. We did like everybody else, we turned on a bunch of tools and we said, go have fun and train around prompt engineering.

(11:42 – 11:55) And that was helpful to some degree, but we began to get more success when we got more structured. And here we did a multi-step approach. So let me kind of describe that.

(11:55 – 12:20) The first one, we did do an offsite or multiple offsites with the leadership team so that they begin to understand what they could do with these new tools. So it started at the top, not just the executive team, but the extended senior leadership team. We spent a day hands-on building new agents and new capabilities.

(12:20 – 12:32) Then we leveraged something that was already there. Quarterly, we have these Udemy days, which are learning days. We pivoted them to be learning AI days.

(12:33 – 13:01) And we realized that it wasn’t enough just to say, go and learn AI. We needed to give them a set of pre-approved secure tools and also give them the ability to make some small wins. So there were structured projects that allowed people to build things in teams, sometimes across different functions in a way that was targeted at a use case.

(13:01 – 13:19) And they could see themselves take progress and be successful. And then as we progress over the year, we ended up removing the structure and then asked them to solve problems that they cared about. And we ended up having some really, really interesting.

(13:19 – 13:42) So I will give you two example in HR. The first one was one of our recruiter came up with a AIassisted agent to create a job description. And in terms of effort, again, not an incredibly new or different idea, but it was tailored, adapted to our needs.

(13:42 – 14:00) It was built by one of our recruiters who’s not technical. And she did it in a way that we can now use it across the whole recruiting team. And it took away about 83% of the work of building these job description.

(14:00 – 14:15) So that was one example. Another one, we created a Ben admin chat bot that was there to answer questions 24 hours a day. Again, there are tools that exist out there, but these are actually super easy to build.

(14:16 – 14:35) And the person who runs Ben admin did that on her own using some pre-approved tools that we had. So really, really interesting to see. We created cohorts so that people could work with each other, encourage each other, share the results.

(14:36 – 14:49) And we managed to get more or less 90% of the organization engaged in some way, shape or form. Not all of them were building AI tools, but many did. And that’s pretty exciting.

(14:49 – 15:08) And we’re continuing to build on that. I’ll give you another example on the next page. We have in our platform, an AI role play, which is the ability to take a scenario and engage with an AI.

(15:09 – 15:27) In this case, and by the way, we now have more than 10,000 AI role play of all types. And internally, we’re using it to practice performance reviews. There’s a rubric that we’ve decided is important in how we do good coaching and good performance reviews.

(15:27 – 15:48) And managers are expected to practice different types of performance reviews, sometimes for great, sometimes with tips, sometimes for right in the middle of the road. And everybody’s doing that with an AI first. And we’re seeing some really, really good improvement in the quality of the conversation that we’re having.

(15:48 – 16:06) Again, keeping the human in the middle and using AI to augment what we would do otherwise. So those are two little stories around how we’re using it. Let me zoom back out and talk about two last topic.

(16:06 – 16:37) The first one, next page, the observation that scaling from POCs to broad-based adoption of AI is pretty common. And I’ve just quoted here an MIT research that highlighted that many organizations are struggling to expand their pilots. And I would offer two suggestions.

(16:37 – 17:07) The first one is you need to be very, very careful about what kind of re-skilling you’re doing. We’re seeing a lot of technical re-skilling around prompt engineering and how to use these different tools, agent force or the ones that are on work day or on co-pilot, whatever. Year over year, 3,000% increase of training.

(17:07 – 17:26) On our Udemy platform around technical skills. And that’s useful and that’s powerful and it helps with AI fluency, but that’s not enough. What we’re seeing, the ones who are being able to scale, they are using some sort of more targeted approach.

(17:27 – 17:47) And in particular, they’re really focused on adaptive skills in the age of AI. And adaptive skills means creative skills, problem solving skills, communication skills, working collaboratively, cross-functionally. And those skills on our Udemy platform, we can monitor the rise of them.

(17:47 – 18:15) It’s not 3,000%, but it is 40% year over year. And what we’re seeing is some of those who have that increase and then the increase in technical skills are having more success expanding beyond the POC. So I would encourage you to consider very carefully what kind of AI fluency programs you have and make sure that you are lending both types of re-skilling.

(18:15 – 18:27) Then the last thing is four thoughts. Oh, the next page, please. Thank you.

(18:30 – 18:53) Four practices to consider as you try to scale. The first one is go beyond the tools and rethink the process. Too often, the question is what can I do with co-pilot or open AI or Anthropic or whatever? That is the wrong question.

(18:54 – 19:43) I think the question is how would I reimagine process, the workflow, the job, now that I have AI? And that’s a much stronger way to get started. The second is AI solves part of the problem, particularly when it’s around getting efficiency and effectiveness, but the human is still the best answer for creative part of the workflow and where judgment matters the most. So the magic becomes how do you keep the human in the flow and how do you orchestrate the work? The third suggestion is, this is maybe for those who are more technical, not a new concept, but in general, I don’t think it’s getting as much press in the HR community.

(19:43 – 20:05) There is a thing called eval and it’s super, super important to understand your eval. Be very good at designing eval and putting a lot of attention on eval as you design or redesign a workflow. And the concept of an eval is not a particularly complicated one.

(20:05 – 20:34) There’s a simple way to explain it. It’s the rubric that your teacher had when they wanted to grade a homework assignment. What are the things that you would expect to be in the paper? So if you’re clever in defining the eval, you can make sure that the output that you’re getting are being tested against that, and then enforces the right kind of outcome and the quality and then the guardrails that you needed.

(20:35 – 20:58) And then if you can demonstrate that your eval are robust, you can demonstrate that your AI enable workflow is robust and then you can have the right level of trust and the right level of accountability. So a very big area of focus and increasing focus this year and next year. The last one is there’s an enormous amount of technology that are floating everywhere.

(20:59 – 21:28) I think it is important to think carefully about the reusability of the models, the reusability of the eval, the reusability of the workflow to make sure that you don’t have an overload of the end user, but also that you have reusability in the components. And we’re seeing some of our clients now being very, very, very thoughtful about the modularity of their AI strategy to get that level and reduce the complexity. So that was a quick tour.

(21:29 – 21:42) I’m gonna wrap it up here and say, thank you. We’re at the beginning of this incredible journey. We’re excited about it and welcome a few questions since we have a bit of time.

(21:49 – 22:12) Let me open up the Q&A maybe. All right, Hugo, I’ll go ahead and relay a couple of questions to you. So we have one question, Hugo, as you experimented with different AI tools, how did you determine when it was time to move on? Was there a structured evaluation process or did you rely more on intuition and results to know when to pivot? Yeah, so great question.

(22:12 – 22:32) Thank you. At the end of the day, it’s all about delivering value to the business. So when we were able to demonstrate that by using this agent or this AI tool, we were able to add value to a use case or the end user, we just moved on.

(22:32 – 23:00) I think here, you don’t need perfection. Sometimes there’s a false dichotomy that’s being put out that you need AI to be more perfect than even the human would be, or the same debate on biases. I think you can have a lot of biases and you need to treat this very, very carefully, but you need to say against what do you compare? And human also have their own limitations.

(23:00 – 23:25) So as long as you can have some degree of confidence that you’re improving the outcomes, you move forward. Next question, which leaders should the HR leader be collaborating with the most to optimize AI agents into their workflows? Yeah, that’s a great question. Obviously it varies by industry.

(23:27 – 23:52) There’s a fair amount of technology, so it’s very obvious to say that partnering with technology leaders in your organization is important. I think the second piece is often what limits the ability to do the AI work is the quality of the data. Everybody struggles with dashboards, everybody struggles with getting good data.

(23:52 – 24:09) So finding what’s the true source of information and data that people can rely on, I think is the second. And then the third is, go where there’s the pull. And there are business leaders who are embracing this.

(24:09 – 24:28) And I would encourage you to lean in on those because once you have a reference case inside your organization, you can get a second, you can get a third, you can get a fourth. You don’t need everybody to move in sync. Great, and here’s another one.

(24:29 – 24:54) For performance reviews, how do you help managers navigate using AI as an assistant, as opposed to solely relying on the AI to do the work for them? We’ve got time for this one and maybe one more. Yeah, so it’s a great question. The way that we’ve done it is we still have the individual conversation to get the 360 feedback.

(24:55 – 25:30) You still have notes and then what you end up doing is you ask AI to help you summarize and refine the recommendation or the suggestions. So the human is still very much in the loop, both in terms of gathering the input and then also validating what kind of suggestion that are being put forward. Ah, AI and skill assessment.

(25:30 – 25:46) It’s a great one. I’ll give you a few ideas. The first one is traditional learning has its own natural limitation.

(25:47 – 26:09) It is delivering content in one way, shape or form and then you listen and then you absorb some information. A better model would be one that would be highly personalized based on what are the knowledge gap. A better model would be one that requires the application of that knowledge.

(26:10 – 26:40) So one of the things that we’re seeing and we’re gonna see more and more is a lot of personalized learning based on assessing where individuals are. It also gives you the opportunity if me and Josh take the same class, if we start with an assessment, you can identify the areas, the gaps and the focus and then tailor the learning to that. So there’s the whole element of using assessment to tailor and personalize the learning experience.

(26:41 – 27:04) The second one is you can do it to make the experience more engaging. So you can auto generate an enormous amount of, you know, I’m gonna call them Quizlet type of question in a way that in the past when an instructor had to do it, it could be very painful and difficult. So now we have the ability and we’re doing that.

(27:05 – 27:27) We’re going through, we have 250,000 courses. So we’re going through all of them and we’re inferring a bunch of very quick quizzes that can kind of prompt the user to move along. And it’s not very different than what you’re seeing in Duolingo and we can have a debate on how much that is a good learning model but it’s certainly an engaging model and I think engagement is a good thing.

(27:27 – 28:06) The last thing I’ll offer on this question is you can do broad skills assessment and we’re doing that with a number of public sector client and increasingly Fortune 500. Upfront, you can bring in chat, AI chat and then put some scenarios and cases and demand from the user to respond. And again, if you have good eval in the background, you can check the skills against the answer you’re getting and get a score based on what you’re seeing.

(28:06 – 28:38) So here it’s a very sophisticated AI powered assessment of a conversation, a dialogue. You can do the same thing in the AI role play as I hinted at the very beginning. Right now we can send somebody to a cohort learning or traditional learning and then you can say over the next three weeks, every week you’re gonna have an AI role play and then you’re gonna be faced with a scenario in the background, there’s a rubric against which we’re gonna score your answer and we’re gonna infer some progress and some understanding.

(28:38 – 28:52) So there’s some really, really exciting things that are gonna be happening on the AI skills assessment. All right, that was fantastic. Thank you so much, Hugo.

(28:52 – 29:08) We’re so lucky that we could tap into your knowledge here. Well, we are starting our next session over in the other room. So if you wanna head back to the lobby, you’ll be able to join Janine and Tonya Jackson to talk about evalators as business leaders.

(29:09 – 29:11) Thank you. Thank you very much. So long.