Are you maximizing the impact of your learning programs? Join us for a compelling Your Brain at Work Live webinar, where we'll explore the power, value, and impact of learning audits. Discover how these audits can revolutionize your approach to organizational learning and development. Webinar Highlights: Understand Learning Efficacy: Learn how audits provide critical insights into the effectiveness of your training initiatives. Drive Organizational Growth: See real-world examples of how learning audits have transformed learning outcomes and business performance. Expert Guidance: Gain valuable tips from our specialists on conducting your own learning audits effectively.
WEBVTT
1
00:00:01.990 --> 00:00:12.220
Shelby Wilburn: Welcome back to another week of your brain at work. Live! I'm your host, Shelby Woburn, for regulars. We're happy to have you back, and for newcomers. Welcome. We're excited to have you here with us today. For the 1st time
2
00:00:12.230 --> 00:00:26.089
Shelby Wilburn: in this episode we'll explore the power value and impact of learning audits and discuss how they can revolutionize your approach to organizational learning and development. Now, as I quickly share housekeeping notes, drop in the comments or chat box where you're joining in from today.
3
00:00:27.720 --> 00:00:52.520
Shelby Wilburn: we are recording today's session. So if you're interested in a replay, be on the lookout for an email later today, that email is also going to include a survey for feedback as well as a number of resources that are aligned with today's conversation. We suggest putting your phone on, do not disturb or quitting out of your email and messaging apps. So you can get the most out of today's discussion. And it's also going to help with your audio and video quality, and we love interaction. So feel free to share your thoughts and comments with us in the chat.
4
00:00:52.670 --> 00:00:55.989
Shelby Wilburn: Now, to get this show underway. I'm going to introduce our speakers.
5
00:00:56.590 --> 00:01:17.650
Shelby Wilburn: Our 1st guest coined the term neuro leadership when he co-founded Nli over 2 decades ago with a professional doctorate 4 successful books under his name, and a multitude of bylines ranging from the Harvard Business Review to the New York Times, and many more. A warm welcome to the co-founder and CEO of the Neuro Leadership Institute. Dr. David Rock. Thanks for being here today, David, as always.
6
00:01:17.650 --> 00:01:20.309
Dr. David Rock: Dolly. Great to be here. It looks a great topic today.
7
00:01:20.680 --> 00:01:49.599
Shelby Wilburn: It is our next guest, for today holds a degree in cognitive neuroscience from Columbia University and multiple certifications. She is vice president of consulting and product at the Neuro Leadership Institute, where she works closely with our largest clients at the strategy level to design custom brand based strategies. Her work has been instrumental to analyze research on organizational growth, mindset culture and leadership, a warm welcome to the Vice President of consulting and product at the Neuro Leadership Institute. Rachel Cardero. Thanks for being here today, Rachel.
8
00:01:49.600 --> 00:01:58.720
Rachel Cardero: Yeah, Hi, Shelby, and Hi, all I see so many of our regular viewers. In the chat from Belgium. DC, it's a pleasure being here.
9
00:01:58.720 --> 00:01:59.840
Shelby Wilburn: It's wonderful.
10
00:02:00.010 --> 00:02:21.359
Shelby Wilburn: And our moderator for today holds a Phd. In neuroscience from New York University. She leads the research team at the Neuro Leadership Institute, where she focuses on translating cognitive and social neuroscience into actionable solutions for organizations. A warm welcome to the senior Director of Research at Nli. Dr. Emma Saro. Thanks for being here today, Emma, and always and I'm passing it over to you.
11
00:02:21.880 --> 00:02:46.039
Dr. Emma Sarro: Shelby, David, and Rachel. It's so great to have you both here. We're incredibly lucky to have Rachel with us today. It's going to be a great discussion. So for anyone who's been here the last 2 weeks, we've had great discussions around the design of learning and design and delivery of learning to large events, so we'd love to hear your insights, as you know. You remember them as I. You've had them over the last couple of weeks
12
00:02:46.070 --> 00:02:57.479
Dr. Emma Sarro: from what we talked about, much of this is anchored on, keeping the brain in mind with our ages, model, designing for insight, integrating the social components, and making sure this happened one habit at a time.
13
00:02:57.480 --> 00:03:19.070
Dr. Emma Sarro: So last week, specifically, we talked about large learning events where we highlighted the focus on on the attendee, not on those on the stage building, more of a tasting menu, not building for the sizzler. So for all of you, what came up for you as you as you listened, and over the course of the week. I don't know for David what came up for you.
14
00:03:19.070 --> 00:03:20.540
Dr. David Rock: Yeah, while people reflect.
15
00:03:20.883 --> 00:03:30.899
Dr. David Rock: I think the the you know, a week later it's always interesting to see what still sticks in your brain. And for me, it's like, you know, design everyone designs for the stage instead of for the seat.
16
00:03:31.592 --> 00:03:55.529
Dr. David Rock: And when you're designing a large event, you want to design for people in the seat, not what's happening on the stage, and it just it just rings so true to me. And I. And I've you know, being a part of other events. Even since then you really really see this this kind of bias that designers have instructional designers and sort of, you know, Od, people that build these events. They're they're really obsessed about making everything that happens on the stage as amazing as possible.
17
00:03:55.858 --> 00:04:21.199
Dr. David Rock: But what you really want to do is is make the experience in the seat as amazing as possible. And just, you know, sitting, watching, stuffing on a stage is not an amazing experience in any way. So just I think that's the one that really jumps out for me and for those, you know, for those of you who weren't here week one and 2, you know week. This is the 3rd in a series, the week one was kind of the foundational science of ages, insight, social learning. And then last week we looked at how it applied to
18
00:04:21.350 --> 00:04:25.500
Dr. David Rock: to larger events. And then and this week we're looking at how it applies
19
00:04:25.520 --> 00:04:27.940
Dr. David Rock: to kind of the whole system of learning
20
00:04:28.288 --> 00:04:48.690
Dr. David Rock: and how you think about learning and and sort of how you assess that. And we're gonna dig in some really interesting things we've never talked about around how you assess the the sort of integrity and and coherence and structure and impact of learning, and really get some data about that as well. That's the that's the plan today. But yeah, any other insights from folks on the line from the last 2 sessions.
21
00:04:49.050 --> 00:05:08.349
Dr. Emma Sarro: Yeah, yeah. And and what are you doing different now? Or what do you plan to do? Different in in the coming year? I know we're we're kind of jumping right into into event season, and I know we're planning our own. So we're thinking about these principles, too, as we design but as we kind of jump into our own topic today.
22
00:05:08.750 --> 00:05:32.540
Dr. Emma Sarro: you know, when was the last time you have actually looked at your own learning programs really dove in and assessed content, and the and the processes that you have in place for your own learning. And that's what we're doing here today. I know we have a pretty great history of designing these these learning audits that we call them, and they've evolved over time as we all do. So. What was that evolution like? When did we start
23
00:05:32.600 --> 00:05:36.150
Dr. Emma Sarro: really really diving into that and designing these audits.
24
00:05:36.340 --> 00:05:57.650
Dr. David Rock: Yeah, it was. It was around 2010 around when we 1st published the ages model. And like a lot of things we do, we just kind of follow the science experiment. Follow the data right? And and the science emerges. You know, new studies come out. And around 2010 we were able to publish ages because of, you know, big body of new research had come out sort of 5 years
25
00:05:57.770 --> 00:06:17.160
Dr. David Rock: previous. And so we started like developing a language around learning with the ages model and and talking about it and presenting it. And you know, companies started to come to us, saying, Hey, I think we might be doing everything completely wrong. Can you help us kind of understand what's going on? And
26
00:06:17.493 --> 00:06:30.150
Dr. David Rock: you know, we we sort of it's evolved over 14 years of doing this. But it's so fascinating the sort of biases that that drive the the decisions. And all this but one of the earliest projects.
27
00:06:30.720 --> 00:06:32.649
Dr. David Rock: It's really really interesting. Was.
28
00:06:32.850 --> 00:06:36.889
Dr. David Rock: you know, a large company, you know, over a hundred 1,000 people, very international.
29
00:06:38.490 --> 00:07:00.729
Dr. David Rock: lots of very. You know very well educated people, and they had something that happens a lot in, particularly in large companies, which is this sort of you know, they had a good core program, at 1 point of, you know, and then it got added to, and then added to and added to and added to. And it was just like was just a mess now. And just in the Dei space there were over 20 different training programs that people could take that all kind of
30
00:07:00.780 --> 00:07:24.109
Dr. David Rock: crossed, huge amount of crossover. No pathway, no no coherence, no connections between, like, you know, literally 20 disparate programs run by 20 different vendors. And you know there's with a very big spend as well. And so, you know, we looked at it, and we we audited that and looked at you know what's really working what's really what what's really sticking
31
00:07:24.494 --> 00:07:47.110
Dr. David Rock: and then so look, we, we think we can actually do this in, we we think we can do better in 3 programs, in just 3 initiatives that cover everything that you're doing and more. But with a really clear pathway like you do this 1st one, then you do the second one, and we'll impact a hundred 1,000 people, not, you know, 5,000 people this way. And it was a, you know, transformational project. So so that was interesting. Then the other one we did was.
32
00:07:47.270 --> 00:07:56.790
Dr. David Rock: I think, really interesting in the same firm. We did something really scary that I recommend you do is that we? We looked at their their manager training separate. Gi.
33
00:07:56.950 --> 00:07:58.579
Dr. David Rock: we looked at the manager training
34
00:07:58.850 --> 00:07:59.643
Dr. David Rock: and we
35
00:08:00.330 --> 00:08:05.699
Dr. David Rock: and we basically interviewed a lot of people. And we we pulled out sort of the central models that were in the training
36
00:08:06.218 --> 00:08:09.249
Dr. David Rock: and we said, How many times a week.
37
00:08:09.280 --> 00:08:11.070
Dr. David Rock: Do you use these ideas?
38
00:08:11.574 --> 00:08:27.210
Dr. David Rock: You know, to people who'd gone through a training, you know earlier, like, how many times a week are you using these ideas. And there were, there was basically only one and a half frameworks or models out of dozens and dozens and dozens of models that was actually in use in the business.
39
00:08:27.932 --> 00:08:47.819
Dr. David Rock: And and so, you know, literally, there were. I don't know how many. We didn't count the number of models, but there was a lot of training, a lot of ideas, and there was only one and a half ideas that that was a fairly consistent, you know idea that because we basically went to, we were basically rebuilding a leadership program for them. We said, Well, let's see what we should keep.
40
00:08:48.140 --> 00:09:07.609
Dr. David Rock: That's really working for you, and we literally found one and a half ideas that they should keep, as you know, in going forward we were blown away. So this is one of the earliest ones we, you know we see all sorts of things, but you know it's evolved and and sort of in many, many ways, Rachel, you know for you. How have you seen it evolve? You've been with us a while. How have you seen that sort of learning audit evolve.
41
00:09:08.030 --> 00:09:13.320
Rachel Cardero: Yeah, I was thinking about that. I think it'll be 11 years in a couple of months
42
00:09:13.720 --> 00:09:36.710
Rachel Cardero: of doing this work, having the pleasure to do this work with so many organizations. It. One of my favorite projects that we had a couple of years ago, you know, was an organization with over 70,000 employees across the globe. A biotech firm with 50% of their population being on a shop floor manufacturing floor.
43
00:09:37.160 --> 00:09:49.610
Rachel Cardero: And you know there was a lot of energy and effort across the business to help the manufacturing population have access to the same resources and learning as those that were sitting in desks
44
00:09:49.930 --> 00:10:04.420
Rachel Cardero: and it a really well meaning Lnd team just opened up all the same offerings to those on the plant floor without changing the modalities, without changing how learning happened.
45
00:10:04.560 --> 00:10:18.049
Rachel Cardero: and nobody went to any of the learning. It didn't make any sense for them. And so I think definitely when you have ages as a north star, as a compass for what to focus on, it's really easy to take
46
00:10:18.390 --> 00:10:24.310
Rachel Cardero: any content and make it good for any audience to give them really that essential insight?
47
00:10:25.030 --> 00:10:43.620
Rachel Cardero: And so that, like, we work with them really carefully to say, what are those main concepts. And then how can you make that digestible for somebody who is in the middle of their shift, or might have a different manager on different days, you know. Let's remove these constructs, remove the noise, and really get into it.
48
00:10:43.620 --> 00:11:12.709
Dr. David Rock: Yeah, yeah, we can dig more into that. People have questions later. You just remind me of one quick thing. Then we should define a learning audit what we sort of assess, so that we know we're all sort of on the same page. But you just remind me one quick story. It was. It was a a big company. It was the the head of learning. And he was at a conference. He basically said, instead of learning we I went as an experiment one day and just switched the Lms off. We just turned it off. Everyone had all these programs, and he said, and nothing happened, and no one complained.
49
00:11:12.770 --> 00:11:26.140
Dr. David Rock: and he turned it off for like quite a while, and literally just crickets. And he was like, Okay, interesting, right? It's it's it's a so I think some of the, you know, learning pathways that are there are sort of
50
00:11:26.471 --> 00:11:37.799
Dr. David Rock: you know, they're they're clearly not central people aren't clearly like, you know, really finding value in them. And that's sort of what we want to look at. So yeah, I guess let's define. You know what what we're talking about. So so what we're talking about is is.
51
00:11:38.110 --> 00:12:05.579
Dr. David Rock: we have, you know, we have a fantastic team of consultants led by Rachel, and we do all sorts of work. But one of the work we'll do is is literally study the the way you're doing learning. Now, this can be just to get clarity. This can be an individual training product like it could be a 1 h, you know, learning experience or a digital product or you know, an in person or a hybrid any sort. So it could be an individual training. Experience could be a pathway of different experiences into, say, you know, a year long leadership program.
52
00:12:05.590 --> 00:12:09.110
Dr. David Rock: or a whole Dei pathway, or it could be the entire
53
00:12:09.464 --> 00:12:20.839
Dr. David Rock: you know, suite of learning that you have right, including soft skills as well as technical skills. Right? So it could be. You know, individual products could be a pathway. It could be the whole thing. And sometimes we'll just look at content.
54
00:12:21.030 --> 00:12:34.550
Dr. David Rock: and and, you know, like advise on on tangles. And you know, duplications and coherence. Sometimes we'll look at process. Only just look at sort of how we're delivering. We're not assessing content. And sometimes we'll look at both
55
00:12:34.790 --> 00:12:40.089
Dr. David Rock: right. Sometimes we'll we'll study like, what are you? You know. What are you trying to teach people? What habits are you trying to build.
56
00:12:40.210 --> 00:12:49.530
Dr. David Rock: and how do they all fit together, and how you're building it? And that's, of course you know, the most interesting. But we, you know, we can look at one product and and give you, you know, real feedback
57
00:12:49.600 --> 00:12:59.989
Dr. David Rock: on the content or the process, or you know all of it. So ultimately, the big, you know, the big thing is sort of looking the entire system at content and and process. And the other thing is, we're increasingly doing this
58
00:13:00.060 --> 00:13:12.870
Dr. David Rock: with partners on their external learning plan. So we have some partners who, who, you know, have digital products or training products or the things. And and they want to know about their their customer facing training. Right? They're doing training for customers.
59
00:13:13.383 --> 00:13:24.370
Dr. David Rock: or or they sell a product that comes with training. Right? So we're increasingly doing that as well. So we're assessing not just an internal company's internal training program, but their external
60
00:13:24.420 --> 00:13:31.190
Dr. David Rock: as well. And there's often a lot of really amazing insights in there. So that's what we're talking about. Basically studying
61
00:13:31.838 --> 00:13:39.360
Dr. David Rock: what they're doing in learning either a very focused granular way, or, more broadly, Rachel. Anything you'd add to that, did I? Did I get that right?
62
00:13:39.660 --> 00:14:03.458
Rachel Cardero: Yeah. And I might just add that we are. I think you mentioned this. But we're seeing a resurgence in the demand for these types of audits. I think you know, many years ago we had people pushing the envelope in terms of innovation. How can how can we be more global? How can we connect with one another after the pandemic people definitely were like, Oh, how can I make my learning remote, virtual, and still useful?
63
00:14:03.830 --> 00:14:29.010
Rachel Cardero: And now we're seeing people like it's time for a refresh. It's time to like, look in the closet and see what's going on. But also balancing these large digital portfolios with people's desire to connect, to have more social learning. And so we're seeing a lot of people have the same questions around, how can I make this good. How can I make this simpler? And how can I make it work? I think.
64
00:14:29.010 --> 00:14:55.930
Dr. David Rock: Yeah, yeah, I think there's been a there's been a sort of a few periods of bursts in learning, you know, one. Obviously, when you know the you know, learning went from all in person to some some digital, you know. Way back in the eighties and nineties there was a burst of innovation. And then, when the Internet got at a good speed right suddenly, you can. You know you can. You can have the Netflix of learning with skill, soft and Harvard management, or and you know you can access digitally. That was a lot of sort of the big burst
65
00:14:56.332 --> 00:15:01.159
Dr. David Rock: the pandemic when the pandemic happened. Everyone had to go virtual.
66
00:15:01.160 --> 00:15:19.480
Dr. David Rock: and many companies really really struggled with it. Well, it turns out we've been working virtually for 20 years. We've been delivering learning virtually for 20 years. At that point, and we'd put a huge range of different kinds of learning experiences into a virtual environment really effectively. So we we had an explosion of interest in in these
67
00:15:19.740 --> 00:15:28.769
Dr. David Rock: learning audits, and a huge amount of advising we did on how to do virtual right, just as the as the pandemic hit. And I think what's happening now is as as Rachel as you're saying.
68
00:15:28.820 --> 00:15:36.819
Dr. David Rock: There's this push to sort of how do we incorporate more in person? But how do we not just over rotate on that? Because it can be expensive, doesn't scale
69
00:15:37.123 --> 00:15:46.769
Dr. David Rock: and in many ways doesn't deliver ages. So it's a problem. But there's this push to sort of how do we incorporate that? There's also this push to cut costs? And there's a push to use AI,
70
00:15:46.850 --> 00:15:59.389
Dr. David Rock: and there's also this incredible pressure on learning. There's just so much more to learn, and in less time. So I think a lot of learning folks are going. Look at that. We spend 2 million dollars a year on learning, you know, that's getting to 12,000 people out of 50,000.
71
00:15:59.687 --> 00:16:13.809
Dr. David Rock: We're chewing up, you know. We're chewing up a thousand, you know, 5,000 people days doing that. There's got to be a better way, right? And we'll we'll look at a number like that, and we'll study it. We'll study everything we'll say. Actually, we think we can do that for for half the price
72
00:16:14.655 --> 00:16:17.000
Dr. David Rock: and and a quarter of the time.
73
00:16:17.410 --> 00:16:19.440
Dr. David Rock: and actually better impact.
74
00:16:19.550 --> 00:16:29.870
Dr. David Rock: And and that's that's our goal with the learning order is looking at. How do we reduce cost, reduce time and increase impact by following the science? And it turns out there are real ways to do that.
75
00:16:30.347 --> 00:16:34.089
Dr. David Rock: And so that's that's our goal, Emma, I guess. Back to you.
76
00:16:34.480 --> 00:16:57.270
Dr. Emma Sarro: Yeah, no. What's coming up? Actually, this just primed me a little bit. Is you mentioned the science that we incorporate? And so, even though, you know, you would suggest that we should be looking at our learning programs consistently because the environment is constantly evolving. So you mentioned all of these challenges we're facing. That's always going to be the case. There'll be different challenges. We'll have to jump back in and re-look at the learning
77
00:16:57.628 --> 00:17:06.229
Dr. Emma Sarro: and how we how and and assess this in different ways. But what are those core research underpinnings that no matter when you look
78
00:17:06.230 --> 00:17:13.880
Dr. Emma Sarro: right 20 years ago now in 10 years they'll still be the case. So what do we? What do we? What lenses do we look through.
79
00:17:14.810 --> 00:17:31.719
Dr. David Rock: Yeah, I'm happy to take that. And then maybe, Rachel, you could add on to it the I mean the 1st one is, basically, how do you? How do you design learning so that it it sustains in the brain? So it sticks. So it's used. We have a particular measure for that which is easy. Recall under pressure.
80
00:17:31.910 --> 00:17:49.340
Dr. David Rock: It's it's not recall. It's easy recall. And it's not just easy recall. It's easy recall under pressure. And that's a particular measure very, very measurable, very easy to test informally. You know, you teach people stuff. And you say, Hey, how often do you use that? What was the idea? How many ideas we went back to a program? Actually.
81
00:17:50.090 --> 00:17:54.510
Dr. David Rock: we went back to a program. A company asked us to assess someone else's training.
82
00:17:54.823 --> 00:18:01.339
Dr. David Rock: About 50 people had been through a 1 day training, and it was getting rave reviews. It was having fantastic net promoter score.
83
00:18:01.440 --> 00:18:07.019
Dr. David Rock: And they were like, we want to know whether to spend, you know, millions of millions of dollars putting a thousand people through it. Now.
84
00:18:07.581 --> 00:18:15.729
Dr. David Rock: You know, it's quite. It's quite expensive training. We're but we're just not sure. So we went and assessed it. And we found 25 different models for communication
85
00:18:15.820 --> 00:18:17.010
Dr. David Rock: and
86
00:18:17.374 --> 00:18:28.610
Dr. David Rock: like, completely different models that didn't, you know fit together? And it was a it was a beautiful one day, like walk through the park. Of all the interesting things happening in science around, you know, communication and management all this.
87
00:18:28.650 --> 00:18:34.510
Dr. David Rock: But when we surveyed people a month after the experience, those 50 people and we got results from more than 30 people of the 50.
88
00:18:34.540 --> 00:18:38.309
Dr. David Rock: We said, how many things you remember like write down what you remember?
89
00:18:39.040 --> 00:18:56.209
Dr. David Rock: Nothing. There was. No, there wasn't even recall, forget easy recall under pressure. There was just no recall. Right? And then we said, How many ideas are you using at least once a week? And there was a couple of people said a couple of things that was it, you know, out of 50. So so ages helps you like design things.
90
00:18:56.745 --> 00:19:07.089
Dr. David Rock: So that you actually get easy recall under pressure later. And so when you design with ages, you should get, you know, 40 out of the 50 saying, oh, yeah, there's multiple things that I'm using every week. Now.
91
00:19:07.100 --> 00:19:18.659
Dr. David Rock: that's that's that's that's kind of a entry point you can actually get up to like 45 out of 50 or 47 out of 50 in some content areas like something easier like growth. Mindset.
92
00:19:19.144 --> 00:19:37.930
Dr. David Rock: So so easy recall under pressure for ages is is all about that. We talked about that in the 1st session a lot more. The second one is designing for insight. We talked about that a bit so designing for insight, really important for motivation, for activation of ideas. It's a whole set of ideas. For how do you design for insight, designing for social learning.
93
00:19:38.263 --> 00:19:45.859
Dr. David Rock: We've been talking about that a lot. So how do you design? So you're really engaging social memory networks that embeds learning better. And then
94
00:19:46.050 --> 00:20:14.730
Dr. David Rock: and then finally, how do you design for coherence? Which is the structural integrity of all the ideas? How they all fit together? So you know, we've written 2 papers on ages. We've written 3 papers on insight. We've written widely on social learning. I don't think we have a paper yet, but we've we've we're. It's on the way. And we've we've written widely on coherence and have a paper on coherence. So those are kind of the big ideas ages, social learning, insight, learning, coherence, are kind of the the Big 4 in a way that you really want to build on. Anything. You would add to that, Rachel.
95
00:20:15.810 --> 00:20:39.340
Rachel Cardero: You know, just like when you say that focus when you mention that focus on easily recallable under pressure. I think one of the things that we see most often when we do these learning audits is that one of the things that many organizations miss, is not only that it's not easily recallable definitely, not under pressure. But what gets in the way of that is that many times they're just awareness building.
96
00:20:39.360 --> 00:20:56.720
Rachel Cardero: And they're not focused on actual behavior change. And so it's impossible to get that easily recallable under pressure because they haven't even been given a behavior or an action to take. It's like with those communication classes. You get a lot of emphasis on why communication matters.
97
00:20:57.079 --> 00:21:11.449
Rachel Cardero: You know that you should communicate more that your people should communicate. But it's kind of like telling somebody, you know, like, just just be better. Just be better be smarter, be fast like people actually need to know
98
00:21:11.450 --> 00:21:19.330
Rachel Cardero: what to do under pressure. They need that behavior. It can't just be communicate like, if we knew we would be doing it, you know.
99
00:21:19.330 --> 00:21:40.331
Dr. David Rock: Yeah, yeah, you know, it's the irony is, I forgot one. There's really 5, not 4. I forgot one. Which is the the fact model, which is fluency, amount, coherence, and time. And sometimes I think of coherence and the fact model sort of interchangeably so. But the fact describes. Is how to work with limited capacity, limited cognitive capacity.
100
00:21:41.239 --> 00:21:44.379
Dr. David Rock: So ages are sort of design, principle, insight, social
101
00:21:44.620 --> 00:21:59.779
Dr. David Rock: design, principles. Coherence is like the study of the system, and then factors about literally working within capacity limits of the human brain. So these 5, you know, bits of research that fit together, that that help you, you know, make these make these different decisions.
102
00:22:01.690 --> 00:22:06.620
Rachel Cardero: And I'll definitely say out of those. I think coherence is my favorite
103
00:22:06.924 --> 00:22:10.630
Rachel Cardero: you know there are so many organizations as well when we do this work
104
00:22:10.670 --> 00:22:24.019
Rachel Cardero: that say, like, there just are some programs that we have that are boring. And like they, they like, we need to deliver them. There's compliance training. There's other foundational skills that people like, you know, we need people to do this.
105
00:22:24.766 --> 00:22:53.789
Rachel Cardero: Sometimes there are organizations, especially ones that have like a a lot of technical experts. We see a lot of pharma companies go through this where, like sometimes, you just need to know certain things. And what so many organizations, miss, is any type of bow to wrap around it, to build that coherence. And so like, I remember working with another like highly technical workforce, large organization. Who had tons of technical training.
106
00:22:53.800 --> 00:23:00.669
Rachel Cardero: And it was a multi-year pathway to help people become, you know, the best scientists that they could be
107
00:23:01.074 --> 00:23:10.679
Rachel Cardero: and it was 2 or 3 years before somebody was able to take a a program or have a learning experience that wasn't highly technical.
108
00:23:11.030 --> 00:23:18.149
Rachel Cardero: and there were no larger labels. There were no grouping. You're just like, kind of going through. You know
109
00:23:18.170 --> 00:23:22.940
Rachel Cardero: how to do this reaction, how to maximize this, how to test for that.
110
00:23:22.980 --> 00:23:31.840
Rachel Cardero: It's so simple and so powerful to give things chunks to give an overarching narrative.
111
00:23:31.840 --> 00:23:56.670
Rachel Cardero: even if you have the same content. Even if it's just as technical. There are little things that we can do to really up the impact of your learning and people's feeling about it like improve, instead of feeling overwhelmed, feeling like, oh, well, you know what there's 4 main things I'm going to master over the next 3 years is so much more manageable than I have 150 credits of technical work to get done in the next 3 years.
112
00:23:56.910 --> 00:24:24.830
Dr. Emma Sarro: Yeah. And it seems a lot of that is is leading back to the engagement. Right? So you also need to have the motivation and engagement of the individuals going through it. So it's design on the front end and making sure it meets all of these pieces. The social learning, the insights all of that, but all of that is really geared to engage and motivate the individuals, to learn and to come back for more, so that they can make it through all of those credits, whatever it is because a lot.
113
00:24:24.830 --> 00:24:31.129
Dr. David Rock: I mean that we talked about this with large events, that there's kind of 3 things you're trying to solve for attention, insight, and action.
114
00:24:31.780 --> 00:24:49.590
Dr. David Rock: Designing everything ultimately, that people really pay close attention, that they have really strong insights. And they really do take action. Those that's fundamentally what you're solving for, and you can. You can caps on that as engagement for sure. From a from a a cognitive perspective. It's those 3 kind of mechanisms that let's get a bit more specific, I guess.
115
00:24:50.287 --> 00:24:51.749
Dr. David Rock: About kind of
116
00:24:52.331 --> 00:25:05.119
Dr. David Rock: what we see when we do these audits. And then maybe what we you know, what we advise. What you know. What are some of the biggest trends, you see, Rachel, as as you're doing these see if we can, if we can sort of bucket these for people a bit.
117
00:25:05.400 --> 00:25:21.778
Rachel Cardero: Yeah. Well, missteps, maybe I'd call them so just to kind of call back to what Emma just said about like focusing on engagement. I think that that's absolutely important. And we have specific methodologies from a brain based perspective on how to get that done.
118
00:25:22.130 --> 00:25:41.740
Rachel Cardero: A misstep. I see all of the time in organizations is that they actually over rotate to engagement. They try and make learning easy. They try and remove all the discomfort they try and really focus on that Nps score. Did you like it? Would you recommend it to somebody else instead of focusing on
119
00:25:41.740 --> 00:25:53.560
Rachel Cardero: the the behavior change? How can we get you to have a deep insight? How can we get you to act on that insight? And so with many organizations, we actually have to go in and say like, Oh, actually, there's a way to help people
120
00:25:53.560 --> 00:26:13.249
Rachel Cardero: manage that discomfort of being called on. There's a way to help them manage the discomfort of sharing with vulnerability and learning from one another. And you ha! That's a dance. It's an art definitely I think the other thing that I see happen all the time is people take.
121
00:26:13.470 --> 00:26:41.960
Rachel Cardero: you know, the same amount of content from in one modality, and then they switch modalities, and they just deliver it the same way. Right? So 5 days in person becomes 5 days on zoom, where everybody's cameras are off, and you're not necessarily taking breaks, and there isn't a social component. But you're asked to like, really hold your attention for all that time. That doesn't work. We really have to tap into what does the learner need again? So like that focus back on the seat?
122
00:26:42.210 --> 00:26:55.470
Rachel Cardero: How can we tailor the content, the brand, you know, the the even the labels of things to make sure that people are having insights, remembering that learning and it's everything isn't just one for one.
123
00:26:56.530 --> 00:27:14.451
Rachel Cardero: And then I I mentioned this already. But I think another big thing that we help organizations with is with the branding, the actual branding of their learning programs, the learning strategy, helping them organize their coursework or their offerings into things that help people make more sense of them.
124
00:27:14.810 --> 00:27:36.989
Rachel Cardero: You know, if you think about reading a textbook, and there's no title on the textbook, and there's no chapters, and there's no headings, you know. Learning is more difficult that way. You need those signals to help people understand. You know. What are you doing? Why are you here? And branding can really do so much branding can create a threat state. It can create a fixed mindset.
125
00:27:37.130 --> 00:28:03.699
Rachel Cardero: and so helping organizations come up with the proper name the proper groupings, and also use that to communicate their learning strategies to their senior stakeholders so that they could more easily tie them to business outcomes and help the business. See the value of like. Please help us get your people into these seats right into these programs. The branding is a huge part of that.
126
00:28:03.700 --> 00:28:25.609
Dr. David Rock: Yeah, I think the the well, said the you know, the learning function should be best friends with the marketing function. Yeah, you know, drawing from them all the time. I think branding is a big piece that you just you know what I what I see is, it's kind of the total wild West. In learning like, it's the absolute Wild West. You can take a you know. You can. You can take a program, a training program and
127
00:28:26.024 --> 00:28:48.235
Dr. David Rock: about. Let's say, you know, conflict resolution. Right? You can take that. That's currently say, a 2 day classroom. And there's nothing to say. You can't turn that into a 1 h, Webinar, and get just as good value. There's no measure there's no like. There's it's just the Wild West out there, right? Or even, you know, we're just gonna send out a 10 min e-learning, and everyone's gonna get it right?
128
00:28:48.540 --> 00:29:09.720
Dr. David Rock: And I think in the absence of good good measurement philosophies. And it it's it's it's just the Wild West. You can do whatever you want, and we see crazy things. I remember, one time I was in India went to see this, you know, really big, important company, hundreds of thousands of employees, and they showed me they proudly showed me their onboarding
129
00:29:09.720 --> 00:29:19.220
Dr. David Rock: process. And we walked through. We were in this in this beautiful, beautiful, you know, learning center, and they showed me there was a sort of
130
00:29:19.576 --> 00:29:34.509
Dr. David Rock: it was almost like a record store. You had all these pamphlets of different training courses sort of in the middle of the room lined up against each other, and they walked. And I was like, oh, what's all this? That's like a lot of content. There was like 40 or 50 different courses.
131
00:29:35.105 --> 00:29:48.030
Dr. David Rock: and you know, and all all good stuff, all meaningful stuff. And I was like, Oh, what's you know? Is this like a a 1 year or 2. Year. Experience is this sort of everything they're like. Oh, no, that's the 2 week onboarding. I'm like what
132
00:29:48.420 --> 00:30:01.680
Dr. David Rock: they're like. No people come in 6 days a week. 9 Am. To 6 pm. And they go through 9 Am. 6 Pm. You know they get, but they get breaks, I'm like, oh, tell me about the break! What do they get? They get? Oh, 45 min for lunch! Any other breaks. Nope.
133
00:30:02.061 --> 00:30:16.710
Dr. David Rock: So 6 days a week, 2 weeks they were going through to me what looked like 2 years of classroom work. I was like, how do people survive that? Does anything stick and so you can do this stuff because there's no universal measure for anything
134
00:30:16.850 --> 00:30:37.559
Dr. David Rock: around. What's an acceptable, you know, success rate for retention. Right? If you have a thousand people going through a learning experience. What should you be targeting for? Rich, you know, for easy recall under pressure a month later. What's what's a good number. So there's no, there's no real sort of metrics in this. We know we're slowly trying to evolve this practice and evolve this thinking
135
00:30:37.897 --> 00:30:50.730
Dr. David Rock: with with some of our measurement philosophies. But it's it's such a a fascinating thing. So I think so. You know. One thing I see way too much content, you know. Way too quickly. Not branded
136
00:30:51.202 --> 00:30:56.749
Dr. David Rock: and a lot of just kind of things got messy. They were clean once, and then they got messy.
137
00:30:57.130 --> 00:31:00.300
Dr. David Rock: and you've got to be sort of tough
138
00:31:00.330 --> 00:31:06.099
Dr. David Rock: as a as a head of L and D, or a talent head of talent, or a C. Hr, you've got to be kind of tough and be good at saying no
139
00:31:06.632 --> 00:31:18.190
Dr. David Rock: cause the challenge. The challenge is, you got a lot of people who just sort of want to bring in new things because new things feel great right? And a lot of like new books come out, new ideas and new stuff.
140
00:31:18.714 --> 00:31:28.379
Dr. David Rock: And you know new new head of learning comes on board. They want to bring their vendors in their market. You know what what looks sort of coherent and made sense now, is just like, really a mess
141
00:31:28.852 --> 00:31:31.270
Dr. David Rock: and so there's a lot of sort of
142
00:31:31.580 --> 00:31:40.160
Dr. David Rock: clean slate work happening. Let's just let's just create a clean slate, you know. And with the tech company I was talking about some years ago now. But you know, we turned
143
00:31:41.280 --> 00:32:05.326
Dr. David Rock: about 20 different Dei programs into 3 programs, right? One for every employee, one for managers and one for you know, folks who wanted some extra work. We didn't call it, for diverse employees or underrepresented employees. But it was just for folks who wanted extra support, and so the 1st one had habits for the whole company just 3 habits. The second one had 3 habits for all managers, the 3rd one had 3 habits for folks who wanted extra support.
144
00:32:05.620 --> 00:32:29.310
Dr. David Rock: and we were able to to give people more than they were getting in the 20 different programs in just 3 pathways and scale that radically cheaper, radically faster. So that's that's, you know. That's a great example of of an outcome that we're sort of advising. Rachel. Any other examples you want to dig into a sort of the from 2 as as a result of so the assessment and what you know, what ended up happening.
145
00:32:29.630 --> 00:32:30.250
Rachel Cardero: Yeah.
146
00:32:30.766 --> 00:32:45.149
Rachel Cardero: so when we do these assessments, when we work with organizations, we usually do them in 3 phases. So the the 1st phase of the 1st step. We spend time getting to know you as an organization, what your goals are for learning.
147
00:32:45.463 --> 00:32:55.176
Rachel Cardero: In that second step or phase. We're doing the work. Teaching you some of the science as we like. Look through your assets, talk to people, review the data itself.
148
00:32:55.490 --> 00:33:20.270
Rachel Cardero: And then in that last phase we a solution together. You'll get findings, recommendations, illustrations, and for some organizations we like really get in there. And we'll build a whole bunch of branding assets, you know. We'll reconfigure your classes for you. It, anything and everything right? So when you brought up onboarding, I just. I'm remembering an organization that we worked with, that when in that 1st phase.
149
00:33:20.450 --> 00:33:24.110
Rachel Cardero: they were talking about how they wanted to become a growth mindset organization
150
00:33:24.300 --> 00:33:34.929
Rachel Cardero: and how they wanted to assess all of their learning, to see if they could, how well they were doing, and how they were either accelerating their growth, mindset as an organization or inhibiting it.
151
00:33:34.970 --> 00:33:54.409
Rachel Cardero: and all of their solutions to date right when they came to us, were about tacking growth mindset on after the fact, everything was like, let's just create more and more complexity on the back end, you know, once you've been here for 4 years, then you take a really complex growth mindset course. And then at 5 years you take an even more complex one
152
00:33:54.440 --> 00:34:21.440
Rachel Cardero: and one of the my favorite solutions. And all the time that I've been doing this work is, we actually figured out that there was this opportunity in onboarding to to give people growth, mindset to teach people growth, mindset. And so there is like one of the things that we do is helping organizations like you mentioned, minimize costs, minimize some of the effort and maximize the impact, like even figuring out
153
00:34:21.590 --> 00:34:39.460
Rachel Cardero: when to give people how to order the learning. If you want a growth mindset organization right like, give it to them. Day one. Don't wait until they've been in your organization for 5 years, and then give them a really heavy course where they have to change their behavior and what they've the habits that they've built over the last 5 years.
154
00:34:39.560 --> 00:35:05.729
Rachel Cardero: even something as simple as that helping under people understand? Where are those missed opportunities? Where are the places where you can eliminate the complexity and get people when they are most motivated to learn about. You know, what should I? What do I need to be successful in this organization. And so that's that's a favorite story of mine, really. Just helping people find, you know, the the hidden gems, the the easy places, the easy wins really.
155
00:35:05.730 --> 00:35:24.809
Dr. Emma Sarro: Yeah, no, that's great. I love these stories. But I'm sure some individuals that are listening are thinking more in the in the realm of in person versus virtual. Only. So what are you? What are you seeing? Maybe in person that's coming up for organizations? Maybe the mistakes they're having, or the stories that you have.
156
00:35:24.810 --> 00:35:25.600
Dr. David Rock: Yeah, for assessment.
157
00:35:25.600 --> 00:35:26.829
Dr. Emma Sarro: And those programs.
158
00:35:27.030 --> 00:35:38.083
Dr. David Rock: Yeah, for sure. So I mean, yeah, let's talk about sort of the the common things we see and what we advise around in person things, and then maybe we can. We can touch on virtual as well. But yeah, with with in person. I mean, there's a lot of
159
00:35:38.370 --> 00:35:58.200
Dr. David Rock: There's a lot of really interesting momentum back to in person. And I think there's some, you know, there's some. There's obviously some really good value in in being in person, the downside with in person is trying to do it all in a block, and it's really hard to fight that. But if there's no spacing. Yes, in ages you're you're missing. You're just not going to get easy recall under pressure. You
160
00:35:58.200 --> 00:36:17.059
Dr. David Rock: barely gonna get recall so so a lot of the time. It's like, let's split this up into 2 days. 2 half days. The other thing is the afternoon of a learning experience. Any day of the week is is just terrible, like people just don't learn much. Right? So you're really much better to just do 2 or 3 h in the morning
161
00:36:17.060 --> 00:36:38.839
Dr. David Rock: of learning, and then let people do their other focus work like really paying attention to new ideas like you should be at the pace you should go at is is very taxing for more than a few hours. So so you know, let's if you've got, if you really think there's an 8 h classroom day. Let's do. 2 h week, 1, 2 h week, 2, 2 h week, 3, 2 h week, 4 way better
162
00:36:38.910 --> 00:36:57.650
Dr. David Rock: than 8 h in one day, way, way, way, way, way, way better. And it might cost more. But you get radically better, better results. Or even if you can't do that, you know, because you're flying people in or something. Let's do like the morning day, one, and in the afternoon let people network with each other and do other things, and then let's do the morning day 2, and then let them fly off
163
00:36:57.790 --> 00:37:13.740
Dr. David Rock: right, just having that like sleep on it. Effect is really helpful. It gives you a little bit of spacing, a little bit of time to digest. Come back re embed. Just that is super super helpful. So you know. Don't do a dinner the night before. Do a dinner on the night between, when you definitely will have everyone
164
00:37:13.780 --> 00:37:25.260
Dr. David Rock: right and do 2 half days, and then you get all this added, you know, networking time, so that it's sort of non obvious. But you know people fly folks in for dinner the night before half the group doesn't make it, and then you do the whole day.
165
00:37:25.540 --> 00:37:31.619
Dr. David Rock: It's not the best way to to design it. Then the other thing is just, you know that we talked about this a little bit last week, just the like
166
00:37:31.680 --> 00:37:38.909
Dr. David Rock: focus time, like 60, 75, no more than 90 min in a block before switching it up, having a short break.
167
00:37:38.990 --> 00:37:44.139
Dr. David Rock: You don't want a 3 h like, you know, 3 h block and then a half hour break.
168
00:37:44.420 --> 00:37:49.749
Dr. David Rock: You don't want a 2 h block and a half hour break. You want a 75 min block and a 15 min, break at the most
169
00:37:49.930 --> 00:37:55.250
Dr. David Rock: right, and then then another hour block, and then a, you know, and then a lunch, or something like that. So
170
00:37:55.410 --> 00:37:59.289
Dr. David Rock: really putting in those short breaks regularly in the afternoon.
171
00:37:59.620 --> 00:38:19.500
Dr. David Rock: A short breaks an hour after lunch, and then another short break, 6 another hour later, and then another short break, you know, like putting in those breaks really move around and moving people around as well. So there's a lot of little things like that sort of managing the flow around in person. And then, you know, room size, and you know, air and all that. Rachel, what do you see with in person? What do we? What do we end up advising.
172
00:38:19.500 --> 00:38:41.059
Rachel Cardero: Yeah, it's funny. Cause I think conceptually, when we talk about adding breaks, everybody's for it until you're in the process of designing the learning. And it's like, Oh, how are we gonna fit this? Let's just extend the day by half an hour. And then how are we gonna fit this? Well, let's start earlier by half an hour, and let's you know we need to fit this in and let's shorten lunch.
173
00:38:41.060 --> 00:38:50.750
Rachel Cardero: And before you know it, like I see I've seen so many sessions go from being 8 h with a 1 h lunch break to all of a sudden 9 h with a limited lunch break.
174
00:38:51.020 --> 00:38:58.809
Rachel Cardero: And so I think that like feeling and it's really it's safety bias, right for those that know our seeds model.
175
00:38:58.820 --> 00:39:15.310
Rachel Cardero: It's risk aversion. It's if I don't put everything in here, how will they learn? I can't create space. I can't lose this opportunity. I have a captive audience. I need to force feed them all these insights? But it actually works against you. I think it's like it's something I see time and time again.
176
00:39:15.890 --> 00:39:29.549
Rachel Cardero: The other thing that I would say with in person. Learning is like something we do at Nli is try and leverage different modalities for their strengths. And so, like you said. You know, if you absolutely do need to have people in person.
177
00:39:29.550 --> 00:39:55.800
Rachel Cardero: use that for the social connection, use that to build relevance to build attention and emotion, the the A and the E and ages, and then follow on with more spaced learning. Where you're doing the you know the exercise, the drills, the skill building you can play with learning in that way. It doesn't have to be all you know. Kind of have the same tone and pace, regardless of the modality.
178
00:39:56.190 --> 00:40:20.439
Dr. David Rock: Yeah, I I think there's, you know, I don't like anything that makes learning formulaic. But there's there's somewhat of a formula that you can follow, of sort of how long you can go and kind of what you want to do at the end of a chapter of learning before you start a new chapter like you want. You want an integration exercise. It can be 3 min can be 10 min. Doesn't need to be more. You know. There's a range of different kinds of integration exercises. But, you know.
179
00:40:20.440 --> 00:40:39.740
Dr. David Rock: look, what we love to do is get people to look over everything that just happened in the last hour and a half, find the strongest insights they had, you know. Share them with some other people, then write them down like these kinds of things are really, you know, really deepen insight and and build those insights. So there's this sort of the brain likes to think in chapters, you know, we finish this topic, we're gonna move to another one.
180
00:40:39.740 --> 00:40:44.809
Dr. David Rock: Let's put a bow on it, you know, in the brain. So so there's a little bit of a formula like that.
181
00:40:45.097 --> 00:40:55.230
Dr. David Rock: And then you're trying to mix things up and obviously not just have people sitting down. You're trying to get people moving around and standing up and doing activities. And you know all this kind of thing. So the best.
182
00:40:55.230 --> 00:41:20.320
Dr. David Rock: you know, the best act. Learning activities are really challenging ones ones that you like think are on the edge, like, I don't know. If people are willing to do this, it should be they should. They should feel really challenging. A lot of the learning programs we look at are just they just feel a little banal, a little bit too easy. We've sort of designed it for not not, you know, making anyone uncomfortable. You need to. As Rachel said, you need to make people a little uncomfortable. Just get permission to to do so.
183
00:41:20.760 --> 00:41:42.519
Dr. Emma Sarro: Yeah, this reminds me a lot of all of this, the concepts that we've been bringing up for the last last couple of weeks. They apply for this as well as we're going back and assessing these programs. So what about when you're when you're designing systems, or when you're advising on the systems, no matter whether they're virtual or in person. How do you advise on what systems to make sure organizations have in place.
184
00:41:43.410 --> 00:41:47.690
Dr. David Rock: Yeah, I mean. So I'll bring it back to the science. Right? Attention.
185
00:41:47.980 --> 00:42:15.269
Dr. David Rock: insight, action, right? Are the 3 things that you're trying to solve for attention, attention, insight, action. So so one of the things that that we're looking at with any of the solutions is just, you know, we're we're trying to study impact. If it's our solution, we have hard data on that right, we know the percentage of of the audience that has a habit. Now? So that's that's important. So that's that's a kind of system. But what we're looking at is is, what do you have in place to make sure that the learning transfers
186
00:42:15.270 --> 00:42:29.210
Dr. David Rock: to the work right to day to day work. What are you? What are you doing there? And you know here you should be using positive social pressure. In lots of creative ways. And so so we're looking at at how you know how
187
00:42:29.260 --> 00:42:58.290
Dr. David Rock: aligned is the overall strategy with with really embedding the habits into day to day skills. And the, you know, the worst example that is a 1 day training give people 10 things they're supposed to do, and there's no follow up. And you say, you know, go and do these 10 assignments. And you have, you know, 5% of people doing anything. The the best case example is, you take a day experience you break it into, say, 3 you know. And by the way, we find we can take a day long classroom and turn it into 3 90 min sessions and get better results than a day long classroom.
188
00:42:58.745 --> 00:43:23.460
Dr. David Rock: In terms of habits. So you you less time, much less cost and you really get better results. So we'll you know better. Example is, you take a 1 day turn into 3, 1 half hour sessions. You have really specific assignments between the people have to do and have to report out on the next. At the start of the next one. Now, you're getting 70, 80, 90% of people doing tasks. You just give them one or 2 things to do. Not a lot, you know, dedicated to that.
189
00:43:23.780 --> 00:43:32.959
Dr. David Rock: You put the social pressure on them, reminding them they're going to have to report out and then, you know, even in the last module you have something where they've got to report out.
190
00:43:33.289 --> 00:43:52.879
Dr. David Rock: You know, to an email address or to a person or something. So you're just. You're just putting in systems to kind of make sure people are paying attention. They're having insights. They're taking action. And some some of those systems are social pressure. Some of it's measurement and and you know some of it sort of going back and telling the stories of what happened as well.
191
00:43:52.970 --> 00:43:54.199
Dr. David Rock: Rachel, what about.
192
00:43:55.910 --> 00:44:23.250
Rachel Cardero: Yeah. And I, we there are some great questions in the chat, too, that I think would be great to parallel with this discussion, for you know how the systems we put in place, as like one of the questions was about bring Cough's success case measures, and I feel like that. That's a really useful methodology. After your learning is designed right to see? Like, okay, how can I? How can I show that we've done a good job.
193
00:44:23.480 --> 00:44:40.860
Rachel Cardero: A lot of what we end up doing. When we're helping organizations is actually many times we're on the front end of that process, actually saying, Okay, like David mentioned, how do we take these this day in person and translate it to something virtual?
194
00:44:40.910 --> 00:45:07.730
Rachel Cardero: And in that case we don't want to experiment and wait till the end to kind of find that success case. We can use science as the source code as the shortcut, as the secret ingredient. You know, we can use that as the checklist to help us make that conversion and like one of the things I think we like in terms of systems, or like operationalized processes that like, if I'd had to name my favorite.
195
00:45:07.960 --> 00:45:14.510
Rachel Cardero: I love using measurement as a strategy for metacognition and sustainment.
196
00:45:14.810 --> 00:45:20.019
Rachel Cardero: You know where we not only help people with the learning itself, but we actually look at
197
00:45:20.110 --> 00:45:36.590
Rachel Cardero: what are your measurement strategies? And then is it position to actually not only give you data and help learners and participants? Or is the timing off or the questions off? And so you can almost do yourself a favor as you collect data
198
00:45:36.700 --> 00:45:45.060
Rachel Cardero: and those data, you know, they're extremely important. But like, that's something systemic, easy to operationalize. If you know what you're looking for.
199
00:45:50.010 --> 00:46:08.750
Dr. Emma Sarro: Yeah. So let's yeah, let's dive into that and talk about how we measure because we have a really well designed measurement strategy. We're measuring for habits, and and how often individuals are engaging in behaviors, which is what we want, right? As opposed to whether they have awareness, as you were mentioning before Rachel.
200
00:46:09.360 --> 00:46:10.789
Dr. David Rock: Yeah, do you want to take this one rich.
201
00:46:11.390 --> 00:46:29.161
Rachel Cardero: Yeah, yeah, absolutely. So there are a couple of things. And a lot of this work, if not, all of this work comes from Emma's wonderful team, our research team really helping us hone in on. You know what those research insights are for creating habit formation and behavior change.
202
00:46:29.961 --> 00:46:55.900
Rachel Cardero: But we really focus on the actual frequency of new behaviors. So we anchor, we talk about Bcp a lot at work at Nli behavior change percentage. And that's our ultimate goal. We're a little obsessed, I would say, with new habits and helping people form those habits and keep those habits. But at the end of any learning experience. We wanna know.
203
00:46:56.150 --> 00:47:04.499
Rachel Cardero: how often are you putting this new behavior into practice? What percentage of learners are actually using the behaviors that you taught.
204
00:47:04.770 --> 00:47:08.640
Rachel Cardero: how are we helping people change their behavior? Not just
205
00:47:08.680 --> 00:47:27.779
Rachel Cardero: gain awareness of, you know, communication we brought up earlier, like, Yeah, I know, communication is is a thing, but we want learners to walk away knowing what do I need to do? And so measuring, how often are you doing? That thing is really the best barometer for knowing. If you've been successful in teaching them the right way.
206
00:47:28.710 --> 00:47:31.800
Rachel Cardero: And, Emma, David anything you'd add there.
207
00:47:32.980 --> 00:47:56.400
Dr. Emma Sarro: Yeah, only that we can. We can also pull apart that that Bcp measure and look a little deeper now. And we're starting to do this. And and if there isn't a change, why is that so? Is it that they? You know there wasn't enough positive social pressure? Or you know, there wasn't that role modeling of others that they were seeing in the organization. So why change, or did they? Did they not know how?
208
00:47:56.400 --> 00:48:09.869
Dr. Emma Sarro: So we're able to dive in and actually provide a bit more guidance on now. Okay, so you didn't get the result you really wanted. So let's let's look and see like under the hood. Why, this is happening, and then we can fix that one piece
209
00:48:09.870 --> 00:48:16.500
Dr. Emma Sarro: so there's so much more than just one number. It gives you kind of a starting point, and then you can kind of fix the problem.
210
00:48:18.470 --> 00:48:45.939
Dr. David Rock: Yeah, the Bcp, I mean, we, we could talk for hours about Bcp behavior change percentage. It's it's been such a useful construct for us. I mean, it's not perfect. Nothing is with measurement. It's a hard thing to do. But we, you know, it feels and looks really valid because we get, you know, really consistent measures across programs, different measures between facilitators, between products different like it. It just. It has this real sensibility in it, even though it's not, you know, no measurements perfect. But being able to go back.
211
00:48:46.201 --> 00:49:03.299
Dr. David Rock: You know, 2 to 4 weeks after a program and say, you know how many times a week. Are you applying this habit now? And then also asking direct reports? That question, how many times is your boss applying? I think just. It's a really really helpful metric that you can keep constant. And then and then do things. And and here's something you can
212
00:49:03.500 --> 00:49:05.670
Dr. David Rock: now take to the bank, because now you can like
213
00:49:05.980 --> 00:49:10.039
Dr. David Rock: put, you know, a hundred people through an in-person training check the Bcp.
214
00:49:10.060 --> 00:49:16.650
Dr. David Rock: Look at the cost per person. 100 people through, you know, a series of hives, you know, 3 90 min sessions with us.
215
00:49:16.870 --> 00:49:29.620
Dr. David Rock: Look at the Bcp. Look at the cost per person. Now do the same with completely digital. Look at the cost. Look at the Bcp. And you can put all that into a spreadsheet. And you can say, Look at that. You can actually look at a cost per habit.
216
00:49:30.390 --> 00:49:44.779
Dr. David Rock: There's there's some pretty basic math in that. You can say, how much does it cost for every habit, and it turns out you'll look at this and you go. Oh, wow! Actually, the virtual programs are drastically cheaper per habit than the in person.
217
00:49:45.032 --> 00:50:03.190
Dr. David Rock: It's a it's it's not that complicated to work it out. So so cost per habit can be worked out and now you have a measure. Now you can say, Hey, facilitator, you know I need you to hit this this standard. If you're not hitting this standard, there's something wrong, because this is. This is the. This is the number of habits we normally see from this program in this way. So we're starting to build
218
00:50:03.507 --> 00:50:31.699
Dr. David Rock: we're starting to build a real like school of thought around, measuring with accountability and measuring with integrity. Around these things. And I think that's you know, we we got a lot of work to do in the space, but we'll we'll present more on this at the summit kind of what's possible with this, and how we're seeing it. But it's it's we should be moving towards a world where it isn't the Wild West like if you don't get 50% of your target audience doing something a month later with the habit.
219
00:50:31.700 --> 00:50:38.209
Dr. David Rock: the program should be shut down or the facilitator shut down or completely redesigned, like you can absolutely get that
220
00:50:38.729 --> 00:50:54.680
Dr. David Rock: and if you're, you know, if you're not getting that, and if you're not getting, you know, 60, 70, 80% from an extended virtual program. Something is really wrong. So I think we're getting to a place where there's there's less of a Wild West. I think in the next year. We'll probably put out some standards like this. We'll challenge ourselves.
221
00:50:54.680 --> 00:51:13.050
Dr. David Rock: Cause a whole lot of ruckus, I'm sure, but we'll challenge ourselves to put some standards out like this and say, Look, this is what you're trying to get to. So we're starting to have, you know, this possibility of a kind of dashboard of a learning where? There's some real accountability in the process, and and being able to study the whole learning strategy, you know, using this data as well.
222
00:51:13.510 --> 00:51:30.610
Dr. Emma Sarro: Yeah. Yeah. And I know that we're designing some events in the future to go into that specifically right, the how to right when you're facilitating designing events. In person, virtual large events. It'll give you all of the how to. With this.
223
00:51:30.770 --> 00:51:55.349
Dr. David Rock: Yeah, there's been so much interest in this. We've we've decided to launch something we've never done before, and kind of open up behind the scenes a bit, and we're we're running a an in person workshop which now you're gonna say, that goes against everything you've said. But we're gonna run an in person workshop for for just instructional designers, facilitators. You know, talent folks, it is for internal people we're not. We're not doing it for external consultants at the moment. But we're doing brain based design facilitation.
224
00:51:55.350 --> 00:52:05.789
Dr. David Rock: We're also launching a virtual version of that which will be 3, 1 half hour sessions that'll be starting early in the New Year. If you're interested in the virtual version of that. Just put
225
00:52:06.080 --> 00:52:07.602
Dr. David Rock: but it's virtual
226
00:52:08.460 --> 00:52:30.580
Dr. David Rock: design, and your company name in the chat. Someone will reach out to you. Just put virtual design company name, and we're gonna have in January, maybe February. We'll have a launch of a virtual version of this. But we're running this in New York and also in London, coming up in in November. An in person workshop is where where? You know, lots of people wanted to get together and have that experience. So that's
227
00:52:30.832 --> 00:52:37.610
Dr. David Rock: that's happening. I think that's yeah. It's probably everything we wanted to cover today. Any, Rachel, what's your biggest insight as we as we wrap this up.
228
00:52:38.580 --> 00:52:49.209
Rachel Cardero: Oh, I'll bring it back to the beginning of the design, for the person in the seat, and not the stage, even if the stage is small, even if the front of the classroom designed for the person in the seat.
229
00:52:50.290 --> 00:53:08.180
Dr. David Rock: Yeah, it's it's it's really big one. Yeah, my, my, I think my insight is the one that's just sort of evolving is is, you know, is that we might be able to lead the charge away from the Wild West into, you know, learning strategies that have real integrity and and accountability right that the real metrics of what you're supposed to achieve
230
00:53:08.180 --> 00:53:27.070
Dr. David Rock: in these different kind of learning modalities, and, you know, cost per habit as a measure. So I think I think there's some exciting things that we can do in that space in the next year or 2, and otherwise. Yeah, thanks so much for the conversation, Rachel and Emma. Oh, very important question. If you're interested in a learning audit, put the word learning order and your company name in the chat. I always forget that kind of thing.
231
00:53:27.070 --> 00:53:28.270
Dr. Emma Sarro: We've been talking about.
232
00:53:28.270 --> 00:53:51.590
Dr. David Rock: Right for the word learning audit and we. We're, you know we're doing one off sessions at times complementary, just to kind of look at what you're doing, and you know, give you some guidance, and then from there we'll know if if there's you know, if there's value we can add to really dig in. And do you know proper projects. But we're happy, for you know, for for our partner clients and potential partners to come in and just do a review with you as well off the clock. So
233
00:53:51.590 --> 00:54:05.099
Dr. David Rock: put put learning audit, or just audit and your company name, and we'll we'll make sure someone reaches out to you. So thanks for a great conversation, Rachel. Emma thanks everyone for being here and look forward to seeing some of you at the summit. We're 6 weeks away.
234
00:54:05.333 --> 00:54:19.819
Dr. David Rock: October 29 and 30. It's gonna be an amazing session. We're in the final throws of of the research and pulling it together with super psyched. So maybe my team can put the the link to the summit in the chat so we can. We can get some people there. Thanks so much. Everyone. Take care of yourselves. Bye, bye.
235
00:54:20.270 --> 00:54:21.450
Dr. Emma Sarro: Back to Shelby.
236
00:54:25.490 --> 00:54:54.789
Shelby Wilburn: Thanks, Emma, David and Rachel, for today's discussion. We really appreciate you as always. We already shared the poll. But just to reiterate summit is on the way. October 29th and 30, th so make sure you check that out. Visit summit. Neural leadership.com for all of your needs. If you enjoyed this conversation, we also have our podcast your brain at work. So listen. Wherever you subscribe, and on behalf of our team behind the scenes. Thank you so much for being here. We really appreciate you each week, and we will see you next week. Have a great weekend.