eLearning Research

At a panel at DevLearn. We have:

• Will Thalheimer, Work Learning Research
  
• Kevin Oakes, I4CP
• Claire Schooley, Forrester
• Chris Howard, Bersin
• Kevin Martin, Aberdeen

Biggest trend is Learning 2.0 trend. A shift in Training Methods. Allow people to work faster, better through informal learning. eLearning 2.0 addresses informal, on-the-job. It's relatively new. Bersin is publishing report on learning 2.0. Collaborative. Learner in charge. Looking out there for the information they need. Find it. Use it. And it sits there. Younger generation used to finding things on their own.

Talent management being discussed in the board room. Performance management. Learning and performance management converging finally. Linking development plans, career paths, learning opportunities. Leap to 2.0 before they get the foundation right.

Good, Chris, called him on this. Learning 2.0 is so different. 2.0 is chaotic. Talent management is structured.

Statistics from ASTD research -

• 86% - web 2.0 technologies likely to use more in learning function than they do today
• 41% - using technology for informal learning at high or very high extent

Kevin Oakes - "learning is getting left behind as people look at talent management." Only 19% say they have integrated talent management to a high or very high extent.

Learning 2.0

• Trust is barrier
• Culture is number one barrier - knowledge sharing is problem
• Reluctance among potential mentors to teach each other
• Time is problem

Qualcomm - employees submit business plans. Have festival where employees vote on best business ideas.

What percentage should they invest in Learning 2.0 - answers ranged from 20 to 50%?

Economy

Do we have to do anything different?

• Shift priorities to align better.
• Use cost lowering technologies (video conferencing, web conferencing) to save money
• Look at criticality of retaining workers
• Improve performance of existing workforce
• Should be good for L&D to be more accountable

GE CEO - told training leaders - need you to step up. That's not the norm. Rest cut training budgets during down economies. Prediction, unemployment is going from 6.4% to 8% over the next 12 months.

Focus on how you can impact sales and cost reduction activities. That's what people care about during tough times.

Talent Management

Where does learning fit in? Is training organization taking a back seat?

Strategic effort - learning is part of it, but not leading it. Broader focus. Especially when you talk "Integrated Talent Management."

Kevin mentions how Learning Management System companies have move towards Talent Management. See Rise and Fall of LMS.

Why are you getting into talent management?

• Compete in marketplace
• Retain
• Need to innovate

Hiring manager focused on better quality candidates. HR may focus more on time-to-hire.

Informal quarterly reviews were differentiator between best in class and those not.

Measurement

Must be done the way the business does itself. Revenue. Profitability.

Not a top priority. Talk about it. Don't actually do as much as they should.

Recognize they are not doing as a good a job as they could, but generally roughly okay.

Best-of-class (top 20%) - want to have metrics. Which means they don't today.

Josh Bersin's book suggested as resource for what impact training has on organizational performance - The Training Measurement Book: Best Practices, Proven Methodologies, and Practical Approaches.

Best in class - training organization design. Hub and spoke is a good approach.

Question from audience: Can learners effectively handle control of their own learning? What does research show?

Some debate, but no research on this yet. Somewhat goes to the fact that this would be hard to measure for most kinds of organizations.

Organizations need to think of Learning 2.0 as a learning style thing.

New Literacies

At the Tim O'Reilly keynote at the DevLearn conference. His outline is very interesting in that he's promising to talk about the "new literacies." I'm hoping this is going to be similar to Work Literacy.

He uses the same quote I often use from William Gibson -

The future is already here, it's just not evenly distributed.

Pattern recognition is key. What do these companies have in common:

• Google
• eBay
• Yahoo
• Amazon
• MapQuest
• Craigslist
• Wikipedia
• YouTube

Built on top of Linux (open source). Services. Not packaged apps. Data aggregators, not just software. Network effects are key to their market dominance.

Dell Ideastorm, Starbucks, Best Buy, SAP (Harmony), YattIt, BeingGirl (P&G) mentioned as examples. Allow people to share ideas. Social media in action.

At DevLearn - tend to equate Web 2.0 with Wiki, Blog, Social Bookmarking, Social Networking, etc. - instead it's really systems that harness network effects. The lock in is data.

Eras - Hardware (IBM), Software (Microsoft), Network/Data (Google). Data is the Intel inside of next generation. He's explaining PageRank.

Wesabe - contributing credit card data. Allows us to see spending at particular stores, repeat at stores. Wow, very cool data. Wonder how we could get access to that?

Mentions Google putting testing over the toilet.

What do you need to teach people? Not about the tools. It's about how to use them. How to leverage the data? Core competency of information age - leveraging collective intelligence.

Get away from old model of data. Hadoop. Computer Science shifted to teaching Hadoop to freshman from teaching SQL. Deals with messy data better.

Statistics. Design. "Big part of the new literacies."

These new literacies are not unlike talking about visual thinking as a literacy, new kinds of synthesis. The level of these "literacies" is interesting to me. Much more challenging to develop these literacies.

Suggestion - study the people who don't need your help. Go find the people who are on the leading edge and study them. The "Alpha Geeks."

Turn the Alpha Geeks into the mentors. They would pair Alpha Geeks with good writers.

Provide self-starters with access to the best resources. Online reference tool - Safari.

Show, then do, with reinforcement from frequent small successes. Get people engaged with the process.

Watch someone doing something right can improve your own performance.

Read in reviews - people talk about themselves. Very cool concept from Kathy Sierra. She is awesome at those kind of ideas.

Importance of hands-on. Magazine - "Make" about playing with technology.

Cites Wikipedia the missing manual. Literally just bookmarked How Wikipedia Works this morning.

People are telling us about these technologies, not telling us how to change. Create peer learning opportunities.

Study success stories.

"Stop fondling the hammer and focus on the house." - It's not about technology. It's about what you are trying to accomplish.

I'm not sure that Tim connected the high level concepts with what each person needs to do. I'll try to do it at 1:30 at my presentation. :)

Per Question Reports

I received a good question from a reader:

Our school district uses Blackboard. We have recently been exploring putting more coursework online and have experimented with the Articulate line of products. Blackboard has done well for us although I am not pleased with their support of the SCORM standards.

I would like to be able to export SCORM test results by user/by question but all Blackboard seems to support is general pass/fail records. Now I can drill down to the individual results via the Bb gradebook but that just won't work for exams that are being giving to hundreds of students.

I have spent many hours researching this issue related to SCORM compliance and more specifically how to get the test results exported out of my LMS for more in depth analysis in Excel or perhaps some type of statistics package.

Would you be able to point in the right direction of a resource to help me get this done or please let me know if I am wasting my time. Perhaps the world has moved onto something else as most of the conversations online I am able to find regarding SCORM matters are 2-4 years old which in Net time as you know may as well be decades.

This brings up a bunch of initial thoughts. First, having worked on custom LMS, LCMS, Authoring projects and being pretty deeply familiar with SCORM, it's easy to blame the LMS for poor reporting on per-question reporting, but its very hard for an LMS to create generic reports that work well across all the variations that you run into. So, if the source of the statement "I'm not happy with their support for SCORM" is only the issues of reporting, that's a bit unfair.

I'd be curious what they are going to do with question level reporting. I've had a lot of clients initially say they want question level reporting. I always drill down on why they are going to do it. The most common answer is that they want to know if there are particular questions that lots of people are missing. But, in many cases (in corporate eLearning) there will be no attempt to go back and fix the course or update the test. So, if you really aren't going to do anything with the data, don't bother. It's a lot of work to get the data, look at it, decipher what it means and do something with it.

So let's get to the more helpful pieces of information. First, it appears that they are getting question level data in the LMS. That's good. And not always the case. In many cases, people will implement a SCORM course and/or LMS to only handle SCO / Module level reporting or even just course level reporting. But in this case, they have the data. Good job Articulate and Blackboard!

So, the easiest thing to do is to get a data extract from the LMS into a CSV that then you can manipulate in Excel. You should absolutely start with this to figure out what you really want to do with the data. You can do this in Blackboard by using the the "Download Results" with the "By Question and User" option. Here's a good plage showing this:

http://kb.blackboard.com/pages/viewpage.action?pageId=14551263

Once you've done this a few times and have figured out what you want your reports to do, then you can define a custom report using Crystal. Sometimes it's a bit hard to get at the data on a per-question basis, but I'm pretty sure there's a good view in Blackboard.

Let me say again, do not bother to try to define your custom report until you've done this manually across several courses/tests that will expose all the different ways that the data may come out. Then you will likely begin to appreciate the complexity. You may end up deciding to just look at the data via CSVs.

With all that, I'd love to hear your thoughts/ideas around:

* Do you use question level data / reports? If so, what do you do with it?
* Generic LMS reports that you have found useful for reporting at the question level?

Concept Worker

Daniel Pink's book A Whole New Mind: Why Right-Brainers Will Rule the Future contains a description of new age - the Conceptual Age.



He describes how our society has gone from agricultural to industrial to the information age. But then he describes how we've really moved on past that to a new age where the dominant value for most organizations are created by high-end knowledge workers working on concepts. He calls them creators and empathizers. That is to make emphasis on his focus on the right-brain aspects. But, I actually think we should be focused on the emphasis on the type of knowledge work and the type of workers. In other words...

Concept Work
and
Concept Worker

I've often been a little bothered by the fact that we categorize the a person working in a call center handling customer service requests in the same category as an engineer working in R&D - they are both called knowledge workers. That's not as helpful as it should be. The person in the call center does quite a bit of routine knowledge work. They will occasionally encounter unusual situations and then need to do more conceptual knowledge work. To me it's all about how easy it is to obtain the answers. The harder it is, the more conceptual. Certainly our work around Work Literacy is all about concept work and concept workers.

I'm sure that if I went back and looked at Davenport's Thinking for a Living: How to Get Better Performances And Results from Knowledge Workers and the various categorizations of knowledge workers he would have several ways to describe the difference between someone in a call center and the engineer and the doctor. And while these are more granular, I'm not sure I hear much of it being discussed. That's why I'm liking the idea of referring to work that involves figuring out unknowns as concept work and the people doing this work as concept workers. This more succinctly and clearly differentiates the issue for me.

eLearning Learning Launched

I'm happy to announce the launch of eLearning Learning. This is the beginning of a community portal where the community will help to collect and organize the best content from blogs and web sites. The goal is to create a place where it's relatively easy to find current content and highly relevant content surrounding eLearning.

To be clear eLearning Learning is a jump off point. The content comes from other places. Right now these are mostly posts from bloggers, but you will soon see other kinds of content appearing. The bottom line is that eLearning Learning is trying to help find and navigate the world of eLearning.

I've selected bloggers who I've been reading and felt provide value on interesting topics.

The following blogs are helping me today to launch this and I've listed a few unsolicited comments on their experience so far.

• Tracy Hamilton - Discovery through eLearning

"simple"

• Collin Kromke - Web 2.0 Work

"I'm LOVING the way the label widget works!"

• Kevin Jones - Engaged Learning

"I like your widget a lot. Kind of a linear tag cloud."

• Wendy Wickham - In The Middle of the Curve

"looks pretty cool"

You will see that these bloggers and soon other bloggers listed as Featured in their blogs. They will show the following badge:



You will also see that most featured bloggers are including the eLearning Learning Organizer Widget. This widget is in my sidebar and it shows the topics that I blog about. As a blogger, this helps me keep my content organized, which has always been a bit challenging.

You might visit a few of the blogs listed above to see the topics they are writing about. For example, looking at top tools:

Wendy

• Moodle (45)
• Captivate (40)
• LMS (33)

Tracy

• LMS (23)
• Second Life (6)
• Twitter (6)

Mine are:

• Wiki (108)
• LMS (83)
• Social Network (57)

Boy, I talk about Wikis a lot.

I hope that you will find eLearning Learning to be a valuable resource. I welcome any thoughts or feedback on the system.

Online Language Training

Over on the Big Question, we are discussing Network Feedback which is basically how can you reach out and get help around a question. Almost the same time I posted the question, I received a question from a reader around Language Learning.

The person works in a corporation in Turkey and is looking to find eLearning to teach employees English. I know there's a lot going on in online language training. In our brief exchange, I mentioned EduFire and Myngle - but the response is that these are more aimed at consumer rather than corporate.

Any suggested solutions?

And, if you have thoughts on how you would go about finding answers to this inquiry, you can use that as your answer to the Big Question. :)

LinkedIn Facebook Twitter - Different Connection Style

As a result of my recent change in my Approach to LinkedIn Connections, I've actually met quite a few people who read my blog, but I don't believe had ever commented. They were comfortable connecting with me via LinkedIn and since it offers mutual value it was worth the effort. It truly does feel like a (virtual) networking event.

Interestingly, one of the people who connected to me suggested we might want to do the same on Facebook and Twitter. This really got me thinking. I described my approach to LinkedIn only having recently changed where I view my connections now as really being more of a combination of:

• PANs = Potentially Active Network
• CANs = Currently Active Network
• FANs = Formerly Active Network

I used to think of LinkedIn (and LinkedIn themselves suggest this is how to think of it) as containing only people who I have a deeper relationship with (CANs and FANs). However, for some of my primary uses Finding Expertise via LinkedIn and LinkedIn Answers to Get Help, it makes more sense to connect to people who it will be good to be connected with in the future (PANs).

However, this is not really the case with the way I use Facebook and Twitter. Both of these tools are much more conversational tools for me.

In Twitter, I have people that I follow in order to listen into their conversation. If you try to listen to too many, or if you listen to people who you don't really have a relationship with, you will experience what I describe in Twitter Mass Follow - Nevermind - basically it feels like noise.

In Facebook, it's much the same. I have my friends who I want to get together with, see what they are doing, have conversations with.

Both Twitter and Facebook really are my CANs and FANs. In some ways, the goal on both is to stay more closely connected with FANs to keep the relationship alive. Something I must say that I'm not good at. I can think of a bunch of folks who I really like, get along really well with and who I won't talk to unless something unusual happens. It's nice when we can get a bit of that ongoing conversation via Facebook or Twitter.

Yes, I could theoretically open up to a much larger network and somewhat ignore the noise. This would get the advantage of being able to broadcast. But, it's really not the way I use these tools today.

Now, it took me 2-3 years to change my thinking about LinkedIn. So, with all fairness, it could be that I've not evolved my thinking about Twitter and Facebook.

When I step back and see the message, I'm sending, I'm a bit worried:

If we are connecting on LinkedIn based on a very loose relationship, maybe we'll have more conversations and get to the point where it makes sense to connect on Facebook and Twitter as well.

Does that sound bad?