I don’t what it is about surveys. Some people loathe them, others get excited about them. Some folks will spend 30 minutes or more completing a mult page survey, others will frown if it goes longer than two minutes.
Giving a “bonus” to have someone complete a survey is equally a hit or miss. Back in the days when people received surveys in the mail, sometimes they showed up with an actual dollar. The reason was simple enough – research showed that it was more likely someone would complete and return a survey (already pre-paid postage) due to receiving something versus someone who would not.
Short and Sweet
Go to any graduate program on market research and they will tell you that the maximum should be one page (10 questions) to retain people’s attention span to complete a survey. But in the age of the quick and fast, not even that will guarantee you the numbers. I’ve seen surveys four pages in depth and others that are a few questions with published results being all over the board – i.e. how many responded.
Data says “Star Trek” (get it?)
The total number or responses which constitute validation is another murky area. Back in my grad days, scientific results were 93% or higher response rate. Thus if you sent out 100 surveys/questionnaires, anything less than 93 being returned was not of scientific quality. However, in the real world – if you landed 93% you would be a miracle worker.
If your survey had 10,000 respondents from anywhere and everywhere and not having any interest per se in the topic, be worth more than if you had 300 respondents via cluster sampling or 300 who were actually involved in some manner with the industry, topics or topics?
You can apply the same logic to focus groups – are five or ten people who meet some criteria of higher value than say 200 random strangers who you picked off the street?
I’ll let you decide on that.
What’s the Point?
The E-Learning 24/7 latest survey is all about e-learning and more importantly learning management systems. Respondents were asked questions tied to mobile, video and yep, even MOOCs.
Results were cross-tabulated and for those who love such items, an added twist – using multivariate anlaysis – Multiple line regression and separately, K-mean method.
For the results below, it was simple tabulation with a bit of cross for further extraction. The other items, were for EL247’s internal purposes (non nefarious I promise you).
The Details – We want the Details
- 484 respondents who completed every question, 40% corporate, 30% education, 8% non-profit, o% from association and 22% e-learning vendors (those were extracted from the below results and will be seen in a future blog)
- An additional 19 respondents did not complete every question and thus were removed from the analysis
- The survey was 100% independent – it was not tied to any LMS vendor or have any ultra hidden agenda meaning (sorry conspiracy ppl)
- The survey initially was to be published in the “State of the LMS Industry 2014” report but due to the size of the report – which appears to in two separate parts, part 1 is 72 pgs, part 2 is close to 300 pages (it is all profiles) – I decided to hold off adding it
In 2014, I believe that video will be hot in the LMS space (amongst others), but it will be on fire watch.
The area is listed in the following manner:
- Header – The question
- Results – Yes, No, Don’t Know (personally, I hate surveys that never include a “don’t know” area, it forces people to select one of the others even if they truly don’t know. To save some time, don’t know will not be listed unless it is significant)
- Analysis (if any, sometimes the data speaks for itself)
People were asked a series of questions regarding video and a LMS – would they want this feature (sub text)
Ability to record video from a web cam and upload the video into a LMS
- 65% said yes
- 29% said no
Upload video from a mobile device or HD camera into a LMS
- 71% said yes
- 27% said no
Analysis: With the success of shooting video from a mobile device and uploading it to such sites as Vine or Instagram Video, this shouldn’t come as a surprise. Consumers are consumers – something that many LMS vendors tend to forget.
Learners can upload their own video
- 71% said yes
- 27% said no
Using cross-tabulation, the learners upload on video was higher with folks in the education sector than corporate.
Learners can share their video with other learners
- 72% said yes
- 28% said no
The twist: Using cross tabulation, Corporate and share video was higher than those in education and share video, non-profit was tossed into corporate and had no real influence
Ability to video edit in a LMS
- 67% said no
- 33% said yes
I was surprised by the data frankly, because it seems to be a bonus to those who do not want to purchase their own software package. That said, for those who want some punch it makes sense – similar to 3rd party authoring tools. The tools will always be far superior (depending on which one you get) than one that is in a LMS.
Screen capturing and recording tool in a LMS
- 65% said yes
- 27% said no
Cross tab – Of those who responded yes, nearly split between Corporate and Education
Analysis – Get it in your system, regardless on whom you are targeting
How often do you use your mobile device to access your LMS?
- 36% said often/very often
- 29% said sometimes
- 31% said rarely/never
- 4% said they do not have a LMS
Irony – Not many vendors have “responsive” with their mobile view – responsive means the layout will change to fit the screen of your mobile device.
Self-Contained and On/Off synch
It is clear to me that we as an industry have a long way to in explaining self-contained (as in what it is and how it works) app and on-off synch. Overwhelmingly, people want on/off synch with higher cross tabulation for corporate and yet at the same time it was nearly 50-50 on using versus not using a self-contained app.
You need to have a self-contained app in order to use on/off synch. It reminds me of a survey a TV show did with Obamacare. They randomly asked people about Obamacare and overwhelmingly people hated it.
They then asked about the Affordable Care Act and overwhelmingly people supported it. The irony? Obamacare and the Affordable Care Act are the same thing.
Just something to consider. Now on to the numbers:
How often would you use a self-contained native app (which you download content, etc. and work on/take) on a mobile device?
- 29% said never
- 6% said rarely
- 33% said sometimes
- 23% said often
- 9% said very often
Add never and rarely together – 35%, add often and very often – 32%, Corporate ruled on the cross tabulation.
How often would you use on/off synch (enables the learner to work on content offline and then when synched back up on the net, the data is pushed back into the LMS)
- 48% very often/often
- 23% sometimes
- 29% never/rarely
Cross tabulation – Another corporate rule
How often would you use an app store where you could select as many apps as you wanted (free/fee) to use in your LMS
- 44% rarely/never
- 29% sometimes
- 27% often/very often
It is hard to tell that if the “fee” was removed whether or not this would decrease the rarely/never and increase often/very often. But anyone who does such an inquiry has to include fee based, because just as in Google Play or iTunes not every app is free.
Corporate was overwhelmingly in favor (often/very often) than education, and equally higher than people who selected rarely/never and listed corporate as their sector. It was split in the “sometimes” and “corporate” sector.
I always am surprised at the number of people who assume that everyone who is involved in someway with e-learning knows the jargon, especially when it comes to high profile terms.
Since there are some vendors in education and corporate who either have already in place or are adding MOOC platforms, the question had to be asked:
Do you know what the term MOOC means?
12% of respondents said no. As a bonus – 50% of those who said “no” were from Education. A double bonus: 50% of those who made up the 12% of “no” were e-learning vendors. Respondents from Corporate knew what the term MOOC meant.
Would you use a MOOC platform in your LMS?
- 56% said yes
- 32% said no
- 12% don’t know
Cross tab: of those who said “yes” in the MOOC platform – very high in education, 38% in corporate.
My take: Not high enough in corporate to warrant an investment, unless you are sure it will work – and work is a big question – not from work in the sense that the feature works, but whether or not the commitment is there long term from the corporate admin or whomever will be in charge of having the SME or instructor stay on task and committed to the time needed to achieve success.
Here are some tasty bits of data to think about on this cold Feb. day – even by LA standards (60).
Thx Craig very interesting findings and mix. 2014 will definately see some interesting “landings” in the e-learning field. Am watching out for Africa to see how these changes will impact our landscape too.
Date: Mon, 3 Feb 2014 18:20:31 +0000 To: [email protected]
Have you heard of waggl.it (http://waggl.it)? It’s a new crowd sourcing style of survey. The results have been extremely good at engaging the target audience and producing conclusions that offer insight. You compare war stories and ideas instead of answering a series random questions. We used it twice, once for a massive company seeking internal ideas and once for a start-up seeking industry insights from nearly 100 Fortune 500 execs, half of which responded. It worked out very well and produced results that we didn’t anticipate.
Comments are closed.