Posted by Discovery calls. Qualifying. Are you our ideal client? Are you a prospect that is perfect for our learning system? LMS? LXP? Learning Platform? Vendors (not just in our industry, but this post is focused on our e-learning industry) in learning systems, learning tech, etc., believe that 15 minutes is sufficient time.
For what exactly?
I’m going to call out a vendor—and yes, other vendors do this—but this vendor is getting the capital—raised quite a bit, getting some fanfare, and the website pushing the narrative of WOW, and this is going to SOLVE via AI.
I believe anyone interested in an AI-focused system would want to know more. Can you, though? A little bit, but there isn’t anyone you can talk to regarding what LLM or LLMs they are using, for example. Can I find that out in my 15 minutes with a sales rep? What about a feedback loop? How do they deal with other AI issues inherent to the LLM or LLMs they have chosen? Oh, didn’t you know that each LLM has strengths and weaknesses? Surprise. Is there gen AI multimodal? Are they using a MaaS or complete open source with (MoE)?
What? I can’t find that out in the 15 minutes because it is a discovery call.
I went to Sana Labs to see the system and learn more—it says so on their website. What I saw under Explore was a misleading chart (and yeah, some of that info is erroneous—traditional software? What CBT is back? Or that text-based game?).
In Explore, nothing showed me the WOW factor of making this far different from other systems with some AI parts. Nevertheless, as a Pirate, I wanted to see it live and ask questions, not rely on their marketing twist. Knowledge Assistant looked like it had part of a feedback loop – hard to tell. There were other items of intrigue. But I was ready to book an intro. Click and go. I should note on a sidebar that in the KA section, there was zero copy that may produce fake or false information, even if it is the content you have added. That’s reality. And that is just one item.
Okay, enough about that—I likely just spent five of my 15 minutes. What I did like was that the person sent an agenda—it consisted of an intro, a topic called What Generative AI or Generative AI (I can’t recall), another topic, information about you, and, I assume, your needs.
Now, can you do all of this in 15 minutes? I doubt it. Their goal is to sell you on their system, not vice versa. You aim to see the system and get the necessary information before considering them. At the discovery phase, it is not useful to explain Gen AI to someone who likely has only heard AI (which is mentioned all over your site and could be machine learning and Gen AI—say using Vertex with Modal Garden).
If a book or an intro is a must, it should be up to 45 minutes. Even at 30 minutes, extracting the info you need and what they need is really to get you to the next step – as anyone in L&D and Training knows is how we think – we want to see! If I like what I see, here is a more in-depth use case beyond the initial one I will provide. Remember, time is their call. I can send the salesperson my challenges, issues, etc., and that info before that “intro call.” Now, they have a foundation. Doesn’t that make more sense than 15 or 30 minutes to ascertain your value?
Sidebar—On the 15-minute or 30-minute discovery calls, don’t be surprised if the salesperson follows a script, especially at a shorter time frame. I’ve seen it on demo calls, where it is evident to everyone that the salesperson is following a script. The only thing missing is the director of this play.
Swashbuckling
Turn the Tide
As a pirate, you take control—not the salesperson or the intro call. It is an introduction to them—not some stuff you can read in marketing copy or off a slide deck (that’s later). If I am the vendor, I want another call with you. Let’s schedule an hour. This way, it can be a more meaningful conversation and a dig deeper perspective. It is a two-way street, as they say. The initial call, as you it is an introduction – tell us about yourself. And spin that way. In that intro call early on, you can say – I’d like to schedule a 60-minute call to go over what we seek in a system, our challenges, what we want to accomplish, and so on. Then you, as the vendor, explain – (see if you can get the use case ahead of time – and as an LMS Pirate, you want to provide them the use case for the 60-minute call ahead of time.
Why?
I am saving time. They can review the use case and extract additional questions or questions they may have and want to know more. The use case will cover your needs, etc., so the vendor needs to ask follow-up questions. This eradicates qualifying you. Perhaps as a vendor, I block 90 minutes and say we may not need the entire 90 minutes, but I thought I’d like to show you some of our system that aligns perfectly/well with your use case.
POW – Cannon in range
It is a win-win for each of you. You do not need to include your RFP here. A pirate does not do that. The use case – detailed (remember no ambiguity) is sent. If you are using additional tools, such as integration with Workday, ADP, or Bog’s Payroll Platform – include them at the end of your use case. Why?
The vendor can state in their discussion that they have had or currently have successful integrations with any of those providers. If it is one they haven’t seen before, they can say that it is possible, but we would need to talk to BPP and get some additional information. Do you have a person we can talk to?
Sure – here you go.
Swashbuckling
Strategy for Success
A great pirate would stay on course. Is the British Navy ahead? We make a strategic move, get close, and either ram them or fire first, then ram. Then we jump over on their ship, and mayhem ensues.
You do not need to do that, so please do not storm the company’s HQ or wear a Pirate outfit.
Before that 60-minute introduction call, ask the vendor to send over any whitepapers, case studies, and other materials about the system, and in the case studies approach – ideally something similar to your scenario. The probability of the last piece is dependent on what vertical you are in.
I hear folks asking me, “I need a system that accepts video, has a training catalog, learning pathways, and administration.” That tells me, honestly, nothing because all systems have that. Yet, not just people ask me about universal items, but I’ve seen it on LinkedIn, where they want five or four items and then expect recommendations, which people give—ignoring about 25 other points the person should ask and will—okay, needs to know.
Point include (oh, as a Pirate, you want to know)
- “Ignore the active user spin” that vendors mention – They will likely bring that up. Unless they invoice monthly, then active users per month is just spin. Thus, you ask the vendor how they bill. Is it upfront – i.e., yearly, or is it monthly? If monthly, is it just for those active users in the system and not any other month? What does the vendor define as an active user? Is it someone who logs into the system? Does it take a piece of content or look at the content? What you are doing – i.e., as the prospect, is forcing the vendor to provide you with info before discussing numbers. If a vendor asks you why you want to know this early on, you can either decline to provide the explanation or tell them that you are very detail-oriented and this is relative as part of your process. Some vendors bill quarterly, but there are no active users over that quarter (actually, one vendor says they do this, but I haven’t seen any validation). Instead, it is just the total cost divided by four. A very well-known vendor follows that logical process.
- Schedule a demo. I know I push this all the time, but I can’t tell you. Okay, I will say there are a lot of people in L&D, Training, and other departments seeking a system, heck, even learning tech, who never push early to see the demo. Demo first, before RFP. A vendor should want to show you the demo. This is their time to shine. If you wish to other people on the call, such as the head of product, a data scientist, or whomever oversees AI, tell them. A vendor should say okay. I’ve seen some that want you to provide reasons for it. Who cares! You are a pirate – a leader, not a follower. Honestly, their goal is to get you as a client. Not as someone to consider to be on a quiz show.
- Why should you consider them as your learning system or learning tech vendor? Again, their job is to woo you. To stand out of the crowd. Any vendor who thinks you are not shopping around needs to see Walt Disney. This partnership is here – you want a winning partner – best in class – not just every day, wishy-washy. The goal of any vendor is to CLOSE THE DEAL. Land the deal. Generate sales. I never understood why vendors who know this is their goal make it challenging due to their approach to prospects.
Swashbuckling
Know thy AI (the hottest feature out there and will continue to be)
It’s coming even if you have zero desire to seek AI in a system. Some vendors already have it, and more do not. I won’t get into what they have with AI in their system, and then they tell you, which always seems to include course creation.
The coolest capability on a sidebar (there is that word again) is the one vendor I saw where you could change the LLM foundation model (They have several), and if you have an OpenAI key, you can add it too. The models were some of the common ones, including the latest. Nevertheless, I thought it was brilliant because every LLM, even SLM, has strengths and weaknesses. I’m sure the vendor told you this with their LLM(s) and how that works with their system.
What they didn’t say or tell you? Don’t be surprised when they don’t because unless you ask, they won’t. And if they don’t know and you want to find out, then ask to speak to someone in the company who knows the answer. Unless your salesperson is an expert in AI (generative in this case), they won’t know. And if nobody knows in the company, that is not a good sign.
You want to know what those LLM(s) are. Sana notes that they have world-class LLMs, but that tells me nothing. Which ones? How often do they add or remove, and what is their process if the LLM is no longer ideal? It seems like irrelevant information – however, in no uncertain terms, it isn’t. The foundation model – that LLM or LLMs, if they are using multiple drives, the AI in their system or tech. If the vendor uses a combination of machine learning (a form of AI) and generative AI (usually stated just as AI), they should tell you.
Personal agents are the future, but their ideal capabilities are not there just yet—regardless of whether they are for learning. Anyway, who tells you that multiple LLMs, an R.A.G., personalized agents, fine-turning, and guardrails eliminate hallucinations is lying? I often hear “reduces hallucinations.” Nice and vague. However, a vendor should state that Gen AI, even multimodal ones, can cause hallucinations.
Thus, you ask the following -, when you see the demo, look for
- Feedback Loop – This is the KING KONG, GODZILLA, and the biggest PLANET at the galaxy level. Bounce over to free GPT-4o (aka 4 omni), and you will see a feedback loop. It will have either a thumbs up or down, or maybe it will be listed as flags and have a prompt window. If the information is wrong, you select the thumbs down and write in the prompt window what it should be or what is erroneous. Then click the submit. Guess what happens next?
The AI learns from that. A couple of learning techs focused on AI have it. One has a thumbs up or down. You need the prompt window. On the system side, it is an embarrassment. Pluralsight has it with a new offering they are launching soon. Another vendor I saw about a week ago, the name escapes me, has it too. Both are on the learner’s side. Since their Gen AI is available for learners, it is relevant.
I haven’t seen the admin side, where you can create a course and an assessment. Overall, I like Cypher Learning’s AI features, but the AI assistant doesn’t have a feedback loop. To me, that is just plain wrong. It creates an assumption that this assistant is 100% correct. How do you know?
A pirate will ask about the token fees. What do you think Gen AI is free? Perhaps the vendor will eat the cost or is eating the cost. Maybe they have hidden it within the add-on – the course creator, for example, is listed as an add-on. When you get to the learner side with Gen AI, those token fees can escalate quickly, even if they are very low in price. Throw in multimodal, and those token costs will go up quickly, too. You do not need to ask for the prompt’s number of characters or parameters. It’s not relevant here – for you.
Pluralsight shows the token costs (in Iris) for what you type in or have in the system. I thought that was highly beneficial (it’s on the admin side).
LLM agnostic—It’s the next big buzzword. Thus, if the vendor states it as such, you, the Pirate, will ask them to have the person overseeing AI connect with you so they can explain the process, how it works, and so on. One vendor told me it was the ability to bring in your own LLM and zing it works without issue. It’s not that simple. What the vendor needs to do is not simple and involves some prompt, severe engineering. Anyway, the point here is to talk to someone who can explain how it works in laypeople’s terms. Feel free to ask them to provide a diagram of the process – it is easier to see a diagram and then ask questions – if you are unfamiliar with the process. And it will depend on the process – i.e., not everyone does it the same way. If the vendor doesn’t mention LLM agnostic, bypass this question area.
Bottom Line
A LMS Pirate (or learning platform, LXP, combo, or whatever the vendor wants to call themselves) drives the discussion. The salesperson’s role is to help them along the way. Be polite, give additional insight, and know the product inside and out (I can tell you many do not). Ask to see the roadmap. Yes, all of this before that RFP.
What’s the point of sending an RFP if you have never seen the system, the tech, or the content? When you decided to do a remodel, did you ask questions, provide insight on what you wanted, ask for examples, and look at the options, OR did you just let the contractor make all those decisions for you? Did you lead, or did you let the contractor push you? Did the contractor try to win your business or say, “I’m the best; you should pick us, and if you don’t, you are not worth my time.”
You are a pirate.
Think as one.
Take out the virtual blade, slice through the schtick, and lead your crew to an outstanding victory.
Because there are only two ways this will turn out
Finding the system that turns into gold
OR
spending 15 minutes stranded on an island
Never to be heard from
Again.
E-Learning 24/7