Tuesday, November 13, 2007

Rearranging Deck Chairs on the Titanic (revisited)

I was recently reminded of the still-outstanding discussion related to Jay Cross' (among others) take on Formal vs. Informal Learning by a new colleague.

It's interesting (to me) how portions of this discussion are still chugging along, but not nearly with the same fervor that they had when Jay's book came out. I'm honestly not sure how to interpret this - have Jay's points been tempered from their original, more extreme stance to be something that should be considered, but not seen as a 'sky is falling' situation?

The following was originally posted to the Tata Interactive Systems' Blog, but it bears repeating for (renewed?) consideration:
-----------------
Rearranging Deck Chairs on the Titanic?

Jay Cross recently posted some interesting (and startling) statistics about the impact of what we, in the custom learning design and development business, make a living doing.

Consider the following blurb (three times - once to allow your amazement to pass; a second time to allow your denial to pass; and a third time to slowly and deeply consider the ramifications, if this is even remotely true):

Formal training programs are not the only learning game in town. CLOs who spend the bulk of their time improving the development and delivery of training might be optimizing the insignificant. Consider this:
  • According to Tom Gilbert and Peter Dean, training only accounts for 10.5 percent of the total potential change in worker behavior. Clarity of objectives, working conditions and other factors are more important.
  • According to the Institute for Research on Learning, at most, formal training only accounts for 20 percent of how people learn their jobs. Most workers learn their jobs from observing others, asking questions, trial and error, calling the help desk and other unscheduled, largely independent activities.
  • According to Robert Brinkerhoff and Stephen Gill, people who do attend formal training never apply 80 to 90 percent of what they learn back on the job. They forget the bulk of what they’re exposed to in a matter of days.

So, formal training accounts for 20 percent x 20 percent x 10 percent of the possible improvements you can make to worker performance. That’s 0.4 percent. To account for potential double-counting and other quirks, let’s say training might influence 1 percent of worker potential. C-level officers who want the human capacity to thrive over the long haul are looking for more.

Now, I'm certainly not lobbying for the end of formal training, but perhaps there is some merit to calling for the end of the *current version of* formal training.

With all that is known about HPT/HPI, Simulations, and Story-based Learning, not to mention long-available but minorly-leveraged knowledge about EPSS, why is it that the vast majority of the eLearning work that is done today resemble electrified versions of what was used to teach our great grandparents? Why are Jay's ideas regarding "informal" and "free-range" learning (for instance) readily accepted in theory, but largely ignored in practice?

We can do better (and it doesn't have to mean a loss of business).

When will learning professionals, who know in their heart that what their client is asking for isn't going to solve their problem or have the desired impact, feel confident to act on their responsibility/obligation to say as much?

When will customers (internal and external) allow the T&D professionals they've hired (again, internal or external) do what they do best, rather forcing them to act as glorified order takers and production shops for knee-jerk/best-guess remedies?

It may be a utopian wish that will never arrive, but, as Robert Browning said, "A man's reach should exceed his grasp or what's a heaven for?"

Update: 11/20/06
Jay's efforts related to Informal Learning have blossomed with the recent publication of his book on the subject. Good stuff!

In reflecting on his points and how they relate to Storytelling (my current area of focus), I ran across a Breeze session he's shared at: http://jaycross.com/informl/ . It's a great overview that clicks at a deep level with many of my thoughts/experiences.

I thought I'd share a shot of one of the slides that give a visual representation of the original point I made above. It may help to communicate just how small a slice of what is done in "formal training" is making any substantive difference.

(to recap the claim: Only about 10% of all potential change factors are related to "training". Of that 10%, only 20% is "formal" (the other 80% is "informal"). Of THAT 20%, only 20% makes its way out of the classroom and into application on the job. So... 10% * 20% * 20% = <1%!)

Tuesday, September 25, 2007

Design is as design does

For a brief period of time after I gave up on Ceramic Engineering as an undergrad major (I had no idea it involved so much Chemistry - not my favorite subject), I seriously considered Industrial Design. Something about form and function, aesthetic and technical really appealed to my nearly even left/right brain split, but the pragmatist in me thought that getting a formal Engineering degree would open more doors, so off I marched towards the Tech Quad...

I never lost my interest in ID (not to be confused with my current day job, Instructional Design), and still get a great kick out of the thumbing through magazines like
I.D. in my local mega-book store.

The latest issue of
Fast Company (Oct '07) is focused on Design of all shapes and sorts and is a really interesting read (IMHO). One thing that struck me as I browsed the pages was a minor realization that although it wasn't an intentional goal (and certainly isn't as classically 'tangible' as other types of design), my interest in software, technology, and their hooks into learning have led me into the broad field of Design in a way that I might have experienced if I'd followed my heart rather than my head as a 19 year old.

Just how different is Instructional Design from Industrial Design, at its root? Just how far can you push the question (Software Programming vs. Interface Design vs. Architecture) before the differences begin to outweigh the similarities? Are there some fundamental commonalities that run, like a thread, through them all? Is it as simple as the urge to 'create' something?

The lead/cover article is on Yves Behar, the creative spark behind the much lauded XO-1 of Nicholas Negroponte's
One Laptop Per Child (OLPC) initiative, among other things. In that article, he shared an insight that kicked the question above into gear for me:

"The simplest definition of design is how you treat your customer. If you acknowledge their intelligence, and treat them well from an environmental, emotional, and aesthetic standpoint, you're probably doing good design." By that standard, he says, few CEOs come close. "They just don't know how hard it is, and what it will take on their part. There's a pain in transformation, pain when you have to do things differently." Most execs hope skillful marketing will make up for design shortfalls, or that word of mouth around an occasional well-conceived product will float the rest of their wares. Such rosy thinking overlooks the tensions that arise when design gets factored into a big business. "Marketing people are incented to come up with great ideas," says Mitch Pergola, fuseproject's vice president and general manager. "Engineers are incented to drive out costs." To resolve those conflicts, somebody at the top has to make the Solomonic calls. "If you want to be design-driven, " Behar says, "the question is, Who's driving?"

Although he is speaking of the design of physical objects, much of the sentiment he captures could just as well apply to Instructional Design (or Software Design, or...). At its core, good design requires some recognition and respect for your customer/audience. In training terms, acknowledgement that your (adult) learners bring something to the educational table, based on their background and experience. They are NOT blank slates that are poised to be filled with the blessings of the wisdom that has been deemed necessary for them to imbibe. Time and time again, when ID's step off their artificial platform of being the fount of knowledge and assume a position of being an enabler and facilitator of information exchange, everyone is richer for it.

Good design isn't window dressing on some more important, underlying core. It is an equal at the table of required elements for success.

Monday, September 24, 2007

Beginnings, Endings, and Head Fakes

Although pretty commonplace and unremarkable these days, I'm in the midst of a job shift and all the associated excitement & nervousness. I am, at once, sad for what I am leaving behind and amped for the potential of what lies ahead. Despite the fact that these sorts of shifts are so commonplace these days as to almost not being worthy of mention, I find myself reflecting on what this transition means to me, what I'm hoping for, and what is fueling me more and more as I get a little grayer.

Early in my career, I had an incredible and rare opportunity to study in an area of extreme interest to me (the combo of technology and human performance/education) at an institute that had a short but intense life, under a dynamic and controversial visionary leader. Out of the worldwide organization I worked for, only a handful of employees were given this golden shot to be 'assigned' to the 'project' of getting an M.S. while staying a full-time (fully salaried/benefitted) employee. Not bad a bad deal, at all! The last hurdle in the competition to procure one of the coveted spots was to have an in-person interview with the head of the program. Without his nod, no amount of corporate support would secure admission.

I was blessed/cursed with the awareness that this was a turning point for me, both professionally and personally. This single interview could determine if I was going to be allowed to comfortably pursue an area of interest in a way I likely wouldn't be able to do otherwise. The interview was as memorable as it was short. In a hurried and distracted fashion, the program director sat down and simply asked, "Why should I give a spot in my program to you?", then fell quiet.

I was tempted to rollout one of the buzzword-laden smooth-talking answers that had historically worked so well in landing summer jobs, undergrad admission, and my current consulting role, but I instantly knew that sort of an approach wouldn't fly with this guy - he'd see right through that sort of answer as the BS it was, and I'd be out on my ear. So I took a moment, did my best to not be (overly) intimidated by the silence in the room or the stare of the hulking presence on the other side of the table, and decided to take a chance on simply speaking from the heart.

I replied (in part), "When my grandkids come to me, sit on my knee, and ask what it was that I did with my working life, I want to be able to say something more than simply, 'I saved companies that made a lot of money even more money.'. That's something that any of hundreds of thousands of people will be able to say. I want to answer that I changed people's lives in some, perhaps small, but meaningful way."

With that, he immediately rose from his chair, returned to desk and began typing. A few seconds passed before I mustered the courage to say, "So, I hope to hear from you soon with your decision...", when he interrupted me. Without lifting his head for a parting glance or his hands for a handshake, he simply said, "You won't be hearing from me. If you hear from anyone, it'll be from someone with your company." He continued typing, head down, and I gathered myself and left his office.

As I shuffled down the hall to the other end of the building, I was sure I'd blown it and just watched a once-in-a-lifetime opportunity evaporate. I returned to where my company's representative was dispatching the next interview victim, probably looking a little shell shocked and crestfallen. She very subtly gave me a smile and quietly whispered, "you made quite an impression... he's said 'yes'."

From that moment on, I have tried to pursue that objective (more or less). There have been times that I felt compelled to be a little more practical and focused on the day-to-day of life, rather than my ideal pursuits (having a child and a mortgage will do that to you), but I've yet to lose the fire that was quietly ignited in that interview. My new employer has a formal policy related to community involvement, allowing for up to 2 hours of paid time per week to be spent helping to better the world in some way. I plan to take them up on the offer and lend my talents in eLearning to our local school system - maybe through Squeak, maybe via Dean Kamen's FIRST, maybe both, maybe something else, but something.

All of this has been brewing in my mind/heart for awhile now, but this morning I ran across a WSJ article and video that brought it all back front-and-center. It was a story about CMU CS Professor Randy Pausch and his 'Last Lecture'. I'd heard of Dr. Pausch before, based on his work with VR and immersive worlds, and knew him to be a legend of sorts. I had also heard of the new trend of some universities to ask their top professors to consider what they would share with their students (and the world) if they only had one 'last lecture' to present. What I didn't know was that Dr. Pausch had been told that he only has a few months to live, due to pancreatic cancer, giving his session a bit more of a literal (and heartbreaking) tie to the series title.


I'd encourage you to at least read the WSJ article and view the abbreviated video of his talk. If you have an hour and change of time, however, I struggle to imagine a better way to spend it than watching the full presentation. I won't attempt to summarize or highlight the talk (although others, including Mark Guzdial, have done a fine job) - suffice it to say that his concept of "the head fake" resonates with me (and is reminiscent of Seymour Papert's and Alan Kay's philosophy related to "hard fun"). And the way he weaves the concept into practice at the end of the talk (twice!) is nothing less than amazing and heart-breaking, considering his circumstance.

We all can't be Pausch's or Papert's or Kay's, but what I can be is the best me the world has ever seen (or will ever see!). I hope I found an inviting place to let some of these aspirations take flight with my new position/employer. I draw a renewed sense of energy and inspiration to make my mark, somehow and in some way, from Dr. Pausch's 'last lecture'.

Maybe you will too...

Monday, August 6, 2007

Water Water Everywhere...

The bottled water industry has been getting it pretty hard from just about everyone of late. If it's not the amount of non-biodegradable waste the bottles account for, it's the fact that most bottled water is just filtered tap water (or should I use the more industry-friendly term of "PWS - Public Water Source", as Pepsi Co has decided to label it's top-selling Aquafina?).

With regard to the first point, I recognize that most (all?) bottled water is packaged in recyclable materials, and that it's the consumers who aren't disposing of the bottles properly. I also recognize that there are many other "drinks" that contribute to the issue (juices, soft drinks, sports drinks, etc.), so the blame should be exclusively left on the bottled water industry's doorstep. On the other hand, an environmental problem of this size isn't minimized just because of a rerouting of blame-assigning finger pointing (as legit as it might be).

But the part that really surprised me is related to the cost, as compared to regular municipal water sources. Consider:




You can get at least 450 gallons of L.A. tap water for the $1.35 you'd pay
for 20 ounces of Aquafina. Turn that around, and 20 ounces of L.A. tap water
costs about one-twentieth of a cent. Would you pay $5 for a gallon of gas in a
pretty container if you could get a plain-wrap gallon for half a penny? When it comes to water, that's pretty much what we do.
(The Atlanta Journal Constitution, 8/5/07)


In San Francisco, the municipal water comes from inside Yosemite National
Park. It's so good the EPA doesn't require San Francisco to filter it. If you bought and drank a bottle of Evian, you could refill that bottle once a day for 10 years, 5 months, and 21 days with San Francisco tap water before that water would cost $1.35. Put another way, if the water we use at home cost what even cheap bottled water costs, our monthly water bills would run $9,000.
(Message in a Bottle, Fast Company, July 2007)


I think it might be time for me to rethink my spending/drinking habits...







Thursday, August 2, 2007

Effective eLearning ID: Tech Skills Req'd?

(originally posted to Tata Interactive Systems' corporate blog)

Alan Kay, a true pioneer/legend in the Computer Science world, once said:

"People who are really serious about software should make their own hardware."

Apple CEO Steve Jobs has referenced this quote several times when challenged with the idea of making Apple more profitable by taking a page from Microsoft's strategic play book and concentrating efforts primarily on software.

Without speaking for Mr. Kay and what it was that he intended with this famous phrase, one potential interpretation is that living in operational silos makes for less than stellar results. The more you have isolated groups who have little/no understanding or appreciation of what goes on in other operational divisions, the less likely you should expect anything groundbreaking or revolutionary to emerge. The best one can expect in such a circumstance is a more finely polished version of what has been seen before, due to a lack of understanding of what is possible and reasonable.

Relating this interpretation to the world of online learning and instructional design, I have gone back and forth over the years regarding if the best (eLearning) IDs also have a more-than-passing familiarity with the basics of computer science, programming, and some of the more popular/powerful authoring tools.

Reflect, for a moment, on the following:

  • Are the IDs you admire/respect the most tech literate?
  • In recruiting, do you explicitly look for tech abilities/understanding as one of the characteristics that are required?
  • Are tech skills something your organization supports developing (in the form of ongoing training) in its ID team members?

Why? Why not?

I'm still not fully convinced one way or another, but I have a heavy leaning...

In my 16+ years in the field, I've seen far too many examples of designs thrown over the wall to developers that detail either mind-numbingly simple interactions (for lack of knowledge that anything better was possible) or amazingly complex pipe-dreams that would require a form of A.I. to actually implement (for lack of understanding of what sorts of logic would necessary). In such cases, I can't help but believe that having a moderate understanding of how development work is done would make for better designs (and ultimately, courseware).

Just as a good architect can't simply design based on what 'looks good', but actually needs to have a basic understanding of the strength of various materials and how they may (not) interact with each other, perhaps the best IDs should know be conversant with Programming Concepts 101 (maybe 201? maybe more?).

Of course, there will always be the counter argument that such a background should be regarded as a 'nice to have', not a 'must have'. After all, didn't Frank Lloyd Wright design beautiful homes that had notoriously leaky flat roofs? (it is said that FLW once told a client to "Move the chair" in response to a complaint of rain leaking through the roof of their house onto the dining table.) But should we build the rule based on the exception?

How would the ID profession change (both positively and negatively) if we suddenly began to require more of a tech-bent? Would we end up cutting our noses off to spite our face? Or would we see a sudden surge in the quality, sophistication, and ingenuity of the instructional solutions that are thrust upon the world?

I'd like to hear YOUR thoughts, regardless of what they are, or how fully fleshed out they might be!

Weigh in!

Personalized Printing --> Personalized Learning Docs?

(originally posted to Tata Interactive Systems' corporate blog)

A couple of months ago, WIRED Magazine made a subtle offer to their subscribers (I've been a loyal subscriber since the first issue showed up in my mailbox for free back in the early 90's):

The first 500(?) people to email a digital photo of themselves to their site would get a personalized cover (using that picture) on the July '07 issue, courtesy of Xerox and their "Custom Xerox" offering.

Well... my personalized issue arrived the other day (see the image on left), and even though it is just a marketing stunt, I have to say that it was pretty neat to have my photo on the cover of my favorite magazine (my son got a kick out of it, too)!

This fun experiment got me to thinking about the implications for learning....

Being primarily focused on technology-based learning, I usually see things through those sorts of glasses, but I am in full agreement with the claim that eLearning is no more a silver bullet solution to education/training woes than the television was when it was introduced. Rarely are there full, single-source solutions to issues, and education is no exception - a careful analysis of goals and objectives, tied to a review of the available tools and techniques is always a critical first step in any learning effort.

With that in mind, I was reflecting on how, even in this age of websites, blogs, and wikis, most people are still much more comfortable and happy with physical documents. For anything over the length of a short email message, most people still prefer to have a hard copy for reading and 'personalizing' (marking up with notes and comments). This is easy enough on a one-off basis for shorter length items using traditional PC printers and blank paper, but what about longer articles or collections of essays that are related?

What would it be like to be able to self-assemble 'personalized learning packets' of related articles from the web that could be bound into a magazine-like format? It's not a rocket science idea (in fact, I don't think there is anything that's been preventing it to date), but just how much more convenient, usable, and useful would such an animal be, compared to stacks of individual articles (usually stapled in a corner, printed with questionable quality on a single side of paper)?

Would YOU value the ability to have a "magazine" of the articles that you wanted/valued? How much would you be willing to pay for it? Would your organization see value in producing customized, high quality collections of articles for its employees (as an internal communication vehicle, performance support tool, or as a blended-learning artifact)? What sorts of avenues of opportunity begin to emerge and open as the ability for greater personalization becomes feasible (both technologically and economically)?

Hmmm.....

(Heck... forget about personalized magazines! What about personalized objects? Where do these trend-lines begin to take us, as learning and performance improvement experts?)

Facts vs. Stories: And the winner is...

(originally posted to Tata Interactive Systems' corporate blog)

Michael Moore's latest movie, Sicko, is a powerful example of how Stories trump Facts when attempting to teach material that is intended to invoke change (mental and/or active).

Although the film won't be in wide release in the US until later this month (June 29th), Moore has been on the promotional circuit non-stop since his latest work debuted at Cannes' annual film festival (where he won top prize of Best Picture three years ago for Fahrenheit 9/11'). This time around, Moore set his sites on the flawed American health care system.

This blog isn't the proper place to comment upon the in/validity of the content within Sicko. Nor is it the right forum for arguing the pros and/or cons of socialized medicine. Besides, a quick search on the 'net will turn up more than enough 'discussion' on these topics.

I'm mentioning this movie here not because of WHAT material is contained in the film, but rather because of HOW that material is presented.

Here, in the US, the shortcomings of the healthcare system aren't something newly uncovered, freshly emerging, or previously unrecognized. The issues associated with this topic have been
discussed and analyzed in great depth, for several years, by both political parties and a myriad of independent agencies (see this, that, and the other for a short, random sampling) . And yet, little has occurred in the way of substantive change on this hot topic.

Why?

While it would be wrong to simplistically distill this complex issue's solution into a blog entry, for the purposes of education, learning, and change (arguably the focus of this blog), there may be
something interesting and illuminating that we can learn here.

Consider the following statistics:

  • "Rates of potentially preventable hospital admissions ranged from more than 10,000 per 100,000 Medicare enrollees in the worst performing states to 5,000 per 100,000 enrollees in the five best... The researchers estimated that if all states could reach low levels of preventable hospital admissions and readmissions for Medicare recipients, hospitalization rates for senior citizens alone could be reduced by 30% to 47% and save Medicare $2 to $5 billion a year." (source)
  • "Between 2000 and 2005, 7.2 million Americans lost their health coverage according to the US Census Bureau.... At the end of 2005, the number of Americans without health insurance reached 47 million." (source)
  • "...the U.S. is an outlier in terms of financial burdens placed on patients. One-half of adults with health problems in the U.S. said they did not see a doctor when sick, did not get recommended treatment, or did not fill a prescription because of cost... Despite these high rates of forgone care, one-third of U.S. patients spent more than $1,000 out-of-pocket in the past year. In contrast, just 13 percent of U.K. adults reported not getting needed care because of costs, and two-thirds had no out-of-pocket costs." (source)

OK... Now consider these profiles from Sicko (as summarized by Robert Weissman):

  • Dawnelle, whose 18-month-old daughter Michelle died because her health plan, Kaiser, insisted Michelle not be treated at the hospital to which an ambulance had taken her, but instead be transferred to a Kaiser hospital. Fifteen minutes after arriving at the next hospital, Michelle died, probably from a bacterial infection that could have been treated with antibiotics.
  • Julie, who works at a hospital, explains how her insurance plan refused to authorize a bone marrow transplant recommended for her cancer-riven husband. He died quickly.
  • Larry and Donna, a late-middle-age couple, find that co-payments and deductibles for treatment after Donna has cancer add up to such a burden that they have to sell their house and move into a small room in their adult daughter's house. The day they move into their daughter's house, her husband leaves to work as a contractor in Iraq.

Which list better captured your attention? Which list made the (claimed) crisis more real/tangible to you? Which list created more outrage... sadness... anger... disappointment... amazement?

From an Instructional Design perspective, which list (or approach: fact vs. stories) is more likely to improve your ability to understand the situation at hand, recall relevant examples of the claims, and invoke behavioral change with your local government representative? (Comprehension, Retention, and Application are three primary legs upon which the fundamental goals of Instructional Design rest).

The questions are effectively rhetorical. In matters of Change Management at nearly any level (personal, corporate, organizational, political), the analytical and quantitative approach of providing stacks of facts and figures in hopes to evoke a shift in behavior is inefficient (at best) and ineffective (at worst). Most change doesn't occur based on evidence - it most often happens based on emotion.


Al Gore's award-winning documentary on global warming, An Inconvenient Truth, was chucked full of statistics and report findings that had been around for years, but those numbers weren't what made the film a catalyst of attention and action - it was the stories and photos (in part) that caused audiences worldwide to speak up and take action.




The dangers of Methamphetamine are well documented, but the factual presentation of definitions and known side effects were largely ineffective against the powerful lure of the drug. Solution: move away from the dry figures and highlight the stories of real people (individuals, as well as their families, children, neighbors, and communities).


The next time you are tempted to force a change in your organization (or at home) by overwhelming your audience with facts, figures, charts, graphs, reports, or (gasp) orders from authority figures ("Why should you do this? Because I/your boss/the CEO said so, that's why!"), take a moment and resist the urge. After all, you know in your heart that sort of an approach is rarely effective beyond the short-term.

Instead, come up with a tangible example (story) that illustrates why action is necessary, what sorts of positive (or negative) results may blossom from the change (or lack thereof), or how the success of the change is intimately tied to the actions of individuals. The difference, not only reception, but in deep understanding and internally-motivated behavioral change/action will be amazing.

In the end, it's easy for data to be questioned and argued to the point that your larger message is lost; to bury your call to action beneath an appeal to facts; to mistakenly assume that your audience is logically-driven rather than emotionally-driven.

When we tell (focus on the quantitative), we force our audience to accept a viewpoint that isn't theirs. When we share (focus on the qualitative), we allow our audience to draw a conclusion
for themselves - to decide, for their own reasons, that change is necessary.

All long-term, meaningful change is grounded in belief, and belief ultimately comes only from within. Stories provide a uniquely effective path to the threshold of internally-motivated change.

The Future of Information Management

(originally posted to Tata Interactive Systems' corporate blog)

We are living in exciting times. We are smack-dab in the middle of a r/evolution of such magnitude that its impact will (likely) only be evident in retrospect.

Three data points have converged recently to evoke this feeling of excitement, amazement, and minor vertigo about what the future holds.

1) Information Management Technologies











In the video above (one of several fantastic presentations shared by TED), Blaise Aguera y Arcas demos some jaw-dropping technology (Photosynth) coming out of Microsoft Live Labs (based on Blaise's previous efforts with Seadragon, which was acquired by MS in '06). This image/info manipulation and organization tool is tough to describe in words, but the wide array of potential applications/uses immediately become apparent via the demo shown in the video.

Suddenly, we see the ability to enable the emergence of information that has never been explicitly defined by someone, but rather is dynamically created based on the novel analysis of large groupings of small pieces. (see the segment about Notre Dame modeled off of Flicker images)

We begin to see novel ways to leverage technology to do truly unique things, rather than using them to recreate old things in new skins. (see the embedding of micro images/print into digital versions of 'traditional' media - it reminds me of the corporations who are inserting long, detailed product information and extended stories in a momentary burst at the tailend of TV ads, viewable only by 'stepping' through them frame-by-frame using a Tivo/DVR remote, as a means to (re)capture eyeballs in the age of 'ad skipping'. For an example, view GE's One Second Theater.)

I recall reading somewhere (can't remember now) that UI beauty is born from spare MIPs - that it was only when processing/computing speeds became sufficiently fast that any substantial attention was directed towards the look & feel of applications. Thus, we had text-based OS's and app's (think DOS as a later example) well before any GUI's were an option. It seems we are experiencing another step along this path, as the power of even average PCs today far outstrip the stress the average user places on them - they (effectively) sit idle, waiting for the next command from the user 95% of the time (I made that figure up, but it's probably an underestimation, if anything). Thus, you see 'grid computing' efforts popping up to take advantage of these spare cycles (one of the older examples is the http://setiathome.berkeley.edu/ initiative). The power of 'gamer' video cards and video gaming systems (XBox, PSP, etc.) is extraordinary by measures set only a few years ago. This fact enables people like Blaise to begin to imagine new capabilities (thanks to Moore's Law... although I think that he came up with some very innovative ideas about how graphics are handled that amplified the pure computing power curve!)


2) Direct Manipulation Interfaces










In video above (again, from TED), Jeff Han demonstrates an amazing and intuitive way of manipulating information directly, rather than the long-in-the-tooth mouse/keyboard method. Again, seeing the demo in video takes the place of pages of (inadequate) descriptive text.

This is very similar (perhaps related deeply?) to the newest Microsoft announcement, Surface.

Even if they aren't related efforts, it may be an example of simulateous convergence of an idea whose time has come (like Newton and Leibniz related to Calculus). This concept of direct manipulation, coupled with widely networked and 'aware' devices (see the three videos on the Surface home page) will open up an entirely new horizon of creating/storing/sharing digital
information.

3) New Paradigms of Information Organization

David Weinberger recently published a fascinating book, Everything is Miscellaneous. At its core, it's a book about classification/categorization and how things change when the items being cataloged are bits rather than atoms (digital rather than physical). It's about the power of metadata, tagging, and in/formal taxonomies.

This basic premise (that we are now afforded the option of describing a single thing in multiple ways, all of which are valid and useful, depending on your objectives) is the fuel behind what makes Blaise's work possible (in part). The ability to search, sort, and shuffle large storehouses of otherwise miscellaneous/random data to reveal new patterns and meaning rests in the 'information about the information'.

These three spikes in the blogosphere converged in my head today and (re)ignited my imagination of what awaits us (and our children) in the not-so-distant future.

How will these samples (along with the hundreds of other gadgets, gizmos, and tools that are popping up daily on the web) influence and impact the way we conceptualize and manage "information", "learning", and "knowledge" in the academic (K-12/16/20+) and corporate domains?

Only time will tell, but it seems clear that the possibilities are limited much more severely by our own imaginations and mental models than by the enabling technologies.

Friday, May 18, 2007

Evil Tools or Evil Uses?

(originally posted to Tata Interactive Systems' corporate blog)


I've noticed a slowly growing trend in the marketplace, and I think it's time it's questioned.

It has become quite in vogue to bad-mouth and shun the use of Powerpoint in presentations.

I was first exposed to this stance in a presentation by Edward Tufte (author of several fantastic books on information visualization and communication). He made a mark for himself by declaring that "Powerpoint is Evil", both in a WIRED article and in a longer whitepaper.

More recently, Elliot Masie dubbed his newly minted "Learning 200X" conferences to be "PPT-free Zones", instructing presenters to leave their slides at home in hopes that it would foster greater discussion and interaction amongst attendees (and this underlying objective may have been accomplished, as the sessions *did* move from being lectures to conversations).

Just last week I spent a terrific day in Washington DC talking about Stories and Conversations at an event hosted by the Smithsonian Associates. One of the great speakers that presented was Larry Prusak of IBM KM fame. I really enjoyed his talk, which was PPT-free, but instead of simply quietly adopting that presentation stance and moving ahead, he made a pointed announcement of his disdain of Powerpoint. He even went so far as to say that he uninstalled the app from the MS Office Suite on his computer.

Finally (and completely serendipitously), this month's 'Big Question' on the Learning Curcuits Blog is precisely on this topic - the What/When/Why of PPT. The BQ seems to have been prompted by an Austrailian press article that uses Dr. John Sweller's Cognitive Load Theory to explain why most PPTs are so head-droppingly awful (and the link makes terrific sense to me).

In all of these cases (and the dozens I haven't mentioned, some of which are here), I can't help but think that the primary flag-wavers of the Anti-Powerpoint movement are projecting their disgust and anger in the wrong direction. Powerpoint is simply a tool, like a hammer, or a lawnmower. In and of itself, it's not really something that can be deemed "evil". It is the way the tool is leveraged that gives it "value" (good or bad).

Don't get me wrong - I've been the victim (and... umm... the perpetrator) of more than a few AWFUL powerpoint presentations. You know the ones - where the audience is given a live (often monotone) narration of (text-laden) slides that are linearly displayed. I am in no way defending this embarrassment of communication - I'm simply pointing out that the fault sits with the *presenter*, not the tool (despite the fact that the tool may make it easy for the lazy to use it in bone-headed ways).

Just because a hammer can be used to both frame a house for the victims of a natural disaster and murder someone, that doesn't make it inherently "evil" or "good". Those value-laden terms can only be reasonably used to describe the users of the tool based on what they chose to do with it. When Larry said he had removed Powerpoint from his computer, for its nature of promoting one-way communication, I wanted to ask him if he had also removed Word? According to the logic of his argument for uninstalling Powerpoint, any word processor should also be shunned - after all, documents are monologues, not dialogs, right?

I'm reminded of a email exchange I recently had with Geetha Krishnan, a colleague at TIS. In our exchange, I was reminded of an old argument Socrates made against the adoption of books:

Socrates: ….If men learn this, it will implant forgetfulness in their souls; they will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks. What you have discovered is a recipe not for memory, but for reminder. And it is not true wisdom that you offer your disciples, but only its semblance, for by telling them of many things without teaching them you will make them seem to know much, while for the most part they know nothing, and as men filled, not with wisdom, but with the conceit of wisdom, they will be a burden to their fellows.

Phaedrus: . . . I agree that the man of Thebes is right in what he said about writing.

Socrates: Then anyone who leaves behind him a written manual, and likewise anyone who takes it over from him, on the supposition that such writing will provide something reliable and permanent, must be exceedingly simple-minded; he must really be ignorant of Ammon's utterance, if he imagines that written words can do anything more than remind one who knows that which the writing is concerned with.

Phaedrus: Very true.

Socrates: You know, Phaedrus, that's the strange thing about writing, which makes it truly analogous to painting. The painter's products stand before us as though they were alive, but if you question them, they maintain a most majestic silence. It is the same with written words; they seem to talk to you as though they were intelligent, but if you ask them anything about what they say, from a desire to be instructed, they go on telling you just the same thing forever. And once a thing is put in writing, the composition, whatever it may be, drifts all over the place, getting into the hands not only of those who understand it, but equally of those who have no business with it; it doesn't know how to address the right people, and not address the wrong. And when it is ill-treated and unfairly abused it always needs its parent to come to its help, being unable to defend or help itself.

Phaedrus: Once again you are perfectly right.
(from The Collected Dialogues of Plato. Eds. Edith Hamilton and H. Cairns. Princeton: Princeton UP, 1963. pp. 520-21. 274D-276B.)

A presentation can be given effectively or ineffectively using the same tool (or without a tool at all). It's time to focus on the actual source of the problem and stop making simple-minded arguments against red herring causes.

Those who create and deliver Powerpoint presentations with packed paragraphs of 10-point text, meaningless clip-art, and irritating spinning and flashing animations should be sent to a class on effective presentations. Simply taking the (abused) tools away leaves these culprits none-the-wiser to the error of their ways, and discounts the communicative power that Powerpoint can render when placed in the right hands.

Just as Socrates threw the baby out with the bathwater regarding the power and benefit of the written word, I think those who argue that Powerpoint should be nuked are making a similar error. This suggestion is analogous to saying that duiring your next presentation, you should require the audience to close their eyes as you speak - to turn off one of the primary means of informational reception.

Visuals play a powerful role in communications, so why would anyone make a broad-brush suggestion that there's benefit in prohibiting them? Can you imagine speaking about Art without visuals? How about good User Interface Design? How about long-horizon trends in any one of a variety of domains? Images often instantly 'say' more than is able to be easily enunciated in words or text.

"A picture is worth a thousand words" isn't just a meaningless phrase, you know...

Thursday, April 26, 2007

Learners Are "Bulletproof"

(originally posted to Tata Interactive Systems' blog)


The title of this entry was inspired by a passage from Craig Wortmann’s terrific book, “What’s Your Story?”. It speaks to both how communication has shifted in recent years and the ineffective nature of the way we often attempt to teach.

Just as lengthy memos were replaced by brief emails, which were, in turn, overtaken by IMing and SMS (thanks to increasingly powerful mobile phones/devices), the way we converse and communicate is becoming more and more staccato’d . Increasingly, we are speaking to each other in fragmented facts and bullets instead of descriptive and nuanced narratives. In pursuit of speed, we have traded away “rich/engaging/compelling” for “fast/efficient/familiar”. In many ways, this is a fool’s bargain, and it’s beginning to show.

Q: What is the longest standing, most tried-and-true, and instinctually natural way of passing knowledge from expert to novice?

A: Leave your laptops, Powerpoints, and Blackberries behind… Forget about books, manuals, and job aids… It’s the ancient art of storytelling.

I’m planning to post a series of entries about the power of Story in Instructional Design in the coming weeks/months, but I figured this observation regarding bullet points was a good place to begin the journey. It speaks to the importance of context and the way we process, store, and recall information.

When you tell someone a set of facts and figures, it can make a temporary impact, but it’s usually quickly forgotten as decontextualized white noise. What’s lacking is meaning and relevance to the listener – in order for something to be understood at a deep level and retained for more than a few days (hours?... minutes!?...), there has to be a personal connection made. Something that relates the new information to old, personal, previously understood information. Something that refines, extends, contradicts, augments, or otherwise changes the existing set of cases and rules that exist in the learner’s mind. Without this comparative review and adjustment, any ‘learning’ that may occur will ultimately be fleeting.

So, we need to move away from the expedient habit of disseminating data in meaningless bullets and rediscover the importance of context, which can be created in the form of stories.

Here's a short and simple example to illustrate the difference.

Consider the following bulleted fact:

  • “Corporate earnings were 3.2 billion (dollars/yen/euros/pounds/rupees) in 2006.”
What does this data point tell you? What image do you have in your head about this company’s standing? What do you understand now that you didn’t understand before? What meaning has this information given you?

OK – now review the following 12 charts (with the red “dot” marking 3.2 billion):

Instantly, and without conscious effort, you probably told yourself a “mini story” to process and comprehend the data represented in each chart. You couldn’t help yourself. Is 3.2 billion a good thing? A bad thing? Status quo? For each chart above, you easily created a plausible tale of “what’s going on” that is grounded in your previous experiences (firsthand and vicarious). This instinctual tendency is part of who we are, at a fundamental level. It provides a compelling insight into how our brains work, and suggests ideas for leveraging this cognitive habit to our instructional advantage.

Imagine the final step in this example chain - instead of being given a bulleted fact, or even a graphical chart, you are provided a compelling narrative of the events that influenced a company's fiscal performance. Maybe a story of how a small oil and gas company played a role in one of the largest bankruptcies in US history? How recognizible and well-understood are terms like "securities fraud" in the post Enron/Worldcom era? Why? Because stories were told - stories of greed, arrogance, fraud, trust, loss, and ruin - that brought obscure accounting terms and practices out of the textbooks and into the personally relevant world of everyday people.

Stories help to add meaning to new data. Meaning is a critical element in increasing retention. Without retention, there's no hope for application.

Thursday, April 19, 2007

The Power of Numeracy and Estimation

I'm a big fan of the ability to estimate and to have a "feel" for numbers. In this day and age of data and info overload, I think it's important that people (specifically, young people) are able to comprehend and process figures that are thrown at them by the modern media. Without it, we become vulnerable to either becoming alarmists at minor things, or complacent when major changes unfold.

For me, a hugely influential book in this area was "Innumeracy" by John Allen Paulos.

It's not uncommon at all to hear people say things like, "A million... a billion... a trillion... it's all the same to me!" or (with a odd sense of pride), "I'm not a numbers person / I've never been good in math / I don't 'do' numbers". We've all heard (uttered?) similar phrases on a regular basis, and don't bat an eyelash.

Yet, imagine the absurdity of hearing someone loudly and proudly proclaim, "I never 'got' that Alphabet thing - all those vowels, consonants, and letters are just too confusing for me!", or "Reading just never made sense to me... leave that stuff to the professors who are in love with their books."

Why is it that illiteracy is universally seen as a serious issue that must be combated (and I think it should), and yet very little is said about innumeracy?

We need to take an active role (especially with our children) to curb this "proud ignorance" and help to establish numeracy as an important foundational element of being a citizen of the world in the 21st century. If you wouldn't dream of skipping more than a night or two of reading your child a story, why not also make it a priority to regularly ask them to engage with numbers (figure out which of two items is a better buy at the store, estimate how many hours/minutes/seconds it is until their next birthday, or figure out how many cents per hour they'll need to save to be able to afford that toy they want)?

It's not that anyone really cares about "how many ping pong balls would it take to fill this room?" or "how many ants are in our front yard?" (I'm the first in line to rail against teaching/learning meaningless facts and figures, but that's a topic for another day's blog :-) ); it's certainly not about getting the "right" answer, either. It's about becoming comfortable enough with numbers that you can make sense of what's "big" and "little" in our daily lives (whether that's the cost of ignoring global warming, money lost in government waste, or just how rich Bill Gates really is.)

One of my favorite efforts in this area was "the power of adding another zero" that was illustrated by Charles and Ray Eames in their "Powers of Ten" video (and Philip and Phyllis Morrison in their book of the same name).

This whole posting was inspired by a chance browsing discovery of a "Powers of Ten '07 (in Flash)" site done by Nikon, called Universcale.

It's creative efforts like PoT and Universcale that help to make numbers, size, and the power of exponential growth (up and down) more relevant and tangible to people.

A few quotes to close:
The reason that children learn a language in a year or two is because it is an environment. There is no reason why physics and mathematics cannot be given the same environmental codification and learned with the same speed and ease.
--Marshall McLuhan

There are three kinds of people in the world - those that understand mathematics and those that don't.
--Ron Zellner

Tuesday, April 17, 2007

OLPC and the Future of Learning




The above is a video of TechTalk given by one of the brains behind the One Laptop Per Child project at Google. Although I'm far enough into the half-life of my university learnings to have a great deal of the geektalk in the presentation go over my head, I can appreciate/understand just enough to be thoroughly wowed. I also *really* liked the context that the speaker took the time to establish in the first 5 mins regarding how the root of the project is about Learning, not Technology.

The One Laptop Per Child (OLPC) initiative has been lauded and ridiculed by various groups since its inception, but it seems to be plugging along towards becoming a reality (thanks to the help of some very bright and innovative people).



All I can say is "bravo!". It's truly inspiring to see an idea as big and (potentially) groundbreaking as this grow from concept to an implemented product before our very eyes in a very short (relatively speaking) period of time. I struggle to imagine an effort that has more potential for fundamental change and empowerment than what these guys have bitten off.



I remember being deeply inspired, many moons ago, the following Apple print ad:





It's embarrassing how quickly time gets away from you and how everyday life pushes youthful aspirations to the side. Does it need to be that way? I'm not THAT old (yet), so why not embrace the old "pick battles big enough to matter but small enough to win" adage and try to make a mark (beyond "I saved company's money")?

Last weekend I attended my 4th FIRST Championship event. It was amazing as usual.

Yesterday, I emailed the guy heading up the FIRST Lego League in the Atlanta area. I'm going to do my best to "be the change I want to see in the world" (Gandhi).

Wish me luck.

Friday, April 13, 2007

Innovation, Failure, and Learning

(originally posted to Tata Interactive Systems' blog)

CNBC has started a new 5-part series on Innovation called, "The Business of Innovation".
The first episode ("Innovators and Iconoclasts") has already aired, but is available online here. It's really quite good, with interviews with many well respected figures from the business and innovation arena. Well worth the time to watch the 4 segments...

So what does this have to do with learning?

Although the series is hosted by Maria Bartiromo, she is assisted by Roger Schank, well-known educational provoker and Artifical Intelligence/Cognitive Science expert. From my perspective, he steals the show (full disclosure: I studied under Schank at Northwestern's Institute for the Learning Sciences).

Consider these snippets from the episode's opening:
  • Innovators don't really fit in very well (with others)
  • Nearly everyone starts out as an Iconoclast (a rule breaker) - little kids don't know the rules; they do what they want to do.
  • Failure is KEY! You have to be failing and failing again.
  • Big companies are always trying to hire people who were in the top 10% at Harvard, under the assumption that they are good. What Harvard graduates are good at is... Harvard.
  • You want someone original from Harvard? Try the BOTTOM 10%. Those kids are smart enough to get it, then decided they weren't playin' the game.

BINGO!

The connection here (among others) is that we need to be willing to experiment and fail in order to break the mold of "what everyone else is doing" or "how it's always been done". If we aren't willing to reach out beyond what's known and assured, then we'll just get more of what we've got.

Failure has such an unfairly bad rep - companies (generally) hate it, employees (mostly) avoid it, recruiters rarely ask about it. Yet it's the home zip code of where learning occurs. If you do something and succeed, all that tells you is that what you already know works - it provides confirmation, but no growth/learning.

The primary time real learning occurs is when we do X, expecting Y, and Z occurs instead.

BOOM!

Discomfort and surprise result! Motivation to find out what the hell happened arises! A new case for your storehouse of experiences is added to your wetware! And a few new synapses are connected! - Learning has occurred!

It's exactly this kind of semi-contrarian stance that has made Roger who he is. Love him or hate him, it's tough to ignore him and the points he makes.

Do yourself a favor and do a quick Google on "Roger Schank" and read what he's written/said, as well as the responses he's generated (pro AND con). I guarantee you will find yourself shifting in your seat and thinking more deeply about some "common/accepted knowledge" than you probably have recently.

Here are a few to get you started:

Shift Happens

(originally posted to Tata Interactive System's blog)



Scott McLeod, of the Univ of Minnesota, recently reworrked and posted a version of a presentation that Karl Fisch created called "Did You Know?". It's really quite good on several different levels (globalization, learning, change, the future) and is worth a view (~6 mins).

Several of the themes resonated especially loudly for me, being an eLearning specialist working for an Indian firm, as well as being the (proud) father of a six year old boy. The world we are living within TODAY is amazing and changing incredibly fast, but it all (may) look like "the good old days" to my son in the same way that today's workforce reflects upon how things were, not in the 1930's, but in the 1830's (or maybe the 1730's!).

The current day statistics and facts he cites are pretty amazing (or scary, depending on your risk profile and comfort with change). And although it's always dangerous business predicting the future (as witnessed by Charles Duell's prediction, as Commissioner of US Patent Office in 1899, that "Everything that can be invented has been invented."), even if only a portion of the forecasts in the latter portion of the presentation are just partially true, we STILL will be in for a wild ride...

(It seems like some of those predictions may have been based on Ray Kurzweil's research on "The Singularity", and how exponential change is powerful but subtle before you hit the "knee of the curve", where change becomes noticable (which is where he claims we all sit currently). He says that we'll experience a century's worth of progress in the next 25 calendar years, and 20,000 years of progress(!) in the next 100 calendar years. Hold on tight...)

So what's the relation to Learning and Development? The scent of it abounds, even if it's not explicitly stated. We must prepare to live/survive/excel in a world that will be changing more rapidly than ever. And in a world of change, it is the person who has learned how to learn who will have the advantage. We need to stop focusing on teaching fact/figures/answers, and begin to embrace the ability to analyze/sythesize/adapt. Good teaching and learning strategies will be at the core of success tomorrow, regardless of the details of what that vista ends up looking like.

In a time of drastic change, it is the learners who inherit the future. The learned find themselves equipped to live in a world that no longer exists.-- Eric Hoffer

I've been cheating...

It looks like I've failed in my New Year's Resolution a mere few weeks into '07, but it's not so... I've just been seeing another blog :-)

I've been posting to my current employer's blog recently, but have decided that I really ought to cross-post here, as well.

So, with that in mind.... Keep on reading!

Monday, February 5, 2007

TED - I wanna GO!

I guess until I start the next Google or YouTube, or win the Lotto, TED is going to be out of my price range, but a boy can dream, can't he?

I'm just happy that they've opted to open a window to the rest of the world by sharing select sessions for free. I've watch nearly every one of the posts and it has rarely left me feeling anything less than amazed and energized. It's actually a great example of (one of) the possibilities the Web offers in terms of knowledge/idea sharing.

To get a better feel for what it's all about, you can watch this news story (by David Pogue, of the NYT, for CBS).

Monday, January 22, 2007

Speed vs. Quality - Still Relevant?

January’s “Big Question” at the Learning Circuits Blog is:

What are the trade offs between quality learning programs and rapid e-learning and how do you decide?

My knee-jerk reaction is that the question makes an unspoken assumption that quality and speed are diametrically opposed – that “rapid” should be equated with “low quality”. Although that certainly may be the conventional wisdom position, I’m not sure it holds water under closer inspection.

It’s similar to asking about the trade-offs between ‘fast’ food and a ‘good’ food. Upon initial inspection, there’s a shared tacit agreement that a drive-through dinner from your local burger franchise won’t (can’t) hold a candle to the quality that would come from a multi-course meal from your favorite fancy sit-down restaurant.

But if you think more deeply about it for a moment, you may discover that (just as with most things) the “trade-offs” are intimately tied to objectives – or more simply put, “it depends”.

If you are in the midst of running a marathon, “good” and “fast” eating are defined quite differently than if you are having an anniversary dinner with your spouse. It’s only when you fully understand the needs and expectations of parties involved in a situation are you able to properly assess what (if any) trade-offs must be made to effectively/properly serve their objectives.

In the case of (e)Learning, the argument stands, as well. It’s generally accepted (in theory, at least) that one solution rarely addresses the needs of more than a few – that what is ‘fast’ and ‘good’ to one person/group isn’t the same as for another. The "value" of a solution is highly dependent upon the needs of target.

Consider, for example, the case of Google and how their primary service has been so deeply adopted into the fabric of society that their company name has morphed into a generic verb meaning “to search”. One of the more dramatic results of the service Google provides is that information on any topic at (nearly) anytime is available to anyone, all with a microsecond response time (rapid). Granted, there’s no guarantees to the quality of the information Google supplies in response to a query, but (and here’s the interesting part) that lack of guarantee also doesn’t preclude quality – it actually says nothing about it at all!

So, we are witnessing an interesting swing away from the monolithic course model, to a JIT search-browse-synthesize-apply model. In this world, ‘rapid’ and ‘quality’ can (and often do) comfortably co-exist. There’s no reason that a similar, albeit more constrained, replica of this model cannot be adopted and leveraged within individual organizations, as well. Instead of leaving the determination of ‘quality’ up to the seeker to discern (as is largely the current case when broadly searching with Google, despite the confidence their ranking algorithms is supposed to proffer), a company could create and offer a multitude of high-quality but quickly accessible & consumable ‘learning bites” on the most frequent/timely/damaging/profitable topics of interest. By aiming for succinct responses to the most leveragable “20” of the 80/20 Pareto Principle of business/sales/performance themes, it would be possible to serve the double-masters of Speed and Quality without any contradiction.

It’s important to note that one element will take precedence over the other in many cases – when the validity and comprehensiveness of information trumps the amount of time necessary to locate and understand it, or when something that half-right today is infinitely more valuable than something that is completely right tomorrow. Again, it all comes down to understanding the underlying objectives of your audience and adjusting your ‘S vs. Q’ knob accordingly.

(My response, obviously, is taking a consumption-oriented interpretation on the BQ. If one was to read the BQ to relate to production, I may have a slightly different stance in answering.

But then again, maybe not… :-) )

Wednesday, January 3, 2007

What's Easy vs. What's Right

I've been a member of the "eLearning" community for over 15 years now. Throughout that time, I've consistently been frustrated with the elephant in the room that no one wants to recognize:

More often than not, T&D departments and the CXO's to whom they report are more interested in going through the motions related to training than with the actual impact and effectiveness of those efforts.

What I mean by this is that there is a disturbingly large amount of money, time, and energy devoted towards getting course completion "checkmarks" for the entire employee population, regardless of whether that tornado of effort and funding actually makes any difference.

For example, why are Multiple-Choice Questions (MCQs) so entrenched in corporate (and K-12) training? Is it because they are especially effective? No. It's because they are efficient. And efficiency and efficacy are rarely related.

MCQs are a terrifically efficient way to move a large number of people through an evaluation process in a quick and easily quantifiable fashion. The problem is that most MCQs are structured in such a ridiculous way that it's often relatively easy to discern the correct answer without any actual knowledge or background in the subject. (I once took an internal "certification exam" that a client used to qualify their employees, sight unseen and with no background in the subject, and scored a passing (78%, if I recall correctly) grade.) Even if the MCQs you create don't fall in this category (and it is possible to create decent MCQs, with a little effort and creativity), the extra effort may not be justified because life and work don't often consist of selecting from menus of options. Recognition doesn't equal Recall, and neither equal Performance. And Performance is (usually) what we are really seeking.

This old saw rant of mine was recently rekindled by a terrific article by Sarah Boehle on the disconnect between L1 eval (smiley sheet) results and the actual effectiveness of that training (L3/L4).

I won't attempt to reframe her arguments, as she makes a strong and pointed case, leveraging a few folks I respect (Roger Chevalier and Will Thalheimer), but I will pull out a couple of items that had me cheering:
"Why do so many companies make this mistake? In all likelihood, they do so because the steps they must take in order to get their hands on the objective information necessary to accurately assess training effectiveness can be complex, difficult, costly and time-intensive. "
and
"From my experience talking to corporate training departments, the calculus in the field seems to be, 'Don't do good evaluations,' " Thalheimer says. "Why would anyone want to? If you evaluate at all levels, only bad things can happen. If you use Level I and get good results, then you're maintaining the status quo and everyone assumes that training is doing a good job. If you evaluate more rigorously and get bad results, however, all hell breaks loose."
There seems to be some sort of unspoken conspiracy/agreement between T&D and upper management of "don't ask, don't tell". T&D keeps shoveling meaningless stats to the boardroom that help to justify their existence, and the boardroom keeps accepting them as a means to help them feel better about the "necessary evil" that they see T&D to be. It's an easier, cleaner, and more friendly way to go about business, but it's an enormous waste of resources all around.

T&D professionals have an obligation to "get dirty" and dig into some of the hard questions and actions associated with making training *matter*. And that involves taking the road less traveled and providing some solid business rationale for why the annual budget has a line item for T&D, whether the CxO is asking for it or not. Until that happens, T&D will never be considered anything more than a "necessary evil" in the boardroom.