I’ve been noodling on the architecture of an open learning network for some time now. I’m making a presentation to my boss today on the subject and I think I have something worth sharing. (Nothing like a high-profile presentation to force some clarity of thought.)

I wrote a post last year exploring the spider-starfish tension between Personal Learning Environments and institutionally run CMSs. This is a fundamental challenge that institutions of higher learning need to resolve. On the one hand, we should promote open, flexible, learner-centric activities and tools that support them. On the other hand, legal, ethical and business constraints prevent us from opening up student information systems, online assessment tools, and online gradebooks. These tools have to be secure and, at least from a data management and integration perspective, proprietary.

So what would an open learning network look like if facilitated and orchestrated by an institution? Is it possible to create a hybrid spider-starfish learning environment for faculty and students?

The diagram below is my effort to conceptualize an “open (institutional) learning network.”

There are components of an open learning network that can and should live in the cloud:

• Personal publishing tools (blogs, personal websites, wikis)
• Social networking apps
• Open content
• Student generated content

Some tools might straddle the boundary between the institution and the cloud, e.g. portfolios, collaboration tools and websites with course & learning activity content.

Other tools and data belong squarely within the university network:

• Student Information Systems
• Secure assessment tools (e.g., online quiz & test applications)
• Institutional gradebook (for secure communication about scores, grades & feedback)
• Licensed and or proprietary institutional content

An additional piece I’ve added to the framework within the university network is a “student identity repository.” Virtually every institution has a database of students with contact information, class standing, major, grades, etc. To facilitate the relationships between students and teachers, students and students, and students and content, universities need to provide students the ability to input additional information about themselves into the institutional repository, such as:

• URLs & RSS feeds for anything and everything the student wants to share with the learning community
• Social networking usernames (probably on an opt-in basis)
• Portfolio URLs (particularly to simplify program assessment activities)
• Assignment & artifact links (provided and used most frequently via the gradebook interface)

Integrating these technologies assumes:

• Web services compatibility to exchange data between systems and easily redisplay content as is or mashed-up via alternate interfaces
• RSS everywhere to aggregate content in a variety of places

As noted in previous posts, we’re in the process of building a stand-alone gradebook app that is consistent with this framework. We’re in the process of deciding which tools come next and whether we build them or leverage cloud apps. After a related and thought-provoking conversation with Andy Gibbons today, I’m also contemplating the “learning conversation” layer of the OLN and how it should be achitected, orchestrated and presented to teachers and learners . . .

While there’s still a lot of work to do, this feels like we’re getting closer to something real and doable. Thoughts?

Lev Gonick, a friend and the CIO of Case Western, is the author of a piece in that appeared today in the Chronicle of Higher Ed today about IT leadership at colleges and universities. It’s a thoughtful, provocative piece which, coupled with a previous piece about leadership in the “wiki-way,” provides an excellent set of principles and directions for university IT leaders.

IT leaders must play an increasingly strategic–and not simply a tactical–role at colleges and universities. Accordingly, I agree with Lev’s assertion that IT leaders deserve a central, strategic role in presidential cabinets at colleges and universities. However, you have to have the right kinds of people (people like Lev) in these positions if that is what you expect of them. The day is past that we can consider IT merely a “support function” of the university. If we think of it as simply auxiliary, we will miss significant opportunities to transform (for the better!) our practices through strategic (not simply tactical) IT initiatives.

I would however, make one addition to Lev’s list of strategic directions for IT, and I’d put it at the top of the list. CIO’s should take an active role in working with the academic community to create a more a flexible, open, integrated toolset to support authentic teaching & learning activities. Our current tools help us manage courses and grades, but we can and must do much more than that to meet the challenges of educating the rising generation.

It is not enough to provide faculty & students with tools to manage the activities that occur inside semester-long courses. That might have been sufficient 5-10 years ago, but it is not today. Today, we need tools that allow students to build relationships with each other, with their teachers and with the content they access. Just as importantly, we need to support students’ creation of new content in the learning process and the discourse around that content. And we need to proactively build bridges between the tools we build, license and provide and the larger, often more dynamic online world in which our students live. CIO’s, IT personnel, and academic technologists must be critical players in the conception, creation, and implementation of tools that support such activities. Otherwise, we’re likely to see repeats past failed technology implementations that were tactically sound but that missed the mark because they were not strategically aligned with the mission of institution.

In a recent Science Daily article, Patricia Greenfield’s research on the impact of technology on learning was summarized under the banner “Is Technology Producing A Decline In Critical Thinking And Analysis?” The piece depicts Greenfield’s research as casting technology in a decidedly negative light when it comes to facilitating the development of critical thinking and analysis skills. The original research report was published in Science (2 January 2009:Vol. 323. no. 5910, pp. 69 – 71), which you may or may not be able to access (depending on rights provided by an educational instituiton or libary with which you are affiliated).

There was an energetic discussion (provoked by Clay Burell’s blog post) about both the Science Daily article and the original research over at http://education.change.org. I won’t rehash in detail here whether or not the Science Daily write-up was a faithful interpretation of the underlying research or not. (For the record, I agree that Clay would have been on firmer ground had he read the original Science piece before writing his critique. But I’m also sympathetic to some of the comments about access to Science being limited.) In any case, it struck me that there are still a lot of folks out there who want to make technology the bogeyman. The Science Daily headline clearly implied that technology was the reason behind declining critical thinking skills. When laptops don’t work in the classroom, it must be that the technology wasn’t appropriate for the classroom. When wireless network initiatives result in distracted students during lectures, we blame the wireless technology. When kids don’t read for pleasure, we blame technology too. If they just weren’t so distracted by video games, cell phones, mp3 players, and social networking sites, maybe they’d read more and think deeper thoughts. Again, we blame the technology, and not the environment in which kids are educated and nurtured.

If you’re even an occasional reader of my ramblings here, you’re probably anticipating the soapbox I’m about to climb up on. Wait for it . . . Here it comes . . .

Technology can’t do anything by itself!

Neither can money! Or cars for that matter! These are all things that humans use for good or for ill. Technology doesn’t “produce” anything! It is the ways we use technology or the ways it is implemented that produce particular kinds of results.  As I said in my response to Clay’s post, you get what you design. If people design boring, process-driven, mindless “learning” experiences for students (with or without technology!), they shouldn’t be surprised that students hate it (or quickly tune out when there’s something more interesting to do, like browsing the web).

This is all simply a design issue and virtually every technology at our disposal is a design tool. If we want critical thinking we should design learning activities that promote and assessments that gauge critical thinking, using the appropriate mix of technologies. Technology can’t do anything by itself, but it seems to be an increasingly convenient bogeyman for people who don’t want to do the hard work of improving learning design and reforming education so learning–and not teaching–are the center of it all.

Larry Seawright and I made our presentation this morning at Educause 2008. Our slides are available here.

Together with Stephanie Allen and Whitney Ransom McGowan, Larry and I have been working on an alternative approach to evaluating the effectiveness of teaching & learning technology. Traditionally, evaluation takes the form of comparative-media studies in which one group of students learns via standard methods (control) and others learn with new, experimental methods (test). Over and over (and over) again, these kinds of studies have found differences that are not statistically significant.

The so-called “NSD” (no significant difference) problem is the bane of teaching & learning evaluators the around the world. A growing group of influential scholars has rejected the comparative-media studies approach in favor of design-based research. Borrowing elements of this approach, we have implemented a goal-driven model of instructional design, technology integration, and evaluation at BYU.

Our approach to evaluating the impact of teaching & learning technology (and getting beyond the NSD problem) begins with the end in mind. The first and essential step in this approach is to begin any teaching & learning with technology project with a carefully articulated goal. Without such a goal, there is no clear, shared understanding of what “success” looks like. Hence, evaluation is virtually impossible–if you don’t know what success looks like, i.e. what should be better as the result of a project, what should you evaluate?

Measuring the impact of teaching & learning technology depends on a clear articulation of learning goals, strategies for accomplishing those goals and tactics for implementing those strategies. The goals can then be re-formulated as teaching & learning “problems” and strategies and tactics become “solutions.” Evaluation is then simply the process of measuring the results implemented solutions, as illustrated below:

To facilitate the consistent articulation of teaching & learning goals, we’ve adopted the Sloan-C’s Five Pillars: (1) Student Learning Outcomes, (2) Cost Effectiveness (Scalability), (3) Access, (4) Student Satisfaction, and (5) Faculty Satisfaction. By choosing to explicitly focus on one or more of these goals in every teaching and learning project, we identify what success should look like and, at the same time, establish an evaluation plan for each project.

As the examples in our slides suggest, there are often serendipitous results of teaching & learning technology implementation efforts. For example, a project aimed at improving access might also improve student learning outcomes and student satisfaction. However, by articulating and staying focused on a clear, shared rationale (and funding justification) for projects, we have been able to consistently measure and demonstrate the impact of our teaching & learning technology projects and get beyond the NSD problem.

It all begins by starting with the end in mind.

I’ve been thinking “big thoughts” lately, a problem brought on by several recent conversations with David Wiley. I realize I’m repeating something I’ve written before, but the idea is so core to the way I see things that I think it bears repeating–the purpose of institutions of higher education (and all of their associated functions and personnel) is student learning. Learners and the knowledge and skills they acquire are the raison d’etre of colleges & universities. Sure there are folks who might argue that university-based research is just as important, but the number of institutions that could send their students home and still make a case for their continued existence is very small.   So why does this matter to an academic technologist? Because at the end of the day, my purpose is to ensure that our investments in technology promote better, more effective and even more efficient learning. While institutional and instructor efficiency and convenience are laudable goals, however, I’m increasingly of the mind that these goals, by themselves, are not very good justifications for technology expenditures. Unless these efficiencies and conveniences have a direct impact on student learning effectiveness and efficiency, I think we’re missing the mark.  For example, if we relieve some of the administrivia for an instructor in an introductory course, we should ask ourselves what the instructor is doing with the saved time. If he or she simply has more discretionary time, that’s a nice thing, but not necessarily worth significant institutional investment. If, on the other hand, that extra time is dedicated to more one-on-one time mentoring and coaching students, working on mentored research projects with students, or teaching smaller sections of upper-division courses, methinks that is a more justifiable use of institutional teaching & learning improvement resources. Some not-so-random observations that have been bouncing around my noggin related to learners, goals & technologies:

1. We should focus on technologies that support LEARNING activities more than we do on technologies that support TEACHING activities.
2. If we can’t readily explain how we expect a particular technology to improve learning, we should rethink what we’re doing.
3. If a technology yields significant institutional or instructor efficiency, we should ask how learners will benefit from that efficiency, i.e. how will learning be improved?

Perhaps these observations are obvious to most readers, but, again, I believe their important enough that they need to be repeated, again and again, so we don’t forget them. Maybe a teaching & learning with technology mantra is in order: “Teaching & learning technology should always improve learning. Teaching & learning technology should always improve learning. Teaching & learning technology . . .” You get the idea. So, how can we tell if technology has actually improved learning? That’s the subject of a presentation I’m making this Friday at Educause 2008. I’ll post about in a couple of days. 

The text “answers” service ChaCha invites user to “simply ask your question like you are talking to a smart friend and ChaCha’s advanced technology instantly routes it to the most knowledgeable person on that topic in our Guide community. Your answer is then returned to your phone as a text message within a few minutes.”

I tried a few questions to see how it works. Here are the results:

Q: What is a confidence interval?
A: A group of continuous or discrete adjacent values is used to estimate a statistical parameter (as a mean or variance)

Q: When was the French Revolution?
A: A major transformation of the society and political system of France, the French Revolution lasted from 1789 to 1799.

Q: How does photosynthesis work?
A: Photosynthesis is the process by which green plants convert carbon dioxide to sugar and oxygen using sunlight for energy.

Q: Who wrote the Book of Mormon? (Thought I’d try a difficult one.)
A: No one can say with certainty who wrote it, or which author wrote which portions. There were too many contributors.

The answers are surprisingly cogent. Where do they come from? According to the site, live “ChaCha Guides” are behind the answers:

ChaCha’s Guides are individuals who are part of a vibrant community dedicated to helping people by sharing their knowledge. To become a ChaCha Guide, you must pass a series of tests that verify that you are a good fit with our Guide community. You are then able to go through ChaCha’s Search University and simulation process to become certified as a live ChaCha Guide. This unique approach aims to ensure that only knowledgeable people who have an interest in sharing their knowledge with others are part of ChaCha’s Guide community. ChaCha’s technology is also learning from each answer that is provided by our guides so that we can deliver accurate answers as quickly as possible

It’s unclear to me if the Guides are volunteers or if they are paid. In any case, this service raises interesting questions about the nature of knowledge. When I first heard of ChaCha and did some investigation, I was primarily concerned that students might use such a service to cheat on exams, quizzes and even homework. And I remain concerned that a student might surreptitiously use a cell phone in his or her pocket to “look up” answers on a test.

But this got me thinking about the nature of knowledge and the importance of recall. If a student can (almost) instantaneously get answers to factual questions, how important is it for us to require them to memorize facts? ChaCha will certainly not be the last or most sophisticated tool that provides just-in-time answers to knowledge questions.  As educators and learning technologists, our challenge is to figure out how to make assessment more meaningful and authentic in a world in which rapidly accessing facts is a trivial matter. When anyone can access any bit of knowledge anywhere, anytime, the real premium will increasingly be knowing what to do with that knowledge. Memorization will increasingly give way to analysis, synthesis and the creation of knew knowledge.

Plus cha-cha change . . .

In my response to Michael Chasen’s response to my post about Blackboard and the innovator’s dilemma, I made the observation that Blackboard’s (and every other CMS vendor’s) problem is that their customers are institutions, not learners.

I’ve been thinking a lot about this notion the past few days. If CMSs are developed and sold (or at least promoted) to institutions, it follows that their functionality and features will be skewed in the direction of meeting institutional needs, including faculty / instructor needs (e.g. easy course content publication, efficient quiz administration, secure grade posting).

Many of the efficiencies that yield benefits to institutions and instructors also provide value to students. But these benefits are mostly serendipitous. Setting aside economic considerations, I’ve been wondering what a CMS (or some other kind of learning software) would look like if it were developed as if the learner was the customer instead of the institution.

Here are some features I think software would include:

1. Learners would “own” their own learning space. They would not be dependent on their institution or some other entity to grant (and continue granting) them access to their learning content. Learner content collections would be managed and maintained by learners according to their changing learning needs. 
2. Access to content and relationships with other learners would persist over time. Access would not be tied to artificial institutional teaching calendars.
3. There would be much more robust tools around note taking, organizing content for study, research and collaboration.
4.  Non-course, non-credit learning tools that support informal learning (e.g. acquiring new technical skills, foreign language acquisition / retention, hobby related learning, etc.).

What do you think? What other features might more learner-centered software applications have? Is it possible to provide these same features via a traditional CMS? Why or why not?

In his opening keynote at Blackboard World yesterday, Steve Wozniak reflected on his experiences growing up at the dawn of the modern computer era. Along the way, he made several references to the intrinsic motivations that drove him to do the things he did. Some examples:

• When he bumped into a new technology, he was instantly driven to figure out how it worked. Not only that, he instantly began working on a way to replicate the technology “using fewer parts”. The intrinsic desire (need?) to know how things work drove him to creatively figure things out, often to the extent of reverse-engineering them.
• He would often go to school, or work, or even sneak into a lab after hours and work all night to solve a challenging problem.
• His first job was with HP. He intended to stay there forever because he wanted to be an engineer. He only left (to start Apple with Steve Jobs) because he was convinced he could remain and engineer and not become a manager.
• After leaving Apple, he became a volunteer technology teacher to help students have the same kinds of experiences he did when he was younger.

His talk was a stream-of-consciousness flow of thoughts, ideas, and recollections. As such, it provided perhaps a more intimate window into his psyche than a carefully constructed speech might have. While emphasizing these sorts of intrinsic motives, he also dropped occasional references to the praise, support, and encouragement he received from his teachers, parents, peers and bosses. In each case, he noted that the praise motivated him to be even more creative the next time he worked on something.

The lesson? Learners learn and create because they want to. If they don’t want to, our efforts to make them learn will largely be in vain. Consequently, teachers and learning technologists need to (as Roger Schank has argued) create the conditions for successful learning by tapping into intrinsic motives, allowing them to essentially take over and drive student learning. That’s when great things happen.

I purposely put off writing an entry about Roger Schank’s visit to BYU (which spanned Monday and Tuesday of this week) because I wanted to mull things over for a couple of days. Sometimes big ideas (and Schank offered up several) take some time to digest. (Kimberly McCollum, one of my former graduate students, has provided an excellent summary of his talks on her blog.)

Well, I’ve thought about my conversations with Schank and the lectures I heard him give and I’ve come to a few conclusions:

1. I spent the better part of two days with Roger during his visit, sharing meals, listening to his prepared lectures and sitting in on informal conversations with faculty and administrators. Today what stands out most to me from all of this is Roger’s unabashed commitment to learning. He really cares about it. He wants people–everyone, everywhere–to learn. I know that sounds trite, but when you meet a larger-than-life scholar (Roger’s the only person I’ve met who has written 25 books) you aren’t quite sure what to think of them. In my estimation, Schank is motivated by a deep-in-the-bones belief and desire that all people should have more and better opportunities to learn than they do today. Not only is he a revolutionary, but he’s a true believer.

2. The key to Schank’s argument is that learning begins and ends with motivation. While every teacher would love to teach classrooms full of intentional learners, the vast majority of students are not motivated to learn for learning’s sake. Schank posits that learning is fundamentally about wanting something. Unfortunately, the reward structure of the educational status quo requires students to please teachers to get favorable grades. Schank calls this arrangement the “tyranny of grades.” Schank estimates that high school and college students are motivated almost exclusively by grades in 90% of their courses, and not by some greater desire to do something.

Citing his recent experience teaching his grandson to crawl by putting a green squeaky frog just out of his reach, Schank reinforced the value of providing students with real-world learning situations. Deriding “required courses” (and plain “courses” for that matter), Schank argues for a story-based curriculum in which students are required to solve the kinds of practical problems they are likely to face in real life. His new science-based high school curriculum, for example, requires students to (among other things) file a report about a crime scene, create an exercise and nutrition plan for a client, and identify and eradicate a mysterious fungus destroying farmer’s crops. As students work toward these goals, they are provided with the facts, information, theories, formulas, etc. they need to succeed. Story-based, goal-driven curriculum doesn’t require artificial, external motivation for students. The motivation is intrinsic in the learning activities themselves.

3. While institutional inertia works against change in higher education (hence Schank’s efforts to reform high school education instead), change is possible and should be pursued by those who care about and work in higher education. Schank’s litany of problems with higher education is familiar:
   • University faculty members want to (and are rewarded to) pursue their specialized research agendas.
   • Students want to acquire meaningful skills that will help them succeed after college. Less altruistically, they want good grades that will help them get good jobs.
   • Since faculty members are rewarded for excellent research and merely adequate teaching, they have no incentive to significantly improve their teaching or student learning.
   • Faculty offer courses that are easy to teach and to grade, not necessarily courses that will substantially prepare students to be successful citizens, family members and employees.
   • Faculty and students collude with each other in a perverse system of rewards by agreeing not to tell on each other–”You don’t tell on me for not really teaching, I won’t tell anyone I’m giving you easy As and Bs.”
   • Universities produce students who frequently need to be “re-trained” by their employers.

So how can administrators, teachers and support staff hope to change such an entrenched system? For starters, we can personally contribute to a culture of learning by refusing to support the status quo. Change, as the aphorism goes, happens one step at a time. In higher ed, change must begin and continue one instructor, one classroom at a time.

We can also pursue systemic change by taking the accreditation focus on learning outcomes seriously. By focusing on what students should be able to DO when they complete degree programs, we can start changing the way we think about curriculum, learning activities and assessment. There are some shining examples of this sort of paradigm shift at BYU and at other institutions across the nation.

Finally, administrators need to begin changing incentive structures to reward faculty activities that are focused on student learning and not just their own research. One very promising avenue is to bring research and teaching together through mentored student research programs (such as the one at BYU).

4. Schank’s ideal model for teachers is the coach or mentor, providing help and motivation for students when they aren’t ready for it or maybe don’t even want it. Although unlooked for, such assistance will almost always be favorably received if the learning context is authentic and the students is, therefore, motivated to accomplish the goals it requires. The role of the teacher is to “put the squeaky green frog in the right place and then make reaching it increasingly difficult” as the student’s capabilities increase.

_____________________

Schank is the kind of thinker who challenges our assumptions and questions the status quo. While he can be (by his own admission) hyperbolic at times, extreme rhetoric is sometimes necessary to wake people up out of complacency. My hope is that his visit to BYU has accomplished just that by reminding us that our primary duty is to help students learn and that we ought to be exploring better ways to do that, even if doing so causes some personal or intellectual discomfort.

 When I think about technology, I always think about problems. Not problems with the technology itself, but about problems that technology can be used to solve. Far too often we technologists get enamored with a cool new technology (”It’s shiny!”) and we immediately begin looking for something to do with it. “Hey, this cool new hammer would work as a bottle opener if you hold it just right!”

But alas, a very long list of bad teaching & learning technology implementations can be cited to demonstrate why the “solution looking for a problem” approach is a bad idea. When I was an instructional designer at BYU’s Center for Instructional Design (now the Center for Teaching & Learning), I constantly fought this battle with faculty members who would come to the Center and announce that they needed a DVD, a website, an immersive 3D simulation, or some other such instructional technology creation. I would politely nod and we’d have a conversation that went something like this:

ME: Okay, so we can build the best (fill in the blank) possible, can you explain to me exactly what it is about your course that’s not going the way you want?

FACULTY: What do you mean?

ME: Well, once we’ve built (fill in the blank), what should be better about your course? Better yet, what should your students know or be able to do that they currently don’t?

FACULTY: Well, that’s a good question. What they really lack is . . .

From there, we’d spend some time talking about student learning and what the faculty member could do to better facilitate it. Then we’d explore how (fill in the blank) would help students learn better or faster. As often as not, we’d decide not to build (fill in the blank). Instead, we’d tweak some things in the course and build something else more appropriate to the problem the faculty member was trying to solve.

Based on experiences like this, I’ve developed a very simple approach to teaching and learning technology, academic technology and technology in general. Simply put, a technology is only as useful as the problems it solves. The trick for technologists, then, is to always begin with the end in mind. We have to get focused on the problem we’re trying to solve and stay focused on it. We have to constantly ask ourselves, “What am I trying to fix or improve? How will I be able to tell when I’ve succeeded?”

Once we know what we’re trying to improve, we have an objective. Let’s call it a “goal.” With our goal firmly in mind, we can then move to strategy formulation. I think of a strategy as a long-term plan of action focused on achieving a goal. From strategy we can then move to specific tactics or operational activities aimed at implementing the strategy.

Let me make this more concrete. Let’s say that the faculty and administrators on a campus are concerned that they’re using up too much classroom time on administrivia, e.g. collecting and returning papers, administering quizzes, etc. A GOAL aimed at addressing this problem would be to reduce time spent on administrivia during class time. One possible STRATEGY for accomplishing this GOAL would be to move most class-administrative activities to an online environment where they could be completed outside of regular class time. One possible TACTIC for implementing this STRATEGY would be to make a particular online course management system (CMS) available for faculty members and students.

The beauty of this approach is that it drives both goal-driven technology implementations (tactics) AND straightforward evaluations of those implementations. Was a technology implementation effective? That question can now be answered, first and foremost, by answering another question: Was the goal accomplished? If the problem is less severe or even non-existent after the strategy and tactics were implemented, you can declare success. If the goal wasn’t accomplished, at least you learned that the tactic (and perhaps the strategy) you picked didn’t work.

Admittedly, this is a simplistic approach to technology planning and evaluation. But it has served me well and I’ll continue to rely on it until something better comes along. The bottom line? Technology should be used to make the world a better place. If we’re not able to demonstrate exactly how and to what extent technology is improving things, all were left with is, “It’s shiny!”