Note: In this post, I reference articles from the Winter 2009 edition of Peer Review, the theme of which was “Assessing Learning Outcomes.” I highly recommend the entire issue if you’re interested in student learning assessment and portfolios. I recently provided an overview of BYU’s loosely-coupled gradebook strategy at TTIX 2009. (We are currently building a standalone gradebook in partnership with Orem, Utah based startup Agilix.) As part of my presentation at TTIX, I also described our plans to leverage the same technology we’re building into the gradebook to implement a loosely-coupled portfolio assessment tool.

The driving purpose behind these efforts is to bridge the gap between our institutional network and the cloud, between the predominant “course management system” (CMS) paradigm and the emerging model of personal learning networks (PLNs), between student-centered and institution-centered portfolios, etc. I maintain that bridging this gap is a necessary condition for the significant transformation of learning via technology. Until learning tools and content become more malleable (i.e. open, modular, and interoperable) we will not realize the full potential of an interconnected, networked world in education.

Student Learning Portfolios & Institutional Assessment

Portfolios are increasingly at the nexus of student learning, institutional assessment, PLNs, CMSs, and assorted other aspects of the higher ed milieu. Student learning portfolios are essential in the movement toward more valid and authentic assessment in higher education. However, the focus on institutional and program assessment has, at least in some instances, diverted our attention from our primary objective of improving student learning. As Trent Batson observed in 2007, that the initial effort to enhance student learning with portfolios has been “hijacked by the need for accountability” to boards of education and accrediting bodies.

This trend is worrisome. If the focus on portfolios shifts primarily to institutional and program assessment, we will have missed out on the essential value of portfolios. Portfolios have the potential to galvanize and enhance student learning. As Miller and Morgaine have observed: “E-portfolios provide a rich resource for both students and faculty to learn about achievement of important outcomes over time, make connections among disparate parts of the curriculum, gain insights leading to improvement, and develop identities as learners or as facilitators of learning.”

Given the potential benefits of portfolios, I believe that student learning portfolios should, first and foremost, belong to and be maintained by individual learners. Gary Brown supports this view, maintaining that portfolios should be student (and not institutionally) operated: “A real student-centered model would put the authority, or ownership, of [ePortfolios] in the hands of the students: They could share evidence of their learning for review with peers, and offer that evidence to instructors for grading and credentialing.” Such an approach increases student ownership and responsibility for learning. It also affords portability and longevity since students are not dependent upon a particular institution (or set of institutions) to provide them with portfolio technology and storage space.

Clark & Enyon have argued that have to get past this tension between student and institutional portfolios: “The e-portfolio movement must find ways to address [institutional assessment] needs without sacrificing its focus on student engagement, student ownership, and enriched student learning.”

Bridging the Gap

So exactly how can we bridge the gap between student-centered learning portfolios and institutional assessment needs? I propose that the loosely-coupled gradebook strategy we’re pursuing can be leveraged to provide a viable solution to this problem.

Here are the dimensions of a loosely-coupled portfolio assessment strategy:

1. Institutions of higher learning should focus on what they do best and on what only they can do. Namely, they should admit and register students, manage course enrollments and  degree program rosters, and maintain secure records and communications tools for faculty and students engaged in the learning process.
2. We can then leverage the best online, third-party applications for student publishing, networking, and portfolio creation. Individual institutions (or even institutions working together) would be hard pressed to produce applications comparable in quality and stability to Google Docs, YouTube, Blogger, Acrobat Online, MS Office Live, Wikispaces, and WordPress.
3. Teachers and learners should embrace the power of the network to enhance, extend and improve learning. Even if institutions could develop and deploy better tools than those freely available online, it would be a bad idea to do so. The fundamental value proposition of these apps is that, since they live in the cloud, they’re accessible anytime, anywhere, by anyone. The openness this affords promotes greater transparency and expanded opportunities for collaboration.
4. Students should be encouraged to be effective, technologically literate, digital citizens who are proficient using a variety of online tools. As they participate in the learning process, they should regularly save and aggregate their work, packaging and repackaging it for various audiences. One of these audiences might be those responsible for degree program review and assessment.
5. Once students have assembled their collections of learning artifacts, metacognitive commentary, and portfolios, they might then be required to simply “register” the URLs of their portfolios and artifacts with their institution. Those conducting program and institutional assessment would then use a lightweight “overlay” tool to review and assess submitted student work.

The various aspects of this approach and the associated work flow might look something like this: Portfolio Assessment Diagram

The Power of a Loosely-Coupled Strategy

The “open learning network” (OLN) strategy I’ve written about from time to time is based on several value propositions. One is that institutions should do what they do best (manage student data, facilitate secure communication between teachers and learners) while leveraging third-party, cloud-based applications for such things as personal publishing and collaboration. The loosely-coupled gradebook strategy described in previous posts is a key component of this larger idea.

Another central value proposition of the OLN is the connectedness it facilitates between teachers and learners both within and without institutional boundaries. When learners not only consume online content, but also refine, improve, remix, mashup, and create new content themselves during the learning process, their learning is more authentic, meaningful, and enduring. And they build deeper, more profound connections between facts, concepts, and the other human beings they interact with. This notion of of connectedness is a fundamental aspect of what we consider education and literacy today.

George Siemens blogged today that:

“Not only are we socially connected in our learning, but the concepts that form our understanding of a subject also reveal network attributes. Understanding is a certain constellation (pattern) of connections between concepts. . . . being a literate person is not so much about what you know, but about how you know things are connected.”

I concur. As we continue to pursue the OLN vision, it is essential that we facilitate opennes in the learning process to promote greater interaction and connections with content, people, cultures, and places. It is with this end in mind that we ought to promote personal learning networks (PLNs), OLNs, and loosely-coupled gradebooks. We want our students to be literate, connected, and efficacious life-long learners who make their homes, their communities, their workplaces and the world better places than they found them.

I’ve recently finished an article with Mike Bush (a colleague here at BYU) in which we coin what we believe to be a new term in the standards debate–”content & tool malleability.” Our piece is modestly titled “The Transformation of Learning with Technology: Learner-Centricity, Content and Tool Malleability, and Network Effects” and will appear in the March-April edition of Educational Technology Magazine. A couple of months after it’s published, I’ll be able to publish the article in its entirety here. For now, I want to provide a preview of our notion of malleability. We suggest that malleability has three key attributes: openness, modularity, and interoperability and that teaching and learning tools and content must become more malleable if they are to become authentically reusable, remixable, redistributable, repurposable, etc.

We cite IBM’s ultimately successful implementation of the principles of modularity and interoperability which enabled the PC-maker to call on outside vendors for parts for their machines, creating an essentially malleable computing platform:

Their rejection of proprietary technology in favor of openness created the opportunity for IBM to call on Microsoft to develop the operating system and for a host of other companies (including Microsoft!) to go on to create thousands upon thousands of software applications, guaranteeing he long-term success of IBM’s initial design. Furthermore, competing companies that chose a proprietary and closed approach for their hardware, software, or both, (e.g., Texas Instruments, Amiga, Atari, Commodore, and Radio Shack) are nowhere to be found among Twenty-First Century personal computers. Even Apple, with the initial version of their innovative Macintosh, came close to meeting disaster until they opened things up with their Macintosh II (Bush, 1996). In the end, the nature of IBM’s approach not only ensured success in their initial venture, but the continued application of the same principles over the years by IBM’s successors also makes it possible for today’s machines to run much of the same software that was created for the original IBM PC.

Among the principles of openness, modularity, and interoperability that brought success to the IBM-PC venture, the importance of modularity seems perhaps preeminent and has been documented in detail by scholars at the Harvard Business School (Baldwin & Clark, 2000). In their initial work, they analyzed how modularity evolved as a set of design principles during the period between 1944 and 1960. Then using Holland’s theory of complex adaptive systems as a theoretical foundation, they explain how the design principles they identified went on to radically transform the information technology industry from the 1960s through the end of the century. They show how modular design and design processes have fostered change in the industry as it moved from one consisting of a few dozen companies and dominated by IBM to one that involves over a thousand companies and in which IBM plays a significantly lesser role. For example, the “packaged software” sector in the information technology industry consisted of about seven firms in 1970 that were valued at just over $1 billion (as measured in constant 2002 dollars). Thirty-two years later that sector had grown to 408 companies with a market capitalization of $490 billion (Baldwin & Clark, 2006).

Unfortunately, the application of the principles that made such developments possible in the computer industry is rare to nonexistent in many areas of education today. The education   technology landscape is best characterized by monolithic, enterprise technology silos with rigid, often impenetrable walls. Course management systems (CMSs), for example, are generally “all-or-nothing” propositions for institutions, teachers, and students. That is, even if you use an open source CMS like Moodle, you are (without significant customization) bound to use Moodle’s content publishing tool, Moodle’s quiz tool, Moodle’s gradebook, etc. Moreover, the CMS paradigm itself, tied as it is to semester calendars and time-bounded learning experiences (courses), severely limits learning continuity and persistence. Teachers and students are not free to choose the right / best / preferred tool for each teaching or learning activity they undertake, thus creating a technology paradigm that artificially limits possibilities and forecloses optimal teaching and learning choices.

The monolithic and rigid nature of today’s learning tools and content mirrors the way content has traditionally been made available to faculty and students—books and other resources (including online courses) have generally been all-or-nothing, take-them-or-leave them propositions. A similar business model was prevalent in pre-Internet days, resulting in CD-ROM databases that were more expensive than many potential consumers could afford. One analysis compared this marketing approach to a public water distribution system that would require selling the whole reservoir to each household rather than placing a meter at individual homes.

New approaches to content distribution, however, particularly the OpenCourseWare (OCW) and Open Educational Resource (OER) movements, promise to make a vast array of content open to instructors and students to reuse, revise, remix, and redistribute. The OCW Consortium, beginning with MIT in 2002, has now grown to include hundreds of institutions around the world that have chosen to place course materials online.iv The efforts of these institutions have spawned a related effort, dubbed Open Educational Resources (OER), to make learning materials and content (as opposed to complete courses) freely available as well (Breck, 2007). Around the world, millions of people, inside and outside of academia, are publishing content under Creative Commons licensing, making that content open for others to use in a variety of ways. We are rapidly approaching the tipping point at which a critical mass of participants in open content and open learning is sufficient to exponentially increase the value of each additional participant in the network (as described in the next section).

The stunning reality of the new standard of openness is that it is quite simple. The key is to create lots and lots of open content and provide open, easy access to it. While technical standards and specifications, such as the Shareable Content Object Reference Model (SCORM), are important when it comes to producing indexing, discovering, sequencing, packaging, and tracking of content, openness by itself is a paradigm shifting approach in the teaching and learning world.

The fact that content is openly available and usable is just as important as any particular technical feature of that content. While openness stands by itself as a radical new innovation, we need to avoid the temptation to downplay the importance of standards and specifications, for they are essential to the realization of the vision of open, modular, and interoperable learning environments.

This reality is not without historical precedent. Printing became affordable and available in large part due to what we today call standards. Indeed, as one scholar declared, “This then—the standardization and rapid multiplication of texts—was what the Fifteenth Century invention of printing made possible” (Bühler, 1952). Bühler also pointed out that printing’s contributions went beyond the replication issue, stating that modern scholarship only became possible with the production of identical copies of texts. Although the value of mass duplication is not to be discounted, the fact that scholars could reference each other’s work represented enormous value. Given this standardization, they were thus able to criticize, comment upon, connect to, and build upon what had come before. In many ways, printing standards facilitated the first widespread appearance of mashups in human history. The existence of identical copies was but one characteristic that facilitated the eventual widespread availability of books. In addition, several other factors contributed to the production process itself, eventually increasing the opportunity for wider distribution.

…

Although SCORM is not perfect, it at least began to address the issue of establishing a framework within which learning content can be made to interoperate in a variety of settings. Just as SIF opens up the opportunity for reuse of information created and used by various operational elements of schools, SCORM still holds the promise to facilitate the sharing of learning content, not only across learning management systems but also across tools that facilitate the design and development of learning content. In addition, common authentication schemes (e.g., OpenID) built upon Web services interoperability will ultimately allow learners to seamlessly navigate multiple Web-based teaching and learning applications, opening up possibilities for personal learning environments in which multiple sources of content and experiences work together to help students learn in ways that are tailored to each individual.

With developments like SCORM 2.0 on the horizon, as well as increasingly powerful software, hardware, and networking tools, technological barriers are falling. The challenge now is to harness these new enabling technologies to create more open, modular, and interoperable learning content as well as production and learning tools that are each malleable with respect to their individual functionality. Together, these technologies will help further the transformation of education from a teaching-oriented enterprise to a learning-centered one.

As noted, the entire piece will be available soon. We hope this notion of malleability helps move the conversation forward. What are your thoughts? Are we on the right track?

References:

Baldwin, C. Y., & Clark, K. B. (2000). Design rules,volume 1: The power of modularity. Cambridge, MA: MIT Press.

Baldwin, C. Y., & Clark, K. B. (2006). Modularity in the design of complex engineering systems. In D. Braha, A. A. Minai, & Y. Bar-Yam (Eds.), Complex engineered systems: Science meets technology (pp. 175–205). New York: Springer.

Breck, J. (2007, Nov./Dec.). Introduction to special issue on opening educational resources. Educational Technology,
47(6), 3–5.

Bühler, C. F. (1952). Fifteenth century books and the twentieth century. New York: The Golier Club.

Bush, M. D. (1996, Nov.). Fear & loathing in cyberspace: Of heroes and villains in the information age. Multimedia
Monitor; http://arclite.byu.edu/digital/heroesa5.html.