by: Brandon Pousley
Over the past 12 months, the TLL has been exploring the potential benefits of ePortfolios within various programs at HGSE. For our purposes, we have defined an ePortfolio as a digital repository for the collection, organization, and sharing of student artifacts to facilitate personal reflection, assessment of learning outcomes, multimodal and peer learning, and academic and career development. We have directly engaged faculty and staff in the CAEL, HTF, and Data Wise programs in technology pilots that have enabled us to surface multiple teaching and learning needs and technical requirements. These pilots have also allowed us to develop best practices, identify and solve technical and pedagogical challenges, and begin measuring impacts on learner outcomes.
Structure of an ePortfolio Technology Pilot
For us, a meaningful technology pilot is more than simply the first implementation of a new tool or platform. It represents an opportunity to explore the real-world impact and unanticipated variables inherent in a new technology. By being deliberate about a process for piloting new tools, we have been able to maintain a strong focus on teaching and learning, identify and apply best practices, and build a stronger and more informed case for expansion into related use cases at the School. In the pilots that we conduct, we adhere to a standard process. Below, we will explore that process as it applied to our exploration of an ePortfolio solution at HGSE.
[For a quick read on the basic mechanics and merits of a modern ePortfolio, view this introductory EDUCAUSE resource.]
Step 1: Needs Analysis
In this first stage, we focus on gathering as much information as possible from the stakeholders of identified use cases for the technology. Stakeholders include project leads, instructors, learners, and school leadership. We are seeking to define our audience, identify intended outcomes, and prioritize pedagogical and technical needs. This step ends with the development of a requirements matrix that lists all of our use cases and identified needs.
For our ePortfolio exploration, this process began a year ago with conversations amongst project stakeholders from CAEL, HTF, Data Wise, Project Zero, and the Ed.L.D. program. These conversations sought to understand what has been explored or implemented in the past, opportunities for exploration, individual and programmatic needs, timelines, and tolerance for risk. Ultimately we discovered there were strong grassroots needs for ePortfolio solutions at the program level. While the use cases had significant overlap in terms of outcomes and functionality, they differed greatly in terms of audience, timeline, modality and pedagogy.
Step 2: Platform Selection
After we know what the various use cases need, we can conduct a market scan of potential solutions and understand where any gaps may exist. We also begin to get a sense of whether a single solution may solve the identified challenge or if we may need to explore multiple options. In this step, consider a wide variety of options, but quickly narrow our focus based on strengths and limitations discovered in various solutions. A common deliverable at the end of this step will be a solutions matrix that maps platform affordances on top of our original requirements matrix.
In regards to ePortfolios, we were able to quickly get a sense of the landscape of solutions through literature reviews, direct vendor conversations, and peer institution networking. We found that a wide spectrum of solutions exists, but only a handful could ultimately serve our identified use cases. We were able to obtain sandbox environments in all of the finalist platforms and to conduct usability testing with students and colleagues to inform our platform selection. This was was conducted over several months and, especially in the later stages, directly involved use case and school leadership stakeholders in the decision-making process.
Step 3: Implementation
Once a solution has been identified, the implementation phase involves direct collaboration with the project or teaching team. This includes custom fitting the solution to meet their needs. Through this process we identify, capture, and track areas where the platform is not performing as expected, or areas where we will need to expand or refine in future iterations. In this stage, we are focused on moving from ‘paper to product’ as quickly as possible to understand the real-world implications of the decisions we have made in the previous steps. At the end of this stage, we have learned as much as possible about the technology and have begun documenting best practices and training and support materials.
Our ePortfolio implementation process began with CAEL. Since the TLL was directly supporting the design and development of CAEL, the integration of an ePortfolio implementation process into the HTF program was well coordinated and integrated. We were able to discover opportunities for ePortfolio integration in an organic way and to ensure we focused on pedagogy and learning experience in addition to technical considerations.
Step 4: Evaluation
In this final step, we seek to evaluate both learner outcomes and technical considerations for the immediate and applicable use cases. We want to understand the impact of the technology implementation on those who were directly engaged with the platform or tool. We also want to gauge the teaching team and administrative impact of the integration. In this stage, we begin to think about next iterations for this use case as well as related projects that can directly or indirectly benefit from these understandings.
Evaluation of our ePortfolio pilot was an embedded component of the overall program evaluation; often it was a standing agenda item at teaching team and course design meetings. Our ongoing evaluation has enabled us to apply lessons learned from the first module to subsequent designs. We have also been able to include other use cases in conversations to consider expanding the pilot to include other projects.
Related: The TLL recently presented at the “Beyond the Box” 2016 AAEEBL Conference in Boston, MA. The presentation featured a panel including: Brandon Pousley and Bonnie Anderson from the TLL and Victor Pereira from HTF. The presentation was titled “Focusing on the Learner Experience: Implementing ePortfolios at Harvard” and featured discussions of learner-focused strategies for technology pilots at HGSE.