Letter from the Vice Provost

Years from now when we're sitting around having a beer and reflecting back on online learning in 2012-13, we'll undoubtedly say, "Remember all the hype about MOOCs?" The question is, will we be making analogies to Beanie Babies and Pet Rocks, or will we talk about how they profoundly changed online education? Will the early MOOCs from Stanford be compared to NCSA Mosaic, which changed the face of the Internet forever? Or will they be remembered as a brief chapter in the history of open access to education?

The numbers are staggering—Coursera has enrolled more than 5.4 million students in its brief history, growing faster than Facebook at a similar point in time. To put this in perspective, the National Center for Education Statistics pegs the total enrollment of U.S. degree-granting institutions at 21 million. Well over $100 million has been invested in MOOC efforts, much of it from the same top-tier Silicon Valley venture capitalists that fueled startups like Google and Amazon. You can't ignore that.

While it will be some time before the post-MOOC future of higher education becomes clear, we're taking steps to learn some things that will help us prepare for that day. This spring, DELTA participated in an "OOC experiment" with Engineering Online and the Electrical and Computer Engineering department. Dr. Paul Franzon volunteered his time to offer his graduate course in ASIC Design as an "open online course" (OOC). The goal was to better understand the technical and operational requirements to support a MOOC, as well as the cost in terms of faculty and staff time. Enrollment was capped at 1,000 students to avoid any impact on NC State's technology infrastructure. Students took the course for free, and could opt in to pay for an authenticated assessment that would earn them a certificate of completion. Approximately 700 students enrolled in the course, and less than 20 opted to pay for a certificate of completion. After all the costs were considered, the "break even" point was around 15 to 20,000 enrollments. You can read more about the OOC experiment in this report, but one thing is clear: sustainable financial models for these types of offerings have a long way to go.

While the jury is still out on "MOOC Mania," the value of digital learning technologies is clear in achieving NC State's #1 goal of enhancing the success of our students through educational innovation. We're seeing impressive gains in student learning outcomes through course redesign. More students are leveraging the flexibility of online courses to accelerate progress to degree. DELTA designers and faculty are collaborating to create powerful learning tools based on game technologies. We're streamlining and seamlessly integrating enterprise learning technologies to improve the user experience for both students and faculty. You will find these stories and more in this year's annual report. Please take time to read the stories, review the data, and learn more about the many ways that DELTA is supporting our faculty as we all work toward the common goal of enhanced student success.

Sincerely,
Dr. Tom Miller
Vice Provost, DELTA

The Mane Event

While learning shouldn't be all horseplay, who says we shouldn't play games to learn about horses? With educational games in higher education on the rise, DELTA's Instructional Innovation Services teamed up with Dr. Shannon Pratt-Phillips, associate professor of equine nutrition and physiology, to create a horse game. The opportunity was the perfect union of DELTA's desire to develop a serious game and Pratt-Phillips' search for a mobile solution to reinforce concepts in her equine science course. And thus the Mane Event was foaled.

Pratt-Phillips approached DELTA through the IDEA Grants program looking for teaching solutions for her course, ANS 110: Introduction to Equine Science. Each semester her students enter the course with varying backgrounds and experience with horses, and she wanted a way to get students up to speed on the basic concepts of the course: nutrition, anatomy, breeds and horse careers. While she initially had a mobile app in mind, Pratt-Phillips embraced the DELTA team's idea to explore a game approach.

"A game is a great way to capture a wide audience," Pratt-Phillips said. "For students who already have some background in the area, the game allows them to build on what they know easily and practice in a fun way. For those students who have less experience, the game helps to reinforce concepts for them, at a pace that they can control."

Developing the Mane Event was a welcome challenge for DELTA's production team. While this small group of full-time staff led the project, they relied heavily on NC State student staff to research, storyboard, design, animate and program the game. The game's artwork and animations were produced entirely by students. Through research, trial and error, and strong staff-faculty-student collaboration, the team saw the game take shape. And while attacking the mountain of technical details, they kept the educational focus and sound learning theories as the highest priorities.

"When you think about chunking content for student learning, mini-games are great ways to learn," said Amanda Robertson, assistant director for educational media development and project lead for the Mane Event. "We could grab these discrete bits of knowledge, like nutrition or anatomy, and build individual mini-games out of them."

This series of 11 mini-games formed the Mane Event, a smart, interactive, visually stunning 2-D game that entertains and teaches at the same time. Players start out as a ranch hand cleaning stables at Hillcrest Ranch and level up as they master increasingly complex concepts of horse care and nutrition. Along they way they encounter the spirited El Diablo, compete against the evil Luchador and earn Horse Bucks for their accomplishments. As they play and compete, they learn.

"I think faculty can become excited about the game development process when they realize that their students might actually play—and continue learning—long after the course is over," Pratt-Phillips said.

The Mane Event Web version is currently available at maneevent.ncsu.edu (new window). The mobile version will be released through the iTunes App Store and Google Play in spring 2014.

Related Link

• The Mane Event (new window)

OOC Experiment

Any good experiment starts with good questions. What will it take to launch an open online course (OOC)? How many staff hours will course development require? Can we manage the necessary infrastructure to host it ourselves? Will it be financially feasible? Will we learn lessons to help NC State find its place in MOOC mania?

These questions and more surfaced when DELTA first learned of Dr. Paul Franzon's plans to launch Digital ASIC (Application Specific Integrated Circuits) Design as an OOC experiment in summer 2013. With the support of Engineering Online, Franzon, professor of electrical computer engineering, had decided to jump into the national trend of massive open online courses (MOOCs) by offering this free, graduate-level course to the public. At DELTA, we saw this endeavor as an opportunity to observe, and be part of, an intriguing experiment.

Step back a few months. By the summer of 2012, MOOCs were making headlines from The New York Times to the Chronicle of Higher Education. Universities across the country were scrambling to assess this disruptive innovation and determine how to participate. In December 2012, DELTA formed an internal MOOC response team, composed of staff with expertise in instructional technology, instructional design, systems development and support, technology infrastructure, and marketing and course administration. This response team was charged with assessing the MOOC landscape and exploring the components of a MOOC, so that DELTA would be ready to assist if an OOC/MOOC opportunity arose at NC State. Franzon's Digital ASIC Design was that opportunity.

"Our goal was to support Dr. Franzon, answer questions, and learn along the way," said Kay Zimmerman, DELTA associate vice provost for marketing and partnership development.

As Franzon worked expeditiously to develop and deliver course content to the 651 students from around the globe who enrolled in the OOC, DELTA staff provided support in the background. Without resources to pay for a partnership with a third-party provider, DELTA used its existing technology platform and infrastructure to host the course. Instructional staff worked with Franzon to develop accessible videos, assessment strategies and group work activities that would scale to a large, international class. Web developers created a website to market the course and assist in registration. And all involved watched closely as the OOC experiment played out.

"The time required was much more than I expected," Franzon said. "Working out the best pedagogical approach and then structuring the content to produce a 'tight' course; preparing boat loads of quiz questions; and then lots of data entry into Moodle. Each step took significant time."

When the 12-week OOC came to an end, the experiment yielded more questions than it did answers. As expected, the team determined that successful deployment of an OOC/MOOC requires a vast amount of time and resources from the faculty and from DELTA. Now we're asking, what business model can provide resources to support MOOC development? How do MOOCs support NC State's and DELTA's strategic plans? How did we benefit from this OOC? As with any good experiment, we will examine the collected data, reflect and cautiously experiment further in search for answers.

Redesigning Courses

Several years ago Dr. Brenda Burns-Williams started to notice her students in MA 111: Precalculus Algebra and Trigonometry becoming disconnected from the course material—quite literally. They didn't want to carry around the enormous course textbook because it was heavy and not particularly useful, and it embarrassingly signaled to others that they were taking a "remedial" math class. More and more, her students left their books at home.

The textbook was symbolic of other problems with MA 111. Burns-Williams, senior lecturer in mathematics, believed her students were not retaining information from her lectures and were not interacting with the material in a way that allowed them to learn course concepts. In this large, predominantly freshman lecture course, around 35 percent of students were receiving Ds and Fs. In 2007, the NC State Registrar's Office pinpointed MA 111 as being high-needs for improvement given the high D and F grades and course withdrawal (DFW) rates and the prevalence of course repeats.

In 2009, Burns-Williams applied for, and was awarded, a DELTA Large Course Redesign (LCR) pilot grant to redesign MA 111. The grant program's purpose was to reconceive and rebuild the delivery of a large enrollment course to increase student learning outcomes, primarily by incorporating learning technology. Over the summer of 2009, Burns-Williams worked with DELTA instructional designers, instructional technologists and LCR coordinators to develop new course materials that would entirely change the way she taught.

When the MA 111 redesigned course piloted in fall 2009, Burns-Williams offered a new learning experience for her students.

"Students can access material in a way that is most comfortable for them," Burns-Williams said. "They're able to get videos or read PDF versions of the notes and they can practice and write things down as they are learning. Instead of having a dry textbook or recalling what we did with the notes in class, they're able to have a more dynamic interaction with the material."

By flipping her course and delivering content through short videos and PDFs on the new course website, Burns-Williams used class time for better purposes, such as small group work, active learning exercises and answering questions.

MA 111 has been taught in a redesigned manner each semester since fall 2009. DELTA has collected and analyzed data to assess student success after redesign, and the results have been impressive. Students' ABC grades have increased as much as 15 percent some semesters. DFW rates have declined. And most exciting to Burns-Williams, students who have taken the redesigned MA 111 are more successful in the subsequent course, MA 141, indicating a positive outcome associated with the course redesign project.

"The results have given DELTA statistical evidence to promote course redesign across campus," said Dr. Traci Temple, DELTA assistant director of planning and assessment. "It serves as encouragement and provides more than anecdotal evidence that course redesign efforts have the potential to increase student retention, improve student performance, and increase mastery of learning and successful transfer of knowledge and skills to future courses."

Sharpening Our Tools

At DELTA, we are almost always upgrading something—a learning management system (LMS), tool, plug-in, or some piece of our extensive enterprise learning technologies. If we're not actively upgrading, we're planning an upgrade. It's enough to make a user wonder, why all the changes?

When it comes to learning technologies, our guiding philosophy is to meet the needs of the learning community at NC State, both faculty and students.

"We strive to identify and address needs within the community, as well as standards of reliability and efficiency, enabling instructors to effectively teach their classes," said Dr. Marty Dulberg, learning management system coordinator.

By meeting needs we mean a wide range of needs, some of which users rarely think about. Of course, we aim to provide tools that are easy to use, widely accessible, flexible and well supported. Then there's the less exciting set of behind-the-scenes requirements we attend to: security, reliability and storage capacity. In FY 2012-13 we took measures to address these needs in our major tools: Moodle, Mediasite, WolfWare and Blackboard Collaborate.

Perhaps the most visible change in DELTA's learning technologies in FY 12-13 was the move to Moodle 2 in the spring and summer of 2013. Because developers in the Moodle community were no longer supporting Moodle 1.9, our previous version, we upgraded our open source LMS to Moodle 2.3. The upgrade provided our users in more than 7,000 course sections with new features, a more modern interface, increased reliability, and improved scalability for our growing community. Plus, with the upgrade, Moodle became mobile, allowing students to easily access course materials on a mobile device on the go. DELTA staff created a course copier in Moodle 2.3, which allows faculty to copy courses from one semester to the next with ease. Throughout the upgrade, we provided support resources, workshops and help desk assistance to create a smooth transition.

Moodle wasn't the only tool that went mobile last year. The Mediasite classroom capture tool upgrade in spring 2013 enabled mobile video playback on any newer Android or iOS mobile device. The upgrade from Mediasite 5.5 to 6.0 required a change in file formats from WMV to MPEG files, which made mobile playback possible. In addition, changes made during the upgrade increased security of our course materials and decreased the amount of storage space required per file, which is crucial given that instructors create roughly one terabyte of Mediasite data per month. With more than 800,000 views of NC State Mediasite recordings during FY 12-13, these tool improvements impacted a significant number of students and their access to course materials.

While managing the major upgrades to Moodle and Mediasite, DELTA smoothly transitioned synchronous learning management systems from Elluminate Live! to Blackboard Collaborate and made minor interface improvements to WolfWare, our primary access point for NC State learning technologies.

All the while, we kept the needs of the community as the top priority.

"We welcome suggestions from anyone with an NC State ID," Dulberg said. "Students, faculty, administrators and staff are all welcome to provide feedback. We want to hear what everybody has to say."

Related Link

• WolfWare (new window)

Supporting Our Faculty

NC State faculty are world-renowned experts in many areas, but they are not always familiar with using learning technologies. That's why they turn to DELTA's Instructional Support Services (ISS) team for help. This team of instructional technologists helps build instructors' skills and confidence as they incorporate technology into their teaching. ISS assists hundreds of instructors each year through DELTA's services, including workshops and seminars, the LearnTech Help Desk and one-on-one instructional consultations. Dr. Mark Keen, teaching assistant professor of microbiology, began attending DELTA workshops during DELTA's 2010 Summer Institute, and he keeps coming back for more.

"Sometimes I attend the same workshop two or three times," Keen said. "I think we're all on a learning curve and I am always learning something new at DELTA."

What Keen has learned from attending DELTA's workshops allows him to more efficiently manage his large general microbiology courses, which generally run between 200 and 300 students per semester. With the experience he gained from Moodle workshops, Keen has greatly improved his efficiency using Moodle's grade book and quizzes. Through Moodle, his students can easily access their grades in real time. Keen has implemented exams and quizzes online, which has eliminated the need to organize, print, grade and return hundreds of paper microbiology tests.

Keen also learned to use Blackboard Collaborate through one of DELTA's workshops, which was taught remotely through Collaborate itself. He found DELTA's instructional method—teaching end users a new technology by completely immersing them in it—to be extremely effective. Now, Keen uses Collaborate to host question sessions for his students prior to exams. Previously, question sessions were held on campus in classrooms reserved on the evening prior to the exam. These classroom sessions restricted the number of participating students and restricted access to many due to time and travel conflicts. Implementing Collaborate sessions provides Keen with a more accessible and inclusive real-time method to address students' questions online the night before an exam.

Stacy Gant, director of DELTA's ISS team, said the workshop and seminar series is continuously evolving to best serve our faculty's needs.

"We are constantly adding to our workshops—improving our classes by updating content to reflect new features and adding new seminars each semester to reflect emerging technologies and hot topics in higher education," Gant said.

In addition to attending DELTA workshops and seminars, Keen frequently consults DELTA for support through the LearnTech Help Desk. When setting up the Moodle grade book for his courses, Keen requested an instructional consultation with a DELTA staff member to resolve his questions. Keen believes the support he has received from ISS has enhanced his effectiveness as a large course instructor.

"I'm a lot luckier than my colleagues who teach at smaller colleges," Keen said. "They just don't have this kind of support service for learning management software. I would have never experienced or learned many of these online learning management technologies without the support I've received from LearnTech and DELTA at NC State."

Service that Scales

When faced with a 20 percent increase in demand for their Distance Education (DE) classroom recording services, Tony Pearson and his team in DELTA's Video Communications Services (VCS) refused to choose between strong customer service and scalability. Instead, they found a way for their team to serve more instructors and students with the same commitment to service that this group has provided for more than 30 years.

"Our forte is to provide customer service behind the scenes," said Pearson, DELTA's associate director for VCS. "We want to be there for the faculty. We want to let them focus on pedagogy, and we'll focus on the technology."

When an instructor walks into a DE classroom, VCS staff are dedicated to making that day's class recording process smooth and seamless. The instructor can concentrate on his or her lecture, the students in the room can focus on learning, and the DE students around the globe can be guaranteed that the course lecture will be uploaded fifteen minutes after the live class ends.

With a little vision, creativity and planning, VCS devised a plan for controlled growth of classroom recording services: the central control room (CCR) model.

In spring 2012, VCS launched production with the CCR, which allows classroom support technicians to remotely monitor and control classroom recordings around campus from their office in Ricks Hall. This new model replaces the previous practice of stationing one technician in a control room at the back of each DE classroom. Not only does the CCR allow a single technician to monitor production quality in multiple classrooms simultaneously, it also frees up physical classroom space for additional student seats.

After launching the CCR, VCS recorded more than 11,000 hours of classroom instruction in FY 2012-13 in its nine DE classrooms, supporting between 60 and 70 DE courses each semester. While this seems like an enormous load, VCS is prepared for even more. To date, only five DE classrooms have been refurbished to integrate with the CCR; by summer 2014 a total of eight classrooms will be operated remotely. And this doesn't even begin to approach the 24-classroom capacity for the CCR.

"DE enrollment growth means increased demand for video recordings that support online instruction and downloads to mobile devices for anytime, anywhere access to course content," said Dr. Rebecca Swanson, associate vice provost for Distance and Distributed Education. "Only by optimizing DE classroom capabilities can we meet the unprecedented demand for video recordings."

With technicians removed from the classroom, you might expect customer service to suffer. Not so. Pearson has measures in place to guarantee excellent instructor support on a larger scale. For example, a technician serving as a runner arrives at the beginning of each class meeting to make sure that the recording is running smoothly. After the runner moves on to the next class, a remote technician is available at the touch of a help button. And in case some equipment fails, each classroom is equipped with a backup recorder poised to capture the classroom experience.

"We pride ourselves on having a lot of redundancy in place," Pearson said. "I can count on one hand the number of sessions that we've lost in the last 30 years. That gives us the quality of service to say to our faculty that you're going to get what you're promised."

This promise of quality service keeps the DE instructor-student experience focused on teaching and learning—not on the technology that makes it all possible.

Making Education Matter

Sarah Egan Warren, assistant director of the English department's Professional Writing Program, struggled to convince her students that what they learned in her class mattered. These future engineers could easily see how they would benefit from strong math skills or a foundation in engineering principles, but they didn't always recognize the value of taking an English course. After all, they weren't training to be writers. Why were they required to pass the professional writing course, English 331: Communication for Engineering and Technology?

While Egan Warren knew the answer to this question, she needed some help getting her message across. She and Sarah Glova, fellow lecturer in the Professional Writing Program, wanted to make videos of professionals in the field discussing how being good communicators can help them professionally. They came to DELTA and asked for help in executing their plan.

When members of DELTA's Video Communication Services (VCS) team began working with Egan Warren and Glova, they considered ways to film this series of videos that would capture the students' attention and drive home their message. Drawing inspiration from the works of filmmaker and designer Hillman Curtis, VCS's Arthur Earnest and John Gordon imagined a way to bring these videos to life that went beyond the traditional educational video. "When people think of education videos, they think of those dry, 16mm cheesy videos from the old times," said Earnest, DELTA technology support technician. "But we have the ability now, and if we have the inspiration, we can make it very cinematic. It doesn't have to be boring."

The result was Cube Talk, a stunning series of five videos that captured three local professionals talking about the impact of solid communication skills on the job. Filmed in a small conference room in an urban office space, the professionals casually chatted about their communication experiences in the workplace, reflecting themes taught in the course. The setting, combined with a modern style of shooting using a digital cinema camera, was designed to help the student audience engage in what seemed like an intimate conversation.

"The videos help 'sell' the importance of this course," Egan Warren said. "When students hear from professionals in their field that communication, collaboration and appropriate use of technology are all important to being successful in the workplace, it seems to carry more weight than if they are only hearing that message from the instructor."

Egan Warren and Glova have shared the Cube Talk videos with 11 instructors in the Professional Writing Program, and the videos are being incorporated into three courses in the curriculum. Because the videos were edited in a simple, clean manner, they are flexible enough to allow each instructor to customize how they use them in their course. According to their students, the videos are having an impact on how they value the course content.

"Overall, the theme that effective communication is a vital aspect of the workplace helps me understand the relevance of this course and makes me appreciate the material covered throughout the semester," a Professional Writing Program student said after watching the Cube Talk videos.

Sharing Best Practices

The word is out: NC State has a reputation for quality online education and the innovative use of learning technologies. As part of a leading research university, DELTA staff members are motivated to keep up with the top-notch faculty we support. For us, that means staying abreast of trends in learning technologies and online education, being leaders in those communities and sharing best practices. DELTA staff are helping other universities improve their online instruction, one conference presentation at a time.

In FY 2012-13, our staff presented at regional, national and international conferences, sharing with peers the results of our experiences at NC State. Explore our presentation topics. View some of the materials. Discover how we are shaping the educational landscape.