research on online courses

Going the Distance: Why Online Learning Works

Learn more about the benefits of online learning, what online learning involves, and why it could be a good option for you.

Mary Sharp Emerson

There’s no debate that online learning has become a staple of higher education. And today’s online courses encompass a wide range of subjects and are more accessible than ever.

From current college students to people seeking a career change to professionals looking to update their skills, there is an online program for everyone. 

More than 6.5 million people were enrolled in some type of distance learning course offered through a degree-granting college or university in 2017, according to the National Center for Education Statistics.  

But why is online learning so popular, what methods make it effective, and where did it all begin? And most critically, how does the online educational experience match up to the traditional classroom? 

Let’s dig into the answers to these questions.

Distance Learning: New Technologies for an Old Practice

Distance learning online may be new, but the desire to learn from field experts from afar is not. In fact, all that’s really changed over the centuries—yes, centuries!—is the speed and style of communication.

For instance, in 1728, distance learning took place through correspondence courses in shorthand. Assignments were distributed and collected via parcel post.

In the early 1900s, radio emerged as a new educational opportunity. In 1919, for example, University of Wisconsin professors began the first federally licensed radio station dedicated to educational broadcasting.

However, truly functional distance learning remained little more than a fringe endeavor until the birth of the World Wide Web in 1991. 

Today, online learning is the fastest growing segment of education , even as overall enrollment at postsecondary institutions declines. It’s no wonder that 65 percent of institutions report that online learning is critical to their long-term strategic plans.

See how Harvard Extension delivers quality programs across the globe

Why Students Choose Online Courses

Learning styles are as varied as people themselves. While there’s no one “best” course format, research suggests that for some, online learning can be a rich experience. 

For instance, studies show that high self-efficacy often leads to success in online courses. In addition, computer literacy, good time management skills, and a positive attitude toward e-learning have been shown to be key characteristics of students who learn effectively in a blended learning environment.

In most cases, online and blended learning are as effective as a traditional classroom setting. But they may offer unmatched benefits for students studying part time or from a distance.

Scheduling flexibility

Most people agree that the biggest benefit of online courses is their flexibility. According to a recent study by Best Colleges , more than half of online learners have children at home or are currently working. Online courses offer the opportunity to complete classwork when it is most convenient, without sacrificing the quality of the educational experience.

Many online courses can be completed in the evenings, over the weekend, or during lunch hour. Assignments can be finished in large blocks or broken down into small sessions over the course of the day or the week. 

In other words, students can choose the schedule that works best for their particular situation.

Easy-to-use technologies facilitate collaboration

Many online courses today rely heavily on online learning platforms or learning management systems that allow collaboration between students and instructors. These are coupled with online discussion boards, one-on-one messaging, and of course, email for soliciting feedback, discussing concepts, and networking.

Most of these technologies are easy to use, a far cry from the unwieldy online conferencing tools of even a decade ago. Most can be downloaded as an app on a smartphone and/or laptop. And once installed, navigating through the course materials—whether live or recorded—is remarkably simple.  

Best of all, the technology used in online courses makes it possible for students with disabilities (whether they be visual, auditory, or physical) to participate fully in classes and discussions.

Plenty of career-boosting course options

Online learning offers thousands of courses designed to meet just about any individual goal, from anywhere with an internet connection. 

Online course technology lets institutions extend their opportunities to people living in regions with limited options. This not only expands educational options for those in remote or under-resourced areas, it allows them to network with professionals and experts in their desired field. 

Plus, the availability of workshops, individual courses, microcredentials, certificates, and degrees lets students tailor their pathway to fill particular gaps. 

Distance programs put learners of every age and background solidly in control of their development, allowing them to take just the courses they need to reach their goals. These factors are why online education is a foundation of the lifelong learning philosophy . 

Have questions? Talk to our Enrollment Services team

What are the different types of online courses? 

Today’s modern online classes offer the same quality of instruction, educational experience, and networking opportunities as the traditional classroom.

Generally speaking, there are three different types of online classes or courses to choose from:

• Asynchronous: Learners read or listen to course material on their own, at their own pace, completing assignments independently
• Synchronous: Students use video-conferencing platforms to interact in real-time with both instructors and classmates
• Blended: A mix of recorded lectures, slideshows, and PDFs can be accessed at any time for self-paced learning, coupled with regularly scheduled video conferencing for real-time interaction; some courses may even include campus visits for additional collaboration

The range of online formats ensures that anyone seeking to enrich their educational background outside of the traditional classroom can do so in the way that best suits them, without sacrificing the quality of the educational experience.

Moreover, the availability of self-paced, yet still interactive, distance education reflects a larger shift in education as a whole toward student-centered learning.

Why Distance Learning Works

Pedagogical theory in general has shifted away from behaviorism, in which learners were viewed as blank slates which could be filled with information and habits through drill and reward.

Today’s cognitive theory and constructivist teaching seek to understand how people build new knowledge within the framework of what they already know and understand. 

Constructivism in particular dictates that learners be at the helm of their own education, while instructors act as mediators who introduce ideas, ask questions, and encourage students to explore new ways of thinking on their own.

Online courses present a rich opportunity for this kind of teaching and learning. Students can absorb new information independently and come together to synthesize that information into real-world knowledge with classmates and instructors.

This ideal match between the theory of learning and the actual implementation of instruction may explain the success of students in online classes. 

Yet whatever the reason, this is certain: Online education is fundamentally changing the paradigm of higher education. It is opening up learning possibilities to students everywhere, in nearly every circumstance, and delivering that education in a way that is proving to be effective, convenient, and empowering.

Ready to take your next step? Explore graduate and undergraduate degrees and certificates

Start your Harvard Extension journey today

About the Author

Digital Content Producer

Emerson is a Digital Content Producer at Harvard DCE. She is a graduate of Brandeis University and Yale University and started her career as an international affairs analyst. She is an avid triathlete and has completed three Ironman triathlons, as well as the Boston Marathon.

6 Strategies for Maintaining Momentum in Chaotic Times

Life coach Deb Levy provides tips for staying grounded and regaining your focus when life gets hectic.

Harvard Division of Continuing Education

The Division of Continuing Education (DCE) at Harvard University is dedicated to bringing rigorous academics and innovative teaching capabilities to those seeking to improve their lives through education. We make Harvard education accessible to lifelong learners from high school to retirement.

• Share full article

Advertisement

Supported by

With Online Learning, ‘Let’s Take a Breath and See What Worked and Didn’t Work’

The massive expansion of online higher education created a worldwide laboratory to finally assess its value and its future.

By Jon Marcus

This article is part of our Learning special report about how the pandemic has continued to change how we approach education.

Kameshwari Shankar watched for years as college and university courses were increasingly taught online instead of face to face, but without a definitive way of understanding which students benefited the most from them, or what if anything they learned.

As an associate professor of economics at City College in New York, Dr. Shankar knew that one of the most important requirements of scientific research was often missing from studies of the effectiveness of online higher education: a control group.

Then came the Covid-19 pandemic, forcing almost everyone on earth online and creating a randomized trial on a planetary scale with a control group so big, it was a researcher’s wildest dream.

“The pandemic and the lockdown — that’s a great natural experiment,” said Dr. Shankar. A study she co-authored called it “a gold mine of evidence.”

Now the results of this experiment are starting to come in. They suggest that online higher education may work better than prepandemic research suggested, and that it is evolving decisively toward a combination of in-person and online, or “blended,” classes.

“For two years we’ve had sort of a petri dish of experimenting with learning online,” said Anant Agarwal, chief platform officer of the online program management company 2U and former CEO of edX, the online provider created by the Massachusetts Institute of Technology and Harvard and sold last year to 2U for $800 million. “Now people are sitting down and saying, ‘Let’s take a breath. Let’s see what worked and didn’t work.’ ”

We are having trouble retrieving the article content.

Please enable JavaScript in your browser settings.

Thank you for your patience while we verify access. If you are in Reader mode please exit and  log into  your Times account, or  subscribe  for all of The Times.

Thank you for your patience while we verify access.

Already a subscriber?  Log in .

Want all of The Times?  Subscribe .

How Effective Is Online Learning? What the Research Does and Doesn’t Tell Us

• Share article

Editor’s Note: This is part of a series on the practical takeaways from research.

The times have dictated school closings and the rapid expansion of online education. Can online lessons replace in-school time?

Clearly online time cannot provide many of the informal social interactions students have at school, but how will online courses do in terms of moving student learning forward? Research to date gives us some clues and also points us to what we could be doing to support students who are most likely to struggle in the online setting.

The use of virtual courses among K-12 students has grown rapidly in recent years. Florida, for example, requires all high school students to take at least one online course. Online learning can take a number of different forms. Often people think of Massive Open Online Courses, or MOOCs, where thousands of students watch a video online and fill out questionnaires or take exams based on those lectures.

In the online setting, students may have more distractions and less oversight, which can reduce their motivation.

Most online courses, however, particularly those serving K-12 students, have a format much more similar to in-person courses. The teacher helps to run virtual discussion among the students, assigns homework, and follows up with individual students. Sometimes these courses are synchronous (teachers and students all meet at the same time) and sometimes they are asynchronous (non-concurrent). In both cases, the teacher is supposed to provide opportunities for students to engage thoughtfully with subject matter, and students, in most cases, are required to interact with each other virtually.

Coronavirus and Schools

Online courses provide opportunities for students. Students in a school that doesn’t offer statistics classes may be able to learn statistics with virtual lessons. If students fail algebra, they may be able to catch up during evenings or summer using online classes, and not disrupt their math trajectory at school. So, almost certainly, online classes sometimes benefit students.

In comparisons of online and in-person classes, however, online classes aren’t as effective as in-person classes for most students. Only a little research has assessed the effects of online lessons for elementary and high school students, and even less has used the “gold standard” method of comparing the results for students assigned randomly to online or in-person courses. Jessica Heppen and colleagues at the American Institutes for Research and the University of Chicago Consortium on School Research randomly assigned students who had failed second semester Algebra I to either face-to-face or online credit recovery courses over the summer. Students’ credit-recovery success rates and algebra test scores were lower in the online setting. Students assigned to the online option also rated their class as more difficult than did their peers assigned to the face-to-face option.

Most of the research on online courses for K-12 students has used large-scale administrative data, looking at otherwise similar students in the two settings. One of these studies, by June Ahn of New York University and Andrew McEachin of the RAND Corp., examined Ohio charter schools; I did another with colleagues looking at Florida public school coursework. Both studies found evidence that online coursetaking was less effective.

About this series

This essay is the fifth in a series that aims to put the pieces of research together so that education decisionmakers can evaluate which policies and practices to implement.

The conveners of this project—Susanna Loeb, the director of Brown University’s Annenberg Institute for School Reform, and Harvard education professor Heather Hill—have received grant support from the Annenberg Institute for this series.

To suggest other topics for this series or join in the conversation, use #EdResearchtoPractice on Twitter.

Read the full series here .

It is not surprising that in-person courses are, on average, more effective. Being in person with teachers and other students creates social pressures and benefits that can help motivate students to engage. Some students do as well in online courses as in in-person courses, some may actually do better, but, on average, students do worse in the online setting, and this is particularly true for students with weaker academic backgrounds.

Students who struggle in in-person classes are likely to struggle even more online. While the research on virtual schools in K-12 education doesn’t address these differences directly, a study of college students that I worked on with Stanford colleagues found very little difference in learning for high-performing students in the online and in-person settings. On the other hand, lower performing students performed meaningfully worse in online courses than in in-person courses.

But just because students who struggle in in-person classes are even more likely to struggle online doesn’t mean that’s inevitable. Online teachers will need to consider the needs of less-engaged students and work to engage them. Online courses might be made to work for these students on average, even if they have not in the past.

Just like in brick-and-mortar classrooms, online courses need a strong curriculum and strong pedagogical practices. Teachers need to understand what students know and what they don’t know, as well as how to help them learn new material. What is different in the online setting is that students may have more distractions and less oversight, which can reduce their motivation. The teacher will need to set norms for engagement—such as requiring students to regularly ask questions and respond to their peers—that are different than the norms in the in-person setting.

Online courses are generally not as effective as in-person classes, but they are certainly better than no classes. A substantial research base developed by Karl Alexander at Johns Hopkins University and many others shows that students, especially students with fewer resources at home, learn less when they are not in school. Right now, virtual courses are allowing students to access lessons and exercises and interact with teachers in ways that would have been impossible if an epidemic had closed schools even a decade or two earlier. So we may be skeptical of online learning, but it is also time to embrace and improve it.

A version of this article appeared in the April 01, 2020 edition of Education Week as How Effective Is Online Learning?

Sign Up for EdWeek Tech Leader

Edweek top school jobs.

Sign Up & Sign In

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Elsevier - PMC COVID-19 Collection

A systematic review of research on online teaching and learning from 2009 to 2018

Associated data.

Systematic reviews were conducted in the nineties and early 2000's on online learning research. However, there is no review examining the broader aspect of research themes in online learning in the last decade. This systematic review addresses this gap by examining 619 research articles on online learning published in twelve journals in the last decade. These studies were examined for publication trends and patterns, research themes, research methods, and research settings and compared with the research themes from the previous decades. While there has been a slight decrease in the number of studies on online learning in 2015 and 2016, it has then continued to increase in 2017 and 2018. The majority of the studies were quantitative in nature and were examined in higher education. Online learning research was categorized into twelve themes and a framework across learner, course and instructor, and organizational levels was developed. Online learner characteristics and online engagement were examined in a high number of studies and were consistent with three of the prior systematic reviews. However, there is still a need for more research on organization level topics such as leadership, policy, and management and access, culture, equity, inclusion, and ethics and also on online instructor characteristics.

• • Twelve online learning research themes were identified in 2009–2018.
• • A framework with learner, course and instructor, and organizational levels was used.
• • Online learner characteristics and engagement were the mostly examined themes.
• • The majority of the studies used quantitative research methods and in higher education.
• • There is a need for more research on organization level topics.

1. Introduction

Online learning has been on the increase in the last two decades. In the United States, though higher education enrollment has declined, online learning enrollment in public institutions has continued to increase ( Allen & Seaman, 2017 ), and so has the research on online learning. There have been review studies conducted on specific areas on online learning such as innovations in online learning strategies ( Davis et al., 2018 ), empirical MOOC literature ( Liyanagunawardena et al., 2013 ; Veletsianos & Shepherdson, 2016 ; Zhu et al., 2018 ), quality in online education ( Esfijani, 2018 ), accessibility in online higher education ( Lee, 2017 ), synchronous online learning ( Martin et al., 2017 ), K-12 preparation for online teaching ( Moore-Adams et al., 2016 ), polychronicity in online learning ( Capdeferro et al., 2014 ), meaningful learning research in elearning and online learning environments ( Tsai, Shen, & Chiang, 2013 ), problem-based learning in elearning and online learning environments ( Tsai & Chiang, 2013 ), asynchronous online discussions ( Thomas, 2013 ), self-regulated learning in online learning environments ( Tsai, Shen, & Fan, 2013 ), game-based learning in online learning environments ( Tsai & Fan, 2013 ), and online course dropout ( Lee & Choi, 2011 ). While there have been review studies conducted on specific online learning topics, very few studies have been conducted on the broader aspect of online learning examining research themes.

2. Systematic Reviews of Distance Education and Online Learning Research

Distance education has evolved from offline to online settings with the access to internet and COVID-19 has made online learning the common delivery method across the world. Tallent-Runnels et al. (2006) reviewed research late 1990's to early 2000's, Berge and Mrozowski (2001) reviewed research 1990 to 1999, and Zawacki-Richter et al. (2009) reviewed research in 2000–2008 on distance education and online learning. Table 1 shows the research themes from previous systematic reviews on online learning research. There are some themes that re-occur in the various reviews, and there are also new themes that emerge. Though there have been reviews conducted in the nineties and early 2000's, there is no review examining the broader aspect of research themes in online learning in the last decade. Hence, the need for this systematic review which informs the research themes in online learning from 2009 to 2018. In the following sections, we review these systematic review studies in detail.

Comparison of online learning research themes from previous studies.

2.1. Distance education research themes, 1990 to 1999 ( Berge & Mrozowski, 2001 )

Berge and Mrozowski (2001) reviewed 890 research articles and dissertation abstracts on distance education from 1990 to 1999. The four distance education journals chosen by the authors to represent distance education included, American Journal of Distance Education, Distance Education, Open Learning, and the Journal of Distance Education. This review overlapped in the dates of the Tallent-Runnels et al. (2006) study. Berge and Mrozowski (2001) categorized the articles according to Sherry's (1996) ten themes of research issues in distance education: redefining roles of instructor and students, technologies used, issues of design, strategies to stimulate learning, learner characteristics and support, issues related to operating and policies and administration, access and equity, and costs and benefits.

In the Berge and Mrozowski (2001) study, more than 100 studies focused on each of the three themes: (1) design issues, (2) learner characteristics, and (3) strategies to increase interactivity and active learning. By design issues, the authors focused on instructional systems design and focused on topics such as content requirement, technical constraints, interactivity, and feedback. The next theme, strategies to increase interactivity and active learning, were closely related to design issues and focused on students’ modes of learning. Learner characteristics focused on accommodating various learning styles through customized instructional theory. Less than 50 studies focused on the three least examined themes: (1) cost-benefit tradeoffs, (2) equity and accessibility, and (3) learner support. Cost-benefit trade-offs focused on the implementation costs of distance education based on school characteristics. Equity and accessibility focused on the equity of access to distance education systems. Learner support included topics such as teacher to teacher support as well as teacher to student support.

2.2. Online learning research themes, 1993 to 2004 ( Tallent-Runnels et al., 2006 )

Tallent-Runnels et al. (2006) reviewed research on online instruction from 1993 to 2004. They reviewed 76 articles focused on online learning by searching five databases, ERIC, PsycINFO, ContentFirst, Education Abstracts, and WilsonSelect. Tallent-Runnels et al. (2006) categorized research into four themes, (1) course environment, (2) learners' outcomes, (3) learners’ characteristics, and (4) institutional and administrative factors. The first theme that the authors describe as course environment ( n  = 41, 53.9%) is an overarching theme that includes classroom culture, structural assistance, success factors, online interaction, and evaluation.

Tallent-Runnels et al. (2006) for their second theme found that studies focused on questions involving the process of teaching and learning and methods to explore cognitive and affective learner outcomes ( n  = 29, 38.2%). The authors stated that they found the research designs flawed and lacked rigor. However, the literature comparing traditional and online classrooms found both delivery systems to be adequate. Another research theme focused on learners’ characteristics ( n  = 12, 15.8%) and the synergy of learners, design of the online course, and system of delivery. Research findings revealed that online learners were mainly non-traditional, Caucasian, had different learning styles, and were highly motivated to learn. The final theme that they reported was institutional and administrative factors (n  = 13, 17.1%) on online learning. Their findings revealed that there was a lack of scholarly research in this area and most institutions did not have formal policies in place for course development as well as faculty and student support in training and evaluation. Their research confirmed that when universities offered online courses, it improved student enrollment numbers.

2.3. Distance education research themes 2000 to 2008 ( Zawacki-Richter et al., 2009 )

Zawacki-Richter et al. (2009) reviewed 695 articles on distance education from 2000 to 2008 using the Delphi method for consensus in identifying areas and classified the literature from five prominent journals. The five journals selected due to their wide scope in research in distance education included Open Learning, Distance Education, American Journal of Distance Education, the Journal of Distance Education, and the International Review of Research in Open and Distributed Learning. The reviewers examined the main focus of research and identified gaps in distance education research in this review.

Zawacki-Richter et al. (2009) classified the studies into macro, meso and micro levels focusing on 15 areas of research. The five areas of the macro-level addressed: (1) access, equity and ethics to deliver distance education for developing nations and the role of various technologies to narrow the digital divide, (2) teaching and learning drivers, markets, and professional development in the global context, (3) distance delivery systems and institutional partnerships and programs and impact of hybrid modes of delivery, (4) theoretical frameworks and models for instruction, knowledge building, and learner interactions in distance education practice, and (5) the types of preferred research methodologies. The meso-level focused on seven areas that involve: (1) management and organization for sustaining distance education programs, (2) examining financial aspects of developing and implementing online programs, (3) the challenges and benefits of new technologies for teaching and learning, (4) incentives to innovate, (5) professional development and support for faculty, (6) learner support services, and (7) issues involving quality standards and the impact on student enrollment and retention. The micro-level focused on three areas: (1) instructional design and pedagogical approaches, (2) culturally appropriate materials, interaction, communication, and collaboration among a community of learners, and (3) focus on characteristics of adult learners, socio-economic backgrounds, learning preferences, and dispositions.

The top three research themes in this review by Zawacki-Richter et al. (2009) were interaction and communities of learning ( n  = 122, 17.6%), instructional design ( n  = 121, 17.4%) and learner characteristics ( n  = 113, 16.3%). The lowest number of studies (less than 3%) were found in studies examining the following research themes, management and organization ( n  = 18), research methods in DE and knowledge transfer ( n  = 13), globalization of education and cross-cultural aspects ( n  = 13), innovation and change ( n  = 13), and costs and benefits ( n  = 12).

2.4. Online learning research themes

These three systematic reviews provide a broad understanding of distance education and online learning research themes from 1990 to 2008. However, there is an increase in the number of research studies on online learning in this decade and there is a need to identify recent research themes examined. Based on the previous systematic reviews ( Berge & Mrozowski, 2001 ; Hung, 2012 ; Tallent-Runnels et al., 2006 ; Zawacki-Richter et al., 2009 ), online learning research in this study is grouped into twelve different research themes which include Learner characteristics, Instructor characteristics, Course or program design and development, Course Facilitation, Engagement, Course Assessment, Course Technologies, Access, Culture, Equity, Inclusion, and Ethics, Leadership, Policy and Management, Instructor and Learner Support, and Learner Outcomes. Table 2 below describes each of the research themes and using these themes, a framework is derived in Fig. 1 .

Research themes in online learning.

Online learning research themes framework.

The collection of research themes is presented as a framework in Fig. 1 . The themes are organized by domain or level to underscore the nested relationship that exists. As evidenced by the assortment of themes, research can focus on any domain of delivery or associated context. The “Learner” domain captures characteristics and outcomes related to learners and their interaction within the courses. The “Course and Instructor” domain captures elements about the broader design of the course and facilitation by the instructor, and the “Organizational” domain acknowledges the contextual influences on the course. It is important to note as well that due to the nesting, research themes can cross domains. For example, the broader cultural context may be studied as it pertains to course design and development, and institutional support can include both learner support and instructor support. Likewise, engagement research can involve instructors as well as learners.

In this introduction section, we have reviewed three systematic reviews on online learning research ( Berge & Mrozowski, 2001 ; Tallent-Runnels et al., 2006 ; Zawacki-Richter et al., 2009 ). Based on these reviews and other research, we have derived twelve themes to develop an online learning research framework which is nested in three levels: learner, course and instructor, and organization.

2.5. Purpose of this research

In two out of the three previous reviews, design, learner characteristics and interaction were examined in the highest number of studies. On the other hand, cost-benefit tradeoffs, equity and accessibility, institutional and administrative factors, and globalization and cross-cultural aspects were examined in the least number of studies. One explanation for this may be that it is a function of nesting, noting that studies falling in the Organizational and Course levels may encompass several courses or many more participants within courses. However, while some research themes re-occur, there are also variations in some themes across time, suggesting the importance of research themes rise and fall over time. Thus, a critical examination of the trends in themes is helpful for understanding where research is needed most. Also, since there is no recent study examining online learning research themes in the last decade, this study strives to address that gap by focusing on recent research themes found in the literature, and also reviewing research methods and settings. Notably, one goal is to also compare findings from this decade to the previous review studies. Overall, the purpose of this study is to examine publication trends in online learning research taking place during the last ten years and compare it with the previous themes identified in other review studies. Due to the continued growth of online learning research into new contexts and among new researchers, we also examine the research methods and settings found in the studies of this review.

The following research questions are addressed in this study.

• 1. What percentage of the population of articles published in the journals reviewed from 2009 to 2018 were related to online learning and empirical?
• 2. What is the frequency of online learning research themes in the empirical online learning articles of journals reviewed from 2009 to 2018?
• 3. What is the frequency of research methods and settings that researchers employed in the empirical online learning articles of the journals reviewed from 2009 to 2018?

This five-step systematic review process described in the U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse Procedures and Standards Handbook, Version 4.0 ( 2017 ) was used in this systematic review: (a) developing the review protocol, (b) identifying relevant literature, (c) screening studies, (d) reviewing articles, and (e) reporting findings.

3.1. Data sources and search strategies

The Education Research Complete database was searched using the keywords below for published articles between the years 2009 and 2018 using both the Title and Keyword function for the following search terms.

“online learning" OR "online teaching" OR "online program" OR "online course" OR “online education”

3.2. Inclusion/exclusion criteria

The initial search of online learning research among journals in the database resulted in more than 3000 possible articles. Therefore, we limited our search to select journals that focus on publishing peer-reviewed online learning and educational research. Our aim was to capture the journals that published the most articles in online learning. However, we also wanted to incorporate the concept of rigor, so we used expert perception to identify 12 peer-reviewed journals that publish high-quality online learning research. Dissertations and conference proceedings were excluded. To be included in this systematic review, each study had to meet the screening criteria as described in Table 3 . A research study was excluded if it did not meet all of the criteria to be included.

Inclusion/Exclusion criteria.

3.3. Process flow selection of articles

Fig. 2 shows the process flow involved in the selection of articles. The search in the database Education Research Complete yielded an initial sample of 3332 articles. Targeting the 12 journals removed 2579 articles. After reviewing the abstracts, we removed 134 articles based on the inclusion/exclusion criteria. The final sample, consisting of 619 articles, was entered into the computer software MAXQDA ( VERBI Software, 2019 ) for coding.

Flowchart of online learning research selection.

3.4. Developing review protocol

A review protocol was designed as a codebook in MAXQDA ( VERBI Software, 2019 ) by the three researchers. The codebook was developed based on findings from the previous review studies and from the initial screening of the articles in this review. The codebook included 12 research themes listed earlier in Table 2 (Learner characteristics, Instructor characteristics, Course or program design and development, Course Facilitation, Engagement, Course Assessment, Course Technologies, Access, Culture, Equity, Inclusion, and Ethics, Leadership, Policy and Management, Instructor and Learner Support, and Learner Outcomes), four research settings (higher education, continuing education, K-12, corporate/military), and three research designs (quantitative, qualitative and mixed methods). Fig. 3 below is a screenshot of MAXQDA used for the coding process.

Codebook from MAXQDA.

3.5. Data coding

Research articles were coded by two researchers in MAXQDA. Two researchers independently coded 10% of the articles and then discussed and updated the coding framework. The second author who was a doctoral student coded the remaining studies. The researchers met bi-weekly to address coding questions that emerged. After the first phase of coding, we found that more than 100 studies fell into each of the categories of Learner Characteristics or Engagement, so we decided to pursue a second phase of coding and reexamine the two themes. Learner Characteristics were classified into the subthemes of Academic, Affective, Motivational, Self-regulation, Cognitive, and Demographic Characteristics. Engagement was classified into the subthemes of Collaborating, Communication, Community, Involvement, Interaction, Participation, and Presence.

3.6. Data analysis

Frequency tables were generated for each of the variables so that outliers could be examined and narrative data could be collapsed into categories. Once cleaned and collapsed into a reasonable number of categories, descriptive statistics were used to describe each of the coded elements. We first present the frequencies of publications related to online learning in the 12 journals. The total number of articles for each journal (collectively, the population) was hand-counted from journal websites, excluding editorials and book reviews. The publication trend of online learning research was also depicted from 2009 to 2018. Then, the descriptive information of the 12 themes, including the subthemes of Learner Characteristics and Engagement were provided. Finally, research themes by research settings and methodology were elaborated.

4.1. Publication trends on online learning

Publication patterns of the 619 articles reviewed from the 12 journals are presented in Table 4 . International Review of Research in Open and Distributed Learning had the highest number of publications in this review. Overall, about 8% of the articles appearing in these twelve journals consisted of online learning publications; however, several journals had concentrations of online learning articles totaling more than 20%.

Empirical online learning research articles by journal, 2009–2018.

Note . Journal's Total Article count excludes reviews and editorials.

The publication trend of online learning research is depicted in Fig. 4 . When disaggregated by year, the total frequency of publications shows an increasing trend. Online learning articles increased throughout the decade and hit a relative maximum in 2014. The greatest number of online learning articles ( n  = 86) occurred most recently, in 2018.

Online learning publication trends by year.

4.2. Online learning research themes that appeared in the selected articles

The publications were categorized into the twelve research themes identified in Fig. 1 . The frequency counts and percentages of the research themes are provided in Table 5 below. A majority of the research is categorized into the Learner domain. The fewest number of articles appears in the Organization domain.

Research themes in the online learning publications from 2009 to 2018.

The specific themes of Engagement ( n  = 179, 28.92%) and Learner Characteristics ( n  = 134, 21.65%) were most often examined in publications. These two themes were further coded to identify sub-themes, which are described in the next two sections. Publications focusing on Instructor Characteristics ( n  = 21, 3.39%) were least common in the dataset.

4.2.1. Research on engagement

The largest number of studies was on engagement in online learning, which in the online learning literature is referred to and examined through different terms. Hence, we explore this category in more detail. In this review, we categorized the articles into seven different sub-themes as examined through different lenses including presence, interaction, community, participation, collaboration, involvement, and communication. We use the term “involvement” as one of the terms since researchers sometimes broadly used the term engagement to describe their work without further description. Table 6 below provides the description, frequency, and percentages of the various studies related to engagement.

Research sub-themes on engagement.

In the sections below, we provide several examples of the different engagement sub-themes that were studied within the larger engagement theme.

Presence. This sub-theme was the most researched in engagement. With the development of the community of inquiry framework most of the studies in this subtheme examined social presence ( Akcaoglu & Lee, 2016 ; Phirangee & Malec, 2017 ; Wei et al., 2012 ), teaching presence ( Orcutt & Dringus, 2017 ; Preisman, 2014 ; Wisneski et al., 2015 ) and cognitive presence ( Archibald, 2010 ; Olesova et al., 2016 ).

Interaction . This was the second most studied theme under engagement. Researchers examined increasing interpersonal interactions ( Cung et al., 2018 ), learner-learner interactions ( Phirangee, 2016 ; Shackelford & Maxwell, 2012 ; Tawfik et al., 2018 ), peer-peer interaction ( Comer et al., 2014 ), learner-instructor interaction ( Kuo et al., 2014 ), learner-content interaction ( Zimmerman, 2012 ), interaction through peer mentoring ( Ruane & Koku, 2014 ), interaction and community building ( Thormann & Fidalgo, 2014 ), and interaction in discussions ( Ruane & Lee, 2016 ; Tibi, 2018 ).

Community. Researchers examined building community in online courses ( Berry, 2017 ), supporting a sense of community ( Jiang, 2017 ), building an online learning community of practice ( Cho, 2016 ), building an academic community ( Glazer & Wanstreet, 2011 ; Nye, 2015 ; Overbaugh & Nickel, 2011 ), and examining connectedness and rapport in an online community ( Bolliger & Inan, 2012 ; Murphy & Rodríguez-Manzanares, 2012 ; Slagter van Tryon & Bishop, 2012 ).

Participation. Researchers examined engagement through participation in a number of studies. Some of the topics include, participation patterns in online discussion ( Marbouti & Wise, 2016 ; Wise et al., 2012 ), participation in MOOCs ( Ahn et al., 2013 ; Saadatmand & Kumpulainen, 2014 ), features that influence students’ online participation ( Rye & Støkken, 2012 ) and active participation.

Collaboration. Researchers examined engagement through collaborative learning. Specific studies focused on cross-cultural collaboration ( Kumi-Yeboah, 2018 ; Yang et al., 2014 ), how virtual teams collaborate ( Verstegen et al., 2018 ), types of collaboration teams ( Wicks et al., 2015 ), tools for collaboration ( Boling et al., 2014 ), and support for collaboration ( Kopp et al., 2012 ).

Involvement. Researchers examined engaging learners through involvement in various learning activities ( Cundell & Sheepy, 2018 ), student engagement through various measures ( Dixson, 2015 ), how instructors included engagement to involve students in learning ( O'Shea et al., 2015 ), different strategies to engage the learner ( Amador & Mederer, 2013 ), and designed emotionally engaging online environments ( Koseoglu & Doering, 2011 ).

Communication. Researchers examined communication in online learning in studies using social network analysis ( Ergün & Usluel, 2016 ), using informal communication tools such as Facebook for class discussion ( Kent, 2013 ), and using various modes of communication ( Cunningham et al., 2010 ; Rowe, 2016 ). Studies have also focused on both asynchronous and synchronous aspects of communication ( Swaggerty & Broemmel, 2017 ; Yamagata-Lynch, 2014 ).

4.2.2. Research on learner characteristics

The second largest theme was learner characteristics. In this review, we explore this further to identify several aspects of learner characteristics. In this review, we categorized the learner characteristics into self-regulation characteristics, motivational characteristics, academic characteristics, affective characteristics, cognitive characteristics, and demographic characteristics. Table 7 provides the number of studies and percentages examining the various learner characteristics.

Research sub-themes on learner characteristics.

Online learning has elements that are different from the traditional face-to-face classroom and so the characteristics of the online learners are also different. Yukselturk and Top (2013) categorized online learner profile into ten aspects: gender, age, work status, self-efficacy, online readiness, self-regulation, participation in discussion list, participation in chat sessions, satisfaction, and achievement. Their categorization shows that there are differences in online learner characteristics in these aspects when compared to learners in other settings. Some of the other aspects such as participation and achievement as discussed by Yukselturk and Top (2013) are discussed in different research themes in this study. The sections below provide examples of the learner characteristics sub-themes that were studied.

Self-regulation. Several researchers have examined self-regulation in online learning. They found that successful online learners are academically motivated ( Artino & Stephens, 2009 ), have academic self-efficacy ( Cho & Shen, 2013 ), have grit and intention to succeed ( Wang & Baker, 2018 ), have time management and elaboration strategies ( Broadbent, 2017 ), set goals and revisit course content ( Kizilcec et al., 2017 ), and persist ( Glazer & Murphy, 2015 ). Researchers found a positive relationship between learner's self-regulation and interaction ( Delen et al., 2014 ) and self-regulation and communication and collaboration ( Barnard et al., 2009 ).

Motivation. Researchers focused on motivation of online learners including different motivation levels of online learners ( Li & Tsai, 2017 ), what motivated online learners ( Chaiprasurt & Esichaikul, 2013 ), differences in motivation of online learners ( Hartnett et al., 2011 ), and motivation when compared to face to face learners ( Paechter & Maier, 2010 ). Harnett et al. (2011) found that online learner motivation was complex, multifaceted, and sensitive to situational conditions.

Academic. Several researchers have focused on academic aspects for online learner characteristics. Readiness for online learning has been examined as an academic factor by several researchers ( Buzdar et al., 2016 ; Dray et al., 2011 ; Wladis & Samuels, 2016 ; Yu, 2018 ) specifically focusing on creating and validating measures to examine online learner readiness including examining students emotional intelligence as a measure of student readiness for online learning. Researchers have also examined other academic factors such as academic standing ( Bradford & Wyatt, 2010 ), course level factors ( Wladis et al., 2014 ) and academic skills in online courses ( Shea & Bidjerano, 2014 ).

Affective. Anderson and Bourke (2013) describe affective characteristics through which learners express feelings or emotions. Several research studies focused on the affective characteristics of online learners. Learner satisfaction for online learning has been examined by several researchers ( Cole et al., 2014 ; Dziuban et al., 2015 ; Kuo et al., 2013 ; Lee, 2014a ) along with examining student emotions towards online assessment ( Kim et al., 2014 ).

Cognitive. Researchers have also examined cognitive aspects of learner characteristics including meta-cognitive skills, cognitive variables, higher-order thinking, cognitive density, and critical thinking ( Chen & Wu, 2012 ; Lee, 2014b ). Lee (2014b) examined the relationship between cognitive presence density and higher-order thinking skills. Chen and Wu (2012) examined the relationship between cognitive and motivational variables in an online system for secondary physical education.

Demographic. Researchers have examined various demographic factors in online learning. Several researchers have examined gender differences in online learning ( Bayeck et al., 2018 ; Lowes et al., 2016 ; Yukselturk & Bulut, 2009 ), ethnicity, age ( Ke & Kwak, 2013 ), and minority status ( Yeboah & Smith, 2016 ) of online learners.

4.2.3. Less frequently studied research themes

While engagement and learner characteristics were studied the most, other themes were less often studied in the literature and are presented here, according to size, with general descriptions of the types of research examined for each.

Evaluation and Quality Assurance. There were 38 studies (6.14%) published in the theme of evaluation and quality assurance. Some of the studies in this theme focused on course quality standards, using quality matters to evaluate quality, using the CIPP model for evaluation, online learning system evaluation, and course and program evaluations.

Course Technologies. There were 35 studies (5.65%) published in the course technologies theme. Some of the studies examined specific technologies such as Edmodo, YouTube, Web 2.0 tools, wikis, Twitter, WebCT, Screencasts, and Web conferencing systems in the online learning context.

Course Facilitation. There were 34 studies (5.49%) published in the course facilitation theme. Some of the studies in this theme examined facilitation strategies and methods, experiences of online facilitators, and online teaching methods.

Institutional Support. There were 33 studies (5.33%) published in the institutional support theme which included support for both the instructor and learner. Some of the studies on instructor support focused on training new online instructors, mentoring programs for faculty, professional development resources for faculty, online adjunct faculty training, and institutional support for online instructors. Studies on learner support focused on learning resources for online students, cognitive and social support for online learners, and help systems for online learner support.

Learner Outcome. There were 32 studies (5.17%) published in the learner outcome theme. Some of the studies that were examined in this theme focused on online learner enrollment, completion, learner dropout, retention, and learner success.

Course Assessment. There were 30 studies (4.85%) published in the course assessment theme. Some of the studies in the course assessment theme examined online exams, peer assessment and peer feedback, proctoring in online exams, and alternative assessments such as eportfolio.

Access, Culture, Equity, Inclusion, and Ethics. There were 29 studies (4.68%) published in the access, culture, equity, inclusion, and ethics theme. Some of the studies in this theme examined online learning across cultures, multi-cultural effectiveness, multi-access, and cultural diversity in online learning.

Leadership, Policy, and Management. There were 27 studies (4.36%) published in the leadership, policy, and management theme. Some of the studies on leadership, policy, and management focused on online learning leaders, stakeholders, strategies for online learning leadership, resource requirements, university policies for online course policies, governance, course ownership, and faculty incentives for online teaching.

Course Design and Development. There were 27 studies (4.36%) published in the course design and development theme. Some of the studies examined in this theme focused on design elements, design issues, design process, design competencies, design considerations, and instructional design in online courses.

Instructor Characteristics. There were 21 studies (3.39%) published in the instructor characteristics theme. Some of the studies in this theme were on motivation and experiences of online instructors, ability to perform online teaching duties, roles of online instructors, and adjunct versus full-time online instructors.

4.3. Research settings and methodology used in the studies

The research methods used in the studies were classified into quantitative, qualitative, and mixed methods ( Harwell, 2012 , pp. 147–163). The research setting was categorized into higher education, continuing education, K-12, and corporate/military. As shown in Table A in the appendix, the vast majority of the publications used higher education as the research setting ( n  = 509, 67.6%). Table B in the appendix shows that approximately half of the studies adopted the quantitative method ( n  = 324, 43.03%), followed by the qualitative method ( n  = 200, 26.56%). Mixed methods account for the smallest portion ( n  = 95, 12.62%).

Table A shows that the patterns of the four research settings were approximately consistent across the 12 themes except for the theme of Leaner Outcome and Institutional Support. Continuing education had a higher relative frequency in Learner Outcome (0.28) and K-12 had a higher relative frequency in Institutional Support (0.33) compared to the frequencies they had in the total themes (0.09 and 0.08 respectively). Table B in the appendix shows that the distribution of the three methods were not consistent across the 12 themes. While quantitative studies and qualitative studies were roughly evenly distributed in Engagement, they had a large discrepancy in Learner Characteristics. There were 100 quantitative studies; however, only 18 qualitative studies published in the theme of Learner Characteristics.

In summary, around 8% of the articles published in the 12 journals focus on online learning. Online learning publications showed a tendency of increase on the whole in the past decade, albeit fluctuated, with the greatest number occurring in 2018. Among the 12 research themes related to online learning, the themes of Engagement and Learner Characteristics were studied the most and the theme of Instructor Characteristics was studied the least. Most studies were conducted in the higher education setting and approximately half of the studies used the quantitative method. Looking at the 12 themes by setting and method, we found that the patterns of the themes by setting or by method were not consistent across the 12 themes.

The quality of our findings was ensured by scientific and thorough searches and coding consistency. The selection of the 12 journals provides evidence of the representativeness and quality of primary studies. In the coding process, any difficulties and questions were resolved by consultations with the research team at bi-weekly meetings, which ensures the intra-rater and interrater reliability of coding. All these approaches guarantee the transparency and replicability of the process and the quality of our results.

5. Discussion

This review enabled us to identify the online learning research themes examined from 2009 to 2018. In the section below, we review the most studied research themes, engagement and learner characteristics along with implications, limitations, and directions for future research.

5.1. Most studied research themes

Three out of the four systematic reviews informing the design of the present study found that online learner characteristics and online engagement were examined in a high number of studies. In this review, about half of the studies reviewed (50.57%) focused on online learner characteristics or online engagement. This shows the continued importance of these two themes. In the Tallent-Runnels et al.’s (2006) study, the learner characteristics theme was identified as least studied for which they state that researchers are beginning to investigate learner characteristics in the early days of online learning.

One of the differences found in this review is that course design and development was examined in the least number of studies in this review compared to two prior systematic reviews ( Berge & Mrozowski, 2001 ; Zawacki-Richter et al., 2009 ). Zawacki-Richter et al. did not use a keyword search but reviewed all the articles in five different distance education journals. Berge and Mrozowski (2001) included a research theme called design issues to include all aspects of instructional systems design in distance education journals. In our study, in addition to course design and development, we also had focused themes on learner outcomes, course facilitation, course assessment and course evaluation. These are all instructional design focused topics and since we had multiple themes focusing on instructional design topics, the course design and development category might have resulted in fewer studies. There is still a need for more studies to focus on online course design and development.

5.2. Least frequently studied research themes

Three out of the four systematic reviews discussed in the opening of this study found management and organization factors to be least studied. In this review, Leadership, Policy, and Management was studied among 4.36% of the studies and Access, Culture, Equity, Inclusion, and Ethics was studied among 4.68% of the studies in the organizational level. The theme on Equity and accessibility was also found to be the least studied theme in the Berge and Mrozowski (2001) study. In addition, instructor characteristics was the least examined research theme among the twelve themes studied in this review. Only 3.39% of the studies were on instructor characteristics. While there were some studies examining instructor motivation and experiences, instructor ability to teach online, online instructor roles, and adjunct versus full-time online instructors, there is still a need to examine topics focused on instructors and online teaching. This theme was not included in the prior reviews as the focus was more on the learner and the course but not on the instructor. While it is helpful to see research evolving on instructor focused topics, there is still a need for more research on the online instructor.

5.3. Comparing research themes from current study to previous studies

The research themes from this review were compared with research themes from previous systematic reviews, which targeted prior decades. Table 8 shows the comparison.

Comparison of most and least studied online learning research themes from current to previous reviews.

L = Learner, C=Course O=Organization.

5.4. Need for more studies on organizational level themes of online learning

In this review there is a greater concentration of studies focused on Learner domain topics, and reduced attention to broader more encompassing research themes that fall into the Course and Organization domains. There is a need for organizational level topics such as Access, Culture, Equity, Inclusion and Ethics, and Leadership, Policy and Management to be researched on within the context of online learning. Examination of access, culture, equity, inclusion and ethics is very important to support diverse online learners, particularly with the rapid expansion of online learning across all educational levels. This was also least studied based on Berge and Mrozowski (2001) systematic review.

The topics on leadership, policy and management were least studied both in this review and also in the Tallent-Runnels et al. (2006) and Zawacki-Richter et al. (2009) study. Tallent-Runnels categorized institutional and administrative aspects into institutional policies, institutional support, and enrollment effects. While we included support as a separate category, in this study leadership, policy and management were combined. There is still a need for research on leadership of those who manage online learning, policies for online education, and managing online programs. In the Zawacki-Richter et al. (2009) study, only a few studies examined management and organization focused topics. They also found management and organization to be strongly correlated with costs and benefits. In our study, costs and benefits were collectively included as an aspect of management and organization and not as a theme by itself. These studies will provide research-based evidence for online education administrators.

6. Limitations

As with any systematic review, there are limitations to the scope of the review. The search is limited to twelve journals in the field that typically include research on online learning. These manuscripts were identified by searching the Education Research Complete database which focuses on education students, professionals, and policymakers. Other discipline-specific journals as well as dissertations and proceedings were not included due to the volume of articles. Also, the search was performed using five search terms “online learning" OR "online teaching" OR "online program" OR "online course" OR “online education” in title and keyword. If authors did not include these terms, their respective work may have been excluded from this review even if it focused on online learning. While these terms are commonly used in North America, it may not be commonly used in other parts of the world. Additional studies may exist outside this scope.

The search strategy also affected how we presented results and introduced limitations regarding generalization. We identified that only 8% of the articles published in these journals were related to online learning; however, given the use of search terms to identify articles within select journals it was not feasible to identify the total number of research-based articles in the population. Furthermore, our review focused on the topics and general methods of research and did not systematically consider the quality of the published research. Lastly, some journals may have preferences for publishing studies on a particular topic or that use a particular method (e.g., quantitative methods), which introduces possible selection and publication biases which may skew the interpretation of results due to over/under representation. Future studies are recommended to include more journals to minimize the selection bias and obtain a more representative sample.

Certain limitations can be attributed to the coding process. Overall, the coding process for this review worked well for most articles, as each tended to have an individual or dominant focus as described in the abstracts, though several did mention other categories which likely were simultaneously considered to a lesser degree. However, in some cases, a dominant theme was not as apparent and an effort to create mutually exclusive groups for clearer interpretation the coders were occasionally forced to choose between two categories. To facilitate this coding, the full-texts were used to identify a study focus through a consensus seeking discussion among all authors. Likewise, some studies focused on topics that we have associated with a particular domain, but the design of the study may have promoted an aggregated examination or integrated factors from multiple domains (e.g., engagement). Due to our reliance on author descriptions, the impact of construct validity is likely a concern that requires additional exploration. Our final grouping of codes may not have aligned with the original author's description in the abstract. Additionally, coding of broader constructs which disproportionately occur in the Learner domain, such as learner outcomes, learner characteristics, and engagement, likely introduced bias towards these codes when considering studies that involved multiple domains. Additional refinement to explore the intersection of domains within studies is needed.

7. Implications and future research

One of the strengths of this review is the research categories we have identified. We hope these categories will support future researchers and identify areas and levels of need for future research. Overall, there is some agreement on research themes on online learning research among previous reviews and this one, at the same time there are some contradicting findings. We hope the most-researched themes and least-researched themes provide authors a direction on the importance of research and areas of need to focus on.

The leading themes found in this review is online engagement research. However, presentation of this research was inconsistent, and often lacked specificity. This is not unique to online environments, but the nuances of defining engagement in an online environment are unique and therefore need further investigation and clarification. This review points to seven distinct classifications of online engagement. Further research on engagement should indicate which type of engagement is sought. This level of specificity is necessary to establish instruments for measuring engagement and ultimately testing frameworks for classifying engagement and promoting it in online environments. Also, it might be of importance to examine the relationship between these seven sub-themes of engagement.

Additionally, this review highlights growing attention to learner characteristics, which constitutes a shift in focus away from instructional characteristics and course design. Although this is consistent with the focus on engagement, the role of the instructor, and course design with respect to these outcomes remains important. Results of the learner characteristics and engagement research paired with course design will have important ramifications for the use of teaching and learning professionals who support instruction. Additionally, the review also points to a concentration of research in the area of higher education. With an immediate and growing emphasis on online learning in K-12 and corporate settings, there is a critical need for further investigation in these settings.

Lastly, because the present review did not focus on the overall effect of interventions, opportunities exist for dedicated meta-analyses. Particular attention to research on engagement and learner characteristics as well as how these vary by study design and outcomes would be logical additions to the research literature.

8. Conclusion

This systematic review builds upon three previous reviews which tackled the topic of online learning between 1990 and 2010 by extending the timeframe to consider the most recent set of published research. Covering the most recent decade, our review of 619 articles from 12 leading online learning journal points to a more concentrated focus on the learner domain including engagement and learner characteristics, with more limited attention to topics pertaining to the classroom or organizational level. The review highlights an opportunity for the field to clarify terminology concerning online learning research, particularly in the areas of learner outcomes where there is a tendency to classify research more generally (e.g., engagement). Using this sample of published literature, we provide a possible taxonomy for categorizing this research using subcategories. The field could benefit from a broader conversation about how these categories can shape a comprehensive framework for online learning research. Such efforts will enable the field to effectively prioritize research aims over time and synthesize effects.

Credit author statement

Florence Martin: Conceptualization; Writing - original draft, Writing - review & editing Preparation, Supervision, Project administration. Ting Sun: Methodology, Formal analysis, Writing - original draft, Writing - review & editing. Carl Westine: Methodology, Formal analysis, Writing - original draft, Writing - review & editing, Supervision

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

1 Includes articles that are cited in this manuscript and also included in the systematic review. The entire list of 619 articles used in the systematic review can be obtained by emailing the authors.*

Appendix B Supplementary data to this article can be found online at https://doi.org/10.1016/j.compedu.2020.104009 .

Appendix A. 

Research Themes by the Settings in the Online Learning Publications

Research Themes by the Methodology in the Online Learning Publications

Appendix B. Supplementary data

The following are the Supplementary data to this article:

References 1

• Ahn J., Butler B.S., Alam A., Webster S.A. Learner participation and engagement in open online courses: Insights from the Peer 2 Peer University. MERLOT Journal of Online Learning and Teaching. 2013; 9 (2):160–171. * [ Google Scholar ]
• Akcaoglu M., Lee E. Increasing social presence in online learning through small group discussions. International Review of Research in Open and Distance Learning. 2016; 17 (3) * [ Google Scholar ]
• Allen I.E., Seaman J. Babson survey research group; 2017. Digital compass learning: Distance education enrollment Report 2017. [ Google Scholar ]
• Amador J.A., Mederer H. Migrating successful student engagement strategies online: Opportunities and challenges using jigsaw groups and problem-based learning. Journal of Online Learning and Teaching. 2013; 9 (1):89. * [ Google Scholar ]
• Anderson L.W., Bourke S.F. Routledge; 2013. Assessing affective characteristics in the schools. [ Google Scholar ]
• Archibald D. Fostering the development of cognitive presence: Initial findings using the community of inquiry survey instrument. The Internet and Higher Education. 2010; 13 (1–2):73–74. * [ Google Scholar ]
• Artino A.R., Jr., Stephens J.M. Academic motivation and self-regulation: A comparative analysis of undergraduate and graduate students learning online. The Internet and Higher Education. 2009; 12 (3–4):146–151. [ Google Scholar ]
• Barnard L., Lan W.Y., To Y.M., Paton V.O., Lai S.L. Measuring self-regulation in online and blended learning environments. Internet and Higher Education. 2009; 12 (1):1–6. * [ Google Scholar ]
• Bayeck R.Y., Hristova A., Jablokow K.W., Bonafini F. Exploring the relevance of single‐gender group formation: What we learn from a massive open online course (MOOC) British Journal of Educational Technology. 2018; 49 (1):88–100. * [ Google Scholar ]
• Berge Z., Mrozowski S. Review of research in distance education, 1990 to 1999. American Journal of Distance Education. 2001; 15 (3):5–19. doi: 10.1080/08923640109527090. [ CrossRef ] [ Google Scholar ]
• Berry S. Building community in online doctoral classrooms: Instructor practices that support community. Online Learning. 2017; 21 (2):n2. * [ Google Scholar ]
• Boling E.C., Holan E., Horbatt B., Hough M., Jean-Louis J., Khurana C., Spiezio C. Using online tools for communication and collaboration: Understanding educators' experiences in an online course. The Internet and Higher Education. 2014; 23 :48–55. * [ Google Scholar ]
• Bolliger D.U., Inan F.A. Development and validation of the online student connectedness survey (OSCS) International Review of Research in Open and Distance Learning. 2012; 13 (3):41–65. * [ Google Scholar ]
• Bradford G., Wyatt S. Online learning and student satisfaction: Academic standing, ethnicity and their influence on facilitated learning, engagement, and information fluency. The Internet and Higher Education. 2010; 13 (3):108–114. * [ Google Scholar ]
• Broadbent J. Comparing online and blended learner's self-regulated learning strategies and academic performance. The Internet and Higher Education. 2017; 33 :24–32. [ Google Scholar ]
• Buzdar M., Ali A., Tariq R. Emotional intelligence as a determinant of readiness for online learning. International Review of Research in Open and Distance Learning. 2016; 17 (1) * [ Google Scholar ]
• Capdeferro N., Romero M., Barberà E. Polychronicity: Review of the literature and a new configuration for the study of this hidden dimension of online learning. Distance Education. 2014; 35 (3):294–310. [ Google Scholar ]
• Chaiprasurt C., Esichaikul V. Enhancing motivation in online courses with mobile communication tool support: A comparative study. International Review of Research in Open and Distance Learning. 2013; 14 (3):377–401. [ Google Scholar ]
• Chen C.H., Wu I.C. The interplay between cognitive and motivational variables in a supportive online learning system for secondary physical education. Computers & Education. 2012; 58 (1):542–550. * [ Google Scholar ]
• Cho H. Under co-construction: An online community of practice for bilingual pre-service teachers. Computers & Education. 2016; 92 :76–89. * [ Google Scholar ]
• Cho M.H., Shen D. Self-regulation in online learning. Distance Education. 2013; 34 (3):290–301. [ Google Scholar ]
• Cole M.T., Shelley D.J., Swartz L.B. Online instruction, e-learning, and student satisfaction: A three-year study. International Review of Research in Open and Distance Learning. 2014; 15 (6) * [ Google Scholar ]
• Comer D.K., Clark C.R., Canelas D.A. Writing to learn and learning to write across the disciplines: Peer-to-peer writing in introductory-level MOOCs. International Review of Research in Open and Distance Learning. 2014; 15 (5):26–82. * [ Google Scholar ]
• Cundell A., Sheepy E. Student perceptions of the most effective and engaging online learning activities in a blended graduate seminar. Online Learning. 2018; 22 (3):87–102. * [ Google Scholar ]
• Cung B., Xu D., Eichhorn S. Increasing interpersonal interactions in an online course: Does increased instructor email activity and voluntary meeting time in a physical classroom facilitate student learning? Online Learning. 2018; 22 (3):193–215. [ Google Scholar ]
• Cunningham U.M., Fägersten K.B., Holmsten E. Can you hear me, Hanoi?" Compensatory mechanisms employed in synchronous net-based English language learning. International Review of Research in Open and Distance Learning. 2010; 11 (1):161–177. [ Google Scholar ]
• Davis D., Chen G., Hauff C., Houben G.J. Activating learning at scale: A review of innovations in online learning strategies. Computers & Education. 2018; 125 :327–344. [ Google Scholar ]
• Delen E., Liew J., Willson V. Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments. Computers & Education. 2014; 78 :312–320. [ Google Scholar ]
• Dixson M.D. Measuring student engagement in the online course: The Online Student Engagement scale (OSE) Online Learning. 2015; 19 (4):n4. * [ Google Scholar ]
• Dray B.J., Lowenthal P.R., Miszkiewicz M.J., Ruiz‐Primo M.A., Marczynski K. Developing an instrument to assess student readiness for online learning: A validation study. Distance Education. 2011; 32 (1):29–47. * [ Google Scholar ]
• Dziuban C., Moskal P., Thompson J., Kramer L., DeCantis G., Hermsdorfer A. Student satisfaction with online learning: Is it a psychological contract? Online Learning. 2015; 19 (2):n2. * [ Google Scholar ]
• Ergün E., Usluel Y.K. An analysis of density and degree-centrality according to the social networking structure formed in an online learning environment. Journal of Educational Technology & Society. 2016; 19 (4):34–46. * [ Google Scholar ]
• Esfijani A. Measuring quality in online education: A meta-synthesis. American Journal of Distance Education. 2018; 32 (1):57–73. [ Google Scholar ]
• Glazer H.R., Murphy J.A. Optimizing success: A model for persistence in online education. American Journal of Distance Education. 2015; 29 (2):135–144. [ Google Scholar ]
• Glazer H.R., Wanstreet C.E. Connection to the academic community: Perceptions of students in online education. Quarterly Review of Distance Education. 2011; 12 (1):55. * [ Google Scholar ]
• Hartnett M., George A.S., Dron J. Examining motivation in online distance learning environments: Complex, multifaceted and situation-dependent. International Review of Research in Open and Distance Learning. 2011; 12 (6):20–38. [ Google Scholar ]
• Harwell M.R. 2012. Research design in qualitative/quantitative/mixed methods. Section III. Opportunities and challenges in designing and conducting inquiry. [ Google Scholar ]
• Hung J.L. Trends of e‐learning research from 2000 to 2008: Use of text mining and bibliometrics. British Journal of Educational Technology. 2012; 43 (1):5–16. [ Google Scholar ]
• Jiang W. Interdependence of roles, role rotation, and sense of community in an online course. Distance Education. 2017; 38 (1):84–105. [ Google Scholar ]
• Ke F., Kwak D. Online learning across ethnicity and age: A study on learning interaction participation, perception, and learning satisfaction. Computers & Education. 2013; 61 :43–51. [ Google Scholar ]
• Kent M. Changing the conversation: Facebook as a venue for online class discussion in higher education. MERLOT Journal of Online Learning and Teaching. 2013; 9 (4):546–565. * [ Google Scholar ]
• Kim C., Park S.W., Cozart J. Affective and motivational factors of learning in online mathematics courses. British Journal of Educational Technology. 2014; 45 (1):171–185. [ Google Scholar ]
• Kizilcec R.F., Pérez-Sanagustín M., Maldonado J.J. Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Computers & Education. 2017; 104 :18–33. [ Google Scholar ]
• Kopp B., Matteucci M.C., Tomasetto C. E-tutorial support for collaborative online learning: An explorative study on experienced and inexperienced e-tutors. Computers & Education. 2012; 58 (1):12–20. [ Google Scholar ]
• Koseoglu S., Doering A. Understanding complex ecologies: An investigation of student experiences in adventure learning programs. Distance Education. 2011; 32 (3):339–355. * [ Google Scholar ]
• Kumi-Yeboah A. Designing a cross-cultural collaborative online learning framework for online instructors. Online Learning. 2018; 22 (4):181–201. * [ Google Scholar ]
• Kuo Y.C., Walker A.E., Belland B.R., Schroder K.E. A predictive study of student satisfaction in online education programs. International Review of Research in Open and Distance Learning. 2013; 14 (1):16–39. * [ Google Scholar ]
• Kuo Y.C., Walker A.E., Schroder K.E., Belland B.R. Interaction, Internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. Internet and Higher Education. 2014; 20 :35–50. * [ Google Scholar ]
• Lee J. An exploratory study of effective online learning: Assessing satisfaction levels of graduate students of mathematics education associated with human and design factors of an online course. International Review of Research in Open and Distance Learning. 2014; 15 (1) [ Google Scholar ]
• Lee S.M. The relationships between higher order thinking skills, cognitive density, and social presence in online learning. The Internet and Higher Education. 2014; 21 :41–52. * [ Google Scholar ]
• Lee K. Rethinking the accessibility of online higher education: A historical review. The Internet and Higher Education. 2017; 33 :15–23. [ Google Scholar ]
• Lee Y., Choi J. A review of online course dropout research: Implications for practice and future research. Educational Technology Research & Development. 2011; 59 (5):593–618. [ Google Scholar ]
• Li L.Y., Tsai C.C. Accessing online learning material: Quantitative behavior patterns and their effects on motivation and learning performance. Computers & Education. 2017; 114 :286–297. [ Google Scholar ]
• Liyanagunawardena T., Adams A., Williams S. MOOCs: A systematic study of the published literature 2008-2012. International Review of Research in Open and Distance Learning. 2013; 14 (3):202–227. [ Google Scholar ]
• Lowes S., Lin P., Kinghorn B.R. Gender differences in online high school courses. Online Learning. 2016; 20 (4):100–117. [ Google Scholar ]
• Marbouti F., Wise A.F. Starburst: A new graphical interface to support purposeful attention to others' posts in online discussions. Educational Technology Research & Development. 2016; 64 (1):87–113. * [ Google Scholar ]
• Martin F., Ahlgrim-Delzell L., Budhrani K. Systematic review of two decades (1995 to 2014) of research on synchronous online learning. American Journal of Distance Education. 2017; 31 (1):3–19. [ Google Scholar ]
• Moore-Adams B.L., Jones W.M., Cohen J. Learning to teach online: A systematic review of the literature on K-12 teacher preparation for teaching online. Distance Education. 2016; 37 (3):333–348. [ Google Scholar ]
• Murphy E., Rodríguez-Manzanares M.A. Rapport in distance education. International Review of Research in Open and Distance Learning. 2012; 13 (1):167–190. * [ Google Scholar ]
• Nye A. Building an online academic learning community among undergraduate students. Distance Education. 2015; 36 (1):115–128. * [ Google Scholar ]
• Olesova L., Slavin M., Lim J. Exploring the effect of scripted roles on cognitive presence in asynchronous online discussions. Online Learning. 2016; 20 (4):34–53. * [ Google Scholar ]
• Orcutt J.M., Dringus L.P. Beyond being there: Practices that establish presence, engage students and influence intellectual curiosity in a structured online learning environment. Online Learning. 2017; 21 (3):15–35. * [ Google Scholar ]
• Overbaugh R.C., Nickel C.E. A comparison of student satisfaction and value of academic community between blended and online sections of a university-level educational foundations course. The Internet and Higher Education. 2011; 14 (3):164–174. * [ Google Scholar ]
• O'Shea S., Stone C., Delahunty J. “I ‘feel’like I am at university even though I am online.” Exploring how students narrate their engagement with higher education institutions in an online learning environment. Distance Education. 2015; 36 (1):41–58. * [ Google Scholar ]
• Paechter M., Maier B. Online or face-to-face? Students' experiences and preferences in e-learning. Internet and Higher Education. 2010; 13 (4):292–297. [ Google Scholar ]
• Phirangee K. Students' perceptions of learner-learner interactions that weaken a sense of community in an online learning environment. Online Learning. 2016; 20 (4):13–33. * [ Google Scholar ]
• Phirangee K., Malec A. Othering in online learning: An examination of social presence, identity, and sense of community. Distance Education. 2017; 38 (2):160–172. * [ Google Scholar ]
• Preisman K.A. Teaching presence in online education: From the instructor's point of view. Online Learning. 2014; 18 (3):n3. * [ Google Scholar ]
• Rowe M. Developing graduate attributes in an open online course. British Journal of Educational Technology. 2016; 47 (5):873–882. * [ Google Scholar ]
• Ruane R., Koku E.F. Social network analysis of undergraduate education student interaction in online peer mentoring settings. Journal of Online Learning and Teaching. 2014; 10 (4):577–589. * [ Google Scholar ]
• Ruane R., Lee V.J. Analysis of discussion board interaction in an online peer mentoring site. Online Learning. 2016; 20 (4):79–99. * [ Google Scholar ]
• Rye S.A., Støkken A.M. The implications of the local context in global virtual education. International Review of Research in Open and Distance Learning. 2012; 13 (1):191–206. * [ Google Scholar ]
• Saadatmand M., Kumpulainen K. Participants' perceptions of learning and networking in connectivist MOOCs. Journal of Online Learning and Teaching. 2014; 10 (1):16. * [ Google Scholar ]
• Shackelford J.L., Maxwell M. Sense of community in graduate online education: Contribution of learner to learner interaction. International Review of Research in Open and Distance Learning. 2012; 13 (4):228–249. * [ Google Scholar ]
• Shea P., Bidjerano T. Does online learning impede degree completion? A national study of community college students. Computers & Education. 2014; 75 :103–111. * [ Google Scholar ]
• Sherry L. Issues in distance learning. International Journal of Educational Telecommunications. 1996; 1 (4):337–365. [ Google Scholar ]
• Slagter van Tryon P.J., Bishop M.J. Evaluating social connectedness online: The design and development of the social perceptions in learning contexts instrument. Distance Education. 2012; 33 (3):347–364. * [ Google Scholar ]
• Swaggerty E.A., Broemmel A.D. Authenticity, relevance, and connectedness: Graduate students' learning preferences and experiences in an online reading education course. The Internet and Higher Education. 2017; 32 :80–86. * [ Google Scholar ]
• Tallent-Runnels M.K., Thomas J.A., Lan W.Y., Cooper S., Ahern T.C., Shaw S.M., Liu X. Teaching courses online: A review of the research. Review of Educational Research. 2006; 76 (1):93–135. doi: 10.3102/00346543076001093. [ CrossRef ] [ Google Scholar ]
• Tawfik A.A., Giabbanelli P.J., Hogan M., Msilu F., Gill A., York C.S. Effects of success v failure cases on learner-learner interaction. Computers & Education. 2018; 118 :120–132. [ Google Scholar ]
• Thomas J. Exploring the use of asynchronous online discussion in health care education: A literature review. Computers & Education. 2013; 69 :199–215. [ Google Scholar ]
• Thormann J., Fidalgo P. Guidelines for online course moderation and community building from a student's perspective. Journal of Online Learning and Teaching. 2014; 10 (3):374–388. * [ Google Scholar ]
• Tibi M.H. Computer science students' attitudes towards the use of structured and unstructured discussion forums in fully online courses. Online Learning. 2018; 22 (1):93–106. * [ Google Scholar ]
• Tsai C.W., Chiang Y.C. Research trends in problem‐based learning (pbl) research in e‐learning and online education environments: A review of publications in SSCI‐indexed journals from 2004 to 2012. British Journal of Educational Technology. 2013; 44 (6):E185–E190. [ Google Scholar ]
• Tsai C.W., Fan Y.T. Research trends in game‐based learning research in online learning environments: A review of studies published in SSCI‐indexed journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (5):E115–E119. [ Google Scholar ]
• Tsai C.W., Shen P.D., Chiang Y.C. Research trends in meaningful learning research on e‐learning and online education environments: A review of studies published in SSCI‐indexed journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (6):E179–E184. [ Google Scholar ]
• Tsai C.W., Shen P.D., Fan Y.T. Research trends in self‐regulated learning research in online learning environments: A review of studies published in selected journals from 2003 to 2012. British Journal of Educational Technology. 2013; 44 (5):E107–E110. [ Google Scholar ]
• U.S. Department of Education, Institute of Education Sciences . InstituteofEducationSciences; Washington,DC: 2017. What Works Clearinghouse procedures and standards handbook, version3.0. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/wwc_procedures_v3_0_standards_handbook.pdf Retrievedfrom. [ Google Scholar ]
• Veletsianos G., Shepherdson P. A systematic analysis and synthesis of the empirical MOOC literature published in 2013–2015. International Review of Research in Open and Distance Learning. 2016; 17 (2) [ Google Scholar ]
• VERBI Software . 2019. MAXQDA 2020 online manual. Retrieved from maxqda. Com/help-max20/welcome [ Google Scholar ]
• Verstegen D., Dailey-Hebert A., Fonteijn H., Clarebout G., Spruijt A. How do virtual teams collaborate in online learning tasks in a MOOC? International Review of Research in Open and Distance Learning. 2018; 19 (4) * [ Google Scholar ]
• Wang Y., Baker R. Grit and intention: Why do learners complete MOOCs? International Review of Research in Open and Distance Learning. 2018; 19 (3) * [ Google Scholar ]
• Wei C.W., Chen N.S., Kinshuk A model for social presence in online classrooms. Educational Technology Research & Development. 2012; 60 (3):529–545. * [ Google Scholar ]
• Wicks D., Craft B.B., Lee D., Lumpe A., Henrikson R., Baliram N., Wicks K. An evaluation of low versus high collaboration in online learning. Online Learning. 2015; 19 (4):n4. * [ Google Scholar ]
• Wise A.F., Perera N., Hsiao Y.T., Speer J., Marbouti F. Microanalytic case studies of individual participation patterns in an asynchronous online discussion in an undergraduate blended course. The Internet and Higher Education. 2012; 15 (2):108–117. * [ Google Scholar ]
• Wisneski J.E., Ozogul G., Bichelmeyer B.A. Does teaching presence transfer between MBA teaching environments? A comparative investigation of instructional design practices associated with teaching presence. The Internet and Higher Education. 2015; 25 :18–27. * [ Google Scholar ]
• Wladis C., Hachey A.C., Conway K. An investigation of course-level factors as predictors of online STEM course outcomes. Computers & Education. 2014; 77 :145–150. * [ Google Scholar ]
• Wladis C., Samuels J. Do online readiness surveys do what they claim? Validity, reliability, and subsequent student enrollment decisions. Computers & Education. 2016; 98 :39–56. [ Google Scholar ]
• Yamagata-Lynch L.C. Blending online asynchronous and synchronous learning. International Review of Research in Open and Distance Learning. 2014; 15 (2) * [ Google Scholar ]
• Yang J., Kinshuk, Yu H., Chen S.J., Huang R. Strategies for smooth and effective cross-cultural online collaborative learning. Journal of Educational Technology & Society. 2014; 17 (3):208–221. * [ Google Scholar ]
• Yeboah A.K., Smith P. Relationships between minority students online learning experiences and academic performance. Online Learning. 2016; 20 (4):n4. * [ Google Scholar ]
• Yu T. Examining construct validity of the student online learning readiness (SOLR) instrument using confirmatory factor analysis. Online Learning. 2018; 22 (4):277–288. * [ Google Scholar ]
• Yukselturk E., Bulut S. Gender differences in self-regulated online learning environment. Educational Technology & Society. 2009; 12 (3):12–22. [ Google Scholar ]
• Yukselturk E., Top E. Exploring the link among entry characteristics, participation behaviors and course outcomes of online learners: An examination of learner profile using cluster analysis. British Journal of Educational Technology. 2013; 44 (5):716–728. [ Google Scholar ]
• Zawacki-Richter O., Backer E., Vogt S. Review of distance education research (2000 to 2008): Analysis of research areas, methods, and authorship patterns. International Review of Research in Open and Distance Learning. 2009; 10 (6):30. doi: 10.19173/irrodl.v10i6.741. [ CrossRef ] [ Google Scholar ]
• Zhu M., Sari A., Lee M.M. A systematic review of research methods and topics of the empirical MOOC literature (2014–2016) The Internet and Higher Education. 2018; 37 :31–39. [ Google Scholar ]
• Zimmerman T.D. Exploring learner to content interaction as a success factor in online courses. International Review of Research in Open and Distance Learning. 2012; 13 (4):152–165. [ Google Scholar ]

EdTakeAways

How effective is online learning what the research does and doesn't tell us.

By Susanna Loeb

Students who struggle will likely struggle more online

The times have dictated school closings and the rapid expansion of online education. Can online lessons replace in-school time?

Clearly online time cannot provide many of the informal social interactions students have at school, but how will online courses do in terms of moving student learning forward? Research to date gives us some clues and also points us to what we could be doing to support students who are most likely to struggle in the online setting.

The use of virtual courses among K-12 students has grown rapidly in recent years. Florida, for example, requires all high school students to take at least one online course. Online learning can take a number of different forms. Often people think of Massive Open Online Courses, or MOOCs, where thousands of students watch a video online and fill out questionnaires or take exams based on those lectures.

Most online courses, however, particularly those serving K-12 students, have a format much more similar to in-person courses. The teacher helps to run virtual discussion among the students, assigns homework, and follows up with individual students. Sometimes these courses are synchronous (teachers and students all meet at the same time) and sometimes they are asynchronous (non-concurrent). In both cases, the teacher is supposed to provide opportunities for students to engage thoughtfully with subject matter, and students, in most cases, are required to interact with each other virtually.

Online courses provide opportunities for students. Students in a school that doesn’t offer statistics classes may be able to learn statistics with virtual lessons. If students fail algebra, they may be able to catch up during evenings or summer using online classes, and not disrupt their math trajectory at school. So, almost certainly, online classes sometimes benefit students.

In comparisons of online and in-person classes, however, online classes aren’t as effective as in-person classes for most students. Only a little research has assessed the effects of online lessons for elementary and high school students, and even less has used the “gold standard” method of comparing the results for students assigned randomly to online or in-person courses.  Jessica Heppen and colleagues  at the American Institutes for Research and the University of Chicago Consortium on School Research randomly assigned students who had failed second semester Algebra I to either face-to-face or online credit recovery courses over the summer. Students’ credit-recovery success rates and algebra test scores were lower in the online setting. Students assigned to the online option also rated their class as more difficult than did their peers assigned to the face-to-face option.

Most of the research on online courses for K-12 students has used large-scale administrative data, looking at otherwise similar students in the two settings. One of these studies, by  June Ahn  of New York University and  Andrew McEachin  of the RAND Corp., examined Ohio charter schools; I did another with colleagues looking at Florida public school coursework. Both studies found evidence that online coursetaking was less effective.

It is not surprising that in-person courses are, on average, more effective. Being in person with teachers and other students creates social pressures and benefits that can help motivate students to engage. Some students do as well in online courses as in in-person courses, some may actually do better, but, on average, students do worse in the online setting, and this is particularly true for students with weaker academic backgrounds.

Students who struggle in in-person classes are likely to struggle even more online. While the research on virtual schools in K-12 education doesn’t address these differences directly, a study of college students that I worked on with Stanford colleagues found very little difference in learning for high-performing students in the online and in-person settings. On the other hand, lower performing students performed meaningfully worse in online courses than in in-person courses.

But just because students who struggle in in-person classes are even more likely to struggle online doesn’t mean that’s inevitable. Online teachers will need to consider the needs of less-engaged students and work to engage them. Online courses might be made to work for these students on average, even if they have not in the past.

Just like in brick-and-mortar classrooms, online courses need a strong curriculum and strong pedagogical practices. Teachers need to understand what students know and what they don’t know, as well as how to help them learn new material. What is different in the online setting is that students may have more distractions and less oversight, which can reduce their motivation. The teacher will need to set norms for engagement—such as requiring students to regularly ask questions and respond to their peers—that are different than the norms in the in-person setting.

Online courses are generally not as effective as in-person classes, but they are certainly better than no classes. A substantial research base developed by Karl Alexander at Johns Hopkins University and many others shows that students, especially students with fewer resources at home, learn less when they are not in school. Right now, virtual courses are allowing students to access lessons and exercises and interact with teachers in ways that would have been impossible if an epidemic had closed schools even a decade or two earlier. So we may be skeptical of online learning, but it is also time to embrace and improve it.

Susanna Loeb is a professor of education and of public affairs at Brown University and the director of the university's Annenberg Institute for School Reform. She studies education policy, and her interests include social inequality.

Researcher Academy provides free access to countless e-learning resources designed to support researchers on every step of their research journey. Browse our extensive module catalogue to uncover a world of knowledge, and earn certificates and rewards as you progress.

Get started

Research preparation

• Research data management
• Research collaborations
• Research design
• Sex and Gender Equity in Research (SAGER) Guidelines

Writing for research

• Fundamentals of manuscript preparation
• Writing skills
• Technical writing skills

Publication process

• Fundamentals of publishing
• Finding the right journal
• Open science
• Publishing in the Chemical Sciences
• How to publish in premium journals
• Research metrics

Navigating peer review

• Fundamentals of peer review
• Becoming a peer reviewer
• Going through peer review
• Certified Peer Reviewer Course

Communicating your research

• Social impact
• Ensuring visibility
• Inclusion and Diversity for Researchers

Shorten your path to publication: Researcher insights from submission to rejection

What is plagiarism and how to avoid it

Trusted publishing vs. predatory journals: What does it mean for the integrity of science?

Promises and pitfalls of online education

• Download the report

Subscribe to the Center for Economic Security and Opportunity Newsletter

Eric bettinger and eric bettinger associate professor of economics of education - stanford graduate school of education, research associate - national bureau of economic research susanna loeb susanna loeb professor and faculty director - scale initiative, stanford university's graduate school of education, founder and executive director - national student support accelerator.

June 9, 2017

• 12 min read

Executive Summary

Online courses have expanded rapidly and have the potential to extend further the educational opportunities of many students, particularly those least well-served by traditional educational institutions. However, in their current design, online courses are difficult, especially for the students who are least prepared. These students’ learning and persistence outcomes are worse when they take online courses than they would have been had these same students taken in-person courses. Continued improvement of online curricula and instruction can strengthen the quality of these courses and hence the educational opportunities for the most in-need populations.

Online courses offer the promise of access regardless of where students live or what time they can participate, potentially redefining educational opportunities for those least well-served in traditional classrooms. Moreover, online platforms offer the promise, through artificial intelligence, of providing the optimal course pacing and content to fit each student’s needs and thereby improve educational quality and learning. The latest “intelligent” tutoring systems, for example, not only assess students’ current weaknesses, but also diagnose why students make the specific errors. These systems then adjust instructional materials to meet students’ needs. 1

Yet today these promises are far from fully realized. The vast majority of online courses mirror face-to-face classrooms with professors rather using technology to better differentiate instruction across students. As one new study that we completed with our colleagues Lindsay Fox and Eric Taylor shows, online courses can improve access, yet they also are challenging, especially for the least well-prepared students. These students consistently perform worse in an online setting than they do in face-to-face classrooms; taking online courses increases their likelihood of dropping out and otherwise impedes progress through college. 2

Online college courses are rapidly growing. One out of three college students now takes at least one course online during their college career, and that share has increased threefold over the past decade. 3  The potential for cost savings and the ease of scaling fuels ongoing investments in online education by both public and private institutions. 4  Online courses have grown in the K-12 sector as well. Florida, for example, requires each high school student to take at least one online course before graduation and the Florida Virtual School offers over 150 classes to students across the state. 5  An estimated 1.5 million K-12 students participated in some online learning in 2010, 6  and online learning enrollments are projected to grow in future years. 7

Non-selective and for-profit higher education institutions have expanded online course offerings particularly quickly. These institutions serve a majority of college-aged students, and these students typically have weaker academic preparation and fewer economic resources than students at other more selective colleges and universities. As such, their ability to provide useful course work, engage students, and build the skills necessary for economic success is particularly important. Their use of online coursework is promising to the extent that it can reach the most students in need and serve them well.

While online course-taking is both prevalent and growing, especially in non-selective higher education institutions, relatively little evidence has examined how taking a course online instead of in person affects student success in college. Our new study is the first of which we are aware to provide evidence on the effects of online courses at-scale at non-selective four-year colleges. It is also the first to assess the effects of online course taking at for-profit institutions. Nearly 2.4 million undergraduate students (full-time equivalent) enrolled at for-profit institutions during the 2011-12 academic year, and the sector granted approximately 18 percent of all associate degrees.

Our study uses data from DeVry University, a large for-profit college with an undergraduate enrollment of more than 100,000 students, 80 percent of whom are seeking a bachelor’s degree. The average DeVry student takes two-thirds of her courses online. The remaining one-third of courses meet in conventional in-person classes held at one of DeVry’s 102 physical campuses. The data include over 230,000 students enrolled in 168,000 sections of more than 750 different courses.

DeVry University’s approach to online education makes it particularly well suited for estimating the effects of taking online courses. Each DeVry course is offered both online and in-person, and each student enrolls in either an online section or an in-person section. Online and in-person sections are identical in most ways: both follow the same syllabus and use the same textbook; class sizes are approximately the same; both use the same assignments, quizzes, tests, and grading rubrics. Many professors teach both online and in-person courses. The contrast between online and in-person sections is primarily the mode of communication. In online sections, all interaction—lecturing, class discussion, group projects—occurs in online discussion boards, and much of the professor’s “lecturing” role is replaced with standardized videos. In online sections, participation is often asynchronous while in-person sections meet on campus at scheduled times. In short, DeVry online classes attempt to replicate traditional in-person classes, except that student-student and student-professor interactions are virtual and asynchronous.

Using variation in course-taking that arises both from changes in course offerings at particular campuses in a particular term and from variation across students in the distance that they have to travel to take in-person courses, we find that taking a course online reduces student grades by 0.44 points on the traditional four-point grading scale, approximately a 0.33 standard deviation decline relative to taking a course in-person (See Figure 1). To be more concrete, students taking the course in-person earned roughly a B- grade (2.8) on average while if they had taken it online, they would have earned a C (2.4). Additionally, taking a course online reduces a student’s GPA the following term by 0.15 points; and, if we look only at the next term GPA for courses in the same subject area or courses for which the course in question is a pre-requisite, we find larger drops of 0.42 points and 0.32 points respectively, providing evidence that students learned less in the online setting.

We also find that taking a course online, instead of in person, increases the probability that a student will drop out of school. In the semester after taking an online course, students are about 9 percentage points less likely to remain enrolled. This reduction is relative to an average of 88 percent of students remaining enrolled in the following term. Moreover, taking a course online reduces the number of credits that students who do reenroll take in future semesters. While this setting is quite different, we can compare the effects on online course taking to other estimates of effects of on college persistence. For example, the literature on financial aid often finds that $1000 in financial aid increases persistence rates by about three percentage points 8  and college mentorship increases persistence rates by five percentage points. 9

The negative effects of online course taking are concentrated in the lowest performing students. As shown in Figure 2, for students with below median prior GPA, the online classes reduce grades by 0.5 points or more, while for students with prior GPA in the top three deciles we estimate the effect as much smaller and, in fact, we cannot tell whether there is negative effect at all for this higher-achieving group. Thus, while online courses may have the potential to differentiate coursework to meet the needs of students with weaker incoming skills, current online courses, in fact, do an even worse job of meeting the needs of these students than do traditional in-person courses.

These analyses provide evidence that students in online courses perform substantially worse than students in traditional in-person courses and that experience in these online courses impact performance in future classes and their likelihood of dropping out of college as well. The negative effects of online course-taking are far stronger for students with lower prior GPA. The results are in line with prior studies of online education in other settings such as community colleges and highly competitive four-year institutions that also show that online courses yield worse average outcomes than in-person courses. 10

The current negative effect of online course taking relative to in-person course taking should not necessarily lead to the conclusion that online courses should be discouraged. On the contrary, online courses provide access to students who never would have the opportunity or inclination to take classes in-person. 11  As one indication, of the 5.8 million students taking online courses in the fall of 2014, 2.85 million took all of their courses online. 12  Moreover, advances in AI offer hope that future online courses can respond to the needs of students, meeting them where they are in their learning and engaging them in higher education even better than in-person courses are currently able to do. 13 Nonetheless, the tremendous scale and consistently negative effects of current offerings points to the need to improve these courses, particularly for students most at risk of course failure and college dropout.

The authors did not receive financial support from any firm or person with a financial or political interest in this article. They are currently not officers, directors, or board members of any organization with an interest in this article.

• Graesser, Arthur C., Mark W. Conley, and Andrew Olney. 2012. “Intelligent tutoring systems.” In APA Educational Psychology Handbook, Vol. 3: Application to Learning and Teaching , edited by Karen. R. Harris, Steve Graham, and Tim Urdan. Washington, DC: American Psychological Association.
• Bettinger, E., Fox, L., Loeb, S., & Taylor, E. (Forthcoming). Changing Distributions: How Online College Classes Alter Student and Professor Performance. American Economic Review .
• Allen, I. Elaine, and Jeff Seaman. 2013. Changing Course: Ten Years of Tracking Online Education in the United States. Newburyport, MA: Sloan Consortium.
• Deming, David J., Claudia Goldin, Lawrence F. Katz, and Noam Yuchtman. 2015. Can Online Learning Bend the Higher Education Cost Curve? American Economic Review, Papers & Proceedings, 105 (5):496-501.
• Jacob, B., Berger, D. Hart, C. & Loeb, S. (Forthcoming). “Can Technology Help Promote Equality of Educational Opportunities?” In K. Alexander and S. Morgan (Editors),  The Coleman Report and Educational Inequality Fifty Years Later.  Russell Sage Foundation and William T. Grant Foundation: New York.
• Wicks, Matthew. 2010. “A National Primer on K-12 Online Learning. Version 2.” Vienna, VA: International Association for K-12 Online Learning.
• Watson, John, Amy Murin, Lauren Vashaw, Butch Gemin, and Chris Rapp. 2012. “Keeping Pace with K-12 Online Learning: An Annual Review of Policy and Practice 2011.” Durango, CO: Evergreen Education Group. And Picciano, Anthony G., Jeff Seaman, Peter Shea, and Karen Swan. 2012. “Examining the Extent and Nature of Online Learning in American K-12 Education: The Research Initiatives of the Alfred P. Sloan Foundation.” The Internet and Higher Education 15(2): 127-35.
• Bettinger, Eric P. 2004. “How Financial Aid Affects Persistence.” In Caroline Hoxby (Ed.), College Choices: The Economics of Where to Go, When to Go, and How to Pay for It . University of Chicago Press.
• Bettinger, Eric P., and Rachel B. Baker. 2013. “The Effects of Student Coaching: An Evaluation of a Randomized Experiment in Student Advising.” Educational Evaluation and Policy Analysis, 36 (1):3-19.
• See for examples: Figlio, David, Mark Rush, and Lu Yin. 2013. “Is It Live or Is It Internet? Experimental Estimates of the Effects of Online Instruction on Student Learning.” Journal of Labor Economics, 31 (4):763-784; Couch, Kenneth A., William T. Alpert, and Oskar R. Harmon. 2014. “Online, Blended and Classroom Teaching of Economics Principles: A Randomized Experiment.” University of Connecticut Working Paper; Xu, Di, and Shanna Smith Jaggars. 2014. “Performance Gaps Between Online and Face-to-Face Courses: Differences Across Types of Students and Academic Subject Areas.” The Journal of Higher Education, 85 (5):633-659; Hart, Cassandra, Elizabeth Friedmann, and Michael Hill. 2014. “Online Course-Taking and Student Outcomes in California Community Colleges.” Working Paper; and Streich, Francie E. 2014. “Online and Hybrid Instruction and Student Success in College: Evidence from Community Colleges in Two States.” University of Michigan Working Paper.
• See, for example, Joshua Goodman, Julia Melkers, and Amanda Pallais, “ Can Online Delivery Increase Access to Education? ” National Bureau of Economic Research working paper 22754, October 2016.
• Online Report Card – Tracking Online Education in the United States , the 2015 Survey of Online Learning conducted by the Babson Survey Research Group and co-sponsored by the Online Learning Consortium (OLC), Pearson, StudyPortals, WCET and Tyton Partners.
• See, for example, the Open Learning Initiative at Carnegie Mellon University.

Education Technology Higher Education

Economic Studies

Center for Economic Security and Opportunity

September 24, 2024

Michael Trucano, Sopiko Beriashvili

September 20, 2024

Megan Kuhfeld, Karyn Lewis

Login to your account

Password reset.

• Board Members
• Management Team
• Become a Contributor
• Volunteer Opportunities
• Code of Ethical Practices

KNOWLEDGE NETWORK

• Search Engines List
• Suggested Reading Library
• Web Directories
• Research Papers
• Industry News

• Become a Member
• Associate Membership
• Certified Membership
• Membership Application
• Corporate Application

• CIRS Certification Program
• CIRS Certification Objectives
• CIRS Certification Benefits
• CIRS Certification Exam
• Maintain Your Certification

• Upcoming Events
• Live Classes
• Classes Schedule
• Webinars Schedules

Education Program

Gold standard of professional online research , aofirs is the world's leading provider for imparting internet research education & training, leading to their credentialing. our programs are accessible online 24x7 in over 80 countries....

• CIRS Certification Syllabus
• CIRS Certification FAQ

Professional Research Certification

The association's education system integrates professional research training programs tailored for knowledge professionals and information users who prefer to conduct their research online. Our certifications aim to educate and train online researchers, a growing group of individuals engaged in various online research activities. In the absence of other formal training in this field, the Association of Internet Research Specialists (AOFIRS) offers the most sought-after career certification: the Certified Internet Research Specialist (CIRS). This certification is designed for individuals pursuing professional online research careers.

Additionally, we now offer a training program focused on Online Investigative Research, catering to those who need to utilize the Internet for investigative purposes and information verification. The association also provides short courses with certificates of course completion, known as Certificates of Proficiency (COP). You may write to [email protected] for more details on these specialized and custom training requirements.

Certified Internet Research Specialist (CIRS)

Professional Certification Program 

The Certified Internet Research Specialist (CIRS) program is a pioneering professional certification for online research practitioners, regarded as the Gold Standard in the field. The course is designed to teach you how to perform professional online research for serious tasks. As a certified specialist, you will become adept at selecting not only the most appropriate search tools and proficient in building efficient Internet search queries but know how to process data and information and understand the legalities surrounding your research.

The CIRS learning program is developed as a Self-study training and an Online Instructor Led training program to help candidates prepare for the CIRS examination. The training guides and education material is available as a comprehensive set of guidebooks to serve as a practical reference for anyone using the Internet as their primary tool for information gathering.

The CIRS program is available in two formats: a self-study course and online instructor-led training classes, making it accessible to a wide range of learners.

Self-study Training Course

Self-study training courses are intended for autonomous learning, enabling students to acquire knowledge and skills without a real-time instructor or a traditional classroom environment. These courses are perfect for people with hectic or unpredictable schedules because they allow students to study at convenient times and at their speed. Students may take charge of their education by using an online platform to access course materials, including videos, online modules, textbooks, quizzes, and practice questions. One of the main characteristics of self-study courses is their accessibility; they are frequently offered online so students can access them from any location with an internet connection. Convenience, affordability, individualized learning opportunities, and the growth of self-discipline via time management and self-motivation are some advantages of self-study. These courses offer access to subjects and specializations that might not be provided nearby, along with extra assistance through online modules, tests, and discussion boards. Self-study training programs offer an adaptable and convenient means of pursuing professional development and ongoing education.

Online Instructor-Led Training

The instructor-led training offers several advantages over the self-study option, significantly enhancing the learning experience and increasing exam pass rates. The instructor-led training for the CIRS program provides a structured and interactive learning environment. Experienced instructors guide you through the course material, offering expert insights and real-time feedback. This format includes live lectures, interactive discussions, and practical exercises, ensuring a thorough understanding of the concepts.

Advantages of Online Instructor-Led Training:

• Higher Pass Rates: Students who enroll in the instructor-led training classes have significantly higher pass rates compared to those in the self-study program, thanks to the comprehensive support and structured learning environment.
• Real-Time Interaction: Engage directly with instructors and peers, ask questions, and participate in discussions that enhance your understanding of the material.
• Expert Guidance: Benefit from the expertise of experienced instructors who provide valuable insights, clarifications, and real-world examples.
• Structured Learning: Follow a well-organized curriculum with scheduled classes and assignments, helping you stay on track and maintain consistent progress.
• Immediate Feedback: Receive instant feedback on assignments and assessments, allowing you to identify and address areas of improvement quickly.
• Enhanced Motivation: The interactive and collaborative nature of instructor-led classes fosters a sense of community and accountability, keeping you motivated throughout the course.
• Comprehensive Support: Access additional resources and support, such as study guides, supplementary materials, and office hours with instructors for personalized assistance.

No prior knowledge or research experience is required to enroll in the CIRS course, making it an excellent starting point for anyone looking to develop expertise in online research. Whether you choose the flexibility of the self-study course or the structured support of the online instructor-led training, the CIRS program equips you with the skills and knowledge to excel in professional online research.

Who can use these Certification?

These Certifications are used by anyone searching the Internet for Free Information sources and valuable online databases that provide reliable and relevant information. Professional Service Providers (such as Lawyers, PI's, Financial Analysts, Writers, and so on), Corporate Executives, Entrepreneurs, and Government and Non-profit Associations are one of the certified professionals.

Corporate & Business

• Senior Business Executives
• Financial Advisors
• Business Consultants
• Retailers and Wholesalers
• Insurance Companies
• Public Companies
• Investors, Shareholders
• Private Companies SMB to Large Size Corporations

Professional Researchers

• Marketing, Advertisement, and Sales
• Marketing Researchers
• Clinical Researchers
• Legal Researchers
• Patent, Copyrights and Intellectual Property
• Company Trends and
• Business Intelligence
• Market Researchers
• Financial Market and Stock
• Researchers

Professional Advisors

• Lawyers, Accountants
• Financial Analysts
• Mortgage Brokers
• Insurance House's
• Business Development Advisors
• Private Investigators
• The Government
• Semi-Government Institutions
• Law Enforcement Agencies
• Military Personnels

Independent Contractors

• Individuals between Jobs
• Independent Researchers
• Web Researchers
• Research Hobbyists
• Stay at Home Mothers
• People with Limited Mobility

Industry Validation

Our flagship certification called CIRS [Certified Internet Research Specialists] is by far the only professional credentials that meet this challenge. The CIRS Certification objective is to educate and train online researchers that now form a significantly large group within different market verticals. Our courses and syllabus have thus far proven to have the most comprehensive curriculum that meets current industry requirements.

Benefits of Certification

Professional credibility.

An undertaking by the CIRS qualified research specialists ties you to abide by the Association's “Professional Code of Conduct” that gives its practitiioners a higher level of credibility in their industry of research.

Professional Certification in Internet Research is the most comprehensive study program recognized in the Online Research Industry. A CIRS Certification demonstrates your extraordinary skills and expertise in conducting a web search.

Career Outlook

The future of the certified Internet Research Specialist (CIRS) is a bright one. Find out if getting your CIRS Certification is the right choice for you.

Save on your Certification

Become an "associate member" and save up to 20% on your all purchases, webinars, seminars and online classes.

World's leading professional association of Internet Research Specialists - We deliver Knowledge, Education, Training, and Certification in the field of Professional Online Research. The AOFIRS is considered a major contributor in improving Web Search Skills and recognizes Online Research work as a full-time occupation for those that use the Internet as their primary source of information.

Get Exclusive Research Tips in Your Inbox

• Privacy Policy
• Terms & Conditions
• Advertising Opportunities
• Knowledge Network

Research Courses

• Social Sciences

Prescription Drug Regulation, Cost, and Access: Current Controversies in Context

Understand how the FDA regulates pharmaceuticals and explore debates on prescription drug costs, marketing, and testing.

Foundations of Clinical Research

This Harvard Medical School six-month, application-based certificate program provides the essential skill sets and fundamental knowledge required to begin or expand your clinical research career.

Critical Issues in Tumor Microenvironment: Angiogenesis, Metastasis and Immunology

The 35th Annual Tumor Course reviews what is currently known about the tumor microenvironment, angiogenesis, metastasis, immunology, targeted therapies, metabolism, microbiome, and toxicity of immunotherapies.

Immuno-oncology

See how the immune system is being used to improve cancer treatment..

Cancer Genomics and Precision Oncology

Learn how cancer treatment is evolving due to advances in genetics..

Gene Therapy

Explore recent advances in gene therapy and learn about the implications for patient care..

Clinical Drug Development

Learning about the process of clinical drug development has important implications for anyone working in health care and related sectors.

Join our list to learn more

You are using an outdated browser. Please upgrade your browser to improve your experience.

• Online Studies

You are in a modal window. Press the escape key to exit.

• News & Events
• See programs

Common Searches

• Why is it called Johns Hopkins?
• What majors and minors are offered?
• Where can I find information about graduate programs?
• How much is tuition?
• What financial aid packages are available?
• How do I apply?
• How do I get to campus?
• Where can I find job listings?
• Where can I log in to myJHU?
• Where can I log in to SIS?
• University Leadership
• History & Mission
• Diversity & Inclusion
• Notable Alumni
• Hopkins in the Community
• Hopkins in D.C.
• Hopkins Around the World
• News from Johns Hopkins
• Undergraduate Studies
• Graduate Studies
• Part-Time & Non-Degree Programs
• Summer Programs
• Academic Calendars
• Advanced International Studies
• Applied Physics Laboratory
• Arts & Sciences
• Engineering
• Peabody Conservatory
• Public Health
• Undergraduate Admissions
• Graduate Admissions
• Plan a Visit
• Tuition & Costs
• Financial Aid
• Innovation & Incubation
• Bloomberg Distinguished Professors
• Undergraduate Research
• Our Campuses
• About Baltimore
• Housing & Dining
• Arts & Culture
• Health & Wellness
• Disability Services
• Calendar of Events
• Maps & Directions
• Contact the University
• Employment Opportunities
• Give to the University
• For Parents
• For News Media
• Office of the President
• Office of the Provost
• Gilman’s Inaugural Address
• Academic Support
• Study Abroad
• Nobel Prize winners
• Homewood Campus
• Emergency Contact Information

You don’t have to be on a Johns Hopkins campus to get the benefits of a Johns Hopkins education. We offer dozens of online courses and programs, some for credit, some for free through our partnership with Coursera.

More information about our online programs is available below

Krieger School of Arts & Sciences

More than 25 master’s and certificate programs are offered fully or partially online via the Krieger School’s Advanced Academic Programs division

Carey Business School

The Flexible MBA program provides a part-time business education for full-time professionals seeking to advance their careers; the business of health care graduate certificate is designed for leaders in health care and medicine

School of Advanced International Studies

Explore the world of international relations through innovative graduate courses and degree programs offered online.

School of Education

Doctoral, master’s, and certificate programs offered fully online, plus a selection of professional development courses

Whiting School of Engineering

More than 140 online courses are available via Johns Hopkins Engineering for Professionals, including programs in computer science, cybersecurity, environmental engineering, and information systems engineering

School of Medicine

The Department of the History of Medicine offers an online master’s and certificate in the History of Medicine, as well as individual courses for credit

School of Nursing

Online prerequisites for health professions—including anatomy, biostatistics, microbiology, and physiology—plus master’s programs and certificate options

Bloomberg School of Public Health

Offering online degree programs, courses, certificates, and training, plus an expanding portfolio of free online courses via OpenCourseWare, Coursera, and iTunes

Choose from more than 60 online classes  offered through Coursera, a leading provider of massive open online courses, or MOOCs. Courses are free and open to anyone who is interested—you don’t have to be enrolled at Johns Hopkins. Topics include bioinformatics, data analysis, the science of patient safety, and caring for people with Alzheimer’s disease.

Continuing Medical Education

The Office of Continuing Medical Education at our School of Medicine offers educational activities—including online courses —designed to help doctors maintain, develop, and/or increase the knowledge and skills they need to best serve their patients, the public, and the profession.

• Johns Hopkins University
• Address Baltimore, Maryland
• Phone number 410-516-8000
• © 2024 Johns Hopkins University. All rights reserved.
• Schools & Divisions
• Admissions & Aid
• Research & Faculty
• Campus Life
• University Policies and Statements
• Privacy Statement
• Title IX Information and Resources
• Higher Education Act Disclosures
• Clery Disclosure
• Accessibility

We're sorry but you will need to enable Javascript to access all of the features of this site.

Stanford Online

Clinical trials: design, strategy, and analysis.

SOM-XCHE0030

Stanford School of Medicine , Stanford Center for Health Education

New medical innovations are constantly hitting the market and changing patient care and research - but what goes into creating these technologies? Researchers need to design, conduct, and report on clinical trials to create impactful technologies that can better our understanding of health and medicine.  From learning frameworks and reporting methods to determining how to accurately collect, manage, and measure data, you’ll gain the knowledge necessary to understand the components that go into running a successful clinical trial. Whether you’re in a patient-facing role looking to implement research into your professional career or seeking how to better communicate and collaborate with colleagues working in the labs, the clinical trials course gives you the tools you need to find answers for your health and medicine-related inquiries. 

Learning Objectives:

• Understand and apply the principles of clinical trial design, such as using frameworks to create research questions and establish study objectives
• Ensure diverse, representative study populations by studying participant selection criteria and recruitment and retention strategies
• Minimize bias and ensure trial reliability and validity through blinding techniques and randomization methods
• Use effective data collection and management techniques to ensure data integrity throughout trials
• Effectively communicate trial findings through an understanding of reporting methods, ethical considerations, and regulatory bodies

Course Outline

Accreditation In support of improving patient care, Stanford Medicine is jointly accredited by the Accreditation Council for Continuing Medical Education (ACCME), the Accreditation Council for Pharmacy Education (ACPE), and the American Nurses Credentialing Center (ANCC), to provide continuing education for the healthcare team. 

Credit Designation American Medical Association (AMA) The Stanford University School of Medicine designates this enduring material for a maximum of 12.00   AMA PRA Category 1 Credits ™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.

View the full accreditation information HERE  from Stanford Medicine

Core Competencies

• Trial Phases
• Study Design
• Hypothesis Testing
• Statistical Analysis

Small Groups and Team Programs

Special Pricing

Enroll as a group or team and learn together. We can advise you on the best group options to meet your organization’s training and development goals and provide you with the support needed to streamline the process. Participating together, your group will develop a shared knowledge, language, and mindset to tackle the challenges ahead.

What Our Learners Are Saying

I recently completed the "Clinical Trials: Design, Strategy, and Analysis" course, and it was outstanding. The presenters were not only knowledgeable but also incredibly engaging, making complex concepts easy to understand and directly applicable to real-world scenarios. Highly recommend for anyone looking to deepen their understanding of clinical trials." - Carole-Anne B., Medical Manager

Teaching Team

Regina Nuzzo

Gallaudet University

Kristin Sainani

Epidemiology and Population Health

Kristin Sainani (née Cobb) is an associate professor at Stanford University and also a health and science writer. After receiving an MS in statistics and a PhD in epidemiology from Stanford University, she studied science writing at the University of California, Santa Cruz. She has taught statistics and writing at Stanford for more than a decade and has received several Excellence in Teaching Awards from the graduate program in epidemiology. Dr. Sainani writes about science and health for a range of audiences. She authored the health column Body News for Allure magazine for a decade. She is also the statistical editor for the journal Physical Medicine & Rehabilitation; and she authors a statistics column, Statistically Speaking, for this journal.

You May Also Like

Design and Conduct of Clinical Trials

Stanford School of Medicine

Thinking Critically: Interpreting Randomized Clinical Trials (CME)

SOM-YCME0019

Introduction to Clinical Data

SOM-XCHE0009

Stanford School of Medicine, Stanford Center for Health Education

Clinical Trial Design: From Initial Concept to Regulatory Approval

• Engineering
• Artificial Intelligence
• Computer Science & Security
• Business & Management
• Energy & Sustainability
• Data Science
• Medicine & Health
• Explore All
• Technical Support
• Master’s Application FAQs
• Master’s Student FAQs
• Master's Tuition & Fees
• Grades & Policies
• HCP History
• Graduate Application FAQs
• Graduate Student FAQs
• Graduate Tuition & Fees
• Community Standards Review Process
• Academic Calendar
• Exams & Homework FAQs
• Enrollment FAQs
• Tuition, Fees, & Payments
• Custom & Executive Programs
• Free Online Courses
• Free Content Library
• School of Engineering
• Graduate School of Education
• Stanford Doerr School of Sustainability
• School of Humanities & Sciences
• Stanford Human Centered Artificial Intelligence (HAI)
• Graduate School of Business
• Stanford Law School
• School of Medicine
• Learning Collaborations
• Stanford Credentials
• What is a digital credential?
• Grades and Units Information
• Our Community
• Get Course Updates

• Online Courses

SOCRA has implemented an online component to the already robust array of educational opportunities. These online courses offer affordable, convenient access to quality education.

SOCRA's online courses are intended to provide access to training and continuing education that will promote quality clinical research, protect the welfare of research participants and improve global health.

Katrina A. Croghan, MS, CCRP

Anatoly Gorkun, MD, PhD, Chartered MCIPD and Hugh Devine, IMIS

Managing Data Management: Lessons from a Large CMS Demonstration Project

Erin O'Kelly Phillips, MPH, CCRP

Annette Oeser, BS, MLAS, CCRP

Training Game Plan: Ensuring Successful Project Team Member Transitions Through Training

Carolyn Rugloski, MSc, CCRP

Amber Ashley Parker, BA, CCRP

Rachel Kingsford, MS, CCRP

Resiliency in the Workplace

Barbara van der Schalie, MS

Prevention = Safety: No Tears Approach to Research Compliance

Angie Price, MSN, CCRC

Laura Holtz, MS, CCRP

Patricia Beers Block, MDEd, BS, BS, CCRP

Amy Jo Jenkins, MS, CCRP

SOCRA Recertification Module 

JoAnn Mick, PhD, RN, NEA-BC

Mtonya Hunter, MBA, CCRP

• Program Information
• Register Online
• Program information
• Complete Learning Module

33RD ANNUAL CONFERENCE

ACHIEVING EXCELLENCE IN CLINICAL RESEARCH :

FORGING STRATEGIC COLLABORATIONS

September 27 to 29

COUNTDOWN TO ANNUAL CONFERENCE

Join us for expert-led sessions, interactive workshops, a peer-driven poster program, an engaging exhibit program and unparalleled networking opportunities!

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• BMJ Journals

You are here

• Volume 109, Issue 10
• Research exposure in current training: where are the gaps?
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Dominic Carr ,
• http://orcid.org/0000-0001-8717-1045 Eva Loucaides
• The London Research, Evaluation and Audit for Child Health (REACH) Network , London , UK
• Correspondence to Dr Dominic Carr; dominic.carr{at}nhs.net

https://doi.org/10.1136/archdischild-2024-326930

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• Child Health
• Health services research
• Paediatrics

Research is a core driver for improving child health. A recent Academy of Medical Sciences report clearly illustrates that investing in child health research confers population-level benefits for national health and prosperity. 1 But beyond pushing boundaries, research literacy is essential to daily clinical practice through informing evidence-based decision-making and facilitating adaptation of guidelines to evolving medical knowledge. It must be a core component of every paediatrician’s skill set and is described in Good Clinical Practice courses as a ‘frontline service’. As far back as 2012, the Royal College of Paediatrics and Child Health (RCPCH) Turning The Tide report highlighted the urgent need for increased provision and reignited enthusiasm for research within paediatric training. Alarmingly, reports have since shown the senior academic paediatric workforce to be shrinking, with only 5.4% of respondents to the 2022 RCPCH UK census reporting research involvement. 2 For child health to flourish, it is essential that the next generation of paediatricians is research literate. Beyond this, we must empower a substantial proportion of trainees to actively participate in and lead research to become skilled academic clinicians of the future. Below, we discuss the state of research within training: the mandates, provisions and shortfalls.

Mandates for research integration in training

Research has rightly been a staple of the training curriculum. RCPCH’s 2023 Progress+ details an increased number of research-oriented key capabilities, mirroring the values outlined by the General Medical Council’s (GMC) Generic Professional Capabilities Framework and further referenced in the Gold Guide.

X @dominicarr, @eloucaid

Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Read the full text or download the PDF:

Alison's New App is now available on iOS and Android! Download Now

Are you a business or organisation seeking to train and upskill your employees?

If yes, check out Alison’s Free LMS here!

header.all_certificate_courses

Personal development, sales & marketing, engineering & construction, teaching & academics.

Become an Alison Affiliate in one click, and start earning money by sharing any page on the Alison website.

• Change Language

Essentials of Research Methodology

This free online course includes:.

• Hours of Learning
• CPD Accreditation
• Final Assessment

Rate This Course & Get Better Recommendations!

Thanks for your review, in this free course, you will learn how to.

This course covers various concepts crucial to scientific research methodology, from the initial formulation of the problem through all the steps of designing and conducting the research to the final stage of writing a report. We introduce you to the definition of ‘scientific research’ and layout its methods, starting with formulating the research problem in a well-written proposal that addresses any potential ethical issues. We then examine the execution of the research, including data collection, processing, analysis and presentation, before concluding with the final stage: writing an accurate, effective research report that does justice to your work and satisfies the standards of a scientific publication.

This course consists of three modules, each containing three or four main topics. The first module explains the definition of ‘scientific research’ and its different types, methods, steps, criteria and ethics. The second module shows you how to prepare for your research project. You learn how to formulate your problem, design your research and select appropriate samples before describing your project in a convincing proposal. The third module unpacks how to collect, process, analyze and present your data and explains what makes up a publishable report.

We provide a complete overview of the process of conducting scientific research, from the initial idea to the final step of writing a report. As a result, we can help college students write their thesis or anyone entering the world of science and academe. Research competence can boost your career and help you understand how scientists work to arrive at the discoveries that affect all our lives. Sign up for this free online course today to learn how to conduct rigorous research.

All Alison courses are free to enrol study and complete. To successfully complete this course and become an Alison Graduate, you need to achieve 80% or higher in each course assessment. Once you have completed this course, you have the option to acquire an official , which is a great way to share your achievement with the world.

Your Alison is:

• Ideal for sharing with potential employers
• Great for your CV, professional social media profiles and job applications.
• An indication of your commitment to continuously learn, upskill & achieve high results.
• An incentive for you to continue empowering yourself through lifelong learning.

Alison offers 3 types of s for completed courses:

• Digital : a downloadable in PDF format immediately available to you when you complete your purchase.
• : a physical version of your officially branded and security-marked , posted to you with FREE shipping.
• Framed : a physical version of your officially branded and security marked in a stylish frame, posted to you with FREE shipping.

All s are available to purchase through the Alison Shop . For more information on purchasing Alison , please visit our FAQs . If you decide not to purchase your Alison , you can still demonstrate your achievement by sharing your Learner Record or Learner Achievement Verification, both of which are accessible from your Account Settings . For more details on our pricing, please visit our Pricing Page

Knowledge & Skills You Will Learn

Complete this cpd accredited course & get your certificate , certify your skills, stand out from the crowd, advance in your career.

Learner Reviews & Feedback For Essentials of Research Methodology

Want to create a customised learning path for your team.

Our dedicated Learning Advisors are here to help you curate a customised learning path tailored to your organisation's needs and goals.

Explore Careers Related To This Course

Not sure where to begin or even what you want to do.

Discover the career most suitable for you and get started in the field with a step-by-step plan.

About Your Alison Course Publisher

- alison stats, more free online courses by this publisher, learners who took this course also enrolled in, explore subjects related to this course.

Join our community of 40 million+ learners, upskill with CPD UK accredited courses, explore career development tools and psychometrics - all for free.

• Reset password form here

Enter your friend's email address

Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

• Accessibility

Course Fees, Registration & Payment: Online Health Inequalities Course

Fees, registration & payment.

Please make sure you have your credit/debit card ready to book. You can pay by any  MasterCard, Visa Credit Card, Switch, Solo or Delta Debit Card.  We are unable to accept payments by American Express or Diner cards.  Please note, we  do not  accept bank transfers.

*  Early booking rate  available until 12pm on Friday 27 September.

TO REGISTER AND PAY, PLEASE CLICK HERE

**Countries which are defined as low-income economies or lower-middle-income economies (LMIC), as defined by The World Bank Group , are eligible for the reduced short course fees.

Fill in the online form to be placed on a waiting list for a future course.

• HERC Online and Classroom Courses 2024/2025
• Online Course: Introduction to Health Economic Evaluation
• Online Course: Integrating Economic Evaluation into Clinical Trials
• Online Course: Applied Methods of Cost-Effectiveness Analysis
• Online Course: Cost-effectiveness Analysis in Stata Using Participant-level Data
• Online Course: Inequality in Health and Health Care: Theoretical and Empirical Considerations
• Online or Classroom-based Course: Understanding and Predicting Choice Behaviour in Health: Preference Elicitation and Analysis
• Classroom-based Applied Methods of Cost-Effectiveness Analysis Course
• RELATED COURSES AT OXFORD UNIVERSITY

3000+ Courses from California Community Colleges

3000+ Online Courses from California Community Colleges (Fall 2024)

Class Central experiments with cataloging online courses from California Community Colleges, offering diverse, affordable, and credit-worthy learning opportunities.

• 7 Best Genetics Courses for 2024
• [2024] 1200+ Free Computer Science Courses from World’s Top Universities
• Learn Something New: 100 Most Popular Courses For October
• 12 Best Microsoft Excel Courses for Beginners for 2024
• 8000+ OpenCourseWare Courses from Top Institutions

600 Free Google Certifications

Most common

• computer science

Popular subjects

Artificial Intelligence

Data Analysis

Popular courses

Introduction to Cyber Security

How to Write Your First Song

Machine Learning for Musicians and Artists

Organize and share your learning with Class Central Lists.

View our Lists Showcase

Clinical Research Courses and Certifications

Learn Clinical Research, earn certificates with paid and free online courses from Harvard, Stanford, University of Michigan, Johns Hopkins and other top universities around the world. Read reviews to decide if a class is right for you.

• Pharmacology Courses
• Biostatistics Courses
• SAS Courses
• Biochemistry Courses
• Public Health Courses
• Microbiology Courses
• Statistics & Probability Courses
• Technical Writing Courses
• With certificate (76)
• Free course (110)
• With free certificate (8)
• University course only (84)
• Beginner (39)
• Intermediate (13)
• Advanced (4)
• < 30 mins (4)
• 30 - 60 mins (15)
• 1 - 2 hours (10)
• 2 - 5 hours (12)
• 5 - 10 hours (18)
• 10+ hours (38)
• English (122)

Understanding Clinical Research: Behind the Statistics

If you’ve ever skipped over`the results section of a medical paper because terms like “confidence interval” or “p-value” go over your head, then you’re in the right place.

• 1 day 3 hours 11 minutes
• Free Online Course (Audit)

Improving Healthcare Through Clinical Research

On this free online course, find out how medical treatments are discovered, tested and evaluated to improve healthcare for all.

• FutureLearn
• 4 weeks, 4 hours a week

What is Health Research?

Explore the world of health research and the role volunteers play in transforming treatments and improving health care.

• 3 weeks, 3 hours a week

Current Regulatory Requirements for Conducting Clinical Trials in India

Comprehensive overview of India's clinical trial regulations, covering key aspects and requirements for conducting ethical and compliant research in the country.

Thinking Critically: Interpreting Randomized Clinical Trials

Enhance critical appraisal skills for evaluating clinical research papers through engaging videos and practical examples. Improve clinical practice, research quality, and peer-review abilities.

• 1 week, 2-3 hours a week

Design and Interpretation of Clinical Trials

Learn to design, conduct, and interpret clinical trials for evaluating medical interventions. Explore trial designs, randomization, masking, outcome selection, analysis methods, ethical considerations, and reporting standards.

Translating Research to Patients

Explore clinical trials, ethical research practices, and regulatory processes in translating scientific discoveries to patient care. Gain insights into human subjects research and its role in advancing medical knowledge.

• 6 hours 18 minutes

Clinical Trials Operations

Develop essential skills for designing, managing, and analyzing clinical trials, covering ethics, safety, recruitment, compliance, and reporting standards in healthcare research.

• 17 weeks, 3 hours a week
• Paid Course

Design and Conduct of Clinical Trials

Comprehensive guide to designing and conducting clinical trials, covering bias control, trial stages, outcome definition, ethical considerations, and participant recruitment and retention strategies.

• 4 hours 12 minutes

Clinical Trials Data Management and Quality Assurance

Learn to manage clinical trial data, ensure quality, and maintain integrity through effective collection, organization, monitoring, and quality assurance practices for successful research outcomes.

• 6 hours 35 minutes

Clinical Trials Management and Advanced Operations

Explore advanced clinical trial management, covering protocol events, regulatory compliance, standardization, transparency, evidence synthesis, and essential study documentation for effective research conduct.

• 4 hours 58 minutes

Clinical Trials Analysis, Monitoring, and Presentation

Learn advanced skills for clinical trial management, including sample size calculation, statistical monitoring, result reporting, and the analyst's role throughout the process.

• 4 hours 42 minutes

Data Management for Clinical Research

Learn critical concepts and practical methods for planning, collecting, storing, and disseminating data in clinical research. Covers electronic data capture, regulatory compliance, and best practices for effective data management.

• 17 hours 25 minutes

Partnering with the Public and Patients in Medical Research

Explore patient-researcher partnerships in medical research, learning strategies for inclusivity, trust-building, and mutual benefit to enhance research outcomes and patient empowerment.

• 5 weeks, 1-2 hours a week

Advanced Knowledge and Skills of Originator and Biosimilar Biologics: For Health Care Providers

Developed by leading researchers and clinicians from across Canada, this course will enhance knowledge and skills for clinicians who are using or beginning to use a biosimilar or biologic medicine in their professional practice setting.

• 16 weeks, 3-6 hours a week

Never Stop Learning.

Get personalized course recommendations, track subjects and courses with reminders, and more.