Please answer the questions you will find in the form below (left side of the page) so you can generate your mixed-methods research design. Please use the information provided on the right side of the page to provide informed answers. After submitting your answers, the system will send you an email with the generated design. You will also receive a link to edit the design as many times as needed. 

Important note: The form below will give you the possibility of saving your work so you can keep on working on it on a different time. To do so, you will have to select "Save to keep on working later," then click "next," and then click on "submit." After doing so, Hopscotch will send you an email with an attached pdf version of your design, as well as a link for you to keep on working in your design later on.

Step 1: Paradigmatic View of the Researcher 

Researchers bring to their studies their particular way of understanding how things work in our world, and the way knowledge is constructed. The worldview of the researcher as well as his/her adscription to a particular Interpretive Community (if so) is going to have a deep impact in the decisions and inquiry procedures he/she will put in practice.


Guba (1990) describes a paradigm or worldview as "a basic set of beliefs that guide action.” That basic set of beliefs of the researcher is based on his ontological (What is the nature of reality?) and epistemological assumptions (What is the nature of knowledge and the relationship between the knower and the would-be known?). Therefore, how one views the constructs of social reality and knowledge affects how they will go about uncovering knowledge of relationships among phenomena and social behavior. Your ontological assumptions inform your epistemological assumptions which inform your methodology and these all give rise to your methods employed to collect data.

From an ontological point of view, post-positivism understands that there is one reality which is knowable within a specific level of probability, while constructivism understands that the nature of reality is multiple and socially constructed. Pragmatism asserts that there is a single reality and that all individuals have their own unique interpretation of reality. Finally, those following a transformative worldview reject cultural relativism and recognize that various versions of reality are based on social positioning.

From an epistemological point of view, post-positivists believe that objectivity is key and than the researcher manipulates and observes in a dispassionate objective manner. Constructivists on the contrary, believe that there should be an interactive link between researcher and participants, and that since knowledge is socially and historically situated, it needs to address issues of power and trust. Pragmatism on its side, understands that relationships in research are determined by what the researcher deems as appropriate to a particular given study. Finally, a transformative worldview acknowledges that since there is an interactive link between researcher and participants, and knowledge is socially and historically situated, there is a clear need to address issues of power and trust.

These ontological and epistemological assumptions have a direct impact in the methodology used in a given study. Post-positivism calls for interventionist quantitative studies, while constructivism prefers qualitative hermeneutical studies, and pragmatists matches methods to specific questions and purposes of research by using mixed methods. In the case of researchers following a transformative worldview, qualitative methods deeply grounded in critical theories, are the most common ones.

The main Worldviews (Creswell, 2013) are:

POST-POSITIVISM

  • This tradition comes from 19th-century (Comte, Mill, Durkheim, Newton, and Locke).
  • It represents the traditional form of research (scientific method).
  • Called post-positivism since it represents the thinking after positivism, challenging the traditional notion of the absolute truth of knowledge (Phillips & Burbules, 2000).
  • Postpositivists hold a deterministic philosophy in which causes (probably) determine effects or outcomes.
  • It is reductionistic in that the intent is to reduce the ideas into a small, discrete set to test, such as the variables that comprise hypotheses and research questions. To test theories.
  • The knowledge that develops through a postpositivist lens is based on empirical observation and measurement of the objective reality that exists “out there” in the world.

CONSTRUCTIVISM

  • Based on the ideas of Mannheim and from works such as Berger and Luekmann’s (1967). The Social Construction of Reality and Lincoln and Guba’s (1985) Naturalistic Inquiry.
  • There is not one objective truth. Truth is socially constructed.
  • Social constructivists believe that individuals seek understanding of the world in which they live and work, developing subjective meanings of their experiences.
  • These meanings are varied and multiple, leading the researcher to look for the complexity of views rather than narrowing meanings into a few categories or ideas.
  • The goal of the research is to rely as much as possible on the participants’ views of the situation being studied..
  • The researcher’s intent is to make sense of (or interpret) the meanings others have about the world.
  • Rather than starting with a theory (as in postpositivism), inquirers inductively develop a theory.

TRANSFORMATIVE

  • This position arose during the 1980´s and 1990´s.
  • It came from individuals who felt postpositivist assumptions imposed structural laws and theories that did not fit marginalized individuals.
  •  In the main, these inquirers felt that the constructivist stance did not go far enough in advocating for an action agenda to help marginalized peoples.
  • It focuses on the needs of groups and individuals in our society that may be marginalized.
  • Historically, the transformative writers have drawn on the works of Marx, Adorno, Marcuse, Habermas, and Freire (Neuman, 2009).
  • No uniform body of literature characterizing this worldview: Includes groups of researchers that are critical theorists; participatory action researchers; Marxists; feminists; racial and ethnic minorities; persons with disabilities; indigenous and postcolonial peoples; and members of the lesbian, gay, bisexual, trans-sexual, and queer communities.
  • This worldview holds that research inquiry needs to be intertwined with politics and a political change agenda to confront social oppression at whatever levels it occurs (Mertens, 2010).

PRAGMATISM

  • Pragmatism derives from the work of Peirce, James, Mead, and Dewey (Cherryholmes,1992).
  • There are many forms of this philosophy, but for many, pragmatism as a worldview arises out of actions, situations, and consequences rather than antecedent conditions (as in postpositivism).
  • Instead of focusing on methods, researchers emphasize the research problem and use all approaches available to understand the problem (Rossman & Wilson, 1985).
  • Individual researchers have a freedom of choice. In this way, researchers are free to choose the methods, techniques, and procedures of research that best meet their needs and purposes.
  • As a philosophical underpinning for mixed methods studies, Morgan (2007), Patton (1990), and Tashakkori and Teddlie (2010) convey its importance for focusing attention on the research problem in social science research and then using pluralistic approaches to derive knowledge about the problem.

Please watch the following clip to clarify the previous concepts:

Step 2: Topics & Goals of the Study

The second step in the generation of your research design implies the definition of your research topic and the goals that will drive the study. Watch the following clip in order to understand the key aspects in defining a good research topic.

 

 

Maxwell (2008) states that goals include motives, desires, and purposes—anything that leads you to do the study or that you hope to accomplish by doing it.)  

Some questions that could help us to better define the goals of our study are:

    • Why is your study worth doing?
    • What issues do you want it to clarify, and what practices and policies do you want it to influence?
    • Why do you want to conduct this study, and why should we care about the results?

According to (Maxwell, 2008), goals serve two main functions for your research:

    • They help guide your other design decisions to ensure that your study is worth doing.
    • They are essential to justifying your study, a key task of a funding or dissertation proposal.

There are three kinds of goals for doing a study (Maxwell, 2008):

    • Personal goals: those that motivate you to do this study; they can include a desire to change some existing situation, a curiosity about a specific phenomenon or event, or simply the need to advance your career.
    • Practical goals: are focused on accomplishing something—meeting some need, changing some situation, or achieving some goal.
    • Intellectual goals: re focused on understanding something, gaining some insight into what is going on and why this is happening.

Watch the following clip to better understand how to define the goals of your study:

 

 

Step 3: Conceptual framework of the study

The conceptual framework of your study is the system of concepts, assumptions, expectations, beliefs, and theories that supports and informs your research. It is a formulation of what you think is going on with what you are studying—a tentative theory of what is happening and why.

 

Theory provides a model or map of why the world is the way it is (Strauss & Corbin, 1990) The function of theory in your design is to inform the rest of the design. The “research problem” is a part of your conceptual framework, and formulating the research problem is often seen as a key task in designing your study. Ravitch & Riggan (2016) define a conceptual framework as an argument about why the topic one wishes to study matters, and why the means proposed to study it are appropriate and rigorous. By argument, they mean that a conceptual framework is a series of sequenced, logical propositions the purpose of which is to ground the study and convince readers of the study’s importance and rigor. Arguments for why a study “matters” vary greatly in scale, depending on the audience. In some scholarly work, the study may only matter to a small, esoteric community, but that does not change the fact that its conceptual framework should argue for its relevance within that community. By appropriate and rigorous, they mean that a conceptual framework should argue convincingly that: (a) the research questions are an outgrowth of the argument for relevance; (b) the research design maps onto the study goals, questions, and context(s); (c) the data to be collected provide the researcher with the raw material needed to explore the research questions; and (d) the analytic approach allows the researcher(s) to effectively address (if not always answer) those questions. In order to start building the conceptual framework for your research project we truly recommend the following book: Ravitch, S. M., & Riggan, M. (2016). Reason & rigor : how conceptual frameworks guide research. Thousand Oaks : Sage Publications. The following figure represents and briefly explains the main components a conceptual framework should have.  Min components of a conceptual framework Steps and Resources to start building  your conceptual framework

 

Step 1: Identify your personal connection with your research topic 
The following are questions that Ravitch & Riggan (2016) encourage you to explore in order to engage in a process of self-examination at the outset of your research and then iteratively throughout the research process. 

What is interesting to me and why? (In terms of my research topic) What personal and professional motivations do I have for engaging in this research? How might these motivations influence how I think about and approach the topic? What are my beliefs about the people, places, and ideas involved in and related to my study? Where do these beliefs come from? What assumptions underlie these beliefs? What orientations to the topic, setting, and concepts do I have? Where do these ideas come from? What is my sense of the relationship between the macro and micro sociopolitical circumstances in which people make meaning and choices in their lives? With respect to the participants in my study specifically? What is my "agenda" for taking up this topic in this setting at this time? (Having an agenda is not necessarily a bad thing. This may be the foundation of your argument!) What influences this agenda? What biases shape this agenda? How might my guiding agenda contribute, both positively and negatively, to my research design? Implementation? Analysis? Findings? What hunches do I have about what I might find and discover? What informs these hunches? What concerns, hopes, and expectations do I have for this research?

In addition to the responses to these questions, you can also take advantage of the goals defined in the second stage of the Hopscotch.  There are three types of goals for doing a study (Maxwell, 2008) that could help you shape your personal connection with the selected research topic:
  • Personal goals: those that motivate you to do this study; they can include a desire to change some existing situation, a curiosity about a specific phenomenon or event, or simply the need to advance your career.
  • Practical goals: are focused on accomplishing something—meeting some need, changing some situation, or achieving some goal.
  • Intellectual goals: re focused on understanding something, gaining some insight into what is going on and why this is happening
Step 2: Identify your identity and positionality as a researcher
This second step is deeply related to the paradigmatic view or worldview that you will be bringing to the studyas a researcher. Please have a look at information provided within the first step of Hopscotch (Step 1: Paradigmatic View of the Researcher) in order to understand the particularities of the main worldviews you might bring to your study. 
Step 3: Literature review
The third and main component of your conceptual framework will be the review of literature. The following  guides that have been generated by Dr. Olga Koz, Graduate Librarian at the Bagwell College of Education (Kennesaw State University) could be a great resource when working in the topical research and theoretical frameworks you will use to justify the relevance of your research topic and the need for the study you are proposing:  
  1. Literature Review for a Dissertation: a step-by-step guide
  2. Databases in education
  3. How to identify scholarly articles 
  4. Searching for education literature
  5. Finding Sources
Ravitch & Riggan (2016) propose two different sub-components in your literature review: a) Topical Research and; b) Theoretical Frameworks. 

"Topical research refers to previous work (most often empirical) that has focused on the topic in which you are interested. While much of this work resides within academic journals and books, it may also be found in policy or government research, or in reports produced through foundations, nonprofits, and advocacy organizations." 

They understand  a theoretical framework "as a set of formal theories and their relationships, that helps you to fill the intellectual bins that make up your conceptual framework." 

The following resource will be of help to identify topical research and theoretical frameworks to justify the relevance of your research topic, its pertinence and its theoretical roots. 
Resource to search for relevant papers related to your research topic
Resource to generate a visual representation of your Literature review
This form will help you build a visual representation of the topics emerging from 10 key readings that are deeply related with your research topic. The generated visual will also help you differentiate between the topical research and theoretical frameworks that will serve you to start building the conceptual framework for your research project. After filling out the form you will receive an email including a pdf document with a visual representation of the work done, as well as a link for you to modify your answers as many times as needed. In the following link you can see an example of the type of document you will get after filling out the form: goo.gl/TL9hsg The visual representation generated will help you build the literature review for your research project. To identify the 10 articles which are most related to your research topic, or that inform your conceptual framework, you will have to read many more than 10. All articles must be primary sources and from peer-reviewed/refereed journals. To do so you can use any of the resources that we have included in the tables below (Resource to identify key topics in your field of research: Open Knowledge Map; Resource to graphically build your conceptual framework; Research guides; Science of Science (Sci2)). You don´t have to use all of the resources provided, just the ones you believe could help you the most based on your previous knowledge. For instance, if you already know how to conduct a literature review, or how to identify scholarly articles, you might not have to use the "Research Guides." On the contrary, if you want to identify existing publications regarding your research topic, you might want to use the "Databases in Education," and "Open Knowledge Maps." The following figure shows an example of the type of visual representation you will get after filling out the form.
Resource to identify key topics in your field of research: Open Knowledge Map
With "Open Knowledge Maps" you will be able to search for key concepts related to your research topic in order to generate a visual representation that will help you identify publications addressing that particular key concept.
Resource to identify key literature in your field of research: Science of Science (Sci2)
The Science of Science (Sci2) Tool is a modular toolset specifically designed for the study of science. It supports the temporal, geospatial, topical, and network analysis and visualization of scholarly datasets at the micro (individual), meso (local), and macro (global) levels. This tool can be used to conduct thorough literature reviews. 
Resource: Research Guides
The following  guides that have been generated by Dr. Olga Koz, Graduate Librarian at the Bagwell College of Education (Kennesaw State University) could be a great resource when working in the topical research and theoretical frameworks you will use to justify the relevance of your research topic and the need for the study you are proposing:  
  1. Literature Review for a Dissertation: a step-by-step guide
  2. Databases in education
  3. How to identify scholarly articles 
  4. Searching for education literature
  5. Finding Sources
Resource to Define your problem statement
One key component of your conceptual framework will be the "Problem Statement." You can use this template to define it after having conducted your review of literature.
Resource to graphically build your conceptual framework
You can use this template to create a visual representation of the main components that should be included in the conceptual framework of your research project. In order to be able to use the template, you will have to log in your Google account so the system can ask you to make a copy of it. 
Step 4: Research design/tradition

Types
Convergent parallel
Explanatory sequential
Exploratory sequential
Embedded
Types

 

There are three main types of mixed-methods designs: Convergent parallel, Exploratory sequential; and Explanatory sequential. You can find a description of each of them in the tabs above. 

 

The previous three basic designs can then be used in more advanced mixed methods strategies.

For instance, transformative mixed methods is a design that uses a theoretical lens drawn from social justice or power as an overarching perspective within a design that contains both quantitative and qualitative data. The data in this form of study could be converged or it could be ordered sequentially with one building on the other.

An embedded mixed methods design involves as well either the convergent or sequential use of data, but the core idea is that either quantitative or qualitative data is embedded within a larger design (e.g., an experiment) and the data sources play a supporting role in the overall design. A multiphase mixed methods design is common in the fields of evaluation and program interventions. In this advanced design, concurrent or sequential strategies are used in tandem over time to best understand a long-term program goal.

Convergent parallel

 

Convergent parallel mixed methods is a form of mixed methods design in which the researcher converges or merges quantitative and qualitative data in order to provide a comprehensive analysis of the research problem. In this design, the investigator typically collects both forms of data at roughly the same time and then integrates the information in the interpretation of the overall results. Contradictions or incongruent findings are explained or further probed in this design.

 

The following tool will help you generate a visual representation of your Convergent parallel mixed methods


Explanatory sequential

 

Explanatory sequential mixed methods is one in which the researcher first conducts quantitative research, analyzes the results and then builds on the results to explain them in more detail with qualitative research. It is considered explanatory because the initial quantitative data results are explained further with the qualitative data. It is considered sequential because the initial quantitative phase is followed by the qualitative phase. This type of design is popular in fields with a strong quantitative orientation (hence the project begins with quantitative research), but it presents challenges of identifying the quantitative results to further explore and the unequal sample sizes for each phase of the study.

 

The following tool will help you generate a visual representation of your Explanatory sequential mixed methods Design


Exploratory sequential

 

Exploratory sequential mixed methods is the reverse sequence from the explanatory sequential design. In the exploratory sequential approach the researcher first begins with a qualitative research phase and explores the views of participants. The data are then analyzed, and the information used to build into a second, quantitative phase. The qualitative phase may be used to build an instrument that best fits the sample under study, to identify appropriate instruments to use in the follow-up quantitative phase, or to specify variables that need to go into a follow-up quantitative study. Particular challenges to this design reside in focusing in on the appropriate qualitative findings to use and the sample selection for both phases of research.

 

The following tool will help you generate a visual representation of your Exploratory sequential mixed methods Design


Embedded

 

An embedded mixed methods design involves as well either the convergent or sequential use of data, but the core idea is that either quantitative or qualitative data is embedded within a larger design (e.g., an experiment) and the data sources play a supporting role in the overall design. A multiphase mixed methods design is common in the fields of evaluation and program interventions. In this advanced design, concurrent or sequential strategies are used in tandem over time to best understand a long-term program goal.

 

The following tool will help you generate a visual representation of your embedded (quantitative) mixed methods design


 

The following tool will help you generate a visual representation of your embedded (qualitative) mixed methods design


 

The following clip summarizes the main types of mixed-methods designs


Once you have picked one mixed-methods research design, you can use the following links in order to generate a visual representation of the different key elements in your design:

 

Step 6: Data Gathering Methods
Step 7: Data Analysis Methods

Data analysis in mixed methods studies involves the analysis of quantitative and qualitative data (watch the clips above).

The following resources might be of help to better understand the way data analysis work in mixed methods research studies:

Qualitative Data Analysis

 

Quantitative Data Analysis

Step 8: Trustworthiness/Validity
MIXED METHODS

When using a mixed methods approach, the researcher should consider the control of validity and reliability in the quantitative portion of the study, as well as trustworthiness strategies for the qualitative portion of the study.


Resources: 

QUALITATIVE RESEARCH METHODS


Resources:

QUANTITATIVE RESEARCH METHODS

In quantitative educational research we usually consider two general dimensions in evaluating a measurement method: reliability and validity.

Reliability is defined as the consistency of the measurements. To what level will the instrument produce the same results under the same conditions every time it is used? Reliability adds to the trustworthiness of the results because it is a testament to the methodology if the results are reproducible. The reliability is often examined by using a test and retest method where the measurement are taken twice at two different times. The reliability is critical for being able to reproduce the results, however, the validity must be confirmed first to ensure that the measurements are accurate. Consistent measurements will only be useful if they are accurate and valid.

The term validity refers to the strength of the conclusions that are drawn from the results. In other words, how accurate are the results? Do the results actually measure what was intended to be measured? There are several types of validity that are commonly examined and they are as follows: 

  • Conclusion validity looks at whether or not there is a relationship between the variable and the observed outcome. 
  • Internal validity considers whether or not that relationship may be causal in nature.
  • Construct validity refers to whether or not the operational definition of a variable actually reflects the meaning of the concept. In other words, it is an attempt to generalize the treatment and outcomes to a broader concept.
  • External validity is the ability to generalize the results to another setting. There are multiple factors that can threaten the validity in a study. They can be divided into single group threats, multiple group threats, and social interaction threats.

For more information regarding the control of the validity of a quantitative study please read chapter 6 in (Price, Jhangiani, & Chiang, 2015)


 

Resources: 

 

 

Step 9: Ethics driving the study

The term “ethics”  derives from the Greek word “ethos” which means character. To engage with the ethical dimension of your research requires asking yourself several important questions:

-What moral principles guide your research?

-How do ethical issues enter into your selection of a research problem?

-How do ethical issues affect how you conduct your research—the design of your study, your sampling procedure, etc.?

-What responsibility do you have toward your research subjects? For example, do you have their informed consent to participate in your project? 

-What ethical issues/dilemmas might come into play in deciding what research findings you publish? 

-Will your research directly benefit those who participated in the study?

 

 

The major principles associated with ethical conduct  are (Litchman, 2011):

1-    Do No Harm: 

-It is the cornerstone of ethical conduct

-There should be a reasonable expectation by those participating in a research study that they will not be involved in any situation in which they might be harmed.

-Often applied to studies involving drugs or a treatment that might be harmful to participants

-The 1971 Stanford Prison Experiment, in which students played the role of guards and prisoners, is one example. When it was found that the guards became increasingly sadistic, the study was terminated.

-Recommendation: It is best to safeguard against doing anything that will harm the participants in your study. If you begin a study and you find that some of your participants seem to have adverse reactions, it is best to discontinue the study, even if it means foregoing your research plan.


2-    Privacy and Anonymity:

-Any individual participating in a research study has a reasonable expectation that privacy will be guaranteed. Consequently, no identifying information about the individual should be revealed in written or other communication. Further, any group or organization participating in a research study has a reasonable expectation that its identity will not be revealed.

-Recommendation: Remove identifying information from your records. Seek permission from the participants if you wish to make public information that might reveal who they are or who the organization is. Use caution in publishing long verbatim quotes, especially if they are damaging to the organization or people in it. Often, these quotes can be located on the Internet and traced to the speaker or author.


3-    Confidentiality: 

-Any individual participating in a research study has a reasonable expectation that information provided to the researcher will be treated in a confidential manner. Consequently, the participant is entitled to expect that such information will not be given to anyone else.

-Recommendation: It is our responsibility to keep the information you learn confidential. If you sense that an individual is in an emergency situation, you might decide that you can waive your promise for the good of the individual or of others. You need to be much more sensitive to information that you obtain from minors and others who might be in a vulnerable position.


4-    Informed Consent: 

-Individuals participating in a research study have a reasonable expectation that they will be informed of the nature of the study and may choose whether or not to participate. They also have a reasonable expectation that they will not be coerced into participation.

-Recommendation: Our responsibility is to make sure that participants are informed, to the extent possible, about the nature of your study. Even though it is not always possible to describe the direction your study might take, it is your responsibility to do the best you can to provide complete information. If participants decide to withdraw from the study, they should not feel penalized for so doing. You need to be aware of special problems when you study people online. For example, one concern might be vulnerability of group participants. Another is the level of intrusiveness of the researcher.


5-    Rapport and Friendship: 

-Once participants agree to be part of a study, the researcher develops rapport in order to get them to disclose information.

-Recommendation: Researchers should make sure that they provide an environment that is trustworthy. At the same time, they need to be sensitive to the power that they hold over participants. Researchers need to avoid setting up a situation in which participants think they are friends with the researcher.


6-    Intrusiveness:

-Individuals participating in a research study have a reasonable expectation that the conduct of the researcher will not be excessively intrusive. Intrusiveness can mean intruding on their time, intruding on their space, and intruding on their personal lives. As you design a research study, you ought to be able to make a reasonable estimate of the amount of time participation will take.

-Recommendation: Experience and caution are the watchwords. You might find it difficult to shift roles to neutral researcher, especially if your field is counseling or a related helping profession.


7-    Inappropriate Behavior:

-Individuals participating in a research study have a reasonable expectation that the researcher will not engage in conduct of a personal or sexual nature.

-Here, researchers might find themselves getting too close to the participants and blurring boundaries between themselves and others. We probably all know what we mean by inappropriate behavior. We know it should be avoided

-Recommendation: If you think you are getting too close to those you are studying, you probably are. Back off and remember that you are a researcher and bound by your code of conduct to treat those you study with respect.


8-    Data Interpretation:

-A researcher is expected to analyze data in a manner that avoids misstatements, misinterpretations, or fraudulent analysis. The other principles discussed involve your interaction with individuals in your study. This principle represents something different. It guides you to use your data to fairly represent what you see and hear. Of course, your own lens will influence you.

-Recommendation: You have a responsibility to interpret your data and present evidence so that others can decide to what extent your interpretation is believable.

9-    Data Ownership and Rewards:

-In general, the researcher owns the work generated. Some researchers choose to archive data and make them available through databanks. Questions have been raised as to who actually owns such data. Some have questioned whether the participants should share in the financial rewards of publishing. Several ethnographers have shared a portion of their royalties with participants.

-Recommendation: In fact, most researchers do not benefit financially from their writing. It is rare that your work will turn into a bestseller or even be published outside your university. But, if you have a winner on hand, you might think about sharing some of the financial benefits with others.