Please answer the questions you will find in the form below ( left side of the page) so you can generate your quantitative research design. Please use the information provided on the right side of the page to provide informed answers. After submitting your answers, the system will send you an email with the generated design. You will also receive a link to edit the design as many times as needed. 

These are the steps we will follow in order to create your quantitative research design:
*Required

    Captionless Image
    This is a required question
    This is a required question
     
    9% completed
Step 1: Paradigmatic View of the Researcher 

Researchers bring to their studies their particular way of understanding how things work in our world, and the way knowledge is constructed. The worldview of the researcher as well as his/her adscription to a particular Interpretive Community (if so) is going to have a deep impact in the decisions and inquiry procedures he/she will put in practice.

Guba (1990) describes a paradigm or worldview as "a basic set of beliefs that guide action.” That basic set of beliefs of the researcher is based on his ontological (What is the nature of reality?) and epistemological assumptions (What is the nature of knowledge and the relationship between the knower and the would-be known?). Therefore, how one views the constructs of social reality and knowledge affects how they will go about uncovering knowledge of relationships among phenomena and social behavior. Your ontological assumptions inform your epistemological assumptions which inform your methodology and these all give rise to your methods employed to collect data.

From an ontological point of view, post-positivism understands that there is one reality which is knowable within a specific level of probability, while constructivism understands that the nature of reality is multiple and socially constructed. Pragmatism asserts that there is a single reality and that all individuals have their own unique interpretation of reality. Finally, those following a transformative worldview reject cultural relativism and recognize that various versions of reality are based on social positioning.

From an epistemological point of view, post-positivists believe that objectivity is key and than the researcher manipulates and observes in a dispassionate objective manner. Constructivists on the contrary, believe that there should be an interactive link between researcher and participants, and that since knowledge is socially and historically situated, it needs to address issues of power and trust. Pragmatism on its side, understands that relationships in research are determined by what the researcher deems as appropriate to a particular given study. Finally, a transformative worldview acknowledges that since there is an interactive link between researcher and participants, and knowledge is socially and historically situated, there is a clear need to address issues of power and trust.

These ontological and epistemological assumptions have a direct impact in the methodology used in a given study. Post-positivism calls for interventionist quantitative studies, while constructivism prefers qualitative hermeneutical studies, and pragmatists matches methods to specific questions and purposes of research by using mixed methods. In the case of researchers following a transformative worldview, qualitative methods deeply grounded in critical theories, are the most common ones.

The main Worldviews (Creswell, 2013) are:

POST-POSITIVISM

  • This tradition comes from 19th-century (Comte, Mill, Durkheim, Newton, and Locke).
  • It represents the traditional form of research (scientific method).
  • Called post-positivism since it represents the thinking after positivism, challenging the traditional notion of the absolute truth of knowledge (Phillips & Burbules, 2000).
  • Postpositivists hold a deterministic philosophy in which causes (probably) determine effects or outcomes.
  • It is reductionistic in that the intent is to reduce the ideas into a small, discrete set to test, such as the variables that comprise hypotheses and research questions. To test theories.
  • The knowledge that develops through a postpositivist lens is based on empirical observation and measurement of the objective reality that exists “out there” in the world.

CONSTRUCTIVISM

  • Based on the ideas of Mannheim and from works such as Berger and Luekmann’s (1967). The Social Construction of Reality and Lincoln and Guba’s (1985) Naturalistic Inquiry.
  • There is not one objective truth. Truth is socially constructed.
  • Social constructivists believe that individuals seek understanding of the world in which they live and work, developing subjective meanings of their experiences.
  • These meanings are varied and multiple, leading the researcher to look for the complexity of views rather than narrowing meanings into a few categories or ideas.
  • The goal of the research is to rely as much as possible on the participants’ views of the situation being studied..
  • The researcher’s intent is to make sense of (or interpret) the meanings others have about the world.
  • Rather than starting with a theory (as in postpositivism), inquirers inductively develop a theory.

TRANSFORMATIVE

  • This position arose during the 1980´s and 1990´s.
  • It came from individuals who felt postpositivist assumptions imposed structural laws and theories that did not fit marginalized individuals.
  •  In the main, these inquirers felt that the constructivist stance did not go far enough in advocating for an action agenda to help marginalized peoples.
  • It focuses on the needs of groups and individuals in our society that may be marginalized.
  • Historically, the transformative writers have drawn on the works of Marx, Adorno, Marcuse, Habermas, and Freire (Neuman, 2009).
  • No uniform body of literature characterizing this worldview: Includes groups of researchers that are critical theorists; participatory action researchers; Marxists; feminists; racial and ethnic minorities; persons with disabilities; indigenous and postcolonial peoples; and members of the lesbian, gay, bisexual, trans-sexual, and queer communities.
  • This worldview holds that research inquiry needs to be intertwined with politics and a political change agenda to confront social oppression at whatever levels it occurs (Mertens, 2010).

PRAGMATISM

  • Pragmatism derives from the work of Peirce, James, Mead, and Dewey (Cherryholmes,1992).
  • There are many forms of this philosophy, but for many, pragmatism as a worldview arises out of actions, situations, and consequences rather than antecedent conditions (as in postpositivism).
  • Instead of focusing on methods, researchers emphasize the research problem and use all approaches available to understand the problem (Rossman & Wilson, 1985).
  • Individual researchers have a freedom of choice. In this way, researchers are free to choose the methods, techniques, and procedures of research that best meet their needs and purposes.
  • As a philosophical underpinning for mixed methods studies, Morgan (2007), Patton (1990), and Tashakkori and Teddlie (2010) convey its importance for focusing attention on the research problem in social science research and then using pluralistic approaches to derive knowledge about the problem.

Please watch the following clip to clarify the previous concepts:

Step 2: Topics & Goals of the Study

Maxwell (2008) states that goals include motives, desires, and purposes—anything that leads you to do the study or that you hope to accomplish by doing it.)

Some questions that could help us to better define the goals of our study are:

  • Why is your study worth doing?
  • What issues do you want it to clarify, and what practices and policies do you want it to influence?
  • Why do you want to conduct this study, and why should we care about the results?

According to (Maxwell, 2008), goals serve two main functions for your research:

  • They help guide your other design decisions to ensure that your study is worth doing.
  • They are essential to justifying your study, a key task of a funding or dissertation proposal.

There are three kinds of goals for doing a study (Maxwell, 2008):

  • Personal goals: those that motivate you to do this study; they can include a desire to change some existing situation, a curiosity about a specific phenomenon or event, or simply the need to advance your career.
  • Practical goals: are focused on accomplishing something—meeting some need, changing some situation, or achieving some goal.
  • Intellectual goals: re focused on understanding something, gaining some insight into what is going on and why this is happening.

 

Step 3: Conceptual framework of the study

The conceptual framework of your study is the system of concepts, assumptions, expectations, beliefs, and theories that supports and informs your research. It is a formulation of what you think is going on with what you are studying—a tentative theory of what is happening and why. Theory provides a model or map of why the world is the way it is (Strauss, 1995) The function of theory in your design is to inform the rest of the design. The “research problem” is a part of your conceptual framework, and formulating the research problem is often seen as a key task in designing your study.

Ravitch & Riggan (2016) define a conceptual framework as an argument about why the topic one wishes to study matters, and why the means proposed to study it are appropriate and rigorous. By argument, they mean that a conceptual framework is a series of sequenced, logical propositions the purpose of which is to ground the study and convince readers of the study’s importance and rigor. Arguments for why a study “matters” vary greatly in scale, depending on the audience. In some scholarly work, the study may only matter to a small, esoteric community, but that does not change the fact that its conceptual framework should argue for its relevance within that community. By appropriate and rigorous, they mean that a conceptual framework should argue convincingly that:
(a) the research questions are an outgrowth of the argument for relevance;
(b) the research design maps onto the study goals, questions, and context(s);
(c) the data to be collected provide the researcher with the raw material needed to explore the research questions; and
(d) the analytic approach allows the researcher(s) to effectively address (if not always answer) those questions.

In order to start building the conceptual framework for your research project we truly recommend the following book: Ravitch, S. M., & Riggan, M. (2016). Reason & rigor : how conceptual frameworks guide research. Thousand Oaks : Sage Publications.

The following figure represents and briefly explains the main components a conceptual framework should have. 

 

Min components of a conceptual framework

 

Steps and Resources to start building  your conceptual framework

Step 1: Identify your personal connection with your research topic 

The following are questions that Ravitch & Riggan (2016) encourage you to explore in order to engage in a process of self-examination at the outset of your research and then iteratively throughout the research process. 

What is interesting to me and why? (In terms of my research topic)
What personal and professional motivations do I have for engaging in this research? How might these motivations influence how I think about and approach the topic?
What are my beliefs about the people, places, and ideas involved in and related to my study? Where do these beliefs come from? What assumptions underlie these beliefs?
What orientations to the topic, setting, and concepts do I have? Where do these ideas come from?
What is my sense of the relationship between the macro and micro sociopolitical circumstances in which people make meaning and choices in their lives? With respect to the participants in my study specifically?
What is my "agenda" for taking up this topic in this setting at this time? (Having an agenda is not necessarily a bad thing. This may be the foundation of your argument!) What influences this agenda? What biases shape this agenda?
How might my guiding agenda contribute, both positively and negatively, to my research design? Implementation? Analysis? Findings?
What hunches do I have about what I might find and discover? What informs these hunches?
What concerns, hopes, and expectations do I have for this research?

In addition to the responses to these questions, you can also take advantage of the goals defined in the second stage of the Hopscotch.  There are three types of goals for doing a study (Maxwell, 2008) that could help you shape your personal connection with the selected research topic:

  • Personal goals: those that motivate you to do this study; they can include a desire to change some existing situation, a curiosity about a specific phenomenon or event, or simply the need to advance your career.
  • Practical goals: are focused on accomplishing something—meeting some need, changing some situation, or achieving some goal.
  • Intellectual goals: re focused on understanding something, gaining some insight into what is going on and why this is happening
Step 2: Identify your identity and positionality as a researcher

This second step is deeply related to the paradigmatic view or worldview that you will be bringing to the studyas a researcher. Please have a look at information provided within the first step of Hopscotch (Step 1: Paradigmatic View of the Researcher) in order to understand the particularities of the main worldviews you might bring to your study. 

Step 3: Literature review

The third and main component of your conceptual framework will be the review of literature. The following  guides that have been generated by Dr. Olga Koz, Graduate Librarian at the Bagwell College of Education (Kennesaw State University) could be a great resource when working in the topical research and theoretical frameworks you will use to justify the relevance of your research topic and the need for the study you are proposing:  

  1. Literature Review for a Dissertation: a step-by-step guide
  2. Databases in education
  3. How to identify scholarly articles 
  4. Searching for education literature
  5. Finding Sources

Ravitch & Riggan (2016) propose two different sub-components in your literature review: a) Topical Research and; b) Theoretical Frameworks. 

"Topical research refers to previous work (most often empirical) that has focused on the topic in which you are interested. While much of this work resides within academic journals and books, it may also be found in policy or government research, or in reports produced through foundations, nonprofits, and advocacy organizations." 

They understand  a theoretical framework "as a set of formal theories and their relationships, that helps you to fill the intellectual bins that make up your conceptual framework." 

The following resource will be of help to identify topical research and theoretical frameworks to justify the relevance of your research topic, its pertinence and its theoretical roots. 

Resource to generate a visual representation of your Literature review

This form will help you build a visual representation of the topics emerging from 10 key readings that are deeply related with your research topic. The generated visual will also help you differentiate between the topical research and theoretical frameworks that will serve you to start building the conceptual framework for your research project. After filling out the form you will receive an email including a pdf document with a visual representation of the work done, as well as a link for you to modify your answers as many times as needed. In the following link you can see an example of the type of document you will get after filling out the form: goo.gl/TL9hsg The visual representation generated will help you build the literature review for your research project.

To identify the 10 articles which are most related to your research topic, or that inform your conceptual framework, you will have to read many more than 10. All articles must be primary sources and from peer-reviewed/refereed journals. To do so you can use any of the resources that we have included in the tables below (Resource to identify key topics in your field of research: Open Knowledge Map; Resource to graphically build your conceptual framework; Research guides; Science of Science (Sci2)).
You don´t have to use all of the resources provided, just the ones you believe could help you the most based on your previous knowledge. For instance, if you already know how to conduct a literature review, or how to identify scholarly articles, you might not have to use the "Research Guides." On the contrary, if you want to identify existing publications regarding your research topic, you might want to use the "Databases in Education," and "Open Knowledge Maps."

The following figure shows an example of the type of visual representation you will get after filling out the form.

Resource to identify key topics in your field of research: Open Knowledge Map

With "Open Knowledge Maps" you will be able to search for key concepts related to your research topic in order to generate a visual representation that will help you identify publications addressing that particular key concept.

Resource to identify key literature in your field of research: Science of Science (Sci2)

The Science of Science (Sci2) Tool is a modular toolset specifically designed for the study of science. It supports the temporal, geospatial, topical, and network analysis and visualization of scholarly datasets at the micro (individual), meso (local), and macro (global) levels. This tool can be used to conduct thorough literature reviews. 

Resource: Research Guides

The following  guides that have been generated by Dr. Olga Koz, Graduate Librarian at the Bagwell College of Education (Kennesaw State University) could be a great resource when working in the topical research and theoretical frameworks you will use to justify the relevance of your research topic and the need for the study you are proposing:  

  1. Literature Review for a Dissertation: a step-by-step guide
  2. Databases in education
  3. How to identify scholarly articles 
  4. Searching for education literature
  5. Finding Sources
Resource to Define your problem statement

One key component of your conceptual framework will be the "Problem Statement." You can use this template to define it after having conducted your review of literature.

Resource to graphically build your conceptual framework

You can use this template to create a visual representation of the main components that should be included in the conceptual framework of your research project. In order to be able to use the template, you will have to log in your Google account so the system can ask you to make a copy of it. 

Step 4: Research design/tradition

There are five main types of quantitative research designs: descriptive, correlational, pre-experimental, quasi-experimental and experimental. The differences between the four types primarily relates to the degree the researcher designs for control of the variables in the experiment. Following is a brief description of each type of quantitative research design.

  • Descriptive Designs
  • Correlational Designs
  • Pre-experimental Designs
  • Quasi-experimental Designs
  • Experimental Designs
Descriptive Designs

 

Descriptive designs are intended to describe an educational situation or reality and / or classify it in a certain category. They are very frequent in education and in general in the field of social sciences. Most of the descriptive studies are carried out through questionnaires or observations. Hence, the types of descriptive designs that can be carried out are survey-research and observational.

An example of descriptive design tries to answer the following question: what is the use of mobile phones by teenagers throughout the day? It is interesting to know the following variables: the daily time dedicated and the differential frequency of use of Whatsapp, social networks, and calls.

Survey-research: The main objective of a survey-research design is to describe features or characteristics of a group or population through the responses provided by the participants to a questionnaire or interview administered by the researcher. They seek to collect information referring to the entire population, and in cases where this is not possible, a sample that represents it. The main instruments in this type of design are questionnaires and structured or semi-structured interviews. The information collected through the questionnaires is usually of two types: a) sociodemographic characteristics of the participants, such as gender, profession, age, etc., and b) attitudes, opinions, perceptions, behaviors, habits, experiences, etc.  Because the objective of these designs is to see the distribution of a certain variable -for example, the opinion of people over 65 years of age regarding the digitalization of the health system in Madrid- in the entire population -persons of 65 who live in Madrid-, it is vitally important that when this is not possible, a sample is selected through the probabilistic sampling procedures that allow to represent the population from which it has been extracted and thus generalize the results. In addition to the rigor in the selection of the sample, another of the decisive questions in this type of research is the validation of the instrument that will be used when it is prepared by the researcher.

Observational designs: Observation is defined by Martínez-González (2007) as the act of looking carefully at something without intervening in its natural course, with the intention of examining it, interpreting it and obtaining conclusions. The fact of being talking about observing, may introduce some confusion with the observation used in qualitative research. In this case, we are referring to an intentional, planned, structured observation,  registered in an objective manner, and seeking the explanation of the observed phenomenon. That is, what is observed is quantified by different criteria such as frequency, intensity, domain, etc. One of the most complicated aspects of an intentional observation is the work that needs to be done to define what needs to be observed.  For example, if we want to observe the attachment behavior of children with their teacher in early childhood education, we must explicitly and clearly specify which behaviors characterize each type of attachment in order to identify them when conducting an observation. It will also be necessary to determine if what matters is the appearance of the behavior, its frequency, or perhaps the registration of patterns of relationship between the teacher and the child. To carry out this process we have to be very sure that a certain behavior expresses what we want to measure. Hence the difficulty to observe internal aspects of the human being, which have a difficult manifestation sometimes to interpret, such as moods, thoughts or even emotions. As described in the case of survey-research studies, the measurements of an observational study can be carried out over time (longitudinal) or at a single moment (cross-sectional). The most frequently used observational designs can be found in the clinical field. The instruments for collecting observational information in a quantifiable way are observational codes, some of them already elaborated, validated and published for use in samples determined according to context and age. Control lists or estimation scales are also common.

You can use the following tool to generate a visual representation of a Descriptive non-experimental design:


Correlational Designs

 

A Correlational Design explores the relationship between variables using statistical analyses. However, it does not look for cause and effect and therefore, is mostly observational in terms of data collection. A correlational study determines whether or not two variables are correlated. This means to study whether an increase or decrease in one variable corresponds to an increase or decrease in the other variable.

There are three types of correlations that are identified:

Positive correlation: Positive correlation between two variables is when an increase in one variable leads to an increase in the other and a decrease in one leads to a decrease in the other. For example, the amount of money that a person possesses might correlate positively with the number of cars he owns.

Negative correlation: Negative correlation is when an increase in one variable leads to a decrease in another and vice versa. For example, the level of education might correlate negatively with crime. This means if by some way the education level is improved in a country, it can lead to lower crime. Note that this doesn't mean that a lack of education causes crime. It could be, for example, that both lack of education and crime have a common reason: poverty.

No correlation: Two variables are uncorrelated when a change in one doesn't lead to a change in the other and vice versa. For example, among millionaires, happiness is found to be uncorrelated to money. This means an increase in money doesn't lead to happiness.

There are essentially two reasons that researchers interested in statistical relationships between variables would choose to conduct a correlational study rather than an experiment. The first is that they do not believe that the statistical relationship is a causal one. The second reason that researchers would choose to use a correlational study rather than an experiment is that the statistical relationship of interest is thought to be causal, but the researcher cannot manipulate the independent variable because it is impossible, impractical, or unethical.

The two main types of correlational designs are:

Explanatory Correlational Design: An explanatory design seeks to determine to what extent two or more variablesco-vary. Co-vary simply means the strength of the relationship of one variable to another. In general, two or more variables can have a strong, weak, or no relationship. This is determined by the product moment correlation coefficient, which is usually referred to as r. The r is measured on a scale of -1 to 1. The higher the absolute value the stronger the relationship.

Prediction Correlational Design: Prediction design has most of the same functions as explanatory design with a few minor changes. In prediction design, we normally do not use the term explanatory and response variable. Rather we have predictor and outcome variable as terms. This is because we are trying to predict and not explain. In research, there are many terms for independent and dependent variable and this is because different designs often use different terms.

 

You can use the following tool to generate a visual representation of your Correlational Design:


Pre-experimental Designs

Pre-experiments are the simplest form of research design. In a pre-experiment either a single group or multiple groups are observed subsequent to some agent or treatment presumed to cause change.

Types of Pre-Experimental Design

One-shot case study design: A single group is studied at a single point in time after some treatment that is presumed to have caused change. The carefully studied single instance is compared to general expectations of what the case would have looked like had the treatment not occurred and to other events casually observed. No control or comparison group is employed.

One-group pretest-posttest design: A single case is observed at two time points, one before the treatment and one after the treatment. Changes in the outcome of interest are presumed to be the result of the intervention or treatment. No control or comparison group is employed.

Advantages of pre-experimental designs: 

As exploratory approaches, pre-experiments can be a cost-effective way to discern whether a potential explanation is worthy of further investigation.

Disadvantages of pre-experimental designs: 

Pre-experiments offer few advantages since it is often difficult or impossible to rule out alternative explanations. The nearly insurmountable threats to their validity are clearly the most important disadvantage of pre-experimental research designs.

 

You can use the following tool to generate a visual representation of your Pre-Experimental Design


Quasi-experimental Designs

Quasi-experimental designs examine cause-and-effect relationships between or among independent and dependent variables. However, one of the characteristics of true-experimental design is missing, typically the random assignment of subjects to groups. Although quasi-experimental designs are useful in testing the effectiveness of an intervention and are considered closer to natural settings, these research designs are exposed to a greater number of threats of internal and external validity, which may decrease confidence and generalization of study's findings.

Nonequivalent control group designs: The most common subset of quasi-experimental research designs are the nonequivalent control group designs.  In one implementation of this design, subjects in the control group are intentionally matched by the researcher to subjects in the treatment group on characteristics which might be associated with the outcome of interest.  This matching can be done at the individual level, resulting in a one-to-one match of individuals in the two groups.  Another approach is aggregate matching, in which researchers select a control group with the same general composition of relevant characteristics (for example, the same proportion of females and the same age distribution) as the treatment group.  These approaches are considered quasi-experimental due to the fact that assignment of subjects to groups is intentional and not random.  Another common approach to this type of quasi-experimental research design is the use of existing groups.  For example, a comparison could be made between students in two classrooms, with the stimulus administered in only one classroom.

Time-series data Designs: Another quasi-experimental approach involves time-series data, in which researchers observe one group of subjects repeatedly both before and after the administration of the treatment.  This can be done in a controlled experimental setting, but this design also lends itself well to a more naturalistic setting in which data are commonly collected on a group of subjects and researchers are interested in the effects of some treatment or intervention which they did not experimentally apply.  For example, researchers might examine the yearly test scores of students at a given school for several years both before and after the implementation of an extended school day; in this situation the yearly tests scores represent the time-series data and the change to an extended school day is the naturally occurring, quasi-experimental treatment.  This approach is an improvement over the single pre-test/post-test design, which is unable to demonstrate long-term effects.  The time-series data design can be further improved by including a control group which is also examined over time but which does not experience the treatment; such a design is termed a multiple time-series design.

While quasi-experimental designs are often more practical to implement than true experiments, they are more susceptible to threats to internal validity.  Special care must be taken to address validity threats, and the use of additional data to rule out alternate explanations is advised.

 

You can use the following tool to generate a visual representation of your Quasi-experimental Design


Experimental Designs

An experiment is a study in which the researcher manipulates the level of some independent variable and then measures the outcome. Experiments are powerful techniques for evaluating cause-and-effect relationships. Many researchers consider experiments the "gold standard" against which all other research designs should be judged. Experiments are conducted both in the laboratory and in real life situations.

True experiments, in which all the important factors that might affect the phenomena of interest are completely controlled, are the preferred design. Often, however, it is not possible or practical to control all the key factors, so it becomes necessary to implement a quasi-experimental research design.

In an experiment, the researcher manipulates the factor that is hypothesized to affect the outcome of interest. The factor that is being manipulated is typically referred to as the treatment or intervention. The researcher may manipulate whether research subjects receive a treatment and the level of treatment.

Example of true experiment: Suppose, for example, a group of researchers was interested in the causes of maternal employment. They might hypothesize that the provision of government-subsidized child care would promote such employment. They could then design an experiment in which some subjects would be provided the option of government-funded child care subsidies and others would not. The researchers might also manipulate the value of the child care subsidies in order to determine if higher subsidy values might result in different levels of maternal employment.

Random Assignment

-Study participants are randomly assigned to different treatment groups
-All participants have the same chance of being in a given condition
-Participants are assigned to either the group that receives the treatment, known as the "experimental group" or "treatment group," or to the group which does not receive the treatment, referred to as the "control group"
-Random assignment neutralizes factors other than the independent and dependent variables, making it possible to directly infer cause and effect

Random Sampling

-Traditionally, experimental researchers have used convenience sampling to select study participants. However, as research methods have become more rigorous, and the problems with generalizing from a convenience sample to the larger population have become more apparent, experimental researchers are increasingly turning to random sampling. In experimental policy research studies, participants are often randomly selected from program administrative databases and randomly assigned to the control or treatment groups.

Advantages of Experimental Designs

The environment in which the research takes place can often be carefully controlled. Consequently, it is easier to estimate the true effect of the variable of interest on the outcome of interest.

Disadvantages of Experimental Designs

It is often difficult to assure the external validity of the experiment, due to the frequently nonrandom selection processes and the artificial nature of the experimental context.

 

You an use the following tool to create your Experimental Design


The following clip shows a description of the main types of quantitative research designs

 

The following decision tree explains briefly the main features of each of the aforementioned research designs in quantitative educational research. 

The following resources might also be of further help in order to decide the quantitative design that best fits the needs of your study:

Once you have picked one quantitative research design, you can use the following links in order to generate a visual representation of the different key elements in your design:

Step 5: Research Questions

Your research questions—what you specifically want to learn or understand by doing your study—are at the heart of your research design.

They connect to all the other components of the design Models of design that place the formulation of research questions at the beginning of the design process, and that see these questions as determining the other aspects of the design, don’t do justice to the interactive and inductive nature of qualitative research.

The research questions in a qualitative study should not be formulated in detail until the goals and conceptual framework (and sometimes general aspects of the sampling and data collection) of the design are clarified, and should remain sensitive and adaptable to the implications of other parts of the design. Two functions:

1- To help you focus the study (the questions’ relationship to your goals and conceptual framework)
2- To give you guidance for how to conduct it (their relationship to methods and validity)

Main problems in the development of Research Questions:

1- A common problem in the development of research questions is confusion between research issues (what you want to understand by doing the study) and practical issues (what you want to accomplish ) (practical goals Vs. intellectual goals). Your research questions need to connect clearly to your practical concerns, but in general an empirical study cannot directly answer practical questions such as, “How can I improve this program?” or “What is the best way to increase students’ knowledge of science?”

2- A second confusion, one that can create problems for interview studies, is that between research questions and interview questions. Your research questions identify the things that you want to understand; your interview questions generate the data that you need to understand these things.

Step 8: Trustworthiness/Validity

In quantitative educational research we usually consider two general dimensions in evaluating a measurement method: reliability and validity.

Reliability is defined as the consistency of the measurements. To what level will the instrument produce the same results under the same conditions every time it is used? Reliability adds to the trustworthiness of the results because it is a testament to the methodology if the results are reproducible. The reliability is often examined by using a test and retest method where the measurement are taken twice at two different times. The reliability is critical for being able to reproduce the results, however, the validity must be confirmed first to ensure that the measurements are accurate. Consistent measurements will only be useful if they are accurate and valid.

The term validity refers to the strength of the conclusions that are drawn from the results. In other words, how accurate are the results? Do the results actually measure what was intended to be measured? There are several types of validity that are commonly examined and they are as follows: 

  • Conclusion validity looks at whether or not there is a relationship between the variable and the observed outcome. 
  • Internal validity considers whether or not that relationship may be causal in nature.
  • Construct validity refers to whether or not the operational definition of a variable actually reflects the meaning of the concept. In other words, it is an attempt to generalize the treatment and outcomes to a broader concept.
  • External validity is the ability to generalize the results to another setting. There are multiple factors that can threaten the validity in a study. They can be divided into single group threats, multiple group threats, and social interaction threats.

For more information regarding the control of the validity of a quantitative study please read chapter 6 in (Price, Jhangiani, & Chiang, 2015) 


Resources: 

Step 9: Ethics driving the study

The term “ethics”  derives from the Greek word “ethos” which means character. To engage with the ethical dimension of your research requires asking yourself several important questions:

-What moral principles guide your research?

-How do ethical issues enter into your selection of a research problem?

-How do ethical issues affect how you conduct your research—the design of your study, your sampling procedure, etc.?

-What responsibility do you have toward your research subjects? For example, do you have their informed consent to participate in your project? 

-What ethical issues/dilemmas might come into play in deciding what research findings you publish? 

-Will your research directly benefit those who participated in the study?

The major principles associated with ethical conduct  are (Litchman, 2011):

1-    Do No Harm: 

-       It is the cornerstone of ethical conduct

-       There should be a reasonable expectation by those participating in a research study that they will not be involved in any situation in which they might be harmed.

-       Often applied to studies involving drugs or a treatment that might be harmful to participants

-       The 1971 Stanford Prison Experiment, in which students played the role of guards and prisoners, is one example. When it was found that the guards became increasingly sadistic, the study was terminated.

-       Recommendation: It is best to safeguard against doing anything that will harm the participants in your study. If you begin a study and you find that some of your participants seem to have adverse reactions, it is best to discontinue the study, even if it means foregoing your research plan.


2-    Privacy and Anonymity:

-       Any individual participating in a research study has a reasonable expectation that privacy will be guaranteed. Consequently, no identifying information about the individual should be revealed in written or other communication. Further, any group or organization participating in a research study has a reasonable expectation that its identity will not be revealed.

-       Recommendation: Remove identifying information from your records. Seek permission from the participants if you wish to make public information that might reveal who they are or who the organization is. Use caution in publishing long verbatim quotes, especially if they are damaging to the organization or people in it. Often, these quotes can be located on the Internet and traced to the speaker or author.


3-    Confidentiality: 

-       Any individual participating in a research study has a reasonable expectation that information provided to the researcher will be treated in a confidential manner. Consequently, the participant is entitled to expect that such information will not be given to anyone else.

-       Recommendation: It is our responsibility to keep the information you learn confidential. If you sense that an individual is in an emergency situation, you might decide that you can waive your promise for the good of the individual or of others. You need to be much more sensitive to information that you obtain from minors and others who might be in a vulnerable position.


4-    Informed Consent: 

-       Individuals participating in a research study have a reasonable expectation that they will be informed of the nature of the study and may choose whether or not to participate. They also have a reasonable expectation that they will not be coerced into participation.

-       Recommendation: Our responsibility is to make sure that participants are informed, to the extent possible, about the nature of your study. Even though it is not always possible to describe the direction your study might take, it is your responsibility to do the best you can to provide complete information. If participants decide to withdraw from the study, they should not feel penalized for so doing. You need to be aware of special problems when you study people online. For example, one concern might be vulnerability of group participants. Another is the level of intrusiveness of the researcher.


5-    Rapport and Friendship: 

-       Once participants agree to be part of a study, the researcher develops rapport in order to get them to disclose information.

-       Recommendation: Researchers should make sure that they provide an environment that is trustworthy. At the same time, they need to be sensitive to the power that they hold over participants. Researchers need to avoid setting up a situation in which participants think they are friends with the researcher.


6-    Intrusiveness:

-       Individuals participating in a research study have a reasonable expectation that the conduct of the researcher will not be excessively intrusive. Intrusiveness can mean intruding on their time, intruding on their space, and intruding on their personal lives. As you design a research study, you ought to be able to make a reasonable estimate of the amount of time participation will take.

-       Recommendation: Experience and caution are the watchwords. You might find it difficult to shift roles to neutral researcher, especially if your field is counseling or a related helping profession.


7-    Inappropriate Behavior:

-       Individuals participating in a research study have a reasonable expectation that the researcher will not engage in conduct of a personal or sexual nature.

-       Here, researchers might find themselves getting too close to the participants and blurring boundaries between themselves and others. We probably all know what we mean by inappropriate behavior. We know it should be avoided

-       Recommendation: If you think you are getting too close to those you are studying, you probably are. Back off and remember that you are a researcher and bound by your code of conduct to treat those you study with respect.


8-    Data Interpretation:

-       A researcher is expected to analyze data in a manner that avoids misstatements, misinterpretations, or fraudulent analysis. The other principles discussed involve your interaction with individuals in your study. This principle represents something different. It guides you to use your data to fairly represent what you see and hear. Of course, your own lens will influence you.

-       Recommendation: You have a responsibility to interpret your data and present evidence so that others can decide to what extent your interpretation is believable.

 

9-    Data Ownership and Rewards:

-       In general, the researcher owns the work generated. Some researchers choose to archive data and make them available through databanks. Questions have been raised as to who actually owns such data. Some have questioned whether the participants should share in the financial rewards of publishing. Several ethnographers have shared a portion of their royalties with participants.

-       Recommendation: In fact, most researchers do not benefit financially from their writing. It is rare that your work will turn into a bestseller or even be published outside your university. But, if you have a winner on hand, you might think about sharing some of the financial benefits with others.