Most Viewed 2014 Posts from the Chronicle of Evidence-Based Mentoring, V

The last of the series...

By Jean Rhodes October 13, 2014

On Methods: What’s the difference between qualitative and quantitative approaches?

Editors Note: We are very fortunate to have Adar Ben-Eliyahu, Ph.D. our new Methods Editor, sharing with us her considerable expertise on research methods. Dr. Ben-Eliyahu completed her doctorate in developmental psychology at Duke University, where she honed very strong skills in methods. She is currently a MacArthur Foundation Postdoctoral Fellow at the Center for Evidence-Based Mentoring at the University of Massachusetts in Boston. Prior to arriving at Boston, Dr. Ben-Eliyahu was a post-doc in the Activation Lab at the Learning Research & Development Center at the University of Pittsburgh. Enjoy!

Understanding different types of research:
What’s the difference between qualitative and quantitative approaches?

Adar Ben-Eliyahu, Ph.D.

In the world of research, there are two general approaches to gathering and reporting information: qualitative and quantitative approaches. The qualitative approach to research is focused on understanding a phenomenon from a closer perspective. The quantitative approach tends to approximate phenomena from a larger number of individuals using survey methods. In this research corner, I describe methods that are generally used in each strand of research. Each approach has its benefits and detriments, and is more suitable to answering certain kinds of questions.

Qualitative Approach

The qualitative approach to gathering information focuses on describing a phenomenon in a deep comprehensive manner. This is generally done in interviews, open-ended questions, or focus groups. In most cases, a small number of participants participate in this type of research, because to carry out such a research endeavor requires many resources and much time. Interviews can vary from being highly structured and guided by open-ended questions, or be less structured and take the form of a conversational interview. Because of the investment in this type of research and the relatively few number of participants, findings from qualitative research cannot be generalized to the whole population. However, such research serves as a spring board for larger studies and deeper understanding that can inform theory, practice, and specific situations.

Example from youth mentoring research:

Ahrens, DuBois, Garrison, Spencer, Richardson, & Lozano (2011) used semi-structured interviews to outline themes of mentor characteristics and factors that youth perceive to influence mentor relationships. They spoke with participants on the phone and asked them open-ended questions. In identifying barriers and facilitators for relationship initiation and maintenance, Ahrens et al. provide important points of inquiry to be used in a larger scale survey-based research. One of the cautions in using qualitative approaches is that the findings apply only to this small group of 23 individuals. This research was crucial in providing evidence that these factors should be examined and further elaborated through quantitative methods prior to making any wide-range recommendation.  (Click to read a summary of this studyHERE).

Benefits of the qualitative approach:

Using open-ended questions and interviews allows researchers and practitioners to understand how individuals are doing, what their experiences are, and recognize important antecedents and outcomes of interest that might not surface when surveyed with pre-determined questions. Although qualitative research can be thought of as anecdotal, when pooled across a number of participants it provides a conceptual understanding and evidence that certain phenomena are occurring with particular groups or individuals.

• Allows identification of new and untouched phenomena
• Can provide a deeper understanding of mechanisms
• Gives a one-on-one and anecdotal information
• Provides verbal information that may sometimes be converted to numerical form
• May reveal information that would not be identified through pre-determined survey questions

Limitations:

• Cannot generalize to the general population
• Challenges in applying statistical methods
• Difficulty in assessing relations between characteristics

Quantitative Approach

The quantitative approach to gathering information focuses on describing a phenomenon across a larger number of participants thereby providing the possibility of summarizing characteristics across groups or relationships. This approach surveys a large number of individuals and applies statistical techniques to recognize overall patterns in the relations of processes. Importantly, the use of surveys can be done across groups. For example, the same survey can be used with a group of mentors that is receiving training (often called the intervention or experimental groups) and a group of mentors who does not receive such a training (a control group). It is then possible to compare these two groups on outcomes of interest, and determine what influence the training had. It is also relatively easy to survey people a number of times, thereby allowing the conclusion that a certain features (like matching) influence specific outcomes (well-being or achievement later in life).

Example from youth mentoring research:

Grossman and Rhodes (2002) examined duration of matched relationships in over 1,100 Big Brothers Big Sisters mentor-mentee matches. Because the information they used was survey-based and numerical, they were able to employ statistical techniques examining how duration of match was related to different outcomes of interest.

In using a variety of statistical techniques, they concluded that “youth who were in [matched mentoring] relationships that lasted a year or longer reported improvements in academic, psychosocial, and behavioral outcomes” (p. 213). If Grossman and Rhodes had not used survey-based quantitative research, they would not have had such a large sample of matches and therefore could not generalize to matches in general. In addition, with a smaller number of participants, it is challenging to apply some statistical techniques to examine emerging patterns across such a large group of mentored matches. The current rule of thumb to using complex statistical modeling is that you need a sample of at least 130 participants. However, for more complex modeling that controls for characteristics, a larger pool of participants is needed.

Benefits of the quantitative approach:

Using survey methods across a large group of individuals enables generalization. For example, if policy makers wanted to instantiate a policy about mentor training, they would likely require some evidence that this training actually works. Interviewing a few individuals, or conducting a focus group with forty matches, might be reflective of specific cases in which the mentoring training worked, however, it would not provide strong evidence that such training is beneficial overall. Stronger support for successful training would be evident if using quantitative methods.

• Enables gathering information from a relatively large number of participant
• Can conduct in a number of groups, allowing for comparison
• Allows generalizing to broader population
• Provides numerical or rating information
• Informative for instantiating policy or guidelines
• Lends to statistical techniques that allow determining relations between variables (think of better word)

Limitations:

• Difficulty in recognizing new and untouched phenomena
• Caution in interpretation without a control group

In summary, the qualitative and quantitative approaches to research allow a different perspective of situations or phenomena. These two main approaches to research are highly informative, especially if used in combination. Each approach has its benefits and detriments, and being aware of the methods used to gather information can help practitioners and policy-makers understand the extent to which research findings can be applied.

http://chronicle.umbmentoring.org/on-methods-whats-the-difference-between-qualitative-and-quantitative-approaches/ 

Ret. 12-23-14

Most Viewed 2014 Posts from the Chronicle of Evidence-Based Mentoring, III

Below is number three reposted.

Beyond “how’s it going?”: A new questionnaire assesses mentors’ satisfaction

Rhodes, J., Schwartz, S., & Wu, M. (2014) Validating a Mentoring Relationship Quality Scale: Does Match Strength Predict Match Length. Youth & Society. 1-14

by Jean Rhodes

Remarkably few studies have focused on mentors’ experiences in relationships. To address this gap, my colleagues and I designed and validated a mentor-reported measure of relationship quality. The result is a new questionnaire that might be useful not only for evaluation but for everyday case management. Notably the scale (the Mentor Strength of Relationship Scale) represents the first mentor-report relationship quality scale with established psychometric properties.

Background

A growing number of studies have highlighted the importance of investigating the quality of the relationships that are formed between mentors and youth. Although several questionnaires have been developed to measure mentor-youth relationships, researchers have tended to focus on mentees’ perspectives. Nonetheless, studies using scales that were originally designed for other caring adults (e.g., for teachers or psychotherapists) have highlighted the importance of assessing mentors’ perspectives as well. As part of the longitudinal data collection efforts of BBBS community-based mentoring programs nationwide, my students and I sought develop and test the properties of the mentor version of the Strength of Relationship (MSoR) scales, which assesses relationship quality from the mentor’s perspective (MSoR).

Data were drawn from a network of 85 of the BBBS agencies running community-based mentoring programs across the United States. Of the 7,757 mentor and youth dyads that formed during this time, a total of 5,222 dyads completed 3-month SoR surveys, and a subsample of 1,294 dyads completed both the three-month and 12-month SoR surveys.

The Mentor Strength of Relationship (MSoR) Scale consists of fourteen mentor-reported items which were developed as part of my year-long stint as a match support specialist at two different mentoring agencies. In this capacity, I maintained ongoing contact with the volunteer mentors, youth, and parents in her caseload, provided ongoing supervision to a wide range of adult-youth dyads, and attended match support meetings where common match issues and their resolution were discussed among agency staff. From this activity, as well as discussions with additional practitioners (thank you Big Sisters of Greater Boston and Big Brothers Big Sisters of Mass Bay) and researchers, a list of 75 items that captured the experiences and frustrations of mentors in relation to their mentees was created. From this item pool, fourteen items that best captured mentors’ common positive and negative perceptions and experiences in relationships were chosen rationally by a panel of three mentoring researchers and three mentoring practitioners for inclusion in the.

On the scale, mentors were prompted to rate the extent to which they agreed with the fourteen items in the final pool, such as “My Little and I are interested in the same things” and “I feel close to my Little”  Answers were scored on a five-point Likert scale, ranging from (1) “strongly disagree” to (5) “strongly agree. Internal consistency for the fourteen items was strong, alpha = .85.

Youth Strength of Relationship (YSoR) Scale was  is a version of the original Relationship Quality Scale (Rhodes, Reddy, Roffman & Grossman, 2005), which was slightly revised to include a more balanced set of positive and negative relationship experiences.

Analyses demonstrated that both the MSoR and the YSoR were significantly associated with relationship duration. These results suggest that early assessment of mentor and youth perceptions of their relationship quality can be used to identify mentor-youth matches that are at greater risk for early termination. This is particularly important in the context of previous research indicating that early terminating relationships may result in negative impacts. Early identification of more vulnerable mentoring relationships could allow programs to provide extra support that could strengthen relationships and possibly avoid early termination.

Moreover, since relationship duration is a well-established benchmark of program effectiveness, which predicts youth impacts in both CBM and SBM programs, these results suggest that the SoR may also predict the benefits that youth will derive from mentoring relationships. It should be noted, however, that the SoR accounted for a relatively small percentage of the variance in relationship duration, raising questions about what other factors, besides strength of relationship, may contribute to how long mentoring relationships last. Nevertheless, these results suggest that negative responses might be used as a screening tool to identify relationships that are at risk for early termination and target them for additional support. Particularly in the context of large caseloads, the items on the MSoR scale might provide an efficient framework for eliciting mentors’ reports of more specific early problems than typically emerge from the more general, “how are things going?,” prompt.

While not the focus of this study, results also provided information about other predictors of mentoring relationship duration. Specifically, child age, mentor age, and mentor education were significantly associated with length of relationship, beyond the variance accounted for by the SoR. These findings are generally consistent with previous research, which indicates that mentoring relationships with younger mentees tend to last longer than those with older mentees and that higher mentor education, income level, and age are predictive of longer-lasting relationships.

Although future research is needed for the continued validation and refinement of the scale, the SoR appears to be a psychometrically sound instrument that can be used to measure relationship quality from the perspective of mentors, and is the first validated instrument to include the mentor’s perspective. It also indicates that quality of relationship, as measured by the SoR at 3 months, can predict relationship duration, suggesting that it may be a useful tool for identifying matches that may be at risk for early endings.

SoR Measure (please cite Rhodes, Schwartz, Willis & Wu, 2014). Items 14 and 15 not part of the scale, but suggested.

http://chronicle.umbmentoring.org/beyond-hows-it-going-a-new-questionnaire-to-assess-mentors-satisfaction/

Ret. 12-23-14