Sample Survey from Mentoring Event

While objective stories tug at people's hearts and sometimes recruit volunteers and/or money, data and evaluation are essential. Below we share a survey template from the FIRST Women in STEM Reception 2016.

Adapt this for any speaker/informational event or share your own evaluation. We have added a few suggestions from the internal actual report about the event.

Thanks to Barbara Wollmershauser of Tulsa Engineering Foundation for sharing.

Robotics Competitor/Mentee

Robotics Competitor/Mentee

SURVEY for MENTEES

Question	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree	Didn’t Answer	Total
1.    Before this event, I knew engineers have various job functions
2.    Engineers are innovative.
3.    Engineers are creative.
4.    Engineers perform hands-on job functions.
5.    Engineers work in many different career fields.
6.    Before this event, I was interested in becoming an engineer.
7.    After this event, I was interested in becoming an engineer.
8.    The guest speaker was informative and engaging.
9.    Engineers do work that enables them to help the community.
10.  I see a connection between my interests/passions & engineering.

EXPENSES *
Printing -
Food -
Giveaways/Door Prizes, e.g., pins, books, etc. -
Signage -
Other -
Staff hours and value if applicable -
Volunteer hours and value -
TOTAL
      * A detailed budget is essential, but reporting can grouped and rounded to nearest dollar.

ATTENDEES
# of  Mentees
# of Mentors
# of Other Volunteers, Staff, and/or Board

OTHER
Excel spreadsheets for relevant detailed data
Organizations represented
Disciplines represented
Sponsors/Donors
Planning committee & tasks assigned
Volunteer hours (before, at and after event)
Sample program and/or brochure

PRE- and POST-PUBLICITY
Which media - newspaper, newsletter, social media, etc.
Frequency or amount, e.g., shares on Facebook, picture with caption, article, etc.


A selected combination of these becomes the blueprint from which to evaluate for both impact and improvement and create future events.

College Grads: Mentors = Game Changers

60,000 college graduates have spoken: Mentors are a game changer!

By Jean Rhodes September 13, 2015

The field of mentoring owes a great debt to the Gallup-Purdue Index, a study whose goal is to “to conduct the largest representative study of college graduates in United States history.” To date, the team has surveyed 60,000 college graduates, resulting in a 2014 report and another report to be released in the coming weeks. The 2014 report highlights the vital role of mentors in college students’ graduation, ongoing engagement in work, and overall wellbeing.

As the lead author notes,  “We learned that if graduates felt “supported” during college — by professors who cared, made them excited about learning and who encouraged them to pursue their goals and dreams — their odds of being engaged in work more than doubled, as did their odds of thriving in their well-being. This finding was true of graduates of all ages and years of graduation; in other words, it’s a career- and life-trajectory game changer.”

In the more recent report, life success was determined by conventional markers (e.g., employment, salary) as well as students’ reports about their satisfaction with 

·         their relationships,

·         their physical health,

·         their community,

·         their economic situation

·         and their sense of purpose.

Once again, college mentors emerged as a key factor accounting for better outcomes overall and greater satisfaction. In particular, as previewed by Frank Bruni in a recent NYTimes column,  satisfaction was predicted, in part by students’ college experience, including whether or not the student

·         developed a relationship with a mentor;

·         took on a project that lasted a semester or more;

·         did a job or internship directly connected to their chosen field;

·         or became deeply involved in a campus organization or activity 

Although readers are cautioned to consider the self-selection biases inherent in survey’s of this nature (i.e., better students seek out mentors and deep involvement in projects), the report found that these effects emerged regardless of the students’ personality.  

Unfortunately,  less than a quarter of national graduates strongly agreed that had received the support of mentors who encouraged them to pursue their goals and dreams . Given the vital influence of mentors for educational achievement, career success, and overall life satisfaction, we need to both teach students to “fish” for a support and to stock the pond with caring adults. Here are a few suggestions.

1.    Teach students to “fish”–although star students will always find their way to faculty mentors, many students are left behind. Some simply don’t see the value or are too busy. Others feel less entitled or are too embarrassed. To resolve this, colleges should provide training in the skills necessary to do so. Along these lines, my colleagues and I have been developing and evaluating a semester-long course in social networking for college students.

2.    Stock the pond–College networks include many caring adults–not just professors but academic staff, graduate students, advisors, alumni, parents, and others who can serve in this capacity. With additional structure, encouragement, incentives, and training, the the true caring potential of college campuses can be more fully realized.

As the Gallup data, mentors are a key active ingredient in college success, and their influence pays forward across the lifetime. As such, we cannot leave this ingredient to chance. Learning how to recruit and effectively engage with mentors and other caring adults, and how to build what Murphy and Kram refer to as a “developmental network,”  is every bit as important as learning many other subjects, perhaps even more so.

http://chronicle.umbmentoring.org/what-the-new-gallup-poll-tells-us-about-student-success/?utm_source=Chronicle+Subscribers&utm_campaign=92f53f5b72-September_179_17_2015&utm_medium=email&utm_term=0_afc94e8372-92f53f5b72-116180833&ct=t(September_179_17_2015)&mc_cid=92f53f5b72&mc_eid=c7177e3427

Ret. 9-18-15

Most Viewed 2014 Posts from the Chronicle of Evidence-Based Mentoring, V

The last of the series...

By Jean Rhodes October 13, 2014

On Methods: What’s the difference between qualitative and quantitative approaches?

Editors Note: We are very fortunate to have Adar Ben-Eliyahu, Ph.D. our new Methods Editor, sharing with us her considerable expertise on research methods. Dr. Ben-Eliyahu completed her doctorate in developmental psychology at Duke University, where she honed very strong skills in methods. She is currently a MacArthur Foundation Postdoctoral Fellow at the Center for Evidence-Based Mentoring at the University of Massachusetts in Boston. Prior to arriving at Boston, Dr. Ben-Eliyahu was a post-doc in the Activation Lab at the Learning Research & Development Center at the University of Pittsburgh. Enjoy!

Understanding different types of research:
What’s the difference between qualitative and quantitative approaches?

Adar Ben-Eliyahu, Ph.D.

In the world of research, there are two general approaches to gathering and reporting information: qualitative and quantitative approaches. The qualitative approach to research is focused on understanding a phenomenon from a closer perspective. The quantitative approach tends to approximate phenomena from a larger number of individuals using survey methods. In this research corner, I describe methods that are generally used in each strand of research. Each approach has its benefits and detriments, and is more suitable to answering certain kinds of questions.

Qualitative Approach

The qualitative approach to gathering information focuses on describing a phenomenon in a deep comprehensive manner. This is generally done in interviews, open-ended questions, or focus groups. In most cases, a small number of participants participate in this type of research, because to carry out such a research endeavor requires many resources and much time. Interviews can vary from being highly structured and guided by open-ended questions, or be less structured and take the form of a conversational interview. Because of the investment in this type of research and the relatively few number of participants, findings from qualitative research cannot be generalized to the whole population. However, such research serves as a spring board for larger studies and deeper understanding that can inform theory, practice, and specific situations.

Example from youth mentoring research:

Ahrens, DuBois, Garrison, Spencer, Richardson, & Lozano (2011) used semi-structured interviews to outline themes of mentor characteristics and factors that youth perceive to influence mentor relationships. They spoke with participants on the phone and asked them open-ended questions. In identifying barriers and facilitators for relationship initiation and maintenance, Ahrens et al. provide important points of inquiry to be used in a larger scale survey-based research. One of the cautions in using qualitative approaches is that the findings apply only to this small group of 23 individuals. This research was crucial in providing evidence that these factors should be examined and further elaborated through quantitative methods prior to making any wide-range recommendation.  (Click to read a summary of this studyHERE).

Benefits of the qualitative approach:

Using open-ended questions and interviews allows researchers and practitioners to understand how individuals are doing, what their experiences are, and recognize important antecedents and outcomes of interest that might not surface when surveyed with pre-determined questions. Although qualitative research can be thought of as anecdotal, when pooled across a number of participants it provides a conceptual understanding and evidence that certain phenomena are occurring with particular groups or individuals.

• Allows identification of new and untouched phenomena
• Can provide a deeper understanding of mechanisms
• Gives a one-on-one and anecdotal information
• Provides verbal information that may sometimes be converted to numerical form
• May reveal information that would not be identified through pre-determined survey questions

Limitations:

• Cannot generalize to the general population
• Challenges in applying statistical methods
• Difficulty in assessing relations between characteristics

Quantitative Approach

The quantitative approach to gathering information focuses on describing a phenomenon across a larger number of participants thereby providing the possibility of summarizing characteristics across groups or relationships. This approach surveys a large number of individuals and applies statistical techniques to recognize overall patterns in the relations of processes. Importantly, the use of surveys can be done across groups. For example, the same survey can be used with a group of mentors that is receiving training (often called the intervention or experimental groups) and a group of mentors who does not receive such a training (a control group). It is then possible to compare these two groups on outcomes of interest, and determine what influence the training had. It is also relatively easy to survey people a number of times, thereby allowing the conclusion that a certain features (like matching) influence specific outcomes (well-being or achievement later in life).

Example from youth mentoring research:

Grossman and Rhodes (2002) examined duration of matched relationships in over 1,100 Big Brothers Big Sisters mentor-mentee matches. Because the information they used was survey-based and numerical, they were able to employ statistical techniques examining how duration of match was related to different outcomes of interest.

In using a variety of statistical techniques, they concluded that “youth who were in [matched mentoring] relationships that lasted a year or longer reported improvements in academic, psychosocial, and behavioral outcomes” (p. 213). If Grossman and Rhodes had not used survey-based quantitative research, they would not have had such a large sample of matches and therefore could not generalize to matches in general. In addition, with a smaller number of participants, it is challenging to apply some statistical techniques to examine emerging patterns across such a large group of mentored matches. The current rule of thumb to using complex statistical modeling is that you need a sample of at least 130 participants. However, for more complex modeling that controls for characteristics, a larger pool of participants is needed.

Benefits of the quantitative approach:

Using survey methods across a large group of individuals enables generalization. For example, if policy makers wanted to instantiate a policy about mentor training, they would likely require some evidence that this training actually works. Interviewing a few individuals, or conducting a focus group with forty matches, might be reflective of specific cases in which the mentoring training worked, however, it would not provide strong evidence that such training is beneficial overall. Stronger support for successful training would be evident if using quantitative methods.

• Enables gathering information from a relatively large number of participant
• Can conduct in a number of groups, allowing for comparison
• Allows generalizing to broader population
• Provides numerical or rating information
• Informative for instantiating policy or guidelines
• Lends to statistical techniques that allow determining relations between variables (think of better word)

Limitations:

• Difficulty in recognizing new and untouched phenomena
• Caution in interpretation without a control group

In summary, the qualitative and quantitative approaches to research allow a different perspective of situations or phenomena. These two main approaches to research are highly informative, especially if used in combination. Each approach has its benefits and detriments, and being aware of the methods used to gather information can help practitioners and policy-makers understand the extent to which research findings can be applied.

http://chronicle.umbmentoring.org/on-methods-whats-the-difference-between-qualitative-and-quantitative-approaches/ 

Ret. 12-23-14

The Need for Mentoring Program Evaluation

In the effort to justify that we have a results-producing mentoring program, we sometimes rely on anecdotes and beliefs. Hard data, sometimes even longitudinal data, is essential for improving programs, recruiting mentors, and securing funding. Note how Carol Travis and Elliot Aronson  gently shake us back into reality. Thanks to Jean Rhodes, Ph.D., for sharing in The Chronicle of Evidence-based Mentoring.

The problem of the benevolent dolphin: Implications for mentoring

 by Jean Rhodes

In their new book, “Mistakes Were Made, But Not by Me,” Carol Travis and Elliot Aronson discuss our how confirmatory biasespredispose us toward evidence that confirms perceptions that we already have. And, in the absence of rigorous, experimental studies, we can easily summon up “evidence” that supports our viewpoints while ignoring evidence that contradicts our beliefs. This has relevance for mentoring programs. If we take as a given that “mentoring works,” and don’t evaluate our efforts, we can easily find evidence to confirm this presumption. To illustrate this point, Travis and Elliot describe the “Problem of the Benevolent Dolphin” notes how, every once in a while, a new story appears about a shipwrecked sailor or swimmer who, on the verge of drowning, is nudged to safety by a dolphin. As they explain:

“It is tempting to conclude that dolphins must really like human beings, enough to save us from drowning. But wait—are dolphins aware that humans don’t swim as well as they do? Are they actually intending to be helpful? To answer that question, we would need to know how many shipwrecked sailors have been gently nudged further out to sea by dolphins, there to drown and never be heard from again. We don’t know about those cases because the swimmers don’t live to tell us about their evil-dolphin experiences. If we had that information, we might that dolphins are neither benevolent nor evil; they are just being playful.”

The authors then take a hard look at psychotherapists, who, in the absence of rigorous, experimental studies, can easily summon up “evidence” that their clients are improving and that their approaches are working. In much the same way, the field of mentoring can, in the absence of data, fall prey to the flawed reasoning of the benevolent dolphin problem.  Not that there are  evil-mentor experiences out there, but we need to hold open the possibility that some of our efforts, especially the more perfunctory approaches to relationships, at best, neutral. We don’t hear about these efforts because youth in matches that don’t click or fall apart prematurely fade quietly from programs, and the failed relationships that they represent are overshadowed by the more compelling success stories of their peers. Indeed, programs often demonstrate their success to funders not by providing decks of slides with evaluation results but by showcasing successful matches and the heartwarming stories they represent. Since youth whose mentors didn’t deliver on their promises typically blame themselves when relationships fail, the shortcomings of programs and the volunteers they recruit go largely unnoticed.

We believe that a close caring relationship improves the lives of youth. But for our theories of change to be scientific they must be stated in ways that it can be shown to be false as well as true. For beliefs to be a matter of science, and not just faith, we need to subject them to the scrutiny of rigorous evaluation. Otherwise, we are no better than the shipwrecked sailors who, having made it to shore, tell us a compelling story that confirm our implicit biases.

http://chronicle.umbmentoring.org/the-problem-of-the-benevolent-dolphin-implications-for-mentoring/ 

Ret. 12-2-14

The Mentoring Effect Report: Definitions

Defining methodology and terms:

THE MENTORING EFFECT

The Mentoring Effect: Young People’s Perspectives on the Outcomes and Availability of Mentoring was commissioned by MENTOR: The National Mentoring Partnership with support from AT&T, and written by Civic Enterprises in partnership with Hart Research. This report is informed by the first-ever nationally representative survey of 1,109 young people on the topic of mentoring, as well as a literature and landscape review with insight from a variety of experts from the mentoring and youth development field.

Methodology

A nationally representative survey was conducted by Hart Research Associates. A total of 1,109 young adults ages 18 to 21 participated in this survey in July and August 2013. To reach out broadly to this highly mobile and technologically savvy group, young adults were contacted and interviewed in three ways: by telephone (landline and cell phone), online, and through in-person interviews. The in-person interviews were conducted with 102 “at-risk” young adults, who tend to be more difficult to reach using traditional survey methods. To reach this highly mobile group, researchers conducted the in-person interviews at 10 diverse locations in four regions across the United States. None of the “at-risk” young adults interviewed in-person completed a college degree. Slight weights were applied to ensure that the sample matched characteristics of young adults in the United States. We are confident that the survey sample, once weighted, represents a true national sample of young adults ages 18 to 21.

At-Risk Youth

There is no field consensus for what factors make a youth “at-risk.” For purposes of this survey, an at-risk youth is a respondent who is, at the time of taking the survey, disconnected (out of school and out of work) and/or has experienced any of the risk factors reflected in the survey screening tool that have been identified as barriers for achieving economic and social mobility. This term was not defined for survey respondents. Respondents were asked if they experienced these conditions (risk factors) when they were in middle or high school:

• Incarcerated parent or guardian
• Regular absenteeism
• Poor academic performance
• Behavioral problems in school
• Delinquency
• Teenage pregnancy
• Homelessness

 

Informal/Unstructured v. Formal/Structured Mentoring

The survey considered two different types of mentoring relationships and defined these terms for survey respondents. In both structured and informal mentoring relationships, the adult is supportive and works with the young person to build a relationship by offering guidance, support, and encouragement to help the young person's positive and healthy development over a period of time.”

Informal/Unstructured: In this type of mentoring an adult comes into a young person's life and they naturally develop an informal mentoring relationship. The adult could be a friend of the family or a teacher with whom the young person maintains a relationship outside of the classroom.

Formal/Structured: In this type of mentoring an organization like a school, a community group, or a faith-based organization matches an adult with a young person with whom they develop a relationship in a structured manner through regular meetings and activities. An example of a structured mentoring program is Big Brothers Big Sisters.

http://www.mentoring.org/mentoringeffect/fact_sheet/

Ret. 2-14-14

The Mentoring Effect Infographic

http://www.mentoring.org/mentoringeffect/infographic/ 

Ret. 2-14-14

Report Supporting Mentoring: The Mentoring Effect 2014

Report from MENTOR, the National Mentoring Partnership...

The Mentoring Effect: Young People’s Perspectives on the Outcomes and Availability of Mentoring

The Mentoring Effect is a compelling new report informed by the first-ever nationally representative survey of young people on the topic of both informal and formal mentoring, as well as a literature and landscape review and insights from a variety of key leaders in business, philanthropy, government, and education. The report was commissioned by MENTOR: The National Mentoring Partnership with support from AT&T, and written by Civic Enterprises in partnership with Hart Research.

The findings of this report are consistent with a powerful mentoring effect as demonstrated by the life experiences of the young people surveyed and mentoring’s link to improved academic, social and economic prospects. This mentoring effect is growing and, if harnessed, it has the potential to help meet a range of national challenges and strengthen our communities and economy.

The survey found that 4.5 million at-risk young people will be matched in mentoring relationships through mentoring programs while they are growing up. In the early 1990s an estimated 300,000 at-risk young people had a structured mentoring relationship. Another 10.5 million at-risk young people have informal mentoring relationships with teachers, coaches, extended family members or neighbors.

Despite this positive trend, one in three young people surveyed did not have a mentor while they were growing up.




Applying their experiences to the U.S. Census demographics for 8-18 year olds, it is projected that 16 million young people, including 9 million at risk young people, will reach adulthood without connecting with a mentor of any kind.



The survey also showed that with each additional risk factor a young person experiences, the less likely he or she is to connect with an informal mentor. This finding suggests a systemic shift to leverage quality mentoring programs to introduce mentors to young people who face a greater number of risk factors is a powerful and necessary strategy.

Mentoring’s Connection to Aspirations and Outcomes

The experiences of the young people surveyed showed significant positive outcomes for those who had a mentor. At-risk young people with mentors were more likely to aspire to attend and to enroll in college. They were more likely to report participating in sports and other extracurricular activities. They were also were more likely to report taking leadership roles in school and extracurricular activities and to regularly volunteer in their communities.

Paths Forward

With input from industry stakeholders and thought-leaders, the report outlines opportunities for the public, private and philanthropic sectors to systemically integrate mentoring as a key youth development strategy. The report describes a series of paths forward that would lead to a society where all young people have access to a quality mentoring relationship and the support they need to succeed in school, work and life. The recommendations include strategies to:

• Utilize mentoring to address national challenges.
• Ensure that young people most in need have a quality mentoring relationship.
• Expand local, state and federal public policies that advance quality mentoring.
• Ensure all structured mentoring is quality mentoring.
• Support and increase private sector engagement in mentoring.
• Facilitate connections between research and practice.
• Explore innovations to close the mentoring gap.

http://www.mentoring.org/mentoringeffect 

Ret. 2-14-14

The Drop Out Crisis

Another excerpt from SNAPSHOT: Youth Mentoring Research and Outcomes

The Dropout Crisis

Nationwide, nearly one‐third of high school students fail to graduate. In total, approximately 1.3

million students drop out each year — averaging 7,200 every school day. Among minority students,

the problem is even more severe with almost 50 percent of African-American and Hispanic students

not completing high school on schedule.  

Experts say that dropping out of high school affects not just students and their families, but also the

country overall — including businesses, government and communities. The Alliance for Excellent

Education estimates that high school dropouts from the class of 2006-07 will cost the U.S. more

than $329 billion in lost wages, taxes and productivity over their lifetimes.  Experts also say that

those who drop out are more likely to be incarcerated, rely on public programs and social services

and go without health insurance than youth who graduate from high school.

 

In summary, high school dropouts are:

• Less likely to have a job or earn less, on average, than high school graduates;

• Less likely to have health insurance than those with more education and more likely to depend on Medicaid or Medicare for their coverage;

• More likely to depend on public assistance; and

• More likely to be incarcerated. 2

2 Social and Fiscal Consequences of the Dropout Crisis. (Updated, 2009). Center for Labor Market Studies, Northeastern University.

Excerpt from:
http://www.mentoring.org/images/uploads/Snapshot%20on%20Mentoring%20FINAL_April%202013.pdf  

Ret. 10-11-13

Return on Mentoring Investment

SNAPSHOT: Youth Mentoring Research and Outcomes

Mentoring is a youth development strategy that is a proven foundational asset for a young person’s

successful path to adulthood. It is also effectively used as an intervention strategy to redirect a

young person’s life toward a healthy and productive future. In both cases, the results save taxpayer

dollars and fuel the economy. In fact, recent research demonstrates that

for every dollar invested in effective

mentoring programs, there is a return of $2.72.

This positive return on investment reflects projected increases in lifetime earnings gained by 

leading at‐risk youth down the path to becoming productive adult citizens, as well as dollars saved 

through evidence‐based mentoring‐related outcomes, such as reduced juvenile delinquency and 

crime, improved school attendance, higher graduation rates, and lowered risk of youth involvement 

in costly behaviors such as drug, alcohol and tobacco use.

In addition, effective mentoring programs also report significant increases among youth served in

self-confidence, self-efficacy and a positive attitude about their futures. 1

1

Anton, P.,Wilder Research & Temple, Judy, Univ. of Minnesota, “Social Return on Investment in Youth Mentoring Programs,” March 2007

An excerpt from
http://www.mentoring.org/images/uploads/Snapshot%20on%20Mentoring%20FINAL_April%202013.pdf