The First in our Series Entitled: Finding the Right Methods
We recently completed a three year evaluation of The Lawson Foundation’s Outdoor Play Strategy, which was designed to build adult capacity to impact outdoor play in early learning and child care (ELCC) and support systemic change. The project gave us amazing opportunities to visit projects across Canada that were demonstrating strategies for getting young children outdoors. It also challenged us to develop new data collection strategies!
What was the data collection challenge?
During these visits, we were keen to speak with people involved in the work, including early childhood educators, post-secondary educators and students, parents, and community partners. We often met them outdoors, as we were walking around, observing programs, and learning about the spaces where the work was happening. These conversations were really rich and powerful, and full of insight, but it was hard to walk through a set of interview questions in a linear way or to take notes.
What method did we use to address it?
We used an informal mixture of methods that we took to calling a “reflection hike.” Before our visits, we shared a simple, high level list of the topics we hoped to learn about. While we were out walking with people on the land, we let the conversation flow naturally, keeping the topics of interest in mind. We often took part in games, sat in on outdoor classes, or tried out a hammock. In this respect, this method was a form of participant observation. Our visits often stretched over two long days and involved multiple visits to several different sites. At the end of each day, we took detailed notes on our experience, focusing not only on the things people said, but also on the social and cultural context, who the people were to one another and how they fit into the work. We also took care to describe the spaces we were in and the activities we did together, as well as our own feelings and reflections. We often met in the evening with our key contacts to process the day’s experiences and share our reflections on what had been said. In these ways, this method was also inspired by the tradition of ethnography in program evaluation. We wrote up our reflections into a short site visit summary report that we shared with our hosts for their feedback.
How does this method seek to meet the standards of the meaningful participation rubric?
This method was especially useful in helping us interrogate biases. Our hosts often drew our attention to information we would not otherwise have noticed (such as the fact that the tree under which they began each class was the largest and oldest maple in the forest). We often became aware of subtle assumptions we had been making about the work that were unwarranted. The evaluation hike also improved representation. It was easy, while walking, to have a quiet side conversation with a person who had not spoken as much, to break into two smaller groups for a while, or to make space for people who wanted to share stories and examples. People often arrived part way through the walk, or left early, joining in in a way that made sense for them.
What did we learn that we wouldn’t have learned with other methods?
Early childhood educators who are skilled in promoting outdoor play don’t plan out a fixed set of activities for the day. Instead, they encourage children to engage with their environment and learn by taking risks and developing their own ways of playing. Educators are reflective about their approach, taking care to document what is being learned for themselves and for children. By using this method, we were applying the same philosophy of outdoor pedagogy to our evaluation work, and this helped us to understand this philosophy in more depth.
Tips for those interested in this method
This is a labour intensive method that requires a lot of planning and follow up. It is important to keep the guiding questions simple. It requires local partners willing to come along for the hike, invite the key people to drop by, and stick around to reflect in the evening. We often found it necessary to send emails or make quick calls after the visit to make sure we had our facts right or to clarify what someone meant. That said, it is a powerful way to develop a deeper understanding of what is really going on inside a program that you are studying.