What Do Practitioners Say About Assessment?

By Betsy Alkenbrack

Practitioner-researchers have really interesting things to say about how they work and how they think about their work. In this article, I look at what practitioner-researchers say about assessment in six RiPAL* reports from Alberta and BC. I will begin by briefly describing the reports, and then discuss three common themes that run through them.

Reports from the Field

Lucy Alderson and Diana Twiss describe a participatory research project that examined the question: "How can literacy activities empower and stabilize the lives of women in the sex trade?" Their report also outlines activities that can promote learning as a strategy for harm reduction. (Alderson and Twiss 2003)

Evelyn Battell reports on the work she and other literacy and Adult Basic Education (ABE) instructors did to develop and test techniques to document non-academic changes that happen in the lives of learners' who participate in literacy instruction. She describes the six techniques, as well as the successes, challenges and lessons she and her colleagues encountered throughout the field test process. (Battell 2001)

Fay Holt Begg describes her work with a learner to explore the extent to which the "Write to Read" teaching method worked with an adult learner in her program. (Holt Begg 2002)

Veronica Park reports on her work with a group of literacy and Adult Basic Education (ABE) students to examine the reason for low participation rates in her community literacy program in the face of statistics indicating a high need. (Park 2000)

Andrea Pheasey describes research she conducted in her community literacy program to find out what literacy students think being literate is and what they want to be like at the end of the program. (Pheasey 2002)

Phyllis Steeves works with one of her program participants to examine ways practitioners can work within a program environment to support change in their learners' . (Steeves 2002)

These reports address different aspects of assessment, including needs assessment, intake interviews, and helping learners' to reflect on their progress. Although they are quite different, they all identify three things as important: non-academic outcomes, learner participation and safety.

Non-academic outcomes are essential

Although Battell's report is the only one that focuses explicitly on non-academic outcomes, all of the studies recognize their importance. All found that, for learners' to make progress in their learning, the non-academic outcomes need to be in place. In her introduction, Battell lists examples of non-academic outcomes that include increased confidence, community involvement, reading to children, interest in studies and ability to reflect. The participants Pheasey worked with talked about "trying," which they describe as "having confidence and being willing to try something new" (p. 17). Steeves talks about self-examination and the ability to "identify and attain other learning goals" (p. 9). Alderson and Twiss learned to broaden their definition of progress to include staying focused, functioning in groups, regular attendance and participating in collective structures (p. 51).

Battell makes the important point that the time taken to nurture these outcomes accounts for the long period of time learners' often spend at the fundamental levels, a fact which is unrecognized and unappreciated by people who do not work in literacy.

Learners must be full participants

All of the reports indicate that learners' need to take a key decision-making role in assessment. For example, Holt Begg talks about how she discusses outcomes with learners' at the beginning of their program, and Pheasey describes sessions at the end of each cycle during which learners' reflect on their progress and identify changes they need to make in their learning plans. The techniques described by Battell "require wholehearted acceptance by the learner and are under the learner's control" (p. 55).

Some of the reports also describe learners' taking an active role in research. For example, Park decided to do her research with literacy and ABE students because

I anticipated that adults who had experienced a need for more education and returned to school could draw on that experience, as well as on their knowledge of the community, to create a vehicle for discovering the needs felt by potential adult students. (p. 186)

In some cases, the researchers were forced to rethink their direction based on learner input. For example, Pheasey was not able conduct a final interview with all her research participants because of conflicts between some of the learners' involved. Steeves abandoned her original plan in favour of one with which the learner, Barb, was more comfortable. In so doing, she moved from a guided-tour approach (following a theoretical framework) to a self-guided pilgrimage (directed by Barb's practical needs and interests).

Alderson and Twiss describe how the chaotic context of women's lives forced them to rethink their plan of conducting a participatory research project:

We found ourselves in a strong collaboration with women about many priorities-literacy, learning, violence, the Missing Women, creating a voice in the WISH [Women's Information and Safe House] organization, and many other issues that played a critical and current role in women's lives. Although the project did not follow the rigorous definition of participatory action research, we were able to achieve many participatory instances in the research process. (p. 13)

Park had planned a project in which participants would develop and conduct a needs assessment, but soon discovered that the learners' felt this was a waste of time. As she switched gears, she learned that:

In terms of participatory practice it is not enough to just ask students for their input: we need to set aside our preconceived ideas and learn to listen. (p. 193)

Holt Begg also encountered resistance to her plan-to test a program designed for children with an adult learner-and had to make adjustments. At first, the learner was reluctant to complain about the process because she was locked into traditional views of the teacher-student relationship:

We had not come to the stage where she was able to reveal feelings or ideas that might be taken as critical. (p. 14)

In all cases, these changes were described as positive learning experiences for the practitioner-researchers.

Pay attention to safety and emotions

Most of the reports mention safety. Alderson and Twiss say it is important to "address the issue of safety in a non-threatening, non-judgmental, safe and realistic way" (p. 25) and Battell reports that the evaluation process was "dominated by need for learners' to feel safe" (p. 55). Steeves describes the need for approaches and techniques to link to emotions. For example, in her first (failed) attempt to work with Barb,

What was missing was a link between the process and the emotions. During a much later conversation, Barb summed up her response to this approach; she covered her heart with both hands and stated, "I felt empty in here." (p. 7)

The reports also make important points about the practitioner's role in creating and maintaining a safe learning environment.

First, it is essential to establish trust. Battell's group found that the techniques were more effective if the instructor knew the learners' and had developed some trust (p. 12). Steeves chose to work with a student with whom she already had a history and common background, and Holt Begg talks about an important lesson the learner Carol taught her:

that adult students will not tell me what they're thinking until mutual trust has been established....Carol didn't share her feelings until she was confident they would be accepted. Saying "I don't know" and "I don't understand" can take great courage. (p. 18)

Alderson and Twiss found that it was difficult to measure progress when the environment is constantly changing:

Women rely on our evenness about their chaotic lives....When we continue to see women as champions of their lives and active learners' in all situations, it breathes optimism into their self-concept. (p. 52)

Secondly, practitioners need to be aware that they may not have the same expectations as the learners' . As Battell says,

Finally we want to record changes, not judge things that don't change. The reason for behaviour may be safety at some very deep level, a level more essential to the learners' than pleasing their current instructors or tutors. (p. 58)

For Alderson and Twiss, it is doubly important to keep expectations realistic. They explain:

When women suddenly stop coming or get heavily back into drugs, it is easy to feel disappointment as literacy instructors and to wonder if we are making any difference. (p. 52)

Practitioners must also take care of themselves. Battell says this is not only a need but also a responsibility, and practitioners should be careful not to choose activities that "trigger unhappy, unsafe feelings" in them (p. 56). For Alderson and Twiss, it is important to create "positive life-affirming activities in our personal lives" (p. 52)

Practitioners make important contributions to the field

In traditional research, findings are reported on and theories developed, but practitioner-researchers want to do more than that. They want to produce something that other practitioners can use. So, for example, Pheasey hoped that "perhaps we can ask better questions, more insightful questions, and hopefully get to what students really want" (p. 2), and Battell and her colleagues dreamed "of having these non-academic outcomes named in such a way that the whole community could make sense of them, and literacy could take its place in a team approach to improving our learners' ' lives" (p. 2).

A big message here is the importance of non-formal, non-academic assessment. Steeves points out that just as there is a bias in favour of formal education and qualifications for learners' , "A parallel within the research community would be the valuing of academic research as opposed to that of practitioners" (p. 4). The six reports I reviewed prove that practitioner research has an important contribution to make. I look forward to reading many more.

* RiPAL = Research and Practice in Adult Literacy    (www.nald.ca/RiPAL/about2.htm)                   

bio:

Betsy Alkenbrack began her career in adult literacy at East End Literacy in Toronto. She also spent eleven years in South Africa helping to build the adult literacy movement, training teachers and developing educational materials. She is currently a doctoral student at UBC, a teacher and a member of RiPAL-BC.

 

Sources:

Alderson, Lucy and Diana Twiss (2003). Literacy for Women on the Streets. Vancouver: Capilano College/WISH.

Battell, Evelyn (2001). Naming the Magic: Non-Academic Outcomes in Basic Literacy. Victoria: Province of British Columbia, Ministry of Advanced Education.

Holt Begg, Fay (2002). Adapting Writing to Read for adult literacy students: It worked for Bill. Will it work for Carol? Edmonton: Learning at the Centre Press.

Park, Veronica (2000). Why Don't People Come? Some Reasons for Non-participation in Literacy Programs. In M. Norton & G. Malicky (Eds.), Learning About Participatory Approaches in Adult Literacy Education. Six research in practice studies. Edmonton: Learning at the Centre Press.

Pheasey, Andrea (2002). What do literacy students think being literate is? Edmonton: RiPAL Network/Learning at the Centre Press.

Steeves, Phyllis (2002). From practice to theory and back again. Edmonton: RiPAL Network/Learning at the Centre Press.

View PDF

Back to Forum readings

Back to table of contents