Development Dialogue 19 | Participatory art as an alternative project monitoring tool? How an INGO is using picture diaries in Timor-Leste

International development projects need to be monitored to see whether they are on the right track. Although the logical framework (log frame), which depends on a standard indicator for monitoring project outcomes, is widely used, it often proves insufficient in capturing progress made by beneficiaries. In this blog article, Young-Gil Kim discusses why alternative monitoring systems are needed and introduces picture diaries as an alternative monitoring tool . He shows how international NGO ThePromise has used these diaries in Timor-Leste to hear from illiterate children and argues that participatory arts have the potential to capture project progress in contexts where conventional monitoring systems frequently encounter challenges.

The log frame: a silver bullet?

While I was working for an aid funding agency (which I did for around eight years), I frequently used the log frame — a monitoring tool in which inputs and activities yield short-term outputs and long-term results — to monitor project progress. Regardless of whom I worked with — UNESCO, the ILO, the UNDP, and so on — the log frame was consistently employed. I kept wondering whether it was truly the silver bullet it was being portrayed as. Briefly speaking, it is a rigid tool  that relies on quantitative surveys. Development practitioners with statistical skills use it to observe causal relationships between the input/activity and output. However, in other development sectors such as governance projects, causal relationships are often more complex, and it takes much longer to see changes.

Some time later, when talking to volunteers working for international NGO ThePromise who were implementing an educational project in Timor-Leste, the question popped up in my mind once again when Jisu An[1], one of the volunteers, told me that monitoring educational progress in illiterate children was challenging because good indicators seemed not to exist. Even if they did, she said, because the children targeted by the project are mostly illiterate, surveying them on paper seemed counterintuitive.

I decided to study the problem by delving into the literature on the topic. I found that the international development arena, saturated with the log frame, leads us to believe that it upholds a profound tradition, while the reality is quite different. First, it was “originally created as a planning tool for military purposes” in the US and later adopted by USAID in the 1970s . Second, the log frame is “virtually unknown outside the development community, and it is noteworthy that it has not been adopted to any great extent elsewhere.” Thus, there is concern that the log frame is “[used] indiscriminately across all programs in the development scene regardless of the nature of the work being measured: from agriculture to human rights, from micro-finance to culture.” This might also be the case for ThePromise, I thought, which might explain why they were facing challenges. I spoke to volunteers such as Jisu An about their work for ThePromise and presented some of my observations and findings at the recent Development Dialogue conference, which I also discuss in this article.

 

Tracking the progress of a teaching programme in rural Timor-Leste

ThePromise is an NGO (with its HQ is in Seoul, South Korea) that seeks to “provide better opportunities” in several developing countries by conducting projects ranging from education, water and sanitation/hygiene, and disaster relief to credit cooperative initiatives. In Timor-Leste, the NGO in 2023 focused on the education sector and was active in a few rural, marginalized areas inundated with challenging educational conditions. This includes mostly illiterate children, teaching methods not provided to many Timorese teachers, insufficient teaching materials, and parents not paying adequate attention to their children’s education. A team of ten South Korean volunteers had been dispatched there, where they taught children in two kindergartens to strengthen the educational environment of the community and improve education standards. Monitoring the project’s progress through the log frame was one of their important tasks.

This proved challenging due to the high illiteracy rate and the young age of many of the children they taught. Hence, they could not complete survey forms specified in their log frame. After having several meetings by themselves, the teachers decided to use picture diaries as an alternative to the survey. After each class, children drew pictures of how they experienced the lesson. The aim was for teachers to monitor their progress effectively using these picture diaries.

 

 

 

 

 

 

These two picture diaries were drawn by a Timorese child named Adu. The left-hand diary depicts Adu’s reflections on his daily class on 15 March 2023, while the picture diary on the right, drawn a mere two months later, demonstrates a noticeable improvement in Adu’s ability to articulate his daily learning experiences — the pictures are more detailed, and Adu’s writing has progressed from simply adding individual words to writing complete sentences.

This led me, as an independent researcher who once questioned the widespread use of the log frame, to ask whether participatory art could serve as an alternative to the log frame tool for monitoring project progress. Participatory art is gaining traction in the international development arena because it offers spaces for envisioning futures and cultivates critical thinking. Inspired by the MSC method, I interviewed four Korean teachers who assessed around 200 children’s picture diaries for four consecutive months. They all felt that the diaries were a good tool for children to express themselves. One teacher for example stated:

I believe that the picture diary is a good tool for monitoring children’s educational progress. When we introduced it in the early stages of our education programme, children rarely expressed themselves; there were no writings and no pictures. As time went by, their ability to express themselves improved. Some of them could articulate their thoughts on sketchbooks in written form as well as through pictures. I also observed that in the early stages, children just added a few words in their sketchbooks, whereas a few weeks later, they started to write in full sentences, articulating themselves better than before. I think the picture diary serves not only as a good tracking tool but also as a means to encourage children to express themselves freely.

Another teacher felt that more research was needed to assess its effectiveness as a monitoring tool, stating, “I […] think that three months is not a sufficient timeline to see any tangible changes in [the children’s way of expressing their experiences].” Overall, the teachers thought that the diaries were primarily a means for children to express themselves. Their effectiveness as an alternative monitoring system — and that of participatory art in general — therefore still needs to be determined. Participatory art could perhaps complement conventional approaches such as the log frame, especially in contexts where surveys cannot be used, until its effectiveness as a monitoring tool has been further investigated.

---

References

Davies, R. (2005). The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use. UK: Care International.

Flower, E. and Kelly, R. (2018). Arts-based research practices and alternatives: reflections on workshops in Uganda and Bangladesh. Changing the Story Working Paper No.3

Fontes, C. (2016). The What and the How: Rethinking Evaluation Practice for the Arts and Development. In Stupples, P. and Katerina Teaiwa (eds). Contemporary Perspectives on Art and International Development. Taylor & Francis. pp. 238–251.

Hailey, J. & Sorgenfrei, M. (2004). Measuring Success: Issues in Performance Management. Occasional Paper Series 44, Oxford: INTRAC

Mkwananzi, W.F., Cin, F.M., and Marovah, T. (2021). Participatory art for navigating political capabilities and aspirations among rural youth in Zimbabwe. Third World Quarterly, 42(12), 2863–2882.

Stupples, P. and Teaiwa, K. (2016). Introduction: On Art and International Development. In Stupples, P. and Katerina Teaiwa (eds). Contemporary Perspectives on Art and International Development. Taylor & Francis. pp. 1–24.

Tools4Dev website: https://tools4dev.org/about/

[1] Thanks go to Jisu An who helped shape my thoughts on the issue through our many interesting discussions and for providing valuable input into the article.

---

Opinions expressed in Bliss posts reflect solely the views of the author of the post in question.

About the author:

Young-Gil Kim is a PhD student at the University of York, UK. He worked as a visiting researcher at the Center for Korean Studies (CKS) in the National University of Timor-Leste (UNTL) in 2023.

 

Are you looking for more content about Global Development and Social Justice? Subscribe to Bliss, the official blog of the International Institute of Social Studies, and stay updated about interesting topics our researchers are working on.