The trouble of a medium-n study—and how I got out of it (part 3)

Alleyway in Vancouver, Canada
Some light again at the end of the book writing alley

It looks like I’m almost done. The last two weeks have been absolutely insane with close to 22,000 words written (and only during normal-ish hours) and the two final chapters finished. Getting the three empirical chapters done clearly was the big hurdle I had to overcome. So how did I pull it off in the end?

The major “mistake” I made was thinking that with all the material I have and all the articles already published it shouldn’t be too complicated to pull together a book. I didn’t realise that in fact I have way too much material and way too many different narratives in those articles for a single book. So for the first two months (January and February) things looked pretty OK. I ploughed my way through writing country-context snapshots (for Australia, India, Malaysia, the Netherlands, Singapore, and the United States) as a background for the reader to better understand the voluntary programs for sustainable buildings and low-carbon city development studied.

I then thought of running a qualitative comparative analysis (QCA) for all 60 programs studied in these countries to come to an evidence based typology of programs that have achieved promising results in attracting participants, and those that have achieved promising results in improving their behaviour (in terms of reduced building related resource consumption and carbon emissions).

This is where I took the first wrong exit. To make readers who are less familiar with QCA understand what I was doing I had to refer them to a lengthy methodological appendix and the chapters to come. After doing this for a while I realised the QCA chapter was becoming fully incomprehensible for people who are not all too familiar with voluntary programs for sustainability, with QCA methodology, or with both. In other words, I was aiming at a very, very small niche market of people who are specialised in both—and I truly think that is me and perhaps a handful of other scholars around the globe. This took me most of March.

A little disillusioned I left the idea of starting the empirical part of the book with a QCA analysis, and decided to first present the voluntary programs studied in a descriptive analysis and then work slowly towards a QCA analysis.  This sounded easy on the outset, but turned out almost impossible to do. The range of programs and countries studied allows for a clustering in either types of programs (with one type per chapter) or countries (with one country per chapter). The country idea didn’t really work out because the programs studied are not spread evenly over all countries—basically I have studied twice as many programs in Australia, the Netherlands, and the United States as in India, Malaysia, and Singapore.

That left me with only one solution: Break up the empirical chapters along the line of a program typology and discuss one type and the examples related to it in a chapter at the time. Yet, coming up with a typology that captured the various programs studied was all but easy. Not so much because I couldn’t refer to existing typologies, but because, again, the spread of cases over the various types that I came up with is not even. This holds all the more for the programs studied in India, Malaysia, and Singapore: Whatever typology I derived from the literature (deductive), the full set of programs I have studied (inductive), or a combination (iterative/abductive), I always ended up with types for which I don’t have examples from the latter group of countries. By now it was late April.

To overcome this sampling problem I have decided to only discuss the voluntary programs studied in Australia, the Netherlands, and the United States in the empirical chapters of the book. I now cluster these in the three dominant types, which cover 26 programs in my study. That leaves me with 14 programs that don’t fit these three dominant types and that are too varied to put in one ‘miscellaneous’ type. In hindsight, they are of interest for specific narratives that I can tell in journal articles but they don’t add many new insights (or counter arguments) to the overarching narrative in the book.

In short, after a lot of writing, moving things around, and getting frustrated because I couldn’t find a happy solution, I have brought back the full pool of programs studied to 26 that are evenly spread over three dominant types, which I now discuss in three empirical chapters. But that wasn’t the end of the story. This still implies that I have some eight voluntary programs to discuss per empirical chapter. I ran, again, in to the problem of a data overkill. In-depth descriptions of eight programs per chapter gets me to 12,000 words per chapter easily without any analysis whatsoever. I didn’t want to burden my readers with that type of book.

The solution to this problem, I realised, was to only discuss a few of the most telling voluntary programs per chapter in depth, and let the others provide context and background—why tell a roughly similar story over and over again? So now I discuss ten of the programs in detail in those three empirical chapters. Coming to terms with that reduction of my data was what has caused so much trouble. But it also provided the solution I was looking for, because now I have a book that, in my very biased opinion as its writer, is still readable.

Having made all those decisions and having written the three empirical chapters (mostly between May and July) things suddenly began to click. After writing the three empirical chapters I realised I could, after all, include a synthesis chapter based on a QCA analysis of the 26 programs discussed. This chapter provides some key insights in how the various conditions I’m interested in—rules, enforcement, rewards, local government involvement, and type of diffusion network—interact in affecting program performance. I wrote that chapter a week ago.

This week I have written the concluding chapter. In it I highlight some of the scholarly advances the book makes, the value of voluntary programs for decarbonising the built environment in developed economies, and provide an answer to the core questions of the book. But, and this is the little win of this week, this chapter also allowed me to reflect on the value of such programs in rapidly developing countries by including insights from 15 programs from the pool of programs that I have studied in Singapore (I know, not really a developing economy), Malaysia, and India.

Long story short, even more than when writing my previous post, I feel that I have accomplished the exact book that I wanted to write: A comparative study of voluntary programs in six countries that builds on QCA methodology, provides relevant insights for a policy and practitioner audience, and critically rethinks and adds to theorising on this approach to (environmental) governance. So I’m hopeful again that I will get the book published by a big name university press after all. What all those insights are? Well, I leave that for another blog post—I’m thinking of doing a six week series of blogposts with the Fifth Estate to share my insights.

I hope to get the introduction to the book done next week, work on the appendixes the week after, and send the manuscript off for review by mid-September. That’s a turnaround time of nine months, which, I have to admit, is not bad at all.


In the end, I managed to give the programs studied in India, Malaysia, and Singapore their own dedicated chapter. This chapter seeks to understand whether the patterns and how the insights related to those patterns found through the QCA analysis in Australia, the Netherlands and the United States resonate in these three other countries. To limit the complexity of this new chapter, I only look at one type of voluntary program (green building and city certification) but in highly different contextual settings. This flips the research design of the chapters prior to the QCA analysis that focus on different types of voluntary programs, but in relatively similar settings (major cities in highly developed economies). With this final empirical chapter done, I have been able to use all my empirical material.

In hindsight, I could have made my life a whole lot easier by thinking through the research design better. But as I have discussed before, when I started this project there was little methodological literature about setting up a medium-n type research project. a key lesson learnt for me is to begin empirically studying the ‘weakest link’ case-study first–that is, the case that will be hardest to study empirically. For the QCA analysis, the case that gives you the least information will set the boundaries to the full analysis. In my study, these were some of the cases from India and Malaysia. Had I started the project with studying these first then they would have set the limits to the data to be obtained from cases elsewhere. That said, by going through the process in all the iterative steps, I have truly learnt a lot about what can and cannot be achieved with a QCA study.

One thought on “The trouble of a medium-n study—and how I got out of it (part 3)

Add yours

  1. Although, I am not a scholar but a ‘simple’ housewife in a rural area, I am currently reading “Governance for Urban Sustainability and Resilience” and it is wonderfully written. A jury of your peers, with an interpreter, would find you guilty of premeditated murder; words were chosen with deadly precision. I LOVE this subject and it causes me to dislike the human species. It is shocking to me how few humans are intelligent enough to think for themselves instead of buying into hedgimony.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

Up ↑

%d bloggers like this: