Measuring the impact of data and information is one of the most important and interesting topics. A recent blog post by Andrej Verity on cue-based decision making has inspired a discussion on how to measure and evaluate the impact of information products. Besides our most recent publication on understanding risk accelerators [pdf preprint; published in LNCS] (presented in Bartel's Keynote at the CAiSE Conference), this post is dedicated to relevance information for decision making. Knowing what information is required, when, by whom, and in what format is key to creating and delivering tailored information products. 

In the past, it was considered as the main challenge to overcome the lack, uncertainty or vagueness of information. The core assumption was that more information and a complete overview of the situation, decision-makers are able to make better decisions. 

And there is more: locally word of mouth, communcations by municipalities or charities; online the digital humanitarians scrambling, curating and assessing information remotely. Information evolves more dynamic than ever before - and our understanding cannot keep pace with the explosion of updates, reports, maps and graphics. 

The question where to set up a hospital has very different characteristics from funding decisions. Both decisions are important, but have very different requirements in terms of information granularity, timeliness, and updates. Which donor would like see a daily update of a few hundred kilobyte operational map? Or what field office needs an infographic? Information providers should ask: which decisions do we want to support? Making the purpose of an information product explicit and transparent is crucial to better operational impact and evaluation. 

Focusing on operations, it is important to understand which information decision-makers use, and even search for. A taxonomy of decision-makers is provided in the Decision-Makers' Needs Report. We propose going even beyond this work - focusing on key decisions, embedded in a personal and organizational network.

In the field, limited bandwidth and time pressure serve as natural filters. Queries from the field are therefore indicators for information that is actually vital, but not provided - at least not in a form that is easy to find or retrieve. 

How can information products be as efficient and reliable as a phone call? 
The answer in this new paradigm: by providing tailored information products, created and designed in near-real time for a purposeful decision in a given context. For us, the greatest potential for innovation and improvements lies in understanding decisions such that we can generate less information to make better decisions.

Now more than half a year has passed, since Typhoon Haiyan / Yolanda hit the Philippines. Our friend and supporter Geert Gijs has recently visited Manila, Tacloban and Palo to monitor the recovery. Indeed, the typhoon has triggered many activities - and it is good to see that part of the efforts of DSWD, OCHA and other agencies are dedicated to learning such as in the SIIEM project. DSWD has invited for an Inter-Agency Forum on Increasing Availability, Quality and Accessibility ofCommon Operational Datasets to Support Disaster Risk Reduction apd Emergency Management in the Philippines, a project that has been supported by DRL in cooperation with MapAction. 

Our leading question in this effort is: if we cannot avoid that natural hazards strike, how can we reduce the consequences? 

In the past weeks, the DRL team has been sharing the impressions from our field research with our crowd funders in a Webinar, we have discussed our findings with the colleagues at our institutes, and presented at the Humanitarian Technology and ISCRAM2014 conferences.

Most of our work is publicly available, including our initial insights from the Philippines, a reflection on our research design and on coordination. For all of us, it was great to discuss how technology can support humanitarian operations (see the keynote presentations of Patrick Meier  and Ed Happ ), to share our visions for advancing science and research, to get a signature of Gisli's new book … but we are aware that a lot remains to be done! 

Practitioners. Our work can only be relevant if we engage in a process of co-creation. This requires from all of us navigating uncharted waters to find a common ground between operational relevance and the generic concerns and questions of research. A way ahead may be the creation of shared learning environments, offering room for reflection and continuous improvements – turning the world into a living lab?!

Walking the line… together. While few researchers have been to the field, many have been collecting and analyzing data about the impact of Haiyan, the response and recovery efforts. We have been trying to understand the context, requirements, needs and to improve technology, tools and systems. Most of this work is still fragmented; also in science, data is often not open. We keep advocating data sharing protocols and better interoperability for emergency managers, and seem utterly unable to open up and share ourselves 

To make headway in both aspects, we will publish in the coming weeks a series of blog posts about the Philippines. This will still be exploratory and anecdotal, including subjective reports and reflections from our respective standpoints. We believe that to achieve academic rigor, we need to make room for information sharing and engage in a discussion. We hope you join! 

“Why did you go to the Philippines?”, asked our colleagues when we presented our work in workshops and seminars; “what is the value of research in the field?”, instead of providing aid, asked a taxi driver. The value of field research is uncontested in domains such as ethnology or sociology - but mathematicians or computer scientists are rather rare. And it is a legitimate question – research in the field is more dangerous, costly, confrontational, and exhausting. The results are much harder to obtain, foresee, plan, and publish in our disciplines; and scientific rigor is hard to maintain.

Why did we leave our desks? Because we believe that any research in disaster management can only be relevant, if it involves a process of joint learning and co-creation. To learn, we have to break norms, overcome barriers of current thinking or “best” practices in both science and practice. This is, at times risky – but we believe that this is (sometimes) for academia and society to take the greatest leap forward In the past 30 years, research and the number of publications on disasters, vulnerability and resilience have grown exponentially. So have the damages from disasters – in any scale you might want to choose; from number of fatalities, to affected population or economic losses. Our scientific results, all the models, and experiments, and tests, have not (yet) lead to better preparedness for and management of disasters. And in the quantitative disciplines, related to modeling or decision support, disasters are a domain that is considered as inaccessible.

We decided to engage with the professionals that we met and interviewed in the field. We are grateful to all our interviewees, who took the time to answer our questions during an ongoing response, and for all of us, this field trip lead to new insights. Being researchers, not professionals or consultants, we started from the aim of understanding the problems in practice, and structuring needs, and requirements – instead of going into the field with ready-made solutions.

Many of the drawbacks of going to the field have been discussed – from the difficulties to publish results, to the actual workload of doing field research on top or besides an academic position. For us, field work is about overcoming silo thinking, and the silence between the professional cultures. Field work requires us to redesign the research paradigms: to reconsider what actually good research is, and how we can achieve societal relevance. It is a long way from the practices of "neutral" observing and the ideal of extracting and purifying knowledge to  co-generation of questions and answers. Yet,with field engagement a whole new world of experience and learning lies ahead of us. This may be daunting for some, since workflows and foundational issues need to be redesigned. It may not be journals and conferences which are our first outlet any more, but communication that is required in a continuous process of professional and academic learning. Yet, for many researchers, our identities of professional self worth, and alos our monetary value are tightly tethered to countable academic achievements. Some accept this, but we will continue to challenge the core assumption that we’re taught to believe: that our engagement in advancing science and implications for the real world is tightly and directly linked, but our structures in academia often sit closer in bed (or at desk?) with academic culture than the real-world itself.

Academia is no longer the sole driver of the knowledge & information – but it is our choice if can be among the leading ones also in future.

Four weeks after the field trip in the Philippines, the team meets at Tilburg University to work on the results from our field research.

Research in the field means that our team was confronted with realities that are very different from our work at a university: instead of trying to analyse the world from our desks, sitting in front of a computer we had to work in an area struck by a disaster: no mobile phone coverage, no access to electricity. No driver, no fuel, no flight, no hotel: logistics and planning needed to be continuously adapted and re-designed. As we went deeper into the field, we had less and less control about when we would be where, and whom we would be able to meet. 

An epistemology for disaster research: often, researchers try to create representativity by large sampling sizes. However, large numbers of representative and comparable interviews are difficult to collect in the field - because of the conditions, but also because there are only few people on the job. 

Being researchers, we are neutral and follow the ideal of objectivity and scientific rigour. Still, we are humans, touched by the stories we hear about a mother struggling to survive, about a barangay captain trying to maintain the order and ensuring the well being of his community - and feeling the need to give something back to these communities. In how far does this affect the process of acquiring and building knowledge? Should we engage with communities via projects such as Radyo Bakdaw - or maintaining our neutral stand point?

Another challenge is the interdisciplinary nature of our work. We have come together to synergize our different backgrounds in health, crisis mapping, information management, decision support, logistics and risk management. Our work aims at avoiding Babylonian confusion of many different disciplinary voices that are only juxtaposed, talking in different languages about different issues to audiences about the same topic: disaster management. 

The key questions we are confronted with are: how do we exploit the knowledge and information from the field in our research? How do we design the next research trips, including roles and responsibilities and rigorous research design? And ultimately: can we embrace the specifics of the field to develop new research methods that enable acknowledging and exploiting information from practice in a transparent and rigorous way? More research needed to find the answers!

Manila, the capital of the Philippines, is also the central place for the coordination of the disaster relief activities between agencies and headquartes, and the relations to the government of the Philippines. Most clusters and organisations meet here, information from the field is evaluated and transmitted from here to Head Quarters and Government. 

Manila is also the place where newcomers - from Australian fire fighters to Danish nurses - make contact with those who come back from the field, tired and happy to go home. For us, this is an opportunity to experience the transition from early to late response and recovery.

Manila itself is a vibrant city, crowded, traffic is dense, and apart from the omni-presence of aid teams, the city hardly touched by Haiyan: Christmas Trees glitter, and we listen to "Rudolphe the Rednosed Reindeer" over breakfast...

This post will report about just one interview, as we are about to leave from Cebu to Manila. We will try to add more as we go.

Interview with Neil Bauman - 
IM for the Shelter Cluster (IFRC/Canadian Red Cross)
Neil has worked in the Philippines since the Bohol earthquake. He is the first referring to Haiyan as the information management disaster. The sheer scope and magnitude of Haiyan has led to an enormous response effort - reflected by thousands and thousands of maps, situation reports, relief projects, Wikis and websites. Haiyan is the first natural hazard declared L3 since Haiti, organisational interests, funds and efforts are according. 
UN-OCHA has implemented its cluster approach to facilitate work in different activities, distribute tasks, and coordinate acitivites and information. 

The shelter cluster was one of the most important in this natural disaster. As of december 16 2013, there are still more than 1.1 million affected houses, and the rainy season will start at the end of January in some affected areas, meaning that short- and medium-term shelter solutions must be found beforehand. 

As an information manager, it was Neils role to act as coordinator (managing relations with cluster partners and agencies working in shelter) - from very technical issues such as engineering to how to address the needs of the population given the context of the Philippines. 

This very first interview gave us important leads for all the others to come: information overload and the trade-off between real-time operational decision-making vs. strategic decisions that need to be well aligned with the other actors, agencies and bureacracies were frequently among the most important issues.