Measuring the Value of Nonprofit Data Portals: An i4 Case Study on Backlinks as a Relevance Metric

Order Here

By Michael Gilbert & Mike Bergfors

The information ecology of civil society is very rich, especially as it concerns the passionate issues that drive nonprofits and their supporters to do the work they do. But when it concerns information about nonprofits themselves, many thoughtful observers have been frustrated by the challenges involved in learning what people think of particular organizations, how well organizations perform, and so forth. Among the prevailing efforts to address these challenges are websites that aggregate information about nonprofits. In this case study, we call them “Nonprofit Data Portals”. We use publicly accessible information on two of these portals to explore a question that is of importance to almost every nonprofit: How do we measure the success of our online content? More specifically, how do we know that our content is relevant to the people we’re trying to inform?

Insight: The prevailing model for evaluating the effectiveness of the content of a web page centers around Pageviews: How often was the page looked at? We offer an alternative model that centers more on the notion of Relevance. We propose Backlinks – links from other sites to a given page – as a viable indicator of Relevance. We argue that, with its basis in behavior rather than opinion, it is a powerful metric for evaluating content – probably superior to the usual combination of Pageviews and surveys in use by many organizations.

Intelligence: We took as our test cases 24 organization profiles at each of two (anonymous) nonprofit data portals. We tested for the number of Backlinks for each of those 48 URLs and compared the results of each of the portals in various ways. Each comparison shows one portal’s pages consistently outperforming the other. We explore the meaning that this business intelligence might have to the leaders charged with making decisions about these portals.

Integrity: Content relevance is important to most organizations, not just data portals, but effective metrics are by no means neutral and they don’t exist in a vacuum. They empower people to make different decisions than they might otherwise make, starting in this case with leadership. They require good data sources, especially if they are to mature into regular operational use. Finally, not all Backlinks are created equal. Some are very weak endorsements indeed. Some indicate relevance to particular communities of practice, but not to others.

Innovation: The first step, of course, is building on what works, with a comparison between highly relevant pages and less relevant pages. But once you open the door to Backlinks as a behavioral indicator of relevance, you begin to embrace (on a much smaller scale) the sorts of analyses that are Google’s bread and butter. Deeper innovation around this metric might involve measuring the quality of different Backlinks, associating relevance with particular communities of practice and the corresponding social network analysis, exploring the role of content syndication, and establishing time-series data for ongoing learning.