Knight News Challenge Proposal: Crowdsourcing Data to Bring OpenBlock to Rural America

At the top of my To Do List this week is the completion of one of the proposals I’ve submitted to the Knight News Challenge this year. I’m posting it here in the hope that you’ll have some feedback on whether/how a service like this would be technically feasible. editorially useful and financially viable. I’m especially interested in hearing from editors of small papers, public records experts, civic/community organizers and anyone who’s worked on the OpenBlock code.

Under what conditions would you volunteer to help a project like this in your community? News organization — how much would you pay for a service like this? What characteristics would it need to have to make it worth your money? What else do you see here that needs further clarification?

(And a big hat-tip here to Penny Abernathy, the Knight Chair in Digital Media Economics here at the UNC-Chapel Hill School of Journalism and Mass Communication. She got this project kicked off with a grant from the McCormick Foundation and who is my co-pilot on this application.)

Here’s our draft pitch:

Crowdsourcing Data to Bring OpenBlock to Rural America

This project would create a co-op to develop and deploy public records databases at news organizations, especially those serving communities of fewer than 75,000 people, preparing those records for presentation and integration in an OpenBlock format.

These rural news organizations are struggling to move to the digital age in part because their staffs are so small they don’t have the capacity to identify, digitize, re-aggregate and map all the various public records available at the state and local level into databases that can be accessed intelligently by both reporters and the reading public.

The project would tackle the lack of capacity at rural papers from two directions. It would create a centralized repository of state, county and city schemas and datafeeds that could be easily used in OpenBlock. This a job well-suited for a small group of experts. In addition, the project will create a statewide corps of amateur data-checkers and records requesters. Data quality assurance and data gathering are jobs well-suited for a crowd of many people, each working on a small piece of the puzzle.

These volunteer citizen-journalists would actually be member-owners of a co-op business. Each task they perform would earn them additional shares in the company’s annual profits. We would generate revenue by charging rural newspapers a fee. The more records and the better their accuracy, the more news organizations would sign on for the service.

In some cases, volunteers would pick up CDs of data from county offices. In others volunteers would scan and upload PDFs of hand-written police incident reports. In still other cases, people would key into a database the information on those PDFs. This job is so big that no single small news organization could do it. But with a corps of member-owners working together, we could create a model for gathering valuable public records from rural America. To individual communities, these records are necessary to foster an informed civic dialog and healthy economy. But in aggregate, these records may also be able to shed light on trends in rural America that would otherwise go unreported.

Improving Delivery of News and Information to Geographic Communities

In small towns and rural America, the local newspaper is more than just a source of information and an engine of commerce.  More importantly, it fosters and builds geographic community and sets the agenda for public policy debate.  This project will foster civic and community engagement — first, by forming a network of knowledgeable volunteer citizen-journalists, and also, by making public records readily available and organized to support decision-making and accountability at all levels of government.

Unmet Needs

In many cases, data that is readily available in GeoRSS or at least CSV format from big cities (such as this example from San Francisco) is simply not available even in print from rural governments. For example, journalism students at the University of North Carolina working last semester to gather and organize public records in two rural counties for an OpenBlock application met with a number of obstacles (which they describe in their blogs) – ranging from significant photocopying fees to inappropriate redactions and denial of access to public information.

Even when acquisition of public datasets is relatively simple – for example, public health restaurant inspections — someone must request that data from a specific county be exported in fielded data format. It is inefficient for each rural news organization to make separate requests for this data in each of North Carolina’s 100  counties. In these cases, our public records coop would outline an initial request for the data for each county.

What’s New?

Currently there is no tool or service that can efficiently gather, format and publish public records on rural news organizations’ sites. In part, this is a technology problem that may soon be overcome with the alpha rollout of OpenBlock later in 2011. But a much bigger piece of the problem is the data itself – neither OpenBlock nor any other technology has the ability to obtain public records as fielded digital data and create a newsworthy user interface for all the various types of records a news organization might need.

Without a project like this there is no indication that OpenBlock will be a viable option for papers in rural communities.

What Will Change?

By the end of the project, we will have

•          at least one member-owner in each county in North Carolina

•          at least 12 news organizations subscribing to the service

•          at least one type of schema for which we’ve collected data from each county

Most importantly, we will have raised public awareness of open government and we will start seeing rural counties and towns publish public data in standardized, machine-readable formats on the Web.

What tasks/benchmarks need to be accomplished to develop your project and by when will you complete them?

How will you measure progress?

Do you see any risk in the development of your project?

How will people learn about what you are doing?

Is this a one-time experiment or do you think it will continue after the grant?

Triangle’s Media Ecosystem Needs Tributaries and Mainstream

Sitting next to News & Observer editor John Drescher last Friday during a forum about the Triangle’s media landscape, I had to feel a bit sorry for him. Of the nearly 20 representatives of news media in the region, he was the most prominent representative of the mainstream media and drew all the fire from the bloggers, entrepreneurs, do-gooders and pontificators who had him easily outnumbered and whose smaller organizations had often beaten his Goliath newsroom on important stories.

But I also envied Drescher. He was also the only one at the table who had ever dropped $200,000 of his company’s money on an investigation of a state agency. And the only one who knew what it was like to spend four years pinging the government for public records before he had a story solid enough to sell to his subscribers and advertisers.

One other thing made Drescher an enviable character in the Triangle’s media ecosystem. Despite their valid criticisms of increasing gaps in The News & Observer’s coverage of our communities many noted without irony in their voices, the small, independent and non-profit news operations had the most impact on public policy when they got the attention of Drescher’s paper or one of the local television stations.

And that made me realize that if our state is going to retain its generation-long reputation as a home for journalism that gives voice to the voiceless and holds powerful people accountable, then we must find a way to foster dozens of new and diverse tributaries of news and information that flow into the big, slow-moving mainstream media. Without the tributaries, the MSM seems likely to evaporate entirely. Without a larger channel into which they can empty, the tributaries seem likely to overwhelm us with a flood of disconnected datapoints.
Continue reading “Triangle’s Media Ecosystem Needs Tributaries and Mainstream”

What’s the Demand for Downballot News?

One of the partners for my Public Affairs Reporting for New Media class this semester was the N.C. Center for Voter Education, long known for its efforts to change the way judges are elected in North Carolina as well as the voting guide it creates in partnership with UNC-TV. That voting guide was the first place I turned for information on candidates in yesterday’s statewide primary for seats on the Court of Appeals. I just presumed that no newspaper had covered the race.

But you know what happens to you and me when you assume things, so I checked it out. Turns out I was mostly right. I’m going to put together a summary of information that got reported about this race, but it got me wondering about this question: How much information – and what kind – of information do North Carolinians need about downballot statewide primary races? Are they getting? From where? Or why not?

After all, if journalism’s worth saving it’s only because of the impact it has on public life. I’ve long been curious about the connection between information and citizen participation. The presumption – not always right, as Samuel Popkin and Michael Schudson might tell you – is that the more information voters have the “better decisions” they will make.

A little more than 700,000 people voted in those races. Some of them might have wanted more information than others? How many had enough? How many would have changed their votes if they had had different information?

And, if we can figure out who needs this information – and what information they need – is there any business model that gets it to them? Do we need independent reporting on downballot races like this or is informing voters the job of the State Board of Elections and the candidates themselves?

How to Cover the Dropout Issue

Perhaps my biggest fear about the subject for this semester’s Public Affairs for New Media class is the danger of mission creep. We’re going to be covering the state’s dropout rate, which anyone who has spent any time with the issue will tell you is not a problem isolated to single moment in a child’s life.

Reading up on the issue, it seemed that people tackled the issue in one of two ways — either as a trailing indicator with roots in pre-kindergarten or as a leading indicator of difficulties that a person will have throughout his or her life staying out of jail, holding down a job, and maintaining a family.

So we run a real danger of trying to wrap our arms around a topic that seems to be correlated to lifelong problems that begin at birth persist throughout life.

On Monday, we’re hosting our newspaper partners in Chapel Hill. We’ll find out then how they see the issue playing out in their communities. But as I educate myself on the topic and have been discussing it this week with students, here are some of the questions I have.

My question to you: What would you like to know about North Carolina’s diploma dilemma? How would you like to see us cover the issue. I welcome your comments.

Continue reading “How to Cover the Dropout Issue”

Leaders — Political and Editorial — Need to Work the Network

The News & Observer in Raleigh today picked up an op-ed I wrote about the need for winning political candidates to follow through on their gestures of online community connectivity. (Hat tip to WCHL for the idea…)

But this challenge isn’t unique to political leaders, it’s also one that journalists must meet and a gesture on which they are following through even less.

Hooked on the promise of the free advertising inventory generated by online comments, more and more newspaper Web sites are deploying  some type of online discussion technology.  What they aren’t deploying is the kind of human  resources that are needed to foster and develop online conversations. Why do most comments on news articles follow Godwin’s Law? Because there is little or no authentic conversational leaders. There is no human being making connections between people and ideas and, um, fact.

Just look at this recent survey of online journalists in North Carolina — online community management ranked as the skill that these editorial staffers said was least important to their jobs.

Here are my quick thoughts on how news organizations should begin to approach online comments.

N.C. Rising Dropout Rate: A Call for Media Partners

Next semester, I’m leading a group of students in a service-learning class at UNC-Chapel Hill that be using online reporting and publishing techniques to dig in to the story of North Carolina’s rising high school dropout rate. As part of this experiment, we’re working with news outlets in the state on a collaboration that will live both on their individual sites and on a centralized site at UNC. If you’re interested in participating, please take a look at our draft plan of attack here .