Great Expectations of Accuracy

I often get the impression that readers are quite frustrated with the accuracy a newspaper is able to provide. Part of that is the fault of marketing, which says, "You should choose us because we're the trustworthy ones who will get it Right." Part of that is because a lot of readers are naive, and believe that news writers are actually without bias or personal opinions. Somehow, the idea that we are bland, mindless automatons that live only to channel The Truth is plausible to a large segment of the population. And, of course, another factor is a general lack of understanding about the medium. Newspapers arrive at truth through ongoing coverage. Magazines are able to step back and look at more of the picture, though they still probably won't find absolute truth.

Journalism as a search for truth is an ongoing process. In an open source journalism model, where that ongoing process is transparent or even participatory, I wonder if readers will continue to expect too much. Perhaps when they have realistic expectations appropriate to the stage of the article, readers will become less alienated from the press.

It could happen!

On a vaguely related note, I was wondering recently: Do most members of the media and press believe in absolute truth? How about you?

Distributed editing: share ratios and eBay-style feedback combine!

So, back to the distributed publishing question I began to explore in the post Following the Story.

In that post, I explored how crowdsourcing could theoretically provide editorial direction, brainstorming, checks and balances on bias and ethics, an extra eye for catching holes in the story, and even, to some extent, some initial fact checking.

The one thing that I couldn't solve with crowdsourcing was what I described as "flow" editing: rearranging the piece so that it flows better. Editing for writing style and clarity would also fit in here.

Well, crowdsourcing may not solve the problem, but borrowing some concepts from peer-to-peer file sharing sites and the eBay feedback system, I might have a barter-based solution that will address the problem.

In essence, most journalists are trained to edit as well as to write. Some are better at one than the other, but I suspect that most of us enjoy at least a little of both. Imagine a system where users had to maintain a specific edit ratio: for every piece they edit, they earn the right to get a piece edited by someone else. Now add on an eBay feedback system, where writers and editors give feedback on the job they each did. Writers in a rush or with a basic news piece would have to take the first editor who came along, but writers who are working on a feature with which they can take their time could hold out for a really good editor. The rating system would motivate users to do a good job editing (and not be too difficult an editee) because if you don't do a good job, it will be harder and harder to find someone to edit your own work. If that didn't motivate people enough, the edit ratios could be partly weighted by the feedback rating of each user.

What about people who only want to edit, or only want to write? Obviously those people can just hire an editor. But if they want to work within this website, people who only want to edit could sell edit credits to people who only want to write.

And, for those of you who aren't fans of crowdsourcing, a system like this would work for other aspects of the editorial process, such as fact checking and copy editing.

Open Source Medicine and the Doctor-Patient relationship

A recent post by Dan Schultz on PBS' MediaShift Idea Lab blog discusses the pros and cons of comments. Schultz mostly sticks to practical tips on how to beef up the comments on your publication, but a belief in the benefits of crowdsourcing seem to underly his advice.

It seems everyone is talking about crowdsourcing in terms of politics and in terms of going hyperlocal. Power to the people via the vast amounts of information available on the internet and the vast sea of experts we can access in only an instant.

I don't see many media people talking about crowdsourcing and data in terms of health. But the medical profession has noticed, and boy are they talking about it.

Let's stop there for a moment for some disclosure. I'm a pushy, inquisitive patient, the kind older doctors hate and young doctors love. I'm pushy and demanding, but that seems to have been effective. Since I started down this path, researching my own health problems and taking a hand in the decision process, my health has improved a great deal. In each case I eventually found a specialist who I trusted - usually because the information that specialist was giving me made sense in the context of the research studies I was reading online. I still try to do some of my own research, but I'm more likely to go with my doctor's advice now that I feel I'm with someone competent.

So what are the doctors saying?

Health providers must avoid frustration about having their role as the sole source of information challenged, or possibly risk losing patients. In one survey of different specialty and general medicine practices, one third of the patients who felt their relationship with their physicians was low in participatory decision-making changed providers within a year.
The Patient-Physician Relationship in the Internet Age: Future Prospects and the Research Agenda, from the Journal of Medical Internet Research

Earlier in the same paper, after observing that patients who have done their own research may be able to get more out of their appointments by skipping the basics and spending more time on the pros and cons of treatment options, the authors write:

"...physicians must be prepared to address alternative possibilities that the patient has learned about from external sources. Instead of saving time, this scenario may require extra discussion when untested approaches need to be debunked (as in the case of some complementary and holistic medicine practices). ... Still, it is yet uncertain whether efficiency improves or declines when patient-acquired Internet information is brought into the decision-making process. This subject warrants further investigation."

A lot of the discussions I'm reading are about the doctor-patient relationship, or impacts on visit duration or visit frequency. This study's results suggests that these fears are unfounded, while this one concludes that "Hype around Internet use by patients appears to exceed the reality of Internet use."

Sure, these results are preliminary, and as internet adoption grows, the way patients use the internet will almost certainly evolve.

But the thing is that the internet doesn't pick and choose what to democratize. In science and medicine there has always been a great divide between the layperson and the expert. Now, as more and more research journals go open source, average people are feeling increasingly confident in their own ability to make judgments about science and medicine. Thanks to the democratizing force of the internet, the chasm between people and scientists appears to be narrowing.

Appearances can be deceiving.

When it comes to science, the learning curve to equal or even approach the expertise of a biologist or physicist (for example) is far too steep for most of us to achieve. The necessary math skills alone present a formidable barrier. But doctors are not necessarily scientists. What does their education consist of, and to what extent can new media equip your average person to fruitfully discuss their health with a doctor?

My own experiences suggest that the ability and inclination to search the internet, decipher article abstracts, assess the quality of evidence, and then weigh that against other evidence, is enough. In fact, more than enough - I often end up teaching doctors (especially general practitioners) new things about my illnesses, although pride may prevent them from learning from it.

When I tell people about this, I'm often told that I'm "different." Because I have a background in science (physics/astrophysics, which has nothing to do with medicine, I should point out), I understand the scientific process and scientific jargon, and I'm not intimidated by it. My friends will even sometimes argue that I'm smarter than your average patient, and that it isn't reasonable to base my assessments of what people can do with the internet's vast sea of data on what I can do.

Mmm, maybe. I certainly think a lot of people (including some doctors) need a crash course in how science works, and in particular, on evidence-based medicine. (by the way, the entire concept of evidence-based medicine originated in Canada at McMaster University in 1992). I'm not sure how smart you need to be, once you have that basic knowledge, to get involved. And hell, who knows: maybe I'm not smart enough. Maybe with all of the pushing I've done to get better healthcare, I've shot myself in the foot and don't know it. I don't think that's the case, but sometimes I entertain the possibility with some degree of horror.

Lately, I've begun to suspect that the general practitioner is essentially a walking talking database of often-obsolete medical information. If that's true, then a skilled internet researcher who understands how to assess medical evidence could potentially provide better medical advice than the general practitioner. I certainly feel like I've done that on occasion.

To get to the bottom of this all, there need to be more studies on the reading comprehension of patients who are familiar with the scientific process. To what extent does their perceived comprehension match their actual comprehension?

I also need to learn more about how general practitioners are educated, and how they continue to educate themselves throughout their careers. Do they learn how to interpret statistics, or how to avoid being influenced by medical literature or drug advertisements? Are they taught about evidence-based medicine these days, and when did that find its way into the curriculum? For those doctors who graduated before EBM, how many of them have learned about it nonetheless? And, as medical information is updated through research, how are general practitioners updated on that information - or are they?

I have a lot more to say about this, but I think it's time to close this post down. I'd love to hear your thoughts on all this, though. Do you use the internet to research health problems? Do you think you can learn as much as some general practitioners? Talk to me.

The honesty double-standard

As I expand my reading list (yes, there will soon be a blogroll), I've been catching up on old posts. One of the blogs I've been reading is PressThink, home to Jay Rosen's incisive (albeit often lengthy) commentary on modern media.

After citizen journalist Mayhill Fowler's run-in with Bill Clinton on June 2, Rosen published an extremely thorough round-up of media reactions to what is now dubbed "PurdumGate."

Fowler committed the heinous crime of failing to inform Clinton that she was a citizen journalist or that she was recording the conversation (her recorder was in her hand, but apparently he didn't notice it). Clinton let his guard down (you can listen here) and spoke honestly and passionately to Fowler about Todd Purdum's Vanity Fair article, The Comeback Id. Needless to say, Clinton was not pleased to discover the conversation, with commentary, published on the Huffington Post's citizen journalist campaign section, Off the Bus. Both Fowler and Clinton regret the incident; Fowler wishes she'd remembered, in the heat of the moment, to identify herself, and Clinton probably wishes he hadn't let his guard down.

The incident hit a nerve with many media members (for a summary, check out Rosen's blog post as linked above).

Reading those reactions, I had a bit of an epiphany.

Old-school reporters hold themselves to a very strict code of ethics. Most believe that we must disclose who we are to sources. If the story is important enough and there's no other way of getting the information, this rule can be bent or in rare cases, broken.

In many of the excerpts Rosen quoted, it's clear that members of the traditional media feel that this is important, because otherwise, politicians would always have to be on their guard. And what a horrible way for them to live!

But wait. Why are they on their guard in the first place? What's so bad about the public seeing them as they really are - humans? The reason is simple. Politicians wear personas the way we might wear hats or masks. The guy we vote for probably doesn't even really exist. He's a carefully crafted ideal designed to capture votes. And most of us expect it, because being an honest and open politician just isn't the done thing.

So we bend over backwards to pretend that there is no man behind the curtain. We strive to be honest so that politicians can continue to be dishonest and lie to us about who they are. We must be honest and disclose at all times, but politicians are expected, always, to present a false facade, a lie. We aid and abet that by allowing them to choose what is and isn't off the record.

I don't mean to suggest that journalists should start lying about who we are. Nor am I suggesting that we should stop disclosing our intent before an interview. But when politicians complain about being on the record all the time (which is increasingly the case what with so many people blogging), we shouldn't feel sorry for them. If they were honest in the first place, and presented themselves as they are, being on the record all the time wouldn't be a problem in the first place, and voters would know who they are really voting for. In essence, elections could be about the issues instead of the image.

Food for thought.

Cha-Cha: The human search engine

The Open Source movement is all about harnessing the power of many, or crowdsourcing. A company called ChaCha is crowdsourcing in a way I never would have thought of: a human-powered search engine.

Just call them up from a mobile phone, or text ChaCha (242242) with your question, and they'll get back to you with an answer within three minutes. The answers are found by human "Guides" who must pass a series of tests, attend ChaCha's "Search University," and go through search simulations before they can begin work as guides. Even then, the guides only get between three and eight dollars per hour.

I'm not clear on what their business model is, and it certainly isn't on their website (although one blogger suggests they plan to profit somehow from advertising). I was going to text 242242 and ask them how they make money, but alas - the service is only available in the United States, and I'm in Canada.

I'd be interested to see something like this that can handle longer form queries and more importantly, answers. There are free services out there like Google Answers, but for whatever reason I haven't been impressed by the quality of answers people get on there. And that's interesting, considering the success of Wikipedia. What makes these two efforts at crowdsourcing different? (Obviously ChaCha is a different animal because the guides are trained and paid, albeit a pittance)

(Via Josh at the Knight Center for Digital Media Entrepreneurship)

Creative niche market campaigning

The Royal Canadian Kilted Yaksman points us towards Sean Tevis, an information architect from Kansas who is running for State Representative.

Tevis is one of a growing handful of politicians who have attempted to harness the internet to reach all the way down to the grassroots for support and funding. Some have been more successful than others, and some have been more well known than others. Here in North America the most familiar name would probably be Howard Dean, a former Governor of Vermont who made an unsuccessful bid for the Democratic nomination for U.S. President.

A campaign webpage fashioned in the style of popular webcombic xkcd has been making the rounds on the Internet. The response has been extremely positive. In fact, Tevis' webpage went down temporarily due to the webcomic generating too much traffic for his server to handle.

At first I was disappointed to see that the comic was not linked from anywhere on the main site. It made me feel like he was pretending to be a geek when in fact he's just another politician. It even seemed a little odd - the response to the comic has been so positive that perhaps not linking to it will lose him votes.

But the more I thought about it, the more I realized that this is a savvy move indeed. Your average voter will not relate to the comic, and this is in many ways a popularity contest. People who read the sites where the comic is being circulated are likely to get the comic, and relate to it. People who don't read the sorts of sites on which the comic has been posted probably wouldn't get the comic. They might even be alienated by it, not because of the issues, but because it's different. But those people don't read those sites, so they won't see it.

I think that Tevis is the real thing. But with niche advertising, there's always the danger of duplicity. Obviously a clever campaign manager could advertise in any number of niche forums or publications, tailoring the advertisement's presentation according to the niche even when the niche has nothing to do with the politician who is campaigning. The question is, would we buy it?

If Stephen Harper's next campaign tried to present him in this way, we might giggle, and think that his campaign manager was clever, but I doubt any of us could be persuaded to believe that Harper is a closet Internet geek.

Following the story

As I read Gillmor's account of Korean phenomenon OhmyNews and Northwestern University's Medill School of Journalism's online projects, I continued to mull over how news stories could be edited without having a full-time editing staff.

Sure, the simple solution would be a repeat of the top-down model we're so familiar with - editors assigned to stories could be entitled to a percentage of any income the story generates.

But what about harnessing the power of the Internet and the power of the readers? I'm hoping I can come open with a theoretical model that will do so, rather than simply mimicking traditional publications in an online forum.

The editorial staff at a publication serve a variety of functions.

1. Direction: Which stories to run, when to kill a story, and how important a story is.
2. Brainstorming: What would be a good angle for the story? What is missing at first draft? Are there any big holes?
3. Flow editing: Rearranging the piece so that it reads better.
4. Copy editing: Maintaining a house style and correcting spelling and grammar.
5. Fact checking: At least in magazines, fact checking is part of the process.
6. Standards: Are the reporters being honest? Are they being ethical? Are they truly invested in coming as close to the truth as possible, rather than forwarding their own agenda? When faced with facts that contradict their bias, will they admit it?

Right now I'd like to talk about step two, brainstorming. Among the wide variety of traditional publications for which I've worked or written, there is an equally wide variety of processes used to brainstorm a story. But all of these processes rely on staff.

Here's a process that might achieve the goals of brainstorming without involving staff members.

1. The story idea is posted on the website's work in progress section
2. Readers can indicate their level of interest in the story idea and/or leave comments suggesting angles, sources, or relevant facts that should be mentioned in the article
3. Using their own reporting skills and reader input, the reporter will then do her reporting and post the story's draft
4. Again, readers will be able to comment on the story with suggestions about questions they feel are left unanswered, things that aren't clear, etc.
5. The reporter uses those comments to write a final draft
6. The final draft gets posted online
7. Professional editors and fact checkers take the final draft and work on the writing style, etc. to create an extremely smooth, well-researched piece, and publish it on a subscription-only portion of the website

What I'd particularly like to see is the story treated as an individual unit. Readers could see a story idea and subscribe to it - they could ask to see every step of the process, or they could ask to see only the final result. Right now RSS feeds, subscriptions, etc. are usually tied to the entire publication, or to an entire section of a publication. But why stop there? Why not let someone focus in on a particular story?

There are lots of problems with concepts like this. For instance, the reporter may be scooped by reporters at traditional publications. Of course, the concept of scooping is arguably outdated. It's unclear, in this day and age, how much it matters to be the first to cover a story. Instantaneous and evolving breaking news coverage started the shift away from 'scooping.' Meanwhile, the ability to tailor what we read using services like Google News Alerts means that more and more, the story is the unit, rather than the publication. And if readers read a story that leaves them with questions, more often than not, they'll go looking for another article that answers those questions. If your story has more information than the person who got there before you, yours is still likely to get read. The extra mile makes the difference.

Another problem is the overwhelming amount of information reporters will have to contend with. Will reporters be drowned in so many comments, he won't be able to sort the signal from the noise? A Slashdot approach to this, allowing readers to rate comments - and respond to them - may help mitigate this problem.

What other problems are there with this approach? Talk to me, cyberspace.

Editorial standards

I've been mulling over the question of maintaining editorial standards in a network of professional blogs. I'm imagining an online publication readers feel they can trust that consists of a network of freelance blogs. Yet, I want to have as little management as possible. A system that runs itself without oversight would be ideal, but is it possible?

Right now I'm reading We the Media by Dan Gillmor, who works at the Knight Center for Digital Media Entrepreneurship. One of the points he keeps on driving home is that the Internet's greatest power is not simply interactivity. Allowing readers to comment? Simple interactivity. Harnessing the knowledge, creativity, and judgment of the readers to enforce editorial standards? That's open source press.

What I've been realizing is that the open source movement is about far more than just coding. Increasingly, individuals are being invested with the tools and knowledge to make change. It's happening in programming and in media. It's also happening in politics. Medicine too - with more and more information available to patients, it is now possible for patients to be far more informed about their more obscure health problems than a general practitioner can possibly be. Meanwhile, smart medical researchers may be looking to health forums for research questions.

Gillmor seems to be saying that as Internet technology evolves, it is handing us more and more tools that make centralization and management less and less crucial. Grassroots civic involvement and a more democratic process are just some of the possible benefits.

At the moment, I'd say that I agree with Gillmor. I'd be interested to read some thoughtful and pointed rebuttals, if anyone can point me towards one.

I'm also on the look out for well-written, succinct information on the open source coding movement. Information on how they do distributed project management, as well as a history of the philosophy and theory of the movement, would be key. Hook me up!