I found passages like the one I just quoted to be puzzling on many levels.

Science blogs pretty much came into existence as a way for scientists themselves to critique bad coverage in traditional media. And, ten years later, that remains a powerful tradition.

The paper presents a romantic, uncritical view of the press. Speaking as a journalist, I can say this is a view we can ill-afford.

What's more, neuroscience blog posts are very often deep, nuanced, and more accurate than "churnalism" driven by glib press releases.

If neuroscientists are indeed avoiding blogs for this reason (no data provided in the paper that this is true), then they are sadly misguided.

That whole paragraph is just nonsense. First: "Scientists may understand that neuroscience stories in legacy media channels are likely to be of higher quality than similar narratives found in blogs."

REALLY? Neuroscience reporting in legacy media is notoriously bad. Here's a paper on that cell.com, which gets cited in the one we're discussing but not in this context. And bloggers, many of whom are active scientists themselves, often do a much better job at correctly reporting neurostories, and/or debunking shoddy coverage.

Then there's the sentence that Carl quoted. I would laugh if it weren't so galling. We've had so many discussions about how long-form pieces can often sweep complexity and nuance under the rug in favour of narratives and protagonists. I really don't want to have to mention Jonah Lehrer here, but there he is.

(Then, there's the irony of making statements about being "bereft of... complexity and nuance" that are themselves bereft of complexity and nuance...)

Another passage I take issue with: "Stories in social channels are often crafted on the fly, without the help of experienced editors who can point out holes in the narrative or who can insist on rewriting and revision."

The reality of daily journalism is that stories have to be crashed sometimes on a tight deadline, at very short size. Editors do the best they can under those conditions, but the result can still be sub-par. It's true that good editors make stories better. I cherish those folks. But we journalists can all share stories of editors who had a faulty notion of a story in their head and forced our articles to fit it, either by changing the text or mangling the headline. So, again, this dichotomy does not reflect reality.

Another puzzling passage:

"we speculate that the scientists in this study may value journalistic narratives because they appreciate that journalism is indifferent to the interests and goals of science. Although this may be perceived as a disadvantage of journalism from the scientists’ point of view, it is actually a key advantage. Their role as external observers affords journalists credibility compared with scientific self-presentation."

I am desperately trying to rack my brain to recall a neuroscientist who ever said to me in an interview, "You know, I appreciate your indifference to the goals of science. You have a credibility that scientific self-presentation lacks."

What, you don't get that? I get it constantly. Ooh, you're so impartial, let me give you a shoulder rub.

Ed, you're going to have to post the audio of those interviews.

Anyway: another question I have about this study is the sample. 126 American and 131 German neuroscientists. 79% of of them were over forty. 78% were men. Brossard and others have documented a clear shift towards online media among younger people, and the paper itself says younger researchers "valued online articles more," but doesn't break out the data. (I can't get at the supplemental file with the tables, since this is a draft paper.) So I am left to wonder if these results just reflect--let me put this jokingly--the biases of a small group of older gentlemen who wonder why all the kids are Facebook. (I'm 46, so I have total immunity for saying this, right?)

<rattles on window 'til he breaks through>

Agree with much said so far - I'll add that as a scientist, I don't get my sci news from legacy media any more. They're slow, and most of the time (hi Mail) awful. Social media's good for getting stuff quick, and gauging general responses to new research. If I want more nuanced opinions? I go to blogs.

Oh, and Ed, Carl. You know, I appreciate your indifference to the goals of science. You have a credibility that scientific self-presentation lacks. ;)

Just like we are behind in studying perceptions of scientists and their use of media, we are even more behind in studying content & nature of science blogging, good, bad & ugly.

It's good to see this study out on scientist behaviors & perceptions, builds on a few previous ones, but we need more in this area across fields.

Matt--I agree. I think Dominique Brossard's work on how comments influence perceptions of science is fantastic and will make me change how I blog. ( sciencenews.org ) So I am baffled by the speculation in this paper based on faulty understanding of both journalism and social media. We don't just need more of this kind of research--we need to hold it to high standard.

I think it's difficult for a new area of research to gain a decent footing. Just look at video game research - it's all over the place. But just because something's new shouldn't mean that we let anything go - it's really critical, in the early stages of this sort of research, to be careful and considered in both methodology and interpretation. The conjecture at the end of this paper worries me that the authors don't have a the right sort of grasp on the current nature of science blogging. I'm looking forward to being able to access the supplementary material to see what the questions were like.

I can't say that I am troubled by the conclusion. There are things that could have been expressed better, but the reality is that apart from some great anecdotal examples to point to relative to science blogging, we don't have any systematic studies of the quality or nature of the science blogging arena. Even work drawing boundaries and distinctions around what we consider science blogging has not been done. Per sampling issues, useful follow up work should do purposive or over-sample of younger scientists and include open-ended interviews, observational & big data analysis.

I am afraid a lot of researchers use their normal methodology (questionnaires etc) without thinking they should actually study the phenomenon themselves for a while. They could find a hundred excellent neuroscience blogs if they just spent five minutes on ScienceSeeker. They could have asked neurobloggers for their own perceptions, history, links to best blogs, links to blog posts that discuss this issue, best questions to ask researchers, etc. We have discussed all of those issues for years, on blogs, at conferences etc. Bloggers can help make research better by pointing at the questions that need answering, and by pointing to best examples, older discussions, etc.

Just to throw in a link - not a study, but a bunch of us have been observing (and participating) for some time and have some things to say.

blogs.scientificamerican.com

Matt--If we don't have any systematic studies, then why was the paper packed with lots of implicit assumptions about the quality of blogging v journalism?

Also, why is "drawing boundaries" so popular in academia? If you want to draw a boundary around science blogging, you will have to slice me, Bora, Ed Yong, and many others right in half!

Defining what counts as a science blog, and the different types matters to defining the population, sampling and other aspects of measurement, observation and analysis. On the article, best person to talk to is Sharon Dunwoody and to elicit her thoughts on journalism and blogging, relevant distinctions.

I think the portion of the Discussion in question obviously touched a nerve with folks here, and while I completely agree with the substance of the criticism, I also think we shouldn't get carried away. The primary issue here is that the article makes a strong claim that isn't supported by data--i.e., that traditional long-form journalism is better than social media commentary. It's not actually that the claim is just *wrong*, because (as far as I know) we don't have good quantitative data to suggest that blogs are better than traditional media (though there's plenty of anecdotal evidence in this thread). Let's wait for good quantitative data before countering one excessively strong claim with others.

For the purposes of preserving science blogs, Library Of Congress will use ScienceSeeker.org, as neither Technorati nor Google Blogsearch, nor impressionistic "I know it when I see it" are good enough for that purpose. If a blog is not registered at ScienceSeeker, it is not "officially" a science blog, and if it should be counted, it needs to apply to editors to be registered. The fact that authors of the paper did not know to draw this boundary tells they did not do their homework.

Hi Bora,
Just so there is no confusion, I am not an author on the study. Also, not sure how knowing the Library of Congress distinction was relevant to the BioScience paper?

Neither claim in the paper nor the counter-claim are supported by data. But sometimes data are not necessary for the purposes of this discussion, or the Discussion section of a paper. The claim is demonstrably false, as the counter-claim can be demonstrated with literally thousands of examples.

I am aware you are not the author, Matt, and was not writing that comment directed at you. I was adding to the discussion on the methodological need for boundaries. The paper should have defined what a science blog is, and should have used the only somewhat official definition of it in existence. Otherwise, they demonstrate they did not do their homework, or they do not know the area of study they are researching.

Also, this is not so new that they could have missed it. We got rid of bloggers vs. journalists at #scio09. This is eight years old: dynamist.com They have no excuse.

Of course data are necessary. Would you have thought it more appropriate if the paper had said "maybe people don't understand that online science writing is much more dynamic, rich, and topical than long-form journalism"? The fact that there are thousands of examples you can point to is not helpful because there are also thousands of examples of stellar long-form journalism one can point to. What's at issue here is what the aggregate perception in the community is of each format (or, even better, direct measurable benefits, which is what we'd really like). For that, yes, you need data.

Who narrowed it down to "long-form" only now? After all, the problem with some blog posts is not that they are too short but that they are too long. Most science reporting is an off-putting, context-less 500-word inverted pyramid crapola. Blog posts tend to be much longer and contain context, details, nuance, expert criticisms of methods, stats and conclusions, etc. As Carl said above, many early science blogs started in order to counteract the crappy reporting in old media. Most of the early science blogs were essentially media criticism blogs.

Of course there can be a paragraph in a paper in the Discussion that is not based on data because data do not exist. In which case, one should go with the obvious, not imagined, reality.

Bora, the point is that what is obvious to you is not obvious to everyone else. There are many people who would claim with just as much conviction as you're displaying here that the *obvious* reality if you look at the state of science blogging today is that it's a chaotic wasteland that only confuses and misleads the reader, and that mainstream media provides a vetted and quality-controlled alternative. Now my own personal view is probably very close to yours, but I recognize that as regular contributors to the online science community (you more than I), we're hardly impartial observers. If there's one lesson psychological science has taught us, is that our intuitions about what's "obvious" are biased and rarely a good guide to reality.

It is not obvious to everyone else because they never looked. By publishing an academic paper, they are establishing themselves as experts on this topic, yet anyone who could have written those couple of paragraphs has only a very tenuous claim on expertise. I know a couple of authors, they are wonderful people who I like and respect. But they are stuck in the 20th century, and it did not occur to them they should become familiar with a topic before embarking on research of it.

The authors have a fundamental misunderstanding about neuroscience portrayed in social media. They seem to think (incorrectly) that it is more akin to a personal blog than a thoughtful, researched and referenced article. The major fault in the paper is assuming this without evidence.

I believe the point was raised that this assumption was in the “discussion” section, a place where you are supposed to speculate on your findings. However, the authors speculate that “scientists may understand that neuroscience stories in legacy media channels are likely to be of higher quality than similar narratives found in blogs.” They present the idea that blogs are of inferior quality as a fact, and then speculate that scientists realize that.

Ignoring the obvious bias in sampling, one conclusion did resonate with me:

“The number of neuroscientists who believed that online channels such as blogs and social networks would have a strong influence on public opinion and political decision making was much higher than the number of neuroscientists who indicated that they use these channels themselves”

Few scientists I know take advantage of social media or blogging to disseminate their science to the broader public. But the results of this study raise a bigger question: if scientists believe that social media and blogging influence public opinion and policy making (aka their own funding), why aren’t they taking advantage of it themselves?

As you know, I'm in the same journalism department as Sharon Dunwoody so I did talk to her about the paper today. She said they wrote the paper about a year ago, before science blog networks such as Scientific American or Discover became so prominent in the discussion. So they were really thinking more about the way neuroscientists evaluated posts from the home blogger, so to speak. Obviously - and she agrees - that landscape has continued to change and, given that, if they wrote the paper today they might have expressed some of that differently.

But she added that one of the things neuroscientists seemed to admire about the old time journalism model is that it has its own system of internal peer review (editors, fact-checkers) that is still not built into even blogging networks today. And this is actually an interesting question as we look at the continued importance of blogging.

Thanks, Deborah. I've invited Sharon to comment, but haven't heard from her.

Just a couple responses...

1. Your note makes it sound as if science blog networks were not prominent last year, when Sharon et al wrote their paper. What happened last year, exactly? I started blogging at Discover in 2008, for example, after having blogged at the Scienceblogs network for three years. I won't comment further, since it would be better to let Sharon respond firsthand. [cont.]

2. Journalism's "internal peer review" is great when it's well-supported and is allowed to work. Of course, many outlets DON'T fact-check, and many editors have little expertise in science. And given all the neuro-nonsense in major magazines and newspapers, journalism's internal peer review is clearly not working perfectly.

On the other hand, blogging networks do not have *internal* fact-checking, but--again, when it works as well as it can--it has *external* fact-checking, in the form of commenters calling BS and other bloggers linking to explain the mistakes. This can even lead to dynamic updates, when bloggers declare mea culpa and learn from their audience--something you RARELY see in traditional media. [cont]

Of course, all this discussion is a bit contorted by the fact that we're framing it in terms of what the surveyed neuroscientists might or might not be thinking....

Carl, thanks for starting this discussion and to all of you thanks for chiming in. I think the important thing to keep in mind here is that the point of the paper was to document the channels used by "productive" neuroscientists to follow science news (and not just neuroscience news), and their perception of the weight of these channels on public opinion and policy-making. As Matt rightfully says, we do not have data to assess the overall quality of science versus mainstream media

Sorry, I meant blogs versus mainstream media coverage of science

Bora, we are interested in the perceptions of random neuroscientists, not in the perceptions of neuroscientists who blog. The latter would be fascinating, but would be a very different study

Yes, most of the paper is about their perceptions and this is good. But the freewheeling paragraphs we are discussing here do not seem to be about them, but statements about the way world is, and those statements are erroneous.

Oh, I think Sharon will definitely offer a better perspective than my brief relay of a brief conversation. I think her point was that neuroscientists themselves were only slowly registering the importance of blog networks. And agreed that external" reviews on blogs do act as excellent fact-checkers. But as a long time former newspaper staff writer, will add that the brainstorming, bounce ideas, correct story direction function of an edit can still be remarkably valuable - and often improves a story (not to mention a book).

As to Carl's note #1 up there, Scienceblogs.com launched in January 2006, at the time when Nature Network and Scientificblogging.org (later renamed Science2.0.org) were already two years old. Discover started a network about a year after Seed. Some others popped up later, e.g., Field of Science and Lab Spaces. August and September 2010 saw the launches of PLOS Blogs, Wired, Guardian, Scientopia and a couple of others. SciAmBlogs launched in July 2011. So all the big events in the blogosphere, and all the discussions about professionalization of science blogs, already happened two years ago in the wake of #Pepsigate.

These paragraphs generated an interesting discussion among the authors as well. I am, for my part, not a big fan of a simple dichotomization of "blogging" versus "journalism." Some academics are still thinking along these lines though and it does raise an interesting empirical question as it relates to trustworthiness and other concepts that have been traditionally attached to, say, the New York Times. So do scientists make these distinctions when they go online and follow science news through blogs?

That is an interesting question. I had two separate discussions on Twitter yesterday on this topic: how much people decide by the banner, how much by the byline. If they arrive at NYTimes site, do they think "it's good", or do they think "let me see, is this Krugman (good) or Brooks (awful)"? People are moving away from trusting brands and toward trusting individual writers. Some are moving faster than the others - and they naturally judge blogs and MSM articles the same way. Academics are the most isolated, slowest-moving cohort, so they are likely to still go by the banner (not just NYTimes, but Science, Cell, PNAS, etc.).

Another slow-moving cohort are old-skool journalists. Thus, as the co-authors of the paper are academics and old-skool journalists, it is not surprising that they missed the developments of the last few decades and naturally assume, without it occurring to them it could be different, that banner beats the byline.

BTW, here is the first blog reaction: blogs.scientificamerican.com I expect there will be more.

I have to say that I find it puzzling that only criticisms are reported on this paper. I thought (I wished) science blogging was a space for constructive debate. The paper's goal was not to go ranting against blogs. Rather, the goal was to depict a changing science landscape in which social media is beginning to have a voice. I believe we are the first ones to report such statistics, based on sound data collected with sound methods. I am disconcerted by the reactions of this community.

Can you provide supplemental data or raw data so we can evaluate properly? Including exact questions in the questionnaire. I am not believing yet that the questions specifically limited the responses to long-form narrative writing usually found only in magazines. Certainly TV, radio and newspapers are not long-form narrative writing. Yet the paper keeps using the term "narrative journalism". If it was not very well defined and specified in the questions, respondents were much more likely to have answered about their consumption of brief, distinctly non-narrative articles, the kind that is typically much shorter and much less detailed than an average blog post.

The DOI link to suppl.data in the PDF is broken, or the DOI has not been assigned yet.

Dominique, I can understand your frustration. Those of us who've been at this for a while have been fighting a long battle for credibility, and although your paper had a different focus, parts of it hit a raw nerve. Hence, the reaction.

The fact that you're willing to take part in the discussion is appreciated.

Thanks for the contextual info, Ed, much appreciated. And Bora, I've just asked Joachim Allgaier (1st author) to send you the pdf of the supplemental material

OK, Scicurious has a good post about the paper here: blogs.scientificamerican.com