M-c Shanahan

“Science educators, here’s what you’re up against. A debate in the comments on this story over whether the movie “Mission to Mars” proves that ancient Martian life was used to seed life on Earth.”

There’s no way I could pass over a Facebook status like this one. My friend K.O. recently made the comment in reference to a Popular Science article called “A Significant Portion of Mars Could Be Friendly to Life, New Models Suggest.”  The article itself is a short summary of a paper published in the journal Astrobiology, which uses models to predict how deep a microbial biosphere might extend into Mars’s surface. And while I might quibble when the author uses the phrase “slam-dunk” to describe the evidence for water on Mars, the interesting story isn’t in the article. The real story, for me and for K.O., is in the comments.

There are 18 comments and they begin innocently enough with speculation about possible future efforts to terraform Mars (“the problem with mars is that it’s geologically dead. even if we did all this, and terraformed it (its underground or its surface) it wouldn’t last very long at all.”). It’s in the sixth comment that things take a turn for the strange:

“nasa stop beating around the bush and admit that life once flourished on Mars surface. there were civilizations just like on earth but an asteroid hit and eradicated all life. watch the movie Mission to Mars it explains a lot of what im talking about. a way of living on Mars is by using its underground Lava tubes. they could protect humans from harmful radiation.”

Jokes, right? Another commenter seemed to think so, responding “It’s a MOVIE. LOL”

Nope. The original commenter came back with “yes a great movies based on actual facts. scientific agree that life could have been ‘seeded’ on mars.”

The discussion continued from there as a speculationfest about NASA holding back information, about junk DNA containing hidden memories of our previous life on Mars and more. Of course, it’s hard to tell which commenters were serious, which had tongues firmly planted in cheek and which were engaging in the art of the troll. Regardless, it left an unpleasant taste and, like reading online comment sections about science can often do, it made me worry about the state of scientific knowledge and literacy.

The original study is speculative as well, using extrapolation to propose a possible scenario. The speculative nature of the study, however, seems to leave so much room there for better comments: comments that ask about the strength of the evidence, comments that challenge the findings or make comparisons to other research, or even comments that ask about why the author from Popular Science chose the term “slam dunk” (considering the article it references used the more appropriate word “suggests” )

Am I asking too much? Does that require a level of knowledge and skill beyond that of the typical reader?

To try to restore my faith and cleanse my palette (and try to answer my question about the possibilities for better commenting) I decided to turn to some data I’ve collected in schools to see what young science students had to say when presented with similar information. One of my research interests is in how students read and respond to different types of scientific texts. I often compare different writing styles, such as factual expository writing that just explains scientific concepts to non-fiction narrative writing that tells the story of scientific findings. One of writing styles I’m really interested in is adapted primary literature. Adapted primary literature takes scientific journal articles and rewrites them, keeping the form and the style of the original research article but simplifying the terminology and making the evidence easier to understand. One of the articles that I’ve adapted was published in 2009 in the journal Geology and called “Recent bright gully deposits on Mars: Wet or dry flow?”.  It presents students with data gathered through models of flowing water and dry sand to explain patterns seen on Mars’s surface (A question that is of ongoing interest).

So, here’s the comparison. Grade 6 students (11-12 years old) in three different schools read different versions of this study and responded by writing letters to the lead author to share their thoughts on the study.  Given the chance, did they respond like the adult commenters with wild speculation and conspiracy? .

Thankfully, no. It turns out that the students provided just the palette cleansing I needed, restoring my faith in how well readers can respond to scientific studies.

The students were interested and enthusiastic, and they expressed support for the researcher. They were positive and excited, showing their own interest in the topic.  Unlike the adult commenters, they also were very specific in their writing. They stuck close to the topic of the article: the possibility that marks on Mars’s surface were made by water flow. They wrote responses like:

“Dear Scientist, I love what you’re researching I have a big interest in the planets too.”[i]

“I thought that your research was very interesting and I liked your idea of making models.”

“I have enjoyed reading your research a lot.  I can’t wait to hear more about if there really is water on Mars.  As well as if you find out how much water (if there is) on Mars.”

“I think that your work has helped the discovery of what is going on, on Mars.  I think you’re awesome.”

And this one, my personal favourite:

“Dear Scientist, I think that you have done good job of your research.  I like how you included the models to help you with your study.  I found it very interesting how Mars could have possible water in it.  If there is water then could there be life?  Your article sort of left me with a question mark and wanting to know more.”

Really, what better reaction could we ask for? The student is obviously interested, has understood the basic method and evidence, and wants to know more.

Beyond this general enthusiasm, some students wrote letters very similar to the first few comments on the article. They engaged in some moderate speculation about what the findings could mean for our future on Mars. Like the adults in the early part of the comment thread, the students presented it as appropriate speculation though with no conspiracy theories attached or science fiction presented as fact.

“Me and my friend have been reading your articles and they are really cool.  We were wondering would it be possible to send indestructible robots to Mars and make it a manmade livable environment, with water, food and oxygen?”

“Could you live on Mars with water?  Could you build a house on Mars, build a farm with a glass case around it and you could survive? “

The most interesting comments, though, were the ones that took a critical view of the evidence, asking for more information, for further justification or making suggestions for how the case could be stronger. This type of comment was completely absent from the adult comment thread. And far from being the outlier, comments like this were among the most common from the students. Almost half of the letters included some element of critical appraisal of the evidence, contemplating whether it was sufficient or whether the type of evidence was convincing. The students wrote comments like:

“I think you made good research.  But I think you should look at more possibilities for the streaks.  Look closely at the streak for other answers.  How do you know it could not be something else other than water or dry materials?  Would you or any other scientists be able to go to Mars to see exactly what the streaks are?”

“You could do more research.  You should take samples and analyze them for a better result.  Where were the streaks?  Could ice have made the Streaks?  How do you know the tests are accurate?”

“I think that was very interesting but you need more proof.  If there is a sighting of water people would have something to compare it to.”

“I think you have lots of evidence and I hope someday you will go to Mars to see if water for real is on it!”

And, from a student who might just be a future scientist or science journalist:

“I heard about your conclusion to the streak on mars.  Though it is smart and it works, I believe you’re not quite done that experiment until you’re sure about what you say with proof such as samples instead of pictures.  Pictures ARE useful, yes, but with using computer models, well those aren’t always quite accurate.  I don’t know much about computer models, so forgive me if I’m wrong, but sand and dirt streaks CAN have straight “tails”, they don’t always have to be finger trails.  It still could be mud but even with high quality pictures and very similar features, you still can’t be sure.  Who knows, it could be something new.” (As a side note, for those who doubt middle school students’ writing ability, I didn’t correct a single thing about this comment)

So, while the adult comment thread degenerated in wild speculation and possible conspiracy, the children’s responses were almost all on topic, specific and with a good dose of critical thought and reflection about evidence.

Ah, that feels better. Faith in the public of tomorrow restored.

To be honest, though, I know that this is an unfair comparison. The students were asked to read an article at school. In the context of the science classroom they probably know that they are expected to read the whole article (not just the headline) and to respond in particular ways (e.g., to be positive instead of negative and to make specific connections to the text). It isn’t surprising that their responses are different from those of adults who happen to come across a Popular Science article and decide, for whatever reason, to leave a comment. Probably many people read the article and thought, like the students, “Cool! That’s interesting. I love reading about planets” but they didn’t leave a comment, whereas the same students had to because it was a school assignment.

So why bother to show the comparison? Partly, because it makes me feel better. Mostly though  because I think there’s a bigger message in it. Reading discussions about science in online comments can be frustrating and disheartening. Is that all readers can do, speculate widely and mistakenly believe evidence from a Hollywood movie? No. I think the students’ responses show that clearly. Even young science students have skills and knowledge to comment intelligently and thoughtfully on scientific research, and there is no real evidence that they lose this somehow just by the mere fact of getting older.

The problem illustrated in the Popular Science article comment thread then becomes a different one that it first seemed. It’s not necessarily a question of mass scientific illiteracy but a question about the kinds of conversations that happen about science. If readers have the basic knowledge and abilities that the students’ writing suggests, where do comment threads like this come from? Why is it, for example, that those with conspiracy views sometimes seem overrepresented in online comments? Why are those comments okay? Where are those with genuinely curious questions or enthusiastic responses? Does it have to do with the amount of attention readers have to give to the science stories they read online? Is the culture of online commenting one that doesn’t always attract sincere, even if simple, questions and answer? If it’s trolling instead of conspiracy, why choose to do that at Popular Science on a short research summary? What’s the point? Ok, that’s a lot of questions. And this is just a start, there are so many more that could be asked. But, like Alice Bell has argued, the most productive question probably isn’t “Why don’t people know more about science?” The way the kids outshine the adults on this one sends strong reminder to ask the right question about why that comment thread is troubling.