Part of this past weekend’s meeting of the Committee on Informing the Public was to evaluate 100+ proposals for “mini-grants” of up to $10,000 for new outreach activities. It wouldn’t be appropriate to go into detail about any of the proposals or what we decided (the PI’s of the proposals we decided to fund will be notified soon), but there was one issue that came up again and again that I think is appropriate for the blog, which is what should be considered as a successful effort, particularly in the online world.
A large number of the proposals we were considering had “new media” components to them– videos to be placed on youTube and the like, web sites with informative descriptions of physics topics, blogs about physics, and so on. One of the things we argued about at length is how to measure the impact of these sorts of items. The obvious metric would be something like the number of times the video got played on YouTube, or the number of hits a web site gets, but then it’s damnably difficult to agree on a number that would constitute success.
Let’s say you put together a video about physics, such as this one of me talking to my dog about the Many-Worlds Interpretation of quantum mechanics:
How many people have to watch this for me to call it a successful example of physics outreach to the general public?
At the time of this writing, that video has been viewed 380 times. Now, on the one hand, you could compare this to the total viewership for YouTube videos that “go viral,” like the LHC Rap, which has drawn more than 6,000,000 views to date. In which case, you would consider my little video a dismal failure.
But then, is that really an appropriate standard to set? Trying to get a video to go viral is a little like trying to find the Higgs Boson by sitting in your room and waiting for a passing cosmic ray to create one. It might happen– there are lots of cosmic rays, and many of them have enough energy to make a Higgs– but it wouldn’t be a great basis for a research program. Viral videos are, more or less by definition, random and unpredictable, so it seems a little unfair to consider something a failure because it didn’t manage a wildly unlikely achievement.
A more realistic standard might be to ask how many people are being reached by this that wouldn’t’ve been reached some other way. For example, you could note that 380 people viewing this video would be close to 70% of an entering class at Union, which is at least three times as many people as I could reasonably expect to teach in the course of my day job. I use this sort of standard a lot when I talk about traffic to the blog— as I note in that post, the average daily number of pageviews for this blog last year was larger than the enrollment of Union, so it’s reaching out to a much bigger audience than I have via my teaching. And that audience is global– I get hits from all around the world.
This does lead to some local variation in standards. For example, in another discussion I noted that Sean Carroll’s talk here drew 100+ people, and one of the other members replied “A hundred people is nothing!” Which might be true, at a state university with a student population of 10,000+, but it’s very good for a science talk at a small liberal arts college. I don’t think there’s any other metric that’s dramatically better, though.
the only other standard that really comes to mind is also a relative measure, namely “how well did this do relative to similar outreach efforts by the same people?” In which case, the video embedded above is a disappointment– my other two book-related videos have been viewed thousands of times, and the Bohr-Einstein puppet video has been viewed around 10,000 times, if you add up all the different versions. But that’s also subject to the viral effect– a large fraction of those 10,000 views came because John Scalzi linked to it from Whatever, which gets 50,000 readers a day. And it’s not remotely reasonable to expect John to promote anything and everything I put on YouTube.
So, in the end, I don’t have a solid idea for how to quantify the effect of outreach programs on the Internet. On the off chance that somebody else does have an idea, though, I’ll throw it out to my wild and worldly readers: What would you say is a reasonable standard for a successful outreach program?
Thanks for this post, Chad. You bring up some fantastic points. I think there is also the question of the impact that one video or one talk has on the people watching it. What if every one of the 100+ people who went to Sean Carroll’s talk left with a desire to learn more about physics and began to listen to pod casts, read books, take courses and so on, but 99.99% of the people who watch the LHC rap just got an annoying song stuck in their heads and never learned a thing and the 600 that did learn something never did anything about it? Would those 100+ people still count as “nothing!” in terms of outreach? Would the LHC rap still be a better “outreach” project? Is outreach impact measured in sheer numbers or also in actual lasting effect?
“At the time of this writing, that video has been viewed 380 times.”
So I went to the page. Still 380. Then I watched it (again – I must have been one of the 380 when you first posted it).
Then I loaded the page again, using a different browser, and a different PC, and a different IP. Still 380.
Quantum Zeno effect?
I couldn’t see the video, I’ve been getting errors but much is likely due to my using Linux-based SeaMonkey (since my Windows-loaded HD crashed), any suggestions? In any case, I am roughly familiar with the MWI concept and the version/s saying that the “world” are still all part of the same space-time just don’t add up (heh.) Consider if you send a photon through a beamsplitter and on toward two detectors the photon can reach. If you say that somehow the photon is really in both detectors, then you violate conservation of mass-energy etc. because there âreally isâ twice as much energy as before. Itâs not just an issue of information, but mass-energy too.
I don’t calim to be “Successful” in outreach terms, but my YouTube channel has around 600,000 views, and the most watched video has more than 150,000, so… those sorts of numbers are not out of reach for a physics or astronomy lecture on YouTube.
One thing one CAN’T tell from YouTube is how many people watched the first 60 seconds and how many watch all 60 minutes.
Numbers aren’t really meaningful unless there’s context.
For example: music. You have a small band of weekend warriors. You play a coffee shop or small club. 100 people show up. Success!! The place is packed! YAY!
A major label picks you up and books a show in a theater that holds 3000 people for your band. 1000 show up. Damn, failure! The record company starts thinking “these guys aren’t the Next Big Thing.”
So 100 is great success and 1000 is failure, depending on the context.
Also, there’s the width vs depth issue. You teach a freshman physics class at State U to 150 people. They paid to be there, tuition,textbooks and all the rest. They are already qualified, plus there’s consequences for them if they don’t take the material seriously and study it.
1000 watch the lecture (or part of it, or at least click on it) on the State U youtube channel. Those people don’t have to know anything about physics, don’t have any consequences if they don’t pay attention, and while some unknown percentage of them probably are interested enough and trained enough in physics to do some follow up activities (even if it’s just watching another video lecture), most likely for the majority it is an isolated event not tied to any sustained course of action or study.
So what is more effective?
It’s nice to have both. I think the main concentration should be on your personal work, and then by posting online you make your work (and yourself) more widely visible. Occassionally somebody will come from the web who is actually a good connection and with whom more can be accomplished, but this is pretty rare compared to the good contacts who will come through offline channels (face-to-face meetings, classes, professional channels, etc).
So I say just concentrate on your experise. Put your stuff online, and be happy when it gets kudos, but don’t obsess about the online results like number of hits, because they aren’t the core of your efforts.
It could very well be that the the video with the lower number of hits is actually a better video than the one the popular blogger reposted for you.
Which brings up another point: number of hits is not a quality indicator. Millions of people click on stupid trash online everyday.
Puppets.