hacking breaking news
Dec. 8, 2013, 4:13 p.m.
Even though I'm no longer working for a news organization, I can't help but continue to spend time thinking about how technology can be used both to build better tools for journalists and communities, and to disseminate information more efficiently.
While there seems to be some debate about whether or not technology is ruining journalism, something I've found is that there tends to be a sweet spot where tech is combined with human curation and oversight. This has the potential to result in something superior to either approach individually. (More modern uses of twitter, and the app circa are both going in the right direction here)
So, I set out to work on a project taking advantage of some of the already existing, curated content sources that are available for free to anybody with an internet connection.
Given a near real-time stream of breaking news headlines, can we algorithmically determine the physical location of the events as they happen, along with some key terms or players associated with the stories?If so, what how can that information be used?
automating breaking news
applying the results
We have a computer program that knows what the most 'important' story is right now, and where that thing is happening.
Given that, we can create something to get updates on the story that are:
- happening at the location of the event
- happening in real-time
- reported by a reputable source
Figuring out (3), which tweets are reliable or not, is the more difficult (and ill defined), especially since the decisions have to be made in realtime as the twitter stream continually provides more and more tweets.
I don't claim to have solved (3), but my attempt was that a twitter user is considered to be reputable if they meet any of the following criteria:
- Are associated with a news organization in their bio
- Have certain keywords in their bio
- Have over N followers
Not surprisingly, the site is most effective when there is a large new story breaking, and is relatively unassuming otherwise. The two images of the program in action (a sporting event, and the Kenya mall attack) display this to some extent
There is plenty of possible future work with stuff like this, including possibly making an API to provide others with realtime and meta-tagged breaking news stories. What else would you do with it?