There’s a pattern you might be forgiven for having missed: most technological developments on the web have been driven by these industries. Think online casinos. Think credit card usage. Think video. Think VR. Users demand greater financial and data privacy — after all, who wants to admit they’re signing up for porn or gambling sites? – and better bang for their buck — a streaming video better be a lot better quality than any of the millions of free sites out there to be worth signing up for, all of which pushes these industries tothe bleeding edge of innovation. Buried in early versions of the Bitcoin software – the cryptocurrency that offers a future of transactions beyond the gaze of banks and governments — are hints of a gambling connection, after all. WebRTC is now no different.

So first off, what is WebRTC, and why should you care? WebRTC is an open source project that embeds real-time voice, texts, and video communications capabilities in web browsers. The technology enables peer-to-peer communication (P2P) among browsers. It does not require specialized software applications or browser plugins for communication. In essence, it’s the engine that powers a lot of the messaging that goes on between apps — think video, audio. The RTC bit in it, after all, stands for Real Time Communication. The Web bit is because the standard at least started life as a way to make using video and audio to communicate through a web browser as easy as using text, without any plug-ins. But it has moved beyond that, and can now sit within apps on mobile phones and elsewhere.

That may seem like quite a modest goal. But it’s only recently, with Apple’s decision to include WebRTC in its Safari browser (from version 11 and in the iOS browser in version 12), that a years-long battle to agree that WebRTC is the standard everyone can agree on is more or less won. Now, you can use a host of services to do Skype-like video and audio chatting via your browser without having to

1. a) register with any service
2. b) install any software
3. c) worry about what kind of browser you’re using.

(Of course, there are still caveats.)

This is quite a feat, if you think about it. Not because you’ve been dying to do this, exactly, but because it means that this kind of capability — real time voice and/or video communication, in real time — can be added to any web page. This means, for example, that you can now stream directly from whatever news event your reporters are covering, or direct from your newsroom, or hold a Q&A with readers, without them having to worry about plug-ins, browser versions, etc. etc.

Beyond the browser

And it doesn’t stop there. WebRTC obviously started with the web, and the browser, but that was because that was where we spent most of our time back then. Now things are mobile, they’re often app-based, and the way we communicate, and the way we consume video, has changed a lot.

WebRTC has ended up being a way for the industry to agree on a bunch of standards which have enabled all sorts of things to happen more quickly and seamlessly than they might have done had WebRTC standards groups not been working away in the background. Varun Singh, CEO of callstats.io, which measures quality of experience of live and real-time media, explained that by agreeing on this standard suite of protocols, WebRTC has quickly wormed its way into pretty much every messaging app, from Snapchat to Facebook Messenger.

And it’s not just there. Dean Bubley, a consultant who has been watching WebRTC since 2009, believes “it’s still lacking in recognition in some quarters; there are many more applications that could benefit from it.” In a white paper funded and published by callstats.io, he explored how real-time communications are becoming embedded into devices and applications, into different formats, into processing (think AI) and more platforms. As users become more accustomed to accepting of voice and video technologies (would you have imagined we would have allowed listening robots called Alexa and Siri into our homes as easily as we have five or ten years ago, or been as comfortable taking selfies or video-ing ourselves?) So these technologies are likely to continue to evolve and, of course, become more commonplace in every day life

“In other words, society is now much more amenable to new use-cases for RTC,” concludes Dean. “There is familiarity with many of the UI tools, and less self-consciousness in front of cameras. This,in turn, means that the “cognitive load” on users’ brains is lower, meaning that the interactions are more natural – and more productive.”

It’s only in media, I suspect, that we’re still a little stuck with old formats: the newsreader, the correspondent talking to the camera, the interviewee stuck in a studio somewhere, or on Skype with a picture of a horse in the background. I strongly believe we have an opportunity to have a chance to break away from this.

In media

I see the opportunities here as at least initially more modest.

Firstly, internally: I have long been frustrated at how internal newsroom discussions can be starved of creative oxygen as much by poor technology decisions as by poor leadership. Reliance on dialling in to a conference call number seems both archaic and wasteful use of resources. More often than not the more creative thinkers on the call are drowned out by the noisier ones. A simple WebRTC link in the browser should solve that, using simple tools like talky.io which require no plugins.

Then there’s content. Varun of callstats.io says WebRTC offers content creators the chance to make content simply, just with a camera, which users can access from the news organisation’s website directly — bypassing YouTube or Twitter. Think webinars, game shows, he says: “it’s fairly trivial, say $50 to set something up, and have it viewed an unlimited number of times.”

So where does porn, gambling and auctioning come into it?

Well, I think the future of WebRTC for media lies in the ability to seamlessly stream events to users as if you were a professional broadcaster. It’s one thing to be able to record or stream a few jerky minutes of a demonstration, but soon enough it will be possible — even expected — that any media organisation, large or small, can, with little preparation, livecast an event with little or no lag.

Alexandre Gouaillard, CTO of a company called Millicast, says these industries are the ones pushing for this. With Adobe no longer supporting the Flash plugin, the mainstay for such industries, there’s a demand for low latency video but at scale: Millicast promises broadcast across the globe in less than 500 milliseconds. The packages range from 10 concurrent viewers (free) to 5,000 ($1,500 a month.)

This may be lower latency than required, but it’s a glimpse, I believe, of the future. I think we’ve been held back from using video in part because of its awkwardness in setting up, and the glitchiness, that makes interactions painful. Those days are coming to a close, mainly because of WebRTC.

You don’t have to be into interactive porn to imagine the possibilities of having people seamlessly integrated by a video connection, wherever they are: if your reporter can walk around with a smartphone mounted on a $150 gimbal, confident that every shot is being beamed to every user; or a Q&A with an editor accompanied by graphics and whiteboard is crystal clear to all 5,000 viewers in a monthly editorial catch-up, then perhaps in a year or two an augmented reality- or virtual reality-, or 360-degree- broadcast from a sports game or political rally — then we’ll be able to thank a porn star or a gambler for marking the way for us, once .

In researching data journalism in the era of big data, I have found myself trying to define what “data” means in this context. Of course when we refer to data we usually envisage significant amounts of numerical information, but as data sets get unimaginably large this itself becomes problematic. So I decided to take a step back, and look at a successful story that is data driven closer to home.

I chose,a cluster of stories,  from Manila, by the Reuters reporting team of Clare Baldwin, Andrew Marshall and Manny Mogato, who covered Philippine president Rodrigo Duterte’s war on drug suspects. (Transparency alert: I was, until earlier this year, an employee of Thomson Reuters.) The three of them won a Pulitzer  in the International Reporting category for this coverage. I wanted to just focus on a couple of stories because I think it helps define what data journalism is. This is how their story of June 29 2017 described how Philippine police were using hospitals to hide drug war killings:

• An analysis of crime data from two of Metro Manila’s five police districts and interviews with doctors, law enforcement officials and victims’ families point to one answer: Police were sending corpses to hospitals to destroy evidence at crime scenes and hide the fact that they were executing drug suspects.
• Thousands of people have been killed since President Rodrigo Duterte took office on June 30 last year and declared war on what he called “the drug menace.” Among them were the seven victims from Old Balara who were declared dead on arrival at hospital.
• A Reuters analysis of police reports covering the first eight months of the drug war reveals hundreds of cases like those in Old Balara. In Quezon City Police District and neighboring Manila Police District, 301 victims were taken to hospital after police drug operations. Only two survived. The rest were dead on arrival.
• The data also shows a sharp increase in the number of drug suspects declared dead on arrival in these two districts each month. There were 10 cases at the start of the drug war in July 2016, representing 13 percent of police drug shooting deaths. By January 2017, the tally had risen to 51 cases or 85 percent. The totals grew along with international and domestic condemnation of Duterte’s campaign.

This is data journalism at its best. At its most raw. The simple process of finding a story, finding the data that supports and illustrates it and then writing that story and using the findings to illuminate and prove it. Of course, the data set we’re talking about here is smaller than other data-driven stories but it’s still the point of the story, the difference between a story and much lesser one.

But how did they come by that data? Andrew tells me it was done the old fashioned way: first, they got the tip, the anecdotal evidence that police were covering up deaths by using ambulances to carry away the dead. Then they went looking for proof — for police reports, which are public information in the Philippines and so can, in theory, be obtained legally. These they found, because they looked early and persistently. Then it was a question of assembling these, and cleaning them up. In some cases, it meant taking photos of barely-legible police blotters at a station entrance.

All their stories, Andrew told me, were driven by the reporters already having a sense of what the story was, and then looking for proof. That means already knowing enough about the topic to have formed an opinion about what to be looking for: about what may have happened, about what angle you’re hoping to be able to prove, about what new fresh evidence you believe the data will unearth for you. A data-driven story doesn’t always mean wandering around the data without a clear idea of what you’re looking for. In fact, it’s better to already know. “The key thing,” he told me, “is that this all grew out of street reporting. We wouldn’t have thought to look for it if we hadn’t heard.”

That’s the first lesson from their experience. Data is something that is there that helps you prove — or refute — something you have already established to be likely from sources elsewhere. 

This is where I think sometimes data journalism can come adrift. By focusing too much on the “data” part of it, we lose sense of the “journalism” part of it. “It’s the blend of street reporting and data analysis that paid the great dividend,” Andrew said.

A  definition of data journalism should probably start somewhere there; but it tends not to. Instead we tend to get: data journalism as a “set of tools’, or “locating the outliers and identifying trends that are not just statistically significant but relevant”, or “a way to tell richer stories” (various recent definitions.)  These are all  good, but I’m not sure they’re enough to help us define how to best use data for journalism.

By emphasizing data over journalism we risk removing and rarifying the craft, creating a barrier where it doesn’t need to exist. As in the previous examples in the Philippines, data is not always something that sits in databases, servers, libraries or archives. Nor is it something that you have to ask for. It’s something you use to gather information to help better storytell and to reinforce the facts in your coverage.  A study by Google last year trumpeted that more than half of newsrooms surveyed had a dedicated data journalist.

Aren’t we all, or shouldn’t we all consider ourselves, data journalists? Shouldn’t we all be looking for data to enrich — if not prove the thesis that underlies — our stories? 

Back to Andrew’s example. For the team it was something of a no-brainer to work on attaining this data. The story would have been unthinkable without it. This might not be part of every journalist’s instinct, but it’s telling in this example, that it became central to their story and took weeks, months to assemble.

The place to start from was with the local police and hospitals to get this data.  To do so was legal. But it wasn’t easy, and became increasingly less so as the work developed.. Clare Baldwin was greeted at one station by homicide detectives who shouted and lifted their shirts to display their guns. Later, Andrew told me, it became much more difficult to have access to this information as the Duterte government realized what it was being used for.

The lesson from this is that data is not necessarily something that is easy to get, or easily given up. Or that arrives in pristine form. It requires some major work in verifying, identifying and compiling. is more akin to the example of Bellingcat, the crowdsourcing website created by journalist Eliot Higgins, which conducts what it calls open-source investigations of data sources, ranging from social media photographs to online databases.

Of course, not all stories are going to be like this, and not all data is going to be like this. But all journalism, data or otherwise, requires thinking that starts from a similar place: a strong knowledge of what the story might be, and where to find it; whether there might be data that might help, to know where to find it, to not be daunted in obtaining it, or by the condition it is in, and to understand the context of the data that you have, and to know what to do with it. And finally, in Andrew’s words, “to use that data quickly, not just to sit on it“.

The aforementioned team’s stories stand on their own.  As another example, the Reuters’ graphics team, led by Simon Scarr, also did some extraordinary visualizations which helped readers understand stories better, and provided additional impact. Visualization and data journalism are obvious bedfellows.

This isn’t to say sometimes the idea for a story doesn’t lie in the data itself. Data journalism can mean taking data as inspiration to explore and write a story — rather than beginning the process by talking to sources.. At its most basic this could be a simple story about a company’s results, or a country’s quarterly trade figures — data-driven stories where the journalist reports the new numbers, compares them with the earlier numbers, and then adds some comment.

But when there is overemphasis on data journalism as a separate part of the news process it can pose problems. There’s been quite a lot written about a backlash against ‘nerd journalists’ and an exodus of those computer-literate staff in newsrooms who are sick of the skepticism and relatively low salaries. I’ve not witnessed this firsthand, but I have seen how little interest there is in learning more about the ‘techie’ side of journalism that might help reporters wrestle with data beyond their familiar charts and tables. Editors are partly to blame: stories that involve dirty or larger data-sets do take longer and so are often unwelcome, unless they fall into a special category. So reporters quickly figure out they’re better off not being overly ambitious when it comes to collecting data.

Data journalism tends to be limited to a handful of really strong players. In my neck of the woods in South and Southeast Asia there’s an impressive array of indigenous (i.e. not one of the big multinational) outfits: Malaysiakini are almost old hands at this process now.  Their sub editor,  Aun Qi Koh told me that as it gets easier in terms of knowing which tools to use and how to use them, so it gets harder because “we want to push ourselves to do more, and to better…and of course, there’s the challenge of trying to streamline a process that often involves a team of journalists, graphic designers, programmers, social media marketers and translators.” she tells me.

This is impressive, and is demonstrating what is possible. News organizations are making the most of governments’ gradual commitment to opening up their data, and to leveraging issues that the public care about. In the Philippines Rappler has been making waves, and won an award for its #SaferRoadsPH campaign, which compiled and visualized statistics on road crash incidents and has led to local police drawing pedestrian lanes outside schools.

These kinds of initiatives are tailor-made for visual data journalism. Not least because journalists don’t have to rely on government data that might be either absent, incomplete or wrong. Or, in some cases, just unreliable. Malaysiakini’s Aun Qi Koh said that the data in a government portal set up in 2014 was neither “organized properly nor updated regularly.” That seems to be par for the course. And while staff everywhere need better training, those that do have the necessary training tend to get snapped up by–and attracted to–private sector companies rather than relatively low paying journalist positions, according to Andrew Thornley, a development consultant in Jakarta.

I’m impressed by all these projects, especially those doing great journalism on a shoestring. But I hope it doesn’t sound churlish if I say I still think this is scratching the surface of what is possible, and that we may not be best preparing ourselves as well as we could for the era of big data.

Take this story as an example: Isao Matsunami of Tokyo Shimbun is quoted in the Data Journalism Handbook as talking about his experience after the earthquake Fukushima nuclear disaster in 2011: “We were at a loss when the government and experts had no credible data about the damage,” he wrote.  “When officials hid SPEEDI data (predicted diffusion of radioactive materials) from the public, we were not prepared to decode it even if it were leaked. Volunteers began to collect radioactive data by using their own devices but we were not armed with the knowledge of statistics, interpolation, visualization and so on. Journalists need to have access to raw data, and to learn not to rely on official interpretations of it.”

The data he’s talking to was created by Safecast, an NGO based in Japan which started building its own devices and deploying its own volunteers when it realised that there was no reliable and granular government data on the radiation around Fukushima. Now it produces its own open source hardware and has one of the largest such data-sets in the world, covering air quality as well, covering sizeable chunks of the world.

The future of data journalism lies, I believe, in exactly this: building early, strong relationships with outside groups — perhaps even funding them. More routinely, journalists should find their own sources of raw data where it’s relevant and practical, and fold the mindset, tools and approach of data journalism into their daily workflows the rest of the time. You can already see evidence of the latter on sites like Medium and Bloomberg Gadfly, where journalists are encouraged to incorporate data and charts into their stories and to build an argument. Much of this is already happening: Google’s survey last year found that 42% of reporters use data to tell stories twice or more per week.

But the kind of data being used may be open to question. Data is no more a journalist’s friend than any source — it has an agenda, it’s fallible, and it can often be misquoted or quoted out of context. As journalists we tend to trust statistics, and interpretation of those statistics, a little too readily.

For the sake of balance, here’s a Reuters story from 2014, still online, that quotes an academic study (“Anti-gay communities linked to shorter lives”) despite the fact that in February this year a considerable correction was posted to the original study. (“Once the error was corrected, there was no longer a significant association between structural stigma and mortality risk among the sample of 914 sexual minorities.”) We are not, as journalists, usually given to expressing skepticism about data provided by academics and similar but maybe we should. (And I suppose we should be better at policing our stories, even if the correction is required years after the story first appeared.)

Tony Nash, founder of one of the biggest single troves of economic and trade data online at CompleteIntel.com, believes journalists tend to let their guard down when it comes to data: “The biggest problem with data journalism is that data is taken at face value. No credible journalist would just print a press release but they’ll republish data without serious probing and validation. Statistics agencies, information services firms, polling firms, etc. all laugh at this.”

Day to day journalism, then, could benefit from being both more skeptical and more ambitious about the data it uses. Tony says he’s tried in vain to interest journalists in using his service to mash stories together, so instead writes his own newsletter, often ‘breaking’ stories long before the media: “In July 2017 I showed that Mexico and China are trade competitors but journos always believe China has an upper hand in trade. For all of 2017, Mexico exported more TVs to the US than China. For the first time. It was not a surprise to us. Most journos still have not woken up to that,” he told me recently.

Coupled with tools that make it easier to combine visuals into their stories — Datawrapper, a chart making tool, for example, has launched an extension called River which makes it easier for journalists to identify stories or add data to breaking stories.

But this is just the start. We are in the era of big data and we are only at the beginning of that era. The Internet of Things (IoT) is a fancy term to cover the trend of devices being connected to the internet (rather than people through their devices, as it were.) There will be sensors on everything, but there will also be light switches, washing machines, pacemakers, weather-vanes, even cartons of milk, telling us whether they’re on or off, full or empty, fresh or sour. All will give off data. Right now only about 10% of that data is being captured. But that will change. According to IDC, a technology consultancy, more than 90 percent of this IoT data will be available on the cloud, meaning that it will be analyzed, by governments, by companies, and possibly by journalists. The market for all this big data, according to IDC, will grow from $130.1 billion in 2018 to over $203 billion in 2020. This market will primarily be one about decision making: a cultural shift, to “data-driven decision making”.

You can see some of this in familiar patterns already: Most of it is being used to better understand users — think Amazon and Netflix getting a better handle on what you want to buy or watch next. But that’s pretty easy. How about harder stuff, such as taking huge disparate data sets — the entire firehose of Twitter, say, along with Google searches, Facebook usage (all anonymized of course) — to be able to slice target audiences very thinly. One Singapore-based company I spoke to has been able to build a very granular — as they call it — picture of who they want to target, down to the particular tower block, their food preference (pizza), music (goth) and entertainment (English premier league). Not only does this make advertisers happy they’re going after the right people, it makes it much cheaper.

But this is just the beginning of big data. Everything will be spitting out data — sensors in cars, satellites, people, buildings; everything we do, say, write etc. Knowing what data there is will be key: Another Reuters graphics story — which won the award for Data visualization of the year at the Data Journalism Awards 2018 involved realising the value of a data-set of GPS coordinates of infrastructure gathered by aid agencies working on the ground at a Rohingya refugee camp in Cox’s Bazaar to analyze the health risks of locating water pumps too close to make-shift toilets. And then there’s knowing whether there might be other data hiding within the data: Buzzfeed’s application of machine learning to Flightradar aircraft data to single out the clues that revealed hidden surveillance flights, which also won a Data Journalism award.

These are small glimpses of the future of the kinds of data journalism we might see.

In the future it will be second nature to journalists to not only know what kind of data is being collected and to turn it to their own uses, but to try to pre-emptively collect it. This will require lateral thinking. Journalists have been using satellite imagery for several years as part of their investigations but this is likely to become even easier, cheaper, and more varied. One entrepreneur I spoke to recently is launching dozens of micro-satellites to monitor methane emissions — data of interest to oil and gas companies worried about gas leaks, governments enforcing greenhouse gas regulations, as well as hedge funds looking for exclusive economic indicators. Imagine if a journalist is able to peruse that data and uncover military activity from heat emissions even before governments know about it.

This is just the tip of the iceberg, and while journalists may not be at the front of the queue for this kind of data, it’s going to be important to know what kind of data is out there. Already the notion of what a “leading indicator” is has begun to change — an investor in China is much more likely to be trawling through data from Baidu than government statistics to get a sense of what is going on, and smart journalists already know that.

The future of data journalism, if it is successful, will still be journalism. And data will still be data. But as the world of data gets bigger, it pays to remember that the relationship between ‘data’ and journalism is still about thinking and acting creatively and quickly to uncover stories others may not want us to tell. 

I’ve been a nomad worker for some time. And I’m shocked at how few journalists seem to be prepared for mobile working. So I thought I’d offer a few tips.

If you can afford it, buy your own equipment.

I’ve been buying my own laptop for nearly 30 years, and while it brings pain to my pocket, I’d never dream of relying on my company’s equipment. In the old days it was because they were too slow and cumbersome, but nowadays it’s mainly because of compliance issues: restrictions on what software you can put on your laptop, as well as what the company is allowed to do and view on its hardware. I would rather retain control over how I organise my information and what apps I use.

Buy your own software.

I’m admittedly a bit of a software addict. (I think it’s probably a thing, I haven’t checked.) But there’s a reason for it: we spend most of our day at our computers, so it makes sense to find the software that best helps you. And with journalists, that’s a broad array of tasks: if you’re a freelance, you want to be measuring your word count and timing how long you’re spending on something. If you’re writing a lot then you want an app that looks aesthetically pleasing (I can’t stand Microsoft Word, and hate it when I see journalists writing stories in it, but that’s me). Then there’s how you collect and store information, be it from the net or from interviews. It needs to go somewhere and it needs to be easily retrievable when you want to write. More on this another time.

Get a decent mouse.

There’s a guy in my co-working space that still uses his Macbook touchpad, that rectangle near the keyboard, to move the mouse around. Very few people are adept at this, so it’s painful to watch guys like my co-worker waste hours a day scrambling around. Buy a mouse. Really. They’re cheap — you can even get a bluetooth one for less than $50 these days, so you don’t even need to take up a USB port. I guarantee it will save you an hour a day.

Save your own neck.

Mobile journalism can mean standing up, moving around, but most of the time it means bringing enough equipment with you to be able to work away from the office — a hotel room, a conference centre, or whatever. This is where I see far too many people hunched over a laptop, looking like Scrooge on Boxing Day. The problem with laptops is they weren’t designed for posture. But you can fix that, with a $20 stand. These are light, foldable, and lift the screen up to a height closer to eye level, which is where it should be. You’ll need to bring an external keyboard with you, but they’re cheap and light too, and your chiropractor will thank you.

While you’re at it get a second screen.

Here’s another tip: Laptop screens are too small to store more than what you’re writing on. If all your source mverkkorahaterial is also stored on your computer, then you’ll need a second screen. You likely have dual monitors in the office, so just because you’re on the road, why should you deny yourself that luxury? There are some good cheap monitors that don’t even require a power supply — plug them into your USB port and they’ll draw the power from there. For several years I had a AOC monitor, which was basic but did the job. I recently upgraded to an Asus monitor which is a beauty, and has made me much more productive and the envy of my co-workers — even the guy fiddling around with the touchpad.

A word of warning to Mac users: recent updates to their operating system have broken the drivers necessary to get the most out of these second screens, but there is a workaround that half fixes it. Email me if you need help.

Be safe.

Being mobile with cool equipment does leave you vulnerable to theft, either financially or politically motivated. Don’t take your main laptop with you to places like China. Have a cheap backup laptop with just the bare essentials on it. Always put your laptop in the room safe, and, if you want to be super clever, buy a small external USB drive to store any sensitive data on, and keep that in your pocket. Samsung do some nice, SSD (solid state, and hence smaller, faster) drives, the latest called T3. I attach mine to the laptop with velcro and then remove it and put it in my pocket when I’m heading off to dinner.

Stay connected.

Don’t trust other people’s wifi. Bring your own. I have a wifi modem, still 3G, which does me fine. Buy a local data SIM card and fire it up. Everyone in your team now has internet access — and the bad guys sniffing the free hotel or coffee shop wireless network will be frustrated.

Finally, stay cool.

By far the most popular thing in my mobile toolkit is a USB fan. Most conference venues are either too hot or too cold, and it’s amazing what a $2 fan can do.