Computers are leaner, meaner and cheaper than ever before. With computing power no longer at a premium, we’re swimming in numbers that describe everything from how a small town in Minnesota behaves during rush hour to the probability of a successful drone strike in Yemen.
The advent of so-called “big data” means that companies, governments and organizations can collect, interpret and wield huge stores of data to an amazing breadth of ends. From shoe shopping to privacy concerns, here’s a look at five ways big data is changing the world:
1. Data as a deadly weapon: The traditional battlefield has dissolved into thin air. In the big data era, information is the deadliest weapon and leveraging massive amounts of it is this era’s arms race. But current military tech is buckling under the sheer weight of data collected from satellites, unmanned aircraft, and more traditional means.
As part of the Obama administration’s “Big Data Initiative,” the Department of Defense launched XDATA, a program that intends to invest $25 million toward systems that analyze massive data sets in record time. With more efficient number crunching, the U.S. military can funnel petabytes of data toward cutting edge advances, like making unmanned drones smarter and more deadly than ever.
2. Saving the Earth: Beyond powering predator drones and increasing retail revenue, big data can do a literal world of good. Take Google Earth Engine, an open source big data platform that allowed researchers to map the first high-resolution map of Mexico’s forests. The map would have taken a traditional computer over three years to construct, but using Google Earth Engine’s massive data cloud it was completed in the course of a day.
Massive sets of data like this can help us understand environmental threats on a systemic level. The more data we have about the changing face of the earth’s ecosystems and weather patterns, the better we can model future environmental shifts — and how to stop them while we still can.
3. Watching you shop: Big data can mean big profits. By understanding what you want to buy today, companies large and small can figure out what you’ll want to buy tomorrow — maybe even before you do.
Online retailers like Amazon scoop up information about our shopping and e-window shopping habits on a huge scale, but even brick and mortar retailers are starting to catch on. A clever company called RetailNext helps companies like Brookstone and American Apparel record video of shoppers as they browse and buy.
By transforming a single shopper’s path into as many as 10,000 data points, companies can see how they move through a store, where they pause and how that tracks with sales.
4. Scientific research in overdrive: Data has long been the cornerstone of scientific discovery, and with big data — and the big computing power necessary to process it — research can move at an exponentially fast clip.
Take the Human Genome Project, widely considered to be one of the landmark scientific accomplishments in human history. Over the course of the $3 billion project, researchers analyzed and sequenced the roughly 25,000 genes that make up the human genome in 13 years. With today’s modern methods of data collection and analysis, the same process can be completed in hours — all by a device the size of a USB memory stick and for less than $1,000.
5. Big data, bigger privacy concerns: You might just be a number in the grand scheme of things, but that adage isn’t as reassuring as it used to be. It’s true that big data is about breadth, but it’s about depth, too.
Web mega-companies like Facebook and Google not only scoop up data on a huge number of users — 955 million, in Facebook’s case — but they collect an incredible depth of data as well. From what you search and where you click to who you know (and who they know, and who they know), the web’s biggest players own data stockpiles so robust that they border on omniscient.
Where technological power, cultural advancement and profit intersect, one thing’s clear: with big data comes even bigger responsibility.