How 3-D Printing is Revolutionizing the Display of Big Data

9. October 2014

Date: 09-10-2014
Source: Technology Review

3D Big DataIf you’ve ever struggled to make sense of an information firehose, perhaps a 3-D printed model could help.

One of the characteristics of our increasingly information-driven lives is the huge amounts of data being generated about everything from sporting activities and Twitter comments to genetic patterns and disease predictions. These information firehoses are generally known as “big data,” and with them come the grand challenge of making sense of the material they produce.

That’s no small task. The Twitter stream alone produces some 500 million tweets a day. This has to be filtered, analyzed for interesting trends, and then displayed in a way that humans can make sense of quickly.

It is this last task of data display that Zachary Weber and Vijay Gadepally have taken on at MIT’s Lincoln Laboratory in Lexington, Massachusetts. They say that combining big data with 3-D printing can dramatically improve the way people consume and understand data on a massive scale.

They make their argument using the example of a 3-D printed model of the MIT campus, which they created using a laser ranging device to measure the buildings. They used this data to build a 3-D model of the campus which they printed out in translucent plastic using standard 3-D printing techniques.

One advantage of the translucent plastic is that it can be illuminated from beneath with different colors. Indeed, the team used a projector connected to a laptop computer to beam an image on the model from below. The image above shows the campus colored according to the height of the buildings.

But that’s only the beginning of what they say is possible. To demonstrate, Weber and Gadepally filtered a portion of the Twitter stream to pick out tweets that were geolocated at the MIT campus. They can then use their model to show what kind of content is being generated in different locations on the campus and allow users to cut and dice the data using an interactive screen. “Other demonstrations may include animating twitter traffic volume as a function of time and space to provide insight into campus patterns or life,” they say.
Read the rest of this entry »


How Network Theory Is Revealing Previously Unknown Patterns in Sport

16. September 2014

Date: 16-09-2014
Source: Technology Review

Analysing the network of passes between soccer players reveals that one of the world’s most successful teams plays an entirely different type of football to every other soccer team on the planet.
Network Theory Soccer
If you’ve ever watched soccer, you’ll know of the subtle differences in tactics and formation between different teams. There is the long ball game, the pressing game, the zone defence and so on. Many teams have particular styles of play that fans admire and hate.

Innovations are common, with teams constantly adopting or abandoning new tactics. And given the international nature of football, new ideas spread rapidly, as players and coaches move from one team and country to another.

So it’s easy to imagine that it’s hard to play a truly unique brand of football, using tactics and skills that no other team copies.

That’s not quite true, say Laszlo Gyarmati at the Qatar Computing Research Institute and a few pals. These guys have used a network theory approach to characterise the play of all the top teams in Spain, Germany, Italy, France and England. And they say this metric reveals that while many teams share similar styles of play, one team stands out as truly unique, playing a style of football that no other team can match.

Football aficionados won’t be surprised to learn that that side is the Spanish team FC Barcelona, one of the most successful soccer teams on the planet. Barcelona have pioneered a type of football called tiki-taka that no other team has been able to master (with the notable exception of the Spanish national side, which generally has a large contingent of Barca players in its ranks).

Tiki-taka is characterised by rapid short passes and fast movement by the players. The idea is to dominate possession of the ball. That’s in sharp contrast to conventional tactics that focus on player formations. Read the rest of this entry »


Love of Labor

25. August 2014

 Typical Nick Carr! (hfk)

Date: 25-08-2014
Source: Technology Review

Automation makes things easier, whether it’s on the factory floor or online. Is it also eroding too many of the valuable skills that define us as people?

WHY IT MATTERS: Automation is creeping into more of our work and our leisure.

Messages move at light speed. maps speak directions. Groceries arrive at the door. Floors mop themselves. Automation provides irresistible conveniences.

Carr Glass CageAnd yet automation can also be cast as a villain. When machines take over work that once required sweat and skill, humans atrophy into mere button-pushing operators. Laments about automation are as familiar as John Henry, the railroad steel-driver of lore who could not outlast a steam-powered version of himself. The latest is The Glass Cage by Nicholas Carr, who worries about the implications as machines and software advance far past the railroad and the assembly line to the cockpit, the courtroom, and even the battle­field. Machines and computers now do much more than rote mechanical work. They monitor complex systems, synthesize data, learn from experience, and make fine-grained, split-second judgments.

What will be left for us to do? While economists and policy makers are debating what automation will mean for employment and inequality (see “How Technology Is Destroying Jobs,” July/August 2013), Carr’s book does not sort out those implications. It is about what he fears will be diminished—our autonomy, our feelings of accomplishment, our engagement with the world—if we no longer have to carry out as many difficult tasks, whether at home or at work.

The centerpiece of his argument is the Yerkes-Dodson curve, which plots the relationship between human performance and the stimulation our tasks provide. Too much stimulation makes us feel panicked and overloaded, but when we have too little stimulation—when our work is too easy—we become lethargic and withdrawn. Activities that provide moderate stimulation yield the highest level of performance and, as Carr argues, turn us into better people in the process.

Things reviewed

The Glass Cage: Automation and Us
BY NICHOLAS CARR
NORTON, 2014 Read the rest of this entry »


Three Questions for J. Craig Venter

31. July 2014

Date: 31-07-2014
Source: Technology Review

Gene research and Silicon Valley-style computing are starting to merge.

WHY IT MATTERS: The number of human genomes being sequenced is increasing exponentially.

Venter CCGenome scientist and entrepreneur J. Craig Venter is best known for being the first person to sequence his own genome, back in 2001.

This year, he started a new company, Human Longevity, which intends to sequence one million human genomes by 2020, and ultimately offer Web-based programs to help people store and understand their genetic data.

Venter says that he’s sequenced 500 people’s genomes so far, and that volunteers are starting to also undergo a battery of tests measuring their strength, brain size, how much blood their hearts pump, and, says Venter, “just about everything that can be measured about a person, without cutting them open.” This information will be fed into a database that can be used to discover links between genes and these traits, as well as disease.

But that’s going to require some massive data crunching. To get these skills, Venter recruited Franz Och, the machine-learning specialist leading Google Translate. Now Och will apply similar methods to studying genomes in a data science and software shop that Venter is establishing in Mountain View, California.

The hire comes just as Google itself has launched a similar-sounding effort to start collecting biomedical data. Venter calls Google’s plans for a biomedical database “a baby step, a much smaller version of what we are doing.”

What’s clear is that genome research and data science are coming together in new ways, and at a much larger scale than ever before. We asked Venter why. Read the rest of this entry »


What Every Company Should Know About Agile Software Development

8. July 2014

Date: 08-07-2014
Source: Technology Review

Does your company make medical devices? How about cars? Or appliances? Or mobile applications? Do you have an external website?

If you answered “yes” to any of these questions, then your company is a software company. As more and more businesses enter this category, the adoption of agile software development practices is becoming mainstream—and not just in the high-tech sector. Across industries, businesses are realizing that, even if their only foray into software is a corporate website, that software is the organization’s public face—and increasingly so, in this digital age of ours. Best to make a good first impression.

Agile software development is no longer a “bleeding-edge” approach. Nor does it mean that a bunch of rogue coders are riding roughshod over the roadmaps, timelines, and standards that are very real for businesses. In fact, the agile software approach requires a cultural embrace and commitment from the corner office all the way to the smallest cubicle.

Agile’s value is well-documented. The approach has helped teams at both large enterprises and small startups deliver high-quality software—on time—for more than a decade. If your organization is not already employing agile software development practices, now is the time to adopt them to maintain a competitive edge and deliver the best possible products to customers.

Why Waterfall Falls Down Read the rest of this entry »


The Limits of Social Engineering

16. April 2014

Date: 16-04-2014

Source: Technology Review By Nicholas Carr
Tapping into big data, researchers and planners are building mathematical models of personal and civic behavior. But the models may hide rather than reveal the deepest sources of social ills.

In 1969, Playboy published a long, freewheeling interview with Marshall McLuhan in which the media theorist and sixties icon sketched a portrait of the future that was at once seductive and repellent. Noting the ability of digital computers to analyze data and communicate messages, he predicted that the machines eventually would be deployed to fine-tune society’s workings. “The computer can be used to direct a network of global thermostats to pattern life in ways that will optimize human awareness,” he said. “Already, it’s technologically feasible to employ the computer to program societies in beneficial ways.” He acknowledged that such centralized control raised the specter of “brainwashing, or far worse,” but he stressed that “the programming of societies could actually be conducted quite constructively and humanistically.”

The interview appeared when computers were used mainly for arcane scientific and industrial number-crunching. To most readers at the time, McLuhan’s words must have sounded far-fetched, if not nutty. Now they seem prophetic. With smartphones ubiquitous, Facebook inescapable, and wearable computers like Google Glass emerging, society is gaining a digital sensing system. People’s location and behavior are being tracked as they go through their days, and the resulting information is being transmitted instantaneously to vast server farms. Once we write the algorithms needed to parse all that “big data,” many sociologists and statisticians believe, we’ll be rewarded with a much deeper understanding of what makes society tick. Read the rest of this entry »


What’s the point of all that data, anyway? It’s to make decisions.

23. January 2014

Date: 22-01-2014
Source: Technology Review
Subject: The Power to Decide

WHY IT MATTERS: The ability to automate decision making will determine winners and losers in many industries.

Back in 1956, an engineer and a mathematician, William Fair and Earl Isaac, pooled $800 to start a company. Their idea: a score to handicap whether a borrower would repay a loan.

It was all done with pen and paper. Income, gender, and occupation produced numbers that amounted to a prediction about a person’s behavior. By the 1980s the three-digit scores were calculated on computers and instead took account of a person’s actual credit history. Today, Fair Isaac Corp., or FICO, generates about 10 billion credit scores annually, calculating 50 times a year for many Americans.

This machinery hums in the background of our financial lives, so it’s easy to forget that the choice of whether to lend used to be made by a bank manager who knew a man by his handshake. Fair and Isaac understood that all this could change, and that their company didn’t merely sell numbers. “We sell a radically different way of making decisions that flies in the face of tradition,” Fair once said. Read the rest of this entry »


The Big Data Conundrum: How to Define It?

4. October 2013

Date: 04-10-2013
Source: Technology Review

Big Data is revolutionizing 21st-century business without anybody knowing what it actually means. Now computer scientists have come up with a definition they hope everyone can agree on.

One of the biggest new ideas in computing is “big data.” There is unanimous agreement that big data is revolutionizing commerce in the 21st century. When it comes to business, big data offers unprecedented insight, improved decision-making, and untapped sources of profit.

And yet ask a chief technology officer to define big data and he or she will will stare at the floor. Chances are, you will get as many definitions as the number of people you ask. And that’s a problem for anyone attempting to buy or sell or use big data services—what exactly is on offer?

Today, Jonathan Stuart Ward and Adam Barker at the University of St Andrews in Scotland take the issue in hand. These guys survey the various definitions offered by the world’s biggest and most influential high-tech organisations. They then attempt to distill from all this noise a definition that everyone can agree on.

Ward and Barker cast their net far and wide but the results are mixed.Formal definitions are hard to come by with many organisations preferring to give anecdotal examples.

In particular, the notion of “big” is tricky to pin down, not least because a data set that seems large today will almost certainly seem small in the not-too-distant future. Where one organizsation gives hard figures for what constitutes “big,” another gives a relative definition, implying that big data will always be more than conventional techniques can handle.

Some organizations point out that large data sets are not always complex and small data sets are always simple. Their point is that the complexity of a data set is an important factor in deciding whether it is “big.”

Here is a summary of the kind of descriptions Ward and Barker discovered from various influential organizations:

1. Gartner. In 2001, a Meta (now Gartner) report noted the increasing size of data, the increasing rate at which it is produced and the increasing range of formats and representations employed. This report predated the term “dig data” but proposed a three-fold definition encompassing the “three Vs”: Volume, Velocity and Variety.This idea has since become popular and sometimes includes a fourth V: veracity, to cover questions of trust and uncertainty.

2. Oracle. Big data is the derivation of value from traditional relational database-driven business decision making, augmented with new sources of unstructured data.

3. Intel. Big data opportunities emerge in organizations generating a median of 300 terabytes of data a week. The most common forms of data analyzed in this way are business transactions stored in relational databases, followed by documents, e-mail, sensor data, blogs, and social media.

4. Microsoft. “Big data is the term increasingly used to describe the process of applying serious computing power—the latest in machine learning and artificial intelligence—to seriously massive and often highly complex sets of information.”

5. The Method for an Integrated Knowledge Environment open-source project. The MIKE project argues that big data is not a function of the size of a data set but its complexity. Consequently, it is the high degree of permutations and interactions within a data set that defines big data.

6. The National Institute of Standards and Technology. NIST argues that big data is data which “exceed(s) the capacity or capability of current or conventional methods and systems.” In other words, the notion of “big” is relative to the current standard of computation.

A mixed bag if ever there was one.

In addition to the search for definitions, Ward and Barker attempted to better understand the way people use the phrase big data by searching Google Trends to see what words are most commonly associated with it. They say these are: data analytics, Hadoop, NoSQL, Google, IBM, and Oracle.

These guys bravely finish their survey with a definition of their own in which they attempt to bring together these disparate ideas. Here’s their defintion:

“Big data is a term describing the storage and analysis of large and or complex data sets using a series of techniques including, but not limited to: NoSQL, MapReduce and machine learning.”

A game attempt at a worthy goal—a definition that everyone can agree is certainly overdue.


Data Won the U.S. Election. Now Can It Save the World?

31. May 2013

Date: 31-05-2013
Source: Technology Review

Data scientist Rayid Ghani helped persuade voters to reëlect President Obama. Now he’s using big data to create a groundswell of social good.

WHY IT MATTERS: Big data hasn’t been widely applied to social needs like raising funds for charity.

Data science and personal information are converging to shape the Internet’s most powerful and surprising consumer products.

As chief scientist for President Obama’s reëlection effort, Rayid Ghani helped revolutionize the use of data in politics. During the final 18 months of the campaign, he joined a sprawling team of data and software experts who sifted, collated, and combined dozens of pieces of information on each registered U.S. voter to discover patterns that let them target fund-raising appeals and ads.

Now, with Obama again ensconced in the Oval Office, some veterans of the campaign’s data squad are applying lessons from the campaign to tackle social issues such as education and environmental stewardship. Edgeflip, a startup Ghani founded in January with two other campaign members, plans to turn the ad hoc data analysis tools developed for Obama for America into software that can make nonprofits more effective at raising money and recruiting volunteers. Read the rest of this entry »


In a Data Deluge, Companies Seek to Fill a New Role: “DATA SCIENTIST”

28. May 2013

Date: 28-05-2013

  Source: Technology Review 

A job invented in Silicon Valley is going mainstream as more industries try to gain an edge from big data. 

WHY IT MATTERS: Communications, computing, and biomedical advances are spurring an explosion of data. How companies use it could determine their own survival. 

Data science and personal information are converging to shape the Internet’s most powerful and surprising consumer products. 

The job description “data scientist” didn’t exist five years ago. No one advertised for an expert in data science, and you couldn’t go to school to specialize in the field. Today, companies are fighting to recruit these specialists, courses on how to become one are popping up at many universities, and the Harvard Business Review even proclaimed that data scientist is the “sexiest” job of the 21st century. 

Data scientists take huge amounts of data and attempt to pull useful information out. The job combines statistics and programming to identify sometimes subtle factors that can have a big impact on a company’s bottom line, from whether a person will click on a certain type of ad to whether a new chemical will be toxic in the human body.  Read the rest of this entry »