Can Computers Analyze Data?
Since “business analytics” has come into vogue, like all newly popular technologies, everyone is talking about it but few are defining what it is. In his inaugural article for the B-EYE-NETWORK, “Today’s ‘Analytic Applications’ — Misnamed and Mistargeted”, Merv Adrian argues that “analytic applications ought to be defined as those which use analysis to deliver business functionality.” He goes on to say that “the promise of the future is truly analytic business applications, software packages that execute automated business processes with and/or without human intervention, based on policies, rules and real-time analytic results.” Although Adrian never defines the terms “analysis” or “analytics” in his article, it is clear that he defines them differently than I do. As I define “analysis,” saying that a computer could do it “without human intervention” makes no sense.
I believe that data analysis is what we do to make sense of data. I could add a few more words around this basic definition to elaborate a bit, but essentially data analysis is the process of sense-making. It’s what we do with information to understand what it means. It’s the process that bridges the gap between information and knowledge. It’s what we do if we want to make informed decisions, based on evidence. Oh yeah, and when applied to business, it’s the heart and soul of business intelligence.
When software automates actions that are carried out in response to rules, it isn’t engaged in analytics or decision-making. Rather, it is simply enforcing a decision that has already been made, based on prior analysis-analysis that was done by a human. The automation of routine responses to specified conditions is a great use of technology, but let’s not confuse this with analysis. Doing so might give people the false impression that by automating such actions, they are addressing their organization’s vital need to understand its data.
I believe that Adrian’s vision of our analytical future involves something that computer applications have been doing for us all along. Software programs instruct computers to do particular things based on particular conditions. This is what all applications do and have always done. Business applications perform actions based on business rules. We shouldn’t confuse this with a process that involves the careful exploration and examination of information to make sense of it-that is, data analysis. The two are fundamentally different. The enforcement of rules is a procedural process that computers excel at performing. Thinking isn’t required.
Data analysis, on the other hand, requires thinking. The only computers that think are those that we read about in science fiction. Until this changes, if it ever does, data analysis will remain a human activity. Computers can support the process by giving us tools that support and augment our ability to think, but they can’t think for us.
Adrian apparently shares my opinion that the business intelligence industry has failed to deliver on its fundamental promise. But, just as he and I define “analysis” differently, we also understand the nature of this failure and its solution differently. If the business intelligence industry continues down its well-worn path of expecting technology to solve problems that are essentially human problems, without taking the time to understand human needs, abilities, and limitations, it will continue to fail in its primary mission.
Why do so few in the business intelligence industry understand this? Perhaps their analysis is faulty. Perhaps they’re coming at it from the wrong perspective.
Take care,
14 Comments on “Can Computers Analyze Data?”
This is a good idea, reserving the word “analysis” to refer to “what humans do”. Perhaps this is why Stephen Wolfram describes the function of Wolfram Alpha (http://blog.wolfram.com/2009/03/05/wolframalpha-is-coming/) as “computing” answers to questions. If a computer can figure out an answer to a query by a deterministic approach, then that is computing, not analysis.
That would mean that the whole market segments of “business analytics” and “web analytics” are mis-labeled. By Wolfram’s definition, these tools do computation, not analysis.
A very interesting post. I feel that analytics software has to make data exploration super easy and fast, to allow a data analyst to really dive in and find insights. Too often, these tools have cumbersome, non-intuitive UI. Reminds me of when Google Maps first came out – the simple mouse dragging and scroll wheeling for navigation blew the competition away.
I think it depends on how seriously you take Bayes Theorem and Jaynes’ work.
Basically, I can see how a BI process/application might take the state of nature that we know of (that which goes into the ETL),use that as prior information within the context of a Bayesian Belief Network to develop a State of Knowledge (quantitative probability information based on the BBN model) and then potentially use that to create a State of Wisdom (therefore, the next actions should be…) using the SoKnowledge BBN posteriors as priors for a second, “SoW” BBN.
Unless I’m mistaken, it’s this sort of information analysis/creation process that the folks at Singularity Institute for Artificial Intelligence get off on.
Great post. There are machine learning algorithms that can do great things, but the human will always enhance what the computers can do. Human-guided analysis is the way to go. In regard to the comment by Jack: In my mind (as a statistician), a really good comprehensive analysis cannot be achieved with only with some simple principles and software with a nice UI. These are great things for a certain level of analysis, but every data set I have modeled has required very specific handling and new adaptations of statistical methods to really get some depth of knowledge from the data, and these aren’t readily available in analytics software. A lot of great knowledge can be gathered by some nice simple software, but much much more can be acquired by delving deeper. These days, the better you understand the data, the bigger your advantage.
Good post as usual. It’s like saying hammers build houses. Technology is a tool, and that’s it.
Mind you, this mindset goes way back to at least some of Herbert Simon’s work on decision-making – there’s a quote of his from the 1950s I can’t put my hand on at the moment that talks about how in 10 years time businesses and factories would all be run by computers, including any management decision making, while we all go off and enjoy some kind of utopian life. He still managed to score a Nobel, though…
Cheers,
Rob.
True, but I may need to add a ‘it depends’ to you thinking.
You seem to have excluded the capability that computers have of data mining. That is, looking at data and applying statistical methods to derive rules.
This is quite a bit different from standard programming where humans have defined the rules, and rather allowing the computer to derive those rules for you.
So, while I agree that technology is a tool which is best driven by people, there is a growing hub of technology which can be utilised to help us people drive better.
G
Gavin,
Actually, I had data mining very much in mind when I wrote this. When computers perform data mining, they simply follow a set of instructions–a data mining algorithm. The computer is not thinking or doing analysis. These instructions tell the computer to look for particular statistical patterns, and when it finds them, to report them. Humans then take over to examine these findings, performing analysis to determine if they are meaningful and significant. The computer does not define rules during the process of data mining; it finds patterns based on explicit instructions, and then humans determine if these patterns actually qualify as rules. This is a great marriage of the strengths of computers with the strengths of human; one that reserves the task of thinking for the humans.
Steve
Data analysis, by your definition, can (theoretically) be performed by a computer. Let me explain with an example. Take a very advanced anti-fraud tool. The tool is provided with access to customer transactions, the results of fraud charge-backs, and the results of declines (i.e. where the customer calls in to manually authorize the transaction); the tool is then empowered to make sense of the data and make the decision to decline transactions automatically. Of course, if the tool is just following pre-programmed rules it is not making sense of the data; if, however, it is discovering coorelations, making rules and acting upon them, then I would submit that the computer is performing business analytics by your definition.
Douglas,
Are you aware of a fraud detection program that discovers patterns and makes up rules on its own, without human intervention. I’ve seen some sophisticated fraud detection programs that are used by banks, and they don’t make up the rules on their own. In fact, the best programs that I’ve seen find suspicious patterns and then display them as visualizations to human beings who go on to investigate them and perhaps write new rules to respond to the situation in the future.
I am working as daily job with a database of economic organizations, and query it daily to make reports to answer different questions about the state of our economy. The questions came from economic experts, and statisticians, who wish to use the answers for their analyses or decisions.
I “compile” their questions to queries and run these queries to give their results to the experts. I can not say, the database “analyses” the data, because if I give a wrong question, I shall given a wrong answer; but I cannot either I analyse data, because I cannot decide the economic decisions to be make from my results. The data are (hopefully) analized by the statiscian, or economic expert who will make her assumptions or decisions on the results I transfer her.
As for Stephen Few: Fraud detections are made by human decisions on “What kind of data situations used to show violation of the rules”, and the detection algorithms was (preferably) written on these decisions. So fraud dectection algorithms will visualize suspicious patterns only for those situations programmers wrote to be show up based on an analysis made by informatics, economics and statistics experts, who analysed earlier data to decide :” What kind of data may show possible situations of illegal activities?”. The analysis was already made before planning of detection program was started.
Hi. I am Neo Anderson and I am from the future. I have seen computers that could invent Matrix that simulated the ‘real’ world to keep the humans race in control… They had the ability to learn from thier mistakes and reinvent Matrix… They could also come up with the idea of getting energy from Humans… They can decide based on a given situation what they think is right to do or what they prefer to do .. etc, etc.. The computers that you see as of now are just “Compute”rs. You will have to wait until sometime in the future, when there will be an invention called “Artificial Intelligence” – a singular conciousness that would spawn an entire race of machines, that can think just as we do.
:P
Though it is not sure if at all human race can ever give birth to artificial intelligence as above.. (but also, one can not say for sure that, there is an end for invention… this is another story) but if rules (based on best practice.. and/or that which takes into consideraiton data available external to the application, such as market condition, climatic condition etc.. etc) are embedded in the application and the applicaiton can suggest or lead to a consequent action, it is AI from the end user point of view. It is not an AI in true sense, Yes. But all that matters is, what the customer feels. When I was small and played car race video game, I was surprised when I found that the computer could play on its own. When I placed my car in its way, it used to find a way around my car. I wondered if the computer could think. Then, it wasnt all that clear to me as to how the computer could do all this. The google search provides stock quote when it finds a companies name in the search (if the google search barely mentioned that it has recognised that it is a companies name, it is of no use..) and it provides context based advertisements ( if google had only mentioned, that it was able to identify some uniques keywords occuring particular number of times in the text, it is of no use). Though we software developers know that it is not Intelligence of the application (AI), the customers do not.
So, if the software automates actions that are carried out in response to rules rather than just showing the data on various charts, then it is definitely Intelligence of the application from the end user point of view. But definitely intelligence as described initially is something that would be most welcome.
Artifical Intelligence is called so since it is not real/actual Intelligence. It is only Intelligence as simulated by the machine to decieve the end user.
Quoting from wikipedia: “Artificial intelligence (AI) is the intelligence of machines…that perceives its environment and takes actions which maximize its chances of success…..The field was founded on the claim that a central property of human beings, intelligence—the sapience of Homo sapiens—can be so precisely described that it can be simulated by a machine.[5]”
Artifical Intelligence as the word suggest is not real/actual intelligence. It is intelligence simulated by the machine to decieve the end user.
Quoting from wikipedia: “Artificial intelligence (AI) is the intelligence of machines ….. that perceives its environment and takes actions which maximize its chances of success.[2]…….. The field was founded on the claim that a central property of human beings, intelligence—the sapience of Homo sapiens—can be so precisely described that it can be simulated by a machine.[5]”
I think most would agree that a human has to be involved at some point. The CPU didn’t create itself. The question is what if a program can be written to the extent that it would produce, at a minimum, the same result as a human? Isn’t that a better form of analysis? It would be a form of analysis that could be replicated, sped up, and consistent. Is an advanced chess program analysis? They have proven to be at least as effective as humans. Of course chess it just a game. What about a medical exam? Air traffic control? Logistics? Could these be programmed? To a large extent I’m sure. What is required is a subject matter expert willing to allow their thought process to be mapped. They have to teach the program what it needs to know. Or…is any human form of repetitive decision making just a “humantation” and not analysis at all? (Great site Steve!)