Share This

Sunday 28 February 2010

Scientists Develop Financial Turing Test

Can humans distinguish between sequences of real and randomly generated financial data? Scientist have developed a new test to find out.
Various economists argue that the efficiency of a market ought to be clearly evident in the returns it produces. They say that the more efficient it is, the more random its returns will be and a perfect market should be completely random.

That would appear to give the lie to the widespread belief that humans are unable to tell the difference between financial market returns and, say, a sequence of coin tosses. A number of experiments seem to back up this belief, showing for example that humans studying randomly generated data very quickly identify 'trends' in the data and develop hypotheses about them.

To find out whether humans can reliably distinguish between real and random market data, Jasmina Hasanhodzic at AlphaSimplex, an investment strategy company in Cambridge, Mass, Andrew Lo at MIT's Sloan School of Management, who founded AlphaSimplex and Emanuele Viola at NorthEastern University, have devised a simple experiment.

They have created a computer game in which a player is shown two time-series of data. One is real data from a financial market such as the US Dollar Index, or the spot price of Gold. The other is the same data randomly rearranged. The player has to guess which is the real series and is immediately told whether the guess is right or wrong.

Hasanhodzic and co call this a financial Turing test and anybody can sign up and take the test on their website.

In their experiment, 78 people took the test, with each contest lasting two weeks.

The results show that that humans are actually rather good at this game. After a few guesses most people quickly learn how to distinguish the real data from the random stuff. "The results provide overwhelming statistical evidence (p-values of at most 0.5%) that humans can quickly learn to distinguish actual price series from randomly generated ones," say Hasanhodzic and co.

It's not hard to see why. In feedback sessions, the players say that the real data was smoother than the randomised data or vice versa and that these patterns were easy to spot after a few goes.

That's an intriguing result but what to make of it? First let's look at what the study does not address. The study does not address any notion of predictability. A truly random market is entirely unpredictable, by definition.

There is good evidence that real markets are not random and that their behaviour can be described by fairly simple principles. That doesn't make them predictable, however (although we have looked at evidence that certain kinds of bubble markets might be predictable here, here and here).

Neither does the study address whether humans are good at making predictions; whether they are better at predicting the future performance of a market than, say, a coin toss.

So what does it show? It shows that humans are good at pattern recognition. Nothing more and nothing less.
Ref: arxiv.org/abs/1002.4592: Is It Real, or Is It Randomized?: A Financial Turing Test

Source: http://newscri.be/link/1029198

Comments


  • It may mean a lot more. . .
    ". . .It shows that humans are good at pattern recognition. Nothing more and nothing less. . ."

    Not so fast. We are talking about MARKETS and the behavior/participation of its various players. And above it all, we have Economic Theories of various types that inform/underpin/hope-to-influence national and global economic policy. And let's not forget that we almost had Depression 2.0

    I often hear economists talk about 'Rational Market Behaviour'. Thing is, that supposition evolved from a time when Traders looked at Ticker Tapes and made their own 'wetware' based analysis of stock trends which were generally based on the common sense desire of companies and their shareholders to have their holdings increase in value-- at which said trader would grab a phone and yell out an order up or down.

    Today-- not only do we have these same trades being executed by computers within seconds, we now have trades being executed algorithmically for mathematical reasons that have nothing to do with company/product trends or even inside info and more to do with graphical stock microjumps up or down. Just a pure 'For X delta, execute Y Action times Z number of stocks'. It doesn't matter if the company makes CPU's or makes Tampons.

    And I'm sure that quite a few of you with PURE Tech stocks may have had a prolonged WTF moment that lasted for DAYS as you watched your beloved stock dive for the oceanic abyss despite it having NOTHING WHAT-SO-EVER to do with Subprime Mortgages. There was NOTHING 'Rational' about it. Nor was it ALL Human wetware Irrationality. 

    Couple that with whole banks of Trading Servers taking the Market on Upward Swoops and deathdefying dives inside a single day with NO MAJOR ECONOMIC NEWS. . .coupled with increasing possibility of Gov't scrutiny on so-called 'Program Trades' this notion of a Financial Turing test could be a match being lite in a dusty room full of dynamite.
    Rate this comment: 12345

    Marrach
    02/26/2010
    Posts:3
    Avg Rating:
    3/5
  • I already built this.
    Three years ago I ran Stock Or Not - 100k games were played. My results were public. http://www.felixsalmon.com/000763.html
    Rate this comment: 12345

    jdigittl
    02/26/2010
    Posts:2
    Avg Rating:
    5/5
  • reinvent the wheel
    So did they actually use Josh Reich's code or simply stumble upon the same idea?
    Rate this comment: 12345

    jd long
    02/26/2010
    Posts:2
    • Re: reinvent the wheel
      I don't claim to be that smart, its a pretty obvious idea. And theirs has Java! mine was cobbled together from bash scripts + R.

      Here is my R code http://i2pi.com/rez/stockOrNot.R
      Rate this comment: 12345

      jdigittl
      02/26/2010
      Posts:2
      Avg Rating:
      5/5
      • Re: reinvent the wheel
        Java, cool! That's so enterprizey! ;)

        Yeah I know it's not exactly novel. But if they got the idea from your game it would have been cool to at least give you a nod.
        Rate this comment: 12345

        jd long
        02/26/2010
        Posts:2
  • EURUSD price line and your predictability
    The current crisis is a crisis linked to the possibility of building high technological products, only possible today. Result of the accessible massive processing and the large data storage available.
    This technological phenomenon as discussed above in the comments, causes meetings between algorithms and makes it sound patterns. They seem random, if we do not see what is going on. There are varying degrees of leverage and space-time. Minor leverages more time and space, opposite is the reverse. All that is smaller than a measure of a day are noisy routines. It is known that all uses standard margins and margin calls. Most of the algorithms act at the beginning or at the end of movements, therefore, most of all buy and sell at the same time, even using the same lagging indicators and/or similar algorithms. With some simple math, one can predict how big is the next step, but not the direction. When a movement starts, just stop in the next step, will never be in the middle ...
    It is a war between powers ...
    Sorry for my poor English.
    Rate this comment: 12345

    lleal2000
    02/26/2010
    Posts:1