Accidental Look-Ahead Bias

Look-Ahead Bias is how temporal models of reality can incorporate hindsight into its decision making process, and thus seem to function exceptionally well when explaining the past but performs poorly when predicting the future.

A well known example is the character Captain Hindsight in the South Park series:

“There’s people trapped in that burning building, Captain Hindsight!” – Firefighter
“And the fire is so massive, we can’t get to them!” – Other Firefighter
“Hm. You see those windows on the right side? They should have built fire escapes on those windows for the higher floors, then people could have gotten down. And then on the roof; they should have built it with a more reinforced structure, so a helicopter could have landed on it.” – Captain Hindsight
“Yes, of course!” – Firefighter
“And then you see that building to the left?” – Captain Hindsight
“Yes!” – Firefighter
“They shouldn’t have built that there, because now you can’t park any firetrucks where you really need to. Well, looks like my job here is done. Goodbye everyone!” – Captain Hindsight
“Thank you, Captain Hindsight!” – Firefighter
*Cheers* – Everyone

 Many climate and financial models accidentally incorporate such hindsight. I recently (re)discovered two such cases: Standardization and Normalization.

Standardization of a time series as a whole means to convert the time series into standard scores – i.e. scaled and shifted versions of the values of the time series. But such scaling and shifting is done by looking at the distribution of values, average value. etc. Such information ought not to be factored into the past of the time series because it is only available in the future.

Normalization is even more worse, because it scales & shifts the values of the time series into between -1 and 1 or 0 and 1, thus explicitly factoring in information about the maximum, minimum of the time series into its past.

Given a time-series which has factored in information about the future, it is relatively easy to come up with a process, that will explain the past. But the process will be unable to predict the future.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s