Wednesday, September 27, 2006

Confidence Intervals and the USDUD's Data

The folks over at Patrick.net have it right about the correct interpretation of the latest US Department of Urban Development statistics (PDF) on new home sales... the confidence intervals around their latest figures are so huge as to make the results statistically meaningless. And how much do you want to bet the numbers will be revised down next month as, lately, they almost always are?

What is a confidence interval? Well, you could read all the mathematical gibberish by going here or you could just think of it this way:

If someone says, for example:
"The average percent change in new housing sales for August for the entire Western region of the US was down a -17%"
What they are really saying is:
"It is, for all practical matters, impossible for us to measure every housing market in the entire Western region of the US and so we must measure a representative sample of markets. By 'representative' we mean that we believe this smaller sample of the overall market is a valid reflection of the entire, overall Western US housing market. Given the results from our sample, we claim that the average for the entire, overall Western US housing market is down a -17%".
If they then tell you that the 90% confidence interval for their measure is, say, 13% what they mean is:
"There is a 90% probability that the true average for the entire, overall Western US housing market (as opposed to just our sample which no one really gives a hoot about) falls somewhere within -17% plus-or-minus 13% or, in other words, somewhere between -30% and -4%; there is a 10% chance that the true average for the overall Western US housing market is somewhere outside this range."
The larger the confidence interval, the more uncertain you should be that the reported average accurately reflects the true average for the entire population (the Western US housing market in this case). E.g., if the confidence interval had been just plus-or-minus 1%, then the claim would be that there is a 90% chance that the true average for the entire Western US housing market falls somewhere between -18% and -16%.

A confidence interval is not restricted to 90%; 95% and 99% are more common and of course reflect greater degrees of certainty. If the USDUD had chosen to use the 95% confidence interval, then the interval range would have been even larger which is probably why they went with the 90% confidence interval which is the lowest confidence interval that still retains a shred of credibility.

Ok, I've done my good deed for the day.

Update: “In the West, prices have held steady there this summer, sales have fallen. ‘Something’s going to give in the West,’ Mr. Lereah said. ‘The correction has not occurred in the West yet. Sellers are stubborn there.’”

13 Comments:

Anonymous Anonymous said...

I read what you wrote....can someone explain what you said! LOL

Sep 27, 2006, 2:29:00 PM  
Blogger Marinite said...

Sigh

Sep 27, 2006, 2:31:00 PM  
Anonymous tom stone said...

god i love statisticians!they are the only people with a more vicious sense of humor than cpa's.

Sep 27, 2006, 2:49:00 PM  
Anonymous Anonymous said...

Good topic and I would add a summary "common sense" observation that avoids numbers.
When the sales volume suddenly drops dramatically the confidence interval becomes much lower, or in other words the standard deviation in sales prices goes way up and the average is less useful for predicting an individual sale price. If all the incentives were factored in you would see that the standard deviation has gone up. Along with this the cost of information in the marketplace goes up because the buyers know less about the actual sale prices. Example I bought my new house for $600k two months ago and my new next door neighbor just bought his for $600k with $100k in hidden incentives. If buyer #3 is looking at a resale on the next block he still thinks $600k is the going number.

Sep 27, 2006, 5:05:00 PM  
Anonymous Anonymous said...

A good explanation on confidence intervals, 90% is barely credible.

Statistics is difficult to get your mind around if you don't have any background in it.

Basically, I always think about confidence intervals like this: Would I feel better if this bridge I'm driving over were 90% safe or 99% safe.....you can substitute bridge for prescription drug or anything of the sort.

Sep 27, 2006, 5:34:00 PM  
Anonymous SF Mechanist said...

Why would statistics be used at all if every single house sold is a measurable data point?

Statistics is useful if you want to generalize information gathered for a small population, onto a larger population. But if the entire population is accounted for then a thirty percent drop in sales is a thirty percent drop in sales and so what is the point of confidence intervals?

Sep 27, 2006, 7:19:00 PM  
Anonymous SF Mechanist said...

I went back and read the article again. I suppose it implies that not all builders are revealing their sales numbers, and so representive samples have to be used, which calls for confidence intervals. The problem though is not with statistics but with builders keeping data regarding their new home sales to themselves.

Sep 27, 2006, 7:26:00 PM  
Blogger Rob Dawg said...

90% confidence is only half the story. you also need what they are confident about. in this case the 4.1% increase over last month's revised down another 6.2% is 90% confidence of the margin of error NOT the 4.1% increase. There's a 1 chance in 10 that the true number is either less than -6% or more than +15%. So if they come back next month or anytime in the next 4 months and revise the +4.1% to -6% their reporting is still statistically valid.

My stats prof used to grade on a curve but return tests with a numerical grade, the std dev and range. If you could tell him what your letter grade should be from the information given he'd raise you a half letter grade.

Sep 28, 2006, 9:32:00 AM  
Blogger fredtobik said...

Interpretation of statistics is equal to expressing your opinion on modern art.


REMINDER: This is a bubble blog.

Sep 28, 2006, 7:30:00 PM  
Blogger sf jack said...

fred -

What are you talking about?

Speaking of statistics on this blog... you used go around prattling about 40% gains since you bought in Marin.

Funny how you suddenly have an opion on "statistics".

Sep 28, 2006, 10:37:00 PM  
Anonymous Anonymous said...

Interpretation of statistics is equal to expressing your opinion on modern art.

And this is coming from the person who previously on this blog admitted to earning a C- in statistics.

Fred, what where you saying again about nitwits in a prvious thread?

Sep 29, 2006, 9:49:00 AM  
Blogger fredtobik said...

Whenever I mentioned my 40% increase it was a joke, I wonder why you would be so offended by it.

and, yes if you took the time to look at my previous posts I gave marinite a hard time about statistics months ago.

Sep 30, 2006, 9:17:00 AM  
Blogger sf jack said...

fred -

Oh... right. Right, OK, I get it.

You were joking.

You mean now, or then?

Oct 2, 2006, 7:46:00 PM  

Post a Comment

Links to this post:

Create a Link

<< Home

Terms of Use: The purpose of the Marin Real Estate Bubble weblog (located at URL http://marinrealestatebubble.blogspot.com/ and henceforth referred to as “MREB” or “this site”) is to present and discuss information relating to real estate and the real estate industry in general (locally, state-wide, nationally, and internationally) as it pertains to the thesis that recent real estate related activity is properly characterized as a “speculative mania” or a “bubble”. MREB is a non-profit, community site that depends on community participation and feedback. While MREB administrators do strive to confirm all information presented here and qualify all doubtful items, the information presented at MREB is neither definitive nor should it be construed as professional advice. All information published on MREB is provided “as is” without warranty of any kind and the administrators of this site shall not be liable for any direct or indirect damages arising out of use of this site. This site is moderated by MREB administrators and the MREB administrators reserve the right to edit, remove, or refuse postings that are off-topic, defamatory, libelous, offensive, or otherwise deemed inappropriate by MREB administrators. You should consult a finance professional before making any decisions based on information found on this site.

The contributors to this site may, from time to time, hold short (or long) positions in mentioned and related companies.