Zillow.com is a popular website many homeowners, and those who intend to be, visit to seek valuation on specific properties. But how accurate can an expert system be that is totally computer-generated without the benefit of actually seeing the property in question, much less the comparables, or even the locale?
It was my brother-in-law, Howard Wilcox, who first told me about Zillow's Zestimate utility when it was brand new back in the summer of 2006. As an appraiser and Realtor in a major metropolitan area for an amazing 50 plus years, he was impressed by how close the Zestimate was to an actual appraisal--this from a guy who has little use for computers in general. I took this with a grain of salt since our company had spent a good deal of effort developing just such a utility that uses statistical analysis to determine value.
To understand the problem it is important to understand the basics of property valuation. Since the savings and loan crisis of the 1980s and 1990s bank appraisals have been done according to a rigid method called the Uniform Standards of Professional Appraisal Practice (USPAP). In general, the specification calls for comparing three to five similar properties within a half-mile and sold within six months. These delineations can be expanded when necessary. Differences are then adjusted to arrive at a valuation. The problem was that given the tight restrictions on which properties the appraiser could use and the small sampling (3 to 5 comparables) the results tended to be like comparing apples to oranges. But like the umpire's call, it may not always be right but it's always the bottom line for lenders.
Our utility takes a different approach by comparing thousands of sales over decades, back to 1991. The concept was to wash out variables by sheer volume (large sampling). Zillow must use something like this because they tout a large sampling of properties (72 million nationally dating back to 2006) although its algorithms are not published.
For our algorithms, differences in the date of a sale are adjusted for area appreciation (and recently depreciation) location is based on school district, which is germane in our market, rather than a restricted area around the subject property which has an unfortunate tendency to cross boundaries between areas of vastly different values. An appraiser that is not wholly familiar with the area--as is common as ordered by online lenders--might compare properties in different school districts or fail to adjust for the proximity of an undesirable element such as a major highway or industrial area even though the comparables fall within the proscribed distance from the subject property. A good appraiser will factor these adjustments in although there is no universal standard other than how a particular geographic feature affects an average of other sale prices of similar properties. USPAP valuations tend to vary by as much as 10% to 15% appraiser to appraiser--especially if each uses different comparables. This is one of the weaknesses of small samplings.
In general, our utility worked well. Interestingly enough--considering how much effort went into its creation--we rarely use it. This is because any statistical analysis ignores the really important esthetic considerations such as warmth and charm, elegance, style, etc. In addition, the utility does not factor in variables like mortgage interest rates, the availability of funds--that which is hindering the present market--and overall consumer confidence. When the market is perceived to be appreciating buyers are willing to pay more but when it's perceived to be depreciating buyers tend to insist on a greater bargain to cover any further erosion of the market. Nor are competing properties presently on the market considered. These tend to have a greater effect on a given sale price than historical comparable. All of these factors have a strong influence on how much a given property will sell for. And yet the Zestimate still manages to come up with a good estimate, at least some of the time.
As of June 13, 2020, the Zestimate is on its fifth algorithm update. But how accurate is it really? After all, a great deal may depend on it. Homebuyers often base their offer on a Zestimate and seller's often base their selling price on the same Zestimate. Fortunately, lenders do not.
As a Realtor, I am often confronted by both buyers and sellers that have placed a great deal of faith in the Zestimate over what a municipal assessment, a Realtor's competitive market analysis (CMA), or even a USPAP appraisal suggests--especially if it's what they want to hear.
To find out just how accurate the Zestimate is--at least in our market--we simply compare the actual closed sale price taken directly from our Multiple Listing Service (MLS) to the Zillow Zestimate using the latest month's sales data for the counties of Albany, Schenectady, Rensselaer, and Saratoga that include the metropolitan areas of Albany, Schenectady, Troy, and Saratoga, New York. Of course, each market is different and may not produce the same results. What we found was that the Zestimate was surprisingly accurate at predicting the actual sales price with some notable exceptions. Zillow uses a similar method to judge its own accuracy. However, Zillow's accuracies are presented as a median error of actual sale prices to the Zestimate. As of their latest algorithm update, they claim a mere 8.5% error. ( http://www.zillow.com/blog/research/2011/06/14/upgrading-the-zestimate ) What is not noted is how far off the Zestimate is when it's off. Let's just say at this point that no appraiser, municipal assessor, or Realtor would be likely to miss the mark by as much as Zillow misses when it misses.
On to the numbers:
In terms of average error what we found was an astounding 2.8% error (overvalue) in all price ranges. In general, the lower price ranges saw a higher Zestimate than actual sale prices while in the higher price range the Zestimate tended to undervalue properties.
As you can see from the table above the undervalue in the higher price ranges washes out the overvalue in the lower price ranges. Or Zestimate overvalued 47% of the time; undervalued 53% about half of the time.
However, when the Zestimate missed it missed by a mile. The largest error was an undervalue of $221,329 or 61% undervalued. The largest overvalue was $137,850 or 44% overvalued. This didn't happen often, as noted.
Bottom line: if it has to be right use an experienced and qualified USPAP appraiser that knows the area. If the use the valuation is intended for is not critical Zillow is great for an excellent ballpark valuation keeping in mind the possibility of a significant error.