Times of crisis, of disruption or constructive change, are not only predictable, but desirable. They mean growth. Taking a new step, uttering a new word, is what people fear most. -Fyodor Dostoevsky
I’ve been thinking about datamining a lot these days, what it means and how it is changing in the context of elections, business and real estate. But, first, some context. In the 1990’s I worked for a (New Canaan) firm that studied consumer behavior in the retail environment. What we learned was to divide the world not just into men, women and children, but to divide consumers into cohorts and group them by how they behave. Rollerblade organized their customers in 4 groups: Fad Followers, First Class, Specialty, and Value shoppers, each motivated differently and with different price sensitivities. Some are motivated by a sale, others buy the latest new trend or color, still others buy specific brands or the best of a category, and some of us look for specific specialty attributes such as sport-specific gear. That’s “Specialty” and it commands a premium. In the 1990’s we followed consumers around the store taking notes with a clipboard. Now, we write an algorithm, an “if-then” statement and combine it with artificial intelligence to get better all the time.
In the 2000’s this science of behavioral cohort analysis made major leaps forward as we began studying consumer shopping behavior and decision-making on the Internet. We had exponentially more data to study, faster computers reacting in real- time. Now, demographic, geographic, and psychographic analysis leads to dynamic pricing of everything from airline seats to movie tickets. Why not houses? Amazon would not exist without advanced and immediate behavioral cohort analysis. And now, Artificial Intelligence is accelerating our understanding of human decision- making.
What about real estate, particularly luxury real estate? Our industry has been slow to adopt technology, almost resistant to the idea, saying that the role of a great agent cannot be replaced with Zillow and the best algorithms. They’ll remind you that real estate is not like buying a car, that intimate local knowledge sets great realtors apart. And, they’ll point to current NAR statistics that show that 97% of all houses sell through a Realtor. These were the same arguments made by Major League Baseball scouts in the days before Moneyball and statistical analysis took over professional sports. For those who did not read Moneyball the point is simple: the intuition of the realtor is being replaced by a better and mathematical understanding of consumer behavior. The tools are getting better every day. They watch you in the Open House, know the details of your current mortgage, and know where you are on the journey. Soon, they’ll replace my gut feeling about what you should offer on that house with a defendable and mathematically more perfect recommendation.
Datamining will focus on two areas, maybe three: the inventory, the buyers, and the agents. Let’s break it down in this market, New Canaan, Connecticut.
First, the supply is finite. There are only 7,000 homes in New Canaan and you can look them all up in the Town Clerk’s database. It is public information when each of them sold, for how much, and how much has been spent on building permits. AI is getting better at analyzing the publicly available photos of the interiors of our houses, enough to make a more precise evaluation of value. If it was easy Zillow would still be in the home-buying business. The technology is not there yet but it is coming. Second, we study the buyers and their behavior. More elusive. It’s done the old-fashioned way today. I ask the buyer’s agent, “Where are they coming from? Why are they moving? How long have they been looking?” and I react, adjusting my “content’: the photos, descriptions, and how I talk about the house and the market. The systems that currently show me “heat maps” of changing demand geographically are getting more sophisticated and armed with a better understanding of buyers, will map it behaviorally. Note, owners of the most expensive homes tend to think that their buyers are coming from far away. It’s statistically not true. They ask if their house will be advertised in our New York, or London, or Dubai offices. The truth is that we know which zip codes move to New Canaan, and why, and what life events cause them to consider moving to New Canaan. Most come from not-so-far-away. Tracking the movements of ultra-high- net-worth individuals (UHNWI) and high net worth (HNWI) used to be for private bankers and the major auction houses tracking art collectors. I would argue that it is becoming essential for the major real estate firms (who are opening offices in every luxury market) to begin studying customer movements so they can get ahead of us with a recommendation or a referral at the most opportune time.
Third, there are the triggers. It’s not enough to know you are wealthy with a Brooklyn zip code. When your child turns 10 you think about upgrading schools. We buy and sell real estate when certain major life events occur. Proctor & Gamble is so successful because they knew you were having a baby before you did, and they dripped offers on you to make you a customer for Pampers. So too we know that young families with Brooklyn zip codes are looking to move to New Canaan for the schools. Which of the major real estate firms in Brooklyn know their names, and are making that transition easier for them? While the business still relies on the agent today, and many of those Brooklyn families are relying on the advice of a friend or family member to choose a realtor, with AI it won’t be long before the agents in Brooklyn and New Canaan are communicating earlier and more efficiently about their shared prospective customer and their agency (or Zillow) will be facilitating that.
Why is the real estate industry not further along in tracking consumer behavior? It’s a structural issue. Agents are independent contractors. They move from agency to agency, and they take their Rolodex and relationships with them. The better agencies are going to provide increasingly sophisticated toolsets and technologies to get stickier with the most capable agents. And the best agents in the future will be the ones who leverage these datamining tools to understand the houses and buyers in these important luxury markets.
In conclusion, I believe that we are not far from the moment where real estate agents in Brooklyn and Vero Beach form a deeper connection with agents in New Canaan, making it easier for me to sell here to a Brooklyn family and move to Florida for retirement. We are already seeing birds of a feather flocking together to points south. Fairfield County families tend to move to a predictable set of southern zip codes. AI of the future, knowing about my kids. their schools and interests, the status of my mortgage, and how much home equity I have in my house will know the towns I’ll be considering and have a ready- made solution for financing, staging, estate sales, moving companies, and insurance. My neighbor recently moved to Florida to be in a very specific neighborhood, near a specific school, close to the squash facility and coaches, and commutable to the office. If the business of real estate were not so fractured I might have made a referral.
What is the takeaway? In an election season where datamining took center stage, where we saw huge sums spent in understanding the behaviors of important zip codes, why not real estate sales? If 2024 was the year lawsuits shook up the real estate industry, forcing agents and agencies to re-think where we add value in the process, 2025 is the year where agencies start getting serious about more than marketing support. Besides reach, the critical way agencies and agents add value is in providing a deeper understanding of each local market and customer behaviors on a granular level within that market. 2025 is the year where Moneyball meets Real Estate.
Notes from the Monday Meeting: Having voted to fold their respective MLS systems, I am hopeful that the New Canaan Board of Realtors and the Darien Board of Realtors combine their historical data in a new local service rather than hand it over to SmartMLS, which is now a monopoly.
John Engel is a former Military Intelligence officer who shifted his focus from studying military behavior to consumer behavior in the early 90’s. He started two Internet companies, e-Media and Paper. com. Now, a Realtor with Douglas Elliman, he is both afraid of being replaced by the machine and embracing the machine. John used spell-check on this article, but did not use ChatGPT. Baby steps.
Check out John Engel’s Podcast, Boroughs & Burbs, the National Real Estate Conversation here.
Read this article on the New Canaan Sentinel website here.
