I believe that in order to actually quit your job and retire early, you will need to make some radical, suboptimal, and tax-inefficient decisions regarding your portfolio.
Please bear with me, as I am still working through this theory (and likely will continue to work through it for the next few decades). I’m only 60% sure of this point. But that’s far enough along for me to publish it and get your feedback.
Here’s my argument: In the course of my career, I’ve talked to dozens (Hundreds? Nearing a thousand?) of individuals who are at, near, or well past the point of financial freedom.
All these people—every one of them—have made choices that are different from what a typical financial planner would advise. All of them have some gaping tax inefficiency in their portfolios. And all of them have some “ace in the hole” on top of the traditional 4% rule portfolio that is the holy grail in the finance community.
Finally, they all built portfolios that were way overkill for financial freedom. And nearly all of them made decisions that would make the typical financial planner cringe.
The ‘Typical’ Portfolio of Legitimately Financially Free Individuals
The exception that proves the rule I’m about to call out is Tim and Amy Rutherford. They are the closest, near-perfect example of a prototypical FIRE (financial independence, retire early) investment portfolio of a couple that actually earns no side income, is “retired,” and is living the travel/FIRE lifestyle that many seem to be chasing.
The Rutherfords truly retired in their 40s on a stock/bond portfolio, a cash cushion, and little else in the way of financial assets. However, even they retired on a substantially larger portfolio than what would be called for by the 4% rule.
If this is the closest to “perfect” in the context of a traditional FIRE portfolio, everyone else who is financially free either continues to work, or, if they truly earn no active income, is way wealthier and generates way more income than their lifestyle demands.
I encounter almost no one, with the notable exception of Amy and Tim, who harvest their equity, in any form, to sustain early retirement.
Despite the sound math and ample history supporting the 4% withdrawal rate on a portfolio, I can count on one hand the number of people I have personally met who are EARLY retirees who are actually selling off chunks of equity to sustain financial freedom.
Nearly everyone else has an ace in the hole. Some of the possibilities:
- A pension plan for a military member
- A paid-off house
- A couple of rental properties that generate enough cash flow to cover living expenses
- Equity in a start-up or other business.
- A side hustle/blog/book or other intellectual property
- A huge savings account and cash cushion
Or it could be a combination of these things.
The ‘Optimal’ Portfolio Designed by Financial Professionals
For every FIRE individual I meet, there is an order of magnitude of potential peers who are equally rich but whose portfolios do not have one of those aces in the hole. Their portfolios do not actually produce spendable liquidity that the individual can wrap their head around or actually use to fund their lifestyle without feeling extremely uncomfortable.
This article is written for the person who is wealthy—in the $1 million-plus range, or who believes they will get there in the next few years—but can’t see a path to actually generate passive income.
I believe there are a lot of you out there. Indeed, 30% of the people reading this are likely to be accredited investors, mostly through net worth. Nearly 100% of the people reading this believe they will become an accredited investor/millionaire within the next 10 years. Why else are you on BiggerPockets?
If the problem I’m about to unpack doesn’t impact you today, it might in a few years. Read on.
A Case Study
Recently, I had an individual reach out and ask me how they could generate $60,000 in passive income. This person is a wealthy executive in California with a great career, a mid-six-figure income, a real estate portfolio, a million-dollar 401(k) and stock portfolio, and a large life insurance policy. This person is about 45 years old.
Now, $60,000 in passive income is a lot of money. It’s almost the median household income in this country on its own. But generating $60,000 in passive income should not be a challenge for someone with $3.5 million in net worth. Literally, at today’s rates, one could stick $1.2 million in a nine-month CD at Ally Bank and generate $60,000 in simple interest. Boom. Done.
Do this at four to five banks, and the entire balance is FDIC-insured. Game over. Sleep well.
However, the fact that the answer to this individual’s question seems so remarkably easy is, I believe, a window into a larger problem that many investors face—a problem that prevents people who work hard for decades to grow their wealth responsibly from actually enjoying that wealth and the financial freedom it is intended to bring about.
So what gives? Why is it so hard for people who are, on paper, in the top 1% to 2% of American household wealth to actually free themselves from dependence on wage income?
As mentioned, I believe that the answer to this puzzle lies in the pursuit of optimization.
The Optimized Portfolio Problem
This $3.5 million portfolio of the individual who reached out to me is roughly divided as follows:
- $1 million in stocks/public equities—mostly in a 401(k) and other tax-advantaged retirement accounts
- $1 million in real estate equity, leveraged at a blended rate of 4.25% on a $3.5 million portfolio.
- A $750,000 life insurance policy
- $250,000 in cash
- $500,000 in home equity, with $500,000 remaining on the mortgage balance at 4.5%. Mortgage PI (not including taxes and insurance) is $2,500 per month.
This is a highly tax-efficient portfolio that was designed, as you’ve probably guessed by now, by a financial planner (who incidentally makes big money managing the $1 million in equities and likely bought their boat selling this life insurance policy five to 10 years ago).
This person is highly likely to grow their net worth by high single digits/low double digits each year in the context of a professional, diversified portfolio. The real estate has done extremely well, appreciating in value and allowing the investor to repeatedly cash-out refinance and purchase additional properties without putting in a ton of cash.
However, their portfolio generates essentially no cash flow—certainly not enough to cover their lifestyle and likely not enough to even cover the costs associated with just their life insurance policy.
The real estate portfolio is highly leveraged—there’s about $4.5 million in real estate (including personal residence) against that $1.5 million in equity. The debt service wipes out essentially all the cash flow, after CapEx, on the investment portfolio.
Sure, there’s a theoretical trickle of maybe $20,000 to $40,000 per year, but that is unpredictable and immaterial compared to the $250,000 in household income.
The life insurance policy premiums suck $60,000 out of their household cash flow each year. The cash in savings generates minimal yield. The home mortgage requires $30,000 in principle and interest alone annually. The stock portfolio produces 2% dividends, and even those are held mostly inside tax-advantaged accounts and reinvested.
Any financial advisor worth their salt would build a portfolio much like this for a client (maybe excluding the whole-life policy if they were a fee-only advisor). There’s honestly a lot to like here. This person is undeniably wealthy. They have a huge liquidity position, no consumer debt, and a big pile of assets. This person is also mitigating their tax bill to the maximum reasonable extent for someone in their position.
In a lot of ways, this is what a “good” portfolio looks like. But this person feels trapped—so trapped that this multimillionaire executive is honestly questioning how to generate $60,000 in passive cash flow. While they don’t live paycheck to paycheck, this person would likely feel like they were running out of money after just 18 months of leaving their job.
The Big, Suboptimal Moves This Person Must Make to Achieve Financial Freedom
To remedy this situation and actually create a world where this person is “free,” we have to make some bold moves. These moves are highly tax-inefficient in the near term and have a high probability of reducing this person’s long-term net worth number. These moves will also result in a big tax bill in the near term and make their (newly hired, fee-only) tax professional stamp their fist on the table with frustration (hopefully, they’ve fired the financial advisor who sold them the life insurance policy yesterday).
Here’s what I’d do.
First, I’d kill the life insurance policy and harvest the equity. Reposition this $750,000 into two to three hard money notes generating 10% to 12% interest. This swing alone removes a $60,000 (!?) annual premium (cash outflow) with an $80,000 to $85,000 cash inflow in the form of simple interest. There will be a $140,000 swing in pre-tax cash flow with this move alone.
However, note that this interest is taxed at a high marginal rate. At a $250,000 base salary, the next dollar of income (simple interest) is likely to be taxed at 35% or more—federally, plus state taxes. This move isn’t likely to be CPA-approved.
Next, I’d sell all but the two highest-cap rate rental properties. This would transform the portfolio from a $3.5 million total asset value to a $1.2 million (using some of the cash and/or equities to pay off small remaining mortgage balances), to a $1 million to $1.2 million cash flow engine generating a cool $50,000 to $60,000 in dependable, (actual) spendable liquidity per year.
Note that in selling, this person will have to pay transaction costs and capital gains taxes. There will be a tax bill coming due. And I’m not recommending a 1031 exchange, which requires the investor to take on new property with the same amount of debt—debt that will be higher interest than their current debt. I’m saying straight up: Pay the tax man, and buy your freedom.
Finally, I’d pay off the mortgage by using some of the stock portfolio (after-tax) or savings over the next year or two. This knocks off $2,500 per month or $30,000 per year.
I get it: Low-interest mortgage and an opportunity to arbitrage the rate. However, this person doesn’t have a net worth problem. They don’t need optimal. They need freedom. This last piece is it for many folks. With a paid-off home and cars and no consumer debt, the cash needed for one’s lifestyle drops drastically.
With these three steps alone, we can generate a net annual increase in cash flow from this portfolio of $30,000 in the form of eliminated mortgage payments, $60,000 from the rental properties, and another $80,000 from simple interest. In addition, we’ve removed the $60,000 annual expense of the whole life policy. That’s a swing of $230,000 of annual liquidity coming into this person’s life. That’s almost this person’s entire pre-tax salary.
All this means a paid-off house and $140,000 in truly spendable passive cash flow. And there’s still $1 million invested in highly tax-advantaged 401(k) investments. If they want more cash flow, they can sell the house, redeploy the equity, and bump up that cash flow number. Imagine what kind of life you can live in that context.
Of course, the tax professionals, financial planners, and optimizers of this community are reading this in disbelief. If this person follows this plan, they will incur capital gains on the real estate sales. They’ve already paid the price of the really bad returns of the whole-life policy. The simple interest they generate on the hard money loans is terribly tax-inefficient, especially for the next few years if they continue to work at their job.
On paper, I’ve probably reduced this person’s wealth by a few hundred thousand dollars with this move on day one. And I’ve potentially reduced their long-term net worth by several million dollars.
But, despite that, I will put my argument out there: I believe a set of moves like this is the price of financial freedom.
I truly believe that most of you reading this would, after my changes, live a life of optionality and freedom—a better life—than if you stuck with the portfolio prior to the changes.
Conclusion
As I stated at the beginning, I’m only 60% sure of this theory, let alone the practical example I used to illustrate this point. Maybe there’s a way to get a better blend of freedom now and long-term wealth later. Maybe there are ways for the “typical” millionaire household to break free earlier. Tell me in the comments. I’d like to hear it.
Until presented with an argument that I understand and that I believe a typical investor can rationalize, my analysis and experience so far tell me that this trade-off is real and does exist. It tells me I want to optimize for a portfolio that generates freedom, not the largest possible long-term wealth number, in the most tax-advantaged way.
It tells me that more investors should consider the cardinal capitalistic sin of knowingly paying more taxes on ordinary income and simple interest than a financial planner would advise. And they should do this for a year or two while still earning a high income to get used to the new source of income.
It tells me that they should consider building toward a portfolio that leads them away from an expensive home mortgage and heavy allocation to 401(k) or retirement accounts and reduce leverage on at least a portion of our core real estate portfolios.
It tells me that investors need to, at some point, determine what “enough” means for them in terms of net worth and stop, many years before reaching that number on paper, optimizing for long-term wealth creation. It tells me that if investors agree with my thesis here, they will need to begin making subtle but important changes to the composition of their portfolio that a tax professional might hate but are, in fact, exactly what is needed to get to a portfolio that reliably, predictably, and sustainably pays all of their lifestyle bills—and then some.
It also tells me that by submitting some of their portfolios to the clutches of Uncle Sam and the generation of cash flow at the expense of paying more in taxes today and forgoing some potential appreciation down the road, they might just actually let themselves free.
And, of course, it tells me that they should totally avoid that whole life insurance policy.
Now, what do you think? Am I crazy? Or am I on to something?
Ready to succeed in real estate investing? Create a free BiggerPockets account to learn about investment strategies; ask questions and get answers from our community of +2 million members; connect with investor-friendly agents; and so much more.
Note By BiggerPockets: These are opinions written by the author and do not necessarily represent the opinions of BiggerPockets.
