Covering a Mortgage in Retirement – How Not Paying Off Your Mortgage Can Lead to More Financial Flexibility and a Larger Investment Portfolio


(This post has been updated with 2021 data!)

 

Conventional wisdom claims that retirees shouldn’t carry debt. If you’re going to enjoy retirement then you want to minimize your non-discretionary expenses and maximize your entertainment!, er, I mean, discretionary spending.

However, military retirees (and any retirees with annuities) have the “unusual” situation of highly reliable retirement income. We know how much we’ll get each month, and we’re relatively confident that it’s going to happen every month. That predictability offers an opportunity to try to earn money from the difference of interest rates (the “spread”) between borrowing and investing. Many Wall Street professionals profited handsomely from arbitrage… until 2008.

Before reading the financial analysis, consider this: Does a mortgage suit your comfort level? Some retirees take great “sleep at night” comfort from having zero debt. They don’t want the risk of paying a mortgage simply because they can invest the money for more profits. They’d rather pass up potential profits and enjoy peace of mind. If you fall into this group, then you may not care to leverage your portfolio with a mortgage. If you’re unhappy with a financial strategy then you won’t stick with it, no matter how compelling the math or logic may be.

But mortgage rates are the lowest in four five decades, and this opportunity won’t might not be repeated in my lifetime.

 

How Investing Instead of Paying Off Our Mortgage Worked for Us

My spouse and I backed into this situation through repeated refinancing. When we bought our house in 2000, the interest rate on our 30-year fixed mortgage was 8.50%. Two years later I retired. In 2010 2017 we finally closed on a 30-year fixed-rate mortgage of 3.625% 3.50% that will be paid off when I’m 80 87 years old. Each mortgage refinance has paid for itself (lower payments minus the closing costs) within a few months to a few years. Over the decade our mortgage payments have dropped by over 40%.

We could have paid off the mortgage before we retired, but our retirement portfolio would have been much smaller. We were also concerned that once we retired (without our employment income) we’d never be able to qualify for a mortgage again. (We were right… especially after 2008.) We had no idea what we’d be doing in retirement, but a mortgage seemed to offer more financial flexibility for the first few years. Financial flexibility is one of the reasons many people choose to carry a mortgage.

We ran many projections on our idea using FIRECalc, cFIREsim, and FinancialEngines. We were concerned about making higher monthly payments, but the calculators indicated that a larger retirement portfolio was more survivable– because our safe withdrawal rate was about the same. However, this didn’t address the emotional perspective of the decision.

In late 2004 I decided to put numbers on the debate, and I started tracking the performance of investing a mortgage in a small-cap value fund. (The mortgage payments, of course, would be covered by pension payments.) I chose the iShares S&P SmallCap 600 Value Index (ticker symbol IJS). Taxes would be paid after each quarterly dividend distribution and the rest would be re-invested. I didn’t count the tax savings of the deduction for mortgage interest. The goal was to earn a higher return than the mortgage interest rate: 30-year fixed rate of 5.5%. Today we have an even lower rate, but back then 5.5% seemed like a reasonable challenge.

Yes, I’ve been updating that spreadsheet for over seven 17 years. Ironically it’s taught me as much about emotions as about math. Every time I entered the data and calculated the APY, I felt as if I was competing in a marathon and struggling to stay in front of the pack.  Every time I edited a formula, I wondered “Am I doing this right?”

Here’s the fund’s performance summary compared to our initial 5.5% benchmark:

  • Date (mostly October) and Compound Annual Growth Rate
  • Oct 2004 (start)
  • 2005 17.1%
  • 2006 13.3%
  • 2007 13.5%
  • 2008 5.3%
  • 2009 2.0% (-7.36% in March 2009!)
  • 2010 3.3%
  • Apr 2011 6.4%
  • Oct 2011 2.3%
  • 2012 5.90%
  • 2013 8.39%
  • 2014 8.18%
  • 2015 7.44%
  • 2016 8.24%
  • 2017 9.04%
  • 2018 9.53%
  • 2019 8.33%
  • 2020 6.22% (even with the pandemic recession)
  • 2021 9.41%

A few more observations:

Between late 2008 and mid 2009 the portfolio’s CAGR briefly dipped below zero. Mortgage arbitrage looked like a horrible mistake.  However, the dividends reinvested during that time were an incredible bargain.

IJS’s dividend rate is slightly over 1%. Over the last seven years, reinvested shares have risen to nearly 10% of the total. By the end of 30 years they may exceed 40% of the total. A small rise in the share price will be magnified 40% by reinvested dividends.

Volatility has dramatically affected APY.  March 2009 and April 2011 are two examples of a short-term changes in the share price causing the ETF’s CAGR to (briefly) fluctuate around the mortgage’s 5.5% interest rate.

Will the portfolio continue to beat its benchmark? History says “Yes”. We’re barely a quarter of the way over halfway through the experiment, and it’s been a tough history. Dividends have been invested at attractive values over the last few years. I think 30 years is enough time for long-term performance to win out.

In late 2017, we restarted the experiment with a new 30-year mortgage at 3.50%. This time we’re invested in a Vanguard total stock market index (VTI) that should be less volatile than the small-cap value ETF. We also appear to have caught a tailwind from a large-cap bull market.

  • Date and CAGR
  • Oct 2017 (start)
  • 2018 16.7%
  • 2019 9.98%
  • 2020 10.42%
  • 2021 17.01%

 

How Should You Design Your Retirement Portfolio for Mortgage Arbitrage?

First, minimize your investment costs. Don’t try to profit from the interest-rate spread if you’re going to be buying/selling a home every decade. (We’re trying to keep this house until I die.) Minimize expense ratios by using passively-managed index funds.

Second, minimize your mortgage costs.  Choose the lowest fixed interest rate on the longest mortgage term.  I’ve only done the analysis for fixed-rate 30-year mortgages. Your investment is compounding for a historically significant length of time and you’re paying back the lender with dollars that are worth less every year due to inflation.  15-20 years may not be long enough for an investment to recover and compound after a recession, and a variable-rate mortgage forces you to accept the inflation risk.

Third, invest in broad asset classes earning a long-term rate higher than the mortgage interest rate. Your asset choices are probably equities and a long-term CD ladder. Even long-term CDs will occasionally fall below the mortgage interest rate. Over 30 years, an index fund of highly-rated bonds will probably not pay more than a 30-year mortgage rate. A single 30-year bond might have a higher interest rate, but unless it’s a Treasury then that also adds in a significant default risk.

Fourth, allocate a meaningful percentage of your portfolio to sectors with long-term returns historically likely to exceed a mortgage rate. This includes equity sectors like small-cap value and international value. It may also include large-cap value index funds, high-yield bonds, high-growth stocks, private equity, and even Berkshire Hathaway. However, the latter three will require a significant effort to screen, select, and monitor.

Finally, diversify. Small-cap value index funds will easily beat a 30-year mortgage rate, but with breathtaking volatility. They might diversify well with blue-chip equity dividend funds (like the iShares Dow Jones Select Dividend fund, DVY), or a rebalanced portfolio of 20-30 dividend-aristocrat stocks, or maybe a REIT.

 

Should You Invest or Pay Off Your Mortgage?

What if you’re not retired yet– should you try mortgage arbitrage? That depends.

First, if you’re on active duty then you probably shouldn’t try to own a home. The costs of buying (and later selling) a house are usually 3-6% of the price for each transaction, and even if you’re moving “only” every 3-5 years that can wipe out any other gains.

Second, if you’re earning a reliable income then you may be willing to invest aggressively. Then you’re likely to earn more on your investments than you’ll pay on your mortgage. However, even in a civilian career you still may not know whether you’re going to stay in one place long enough to beat the costs of buying/selling a home.  If you’re laid off  without other sources of reliable income then you definitely don’t want to be exposed to the risks of mortgage arbitrage.

Third, your aggressively-invested portfolio will be quite volatile. You’re able to handle that if you’re still earning a paycheck and saving money every month, but the rapid price swings may cause some investors to have trouble sleeping at night. Again, if you’re not happy about what you’re doing then you’re unlikely to stick with it.

Whenever I talk about mortgage arbitrage I get two more questions:

  • Do you really need to take this much risk?
  • Do you really need to work this hard?!?

Every investor has to choose their own risk tolerance. My spouse and I have “enough” to live our low-key lifestyle, so we don’t mind taking on more risk with the money that we’re not using. If you’re not hard-wired to enjoy this type of research and tracking, then you shouldn’t feel obligated to attempt mortgage arbitrage.

A military pension is an extraordinary opportunity to arbitrage the century’s lowest interest rates with reliable income. If you’re already living a lifestyle that supports mortgage arbitrage, and if you think it’s worth risking money that you don’t expect to need, then over a 30-year mortgage you should make a profit! You have the time to let the investment compound, and at any point in the process you can cash out to pay off the remaining mortgage. In our case, the potential rewards are worth the risks and the effort.

 

Related articles:
ESIMoney:  If You’ve Won the Game, Stop Playing
White Coat Investor:  The Wrong Way to Think About Debt

 

Does this post help? Sign up for more free military retirement tips via e-mail, Facebook, or Twitter!

About Doug Nordman

Author of "The Military Guide to Financial Independence and Retirement" and co-author of "Raising Your Money-Savvy Family For Next Generation Financial Independence."
This entry was posted in Mortgage & Real Estate. Bookmark the permalink.

4 Responses to Covering a Mortgage in Retirement – How Not Paying Off Your Mortgage Can Lead to More Financial Flexibility and a Larger Investment Portfolio

  1. Doug Nordman says:

    Thanks, good question! The 1990s military drawdown was more about the end of the Cold War, not the politics. Remember the “peace dividend”?

    Your question about “retired benefits” leads right into the post scheduled for 5 AM HST Wed 14 Dec. And while there will be a slash & burn, I don’t think it’ll impact today’s active-duty personnel as badly as it did in the 1970s. I think the Reserves & National Guard will feel pain in their training, equipment, & travel budgets but not in the expenses for current personnel.

    I’ll put some thoughts together for a “drawdown predictions” post.

  2. m.s. russo says:

    What do you think is going to happen with the “retired benefit” debate within the greater context of the DoD portion of the Federal Budget? Will it be another Clinton-Era slash and burn? What does your financial sense say?

  3. Mel M. says:

    Nords,
    When we first bought out house here in Hawaii back in early 2010, our interest rate was 5.125% for a 30-year fixed mortgage. We refinanced last year to a 20-year fixed at 3.5%…then when the rates kept going down, we pulled the trigger and got a 15-year fixed at 2.5% which we closed on a couple of weeks ago. While my wife and I are both on active duty, we pay an additional $1000 per month on the principal but will likely stop that when I retire in a couple of months. Like you, we could have paid for the house but our retirement accounts would have taken a big hit.

    At retirement, with both my wife and I receiving a decent amount from our combined military pensions, we anticipate (at least based on my spreadsheets) that our monthly expenses would be covered (to include SBP payments and taxes factored in) by our pensions with some surplus. I do not anticipate we would need to tap our nest egg, unless we wanted to splurge on some extended vacation of some sort.

    This mortgage arbitrage is new to me and I don’t think I will devote much effort in truly understanding and applying it. I plugged in figures on FIRECalc and looking forward to 15 or less years down the road when we would be free of a mortgage payment…all before I turn 60 years old!

    • Doug Nordman says:

      Thanks, Mel!

      The comfort factor is a big part of the “pay off the mortgage or invest” decision. I still struggle to figure out how best to use the money that we don’t “need”, whether that’s for charity or long-term care insurance or our daughter’s inheritance or just compounding in the stock market.

      I think you’re encountering what a lot of dual-military retirees experience: more pension income than expenses. Good problem to have now, but we certainly earned it over the years.

Please leave a comment here!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s