# Jim Otar's method of retirement modeling



## james4beach (Nov 15, 2012)

Jim Otar came up in another thread. He is a University of Toronto educated mechanical engineer who sells books and software for retirement modelling. He's also published articles in Canadian MoneySaver magazine.

His method: he calculates the return and depletion of a retirement portfolio starting at _every year_ starting from 1900. With all the results available, he focuses on the worst case scenario. This sounds like a good method to me and is similar how I'd want to do this, myself. It will give more pessimistic answers than the typical sustainable withdrawal rate (SWR) studies, Trinity studies, etc. As an engineer who is constantly modelling data and who has thought a lot about investment modelling, I strongly agree with his criticism of the Monte Carlo simulations used in the well-known SWR studies:



> Secular trends can last as long as 20 years (up down or sideways). The randomness of the markets are piggybacked onto these secular trends. Assuming an average growth and adding randomness to it does not provide a good model for the market behavior over the long term and it makes the model to "forget" the black swan events.


Some time ago I also arrived at the same thing he describes here. Basically, the Monte Carlo style simulation does not do justice to the actual financial markets and results in an overly optimistic outcome. Black swans *and* long-lasting bear markets are very important but are dismissed in typical retirement planning, and SWR studies also _de-emphasize their effects_. I posted this blog article before that reminds how multiple decade bear or sideways markets are very common, historically.

My question to the forum:

I'm not rushing to buy books or software. I'm curious if others here have investigated Otar's work and mathematical approach. I'm a bit disappointed that Otar has not published any papers in peer-reviewed academic journals about his method and findings. This would give me more confidence that the method is sound.

Thoughts?


----------



## james4beach (Nov 15, 2012)

By the way, I work with experts in statistics and data modelling. One of my colleagues always reminds me: outcomes are only are good as the models. It's important to choose a model that is appropriate and true-to-form for what you're modelling.

Otar is correct in pointing out that financial markets have secular (multi-year or multi-decade) trends. *If he is correct that Monte Carlo (MC) simulations like Trinity study, Pfau, etc fail to model these secular trends*, that seems like a big deal to me. He has other points of criticism too (overly simplistic distributions; and sequence of outcomes is wrong since secular trends are not modelled) which seem plausible. He demonstrates that these all result in much more optimistic outcomes than is sensible.

Here is his article describing why MC simulations don't accurately model markets. Why would such an important analysis not get published in academic journals?


----------



## Just a Guy (Mar 27, 2012)

If people could accurately model the market, I see one of two things happening...a few people would be very rich (since it wouldn't be I their best interest to share) or...

The predictions would break the market as, by investing according to the predictions, the market would no longer react based on market conditions but rather become the investment model as the money flooding in would overwhelm the market.

As for a reason why the models don't work, heck most of the companies from the 1900's don't exist anymore. Models can't predict major advancements (personal computers, electricity, the internet, plastic, Graphene, etc.) none of these potential "game changers" existed at the beginning, not to mention larger socio issues such as world wars, the collapse of the USSR, reunification of Germany, etc.

Looking backwards, it's often easy to spot "obvious patterns", our mind is actually designed to spot patterns...the problem is, just because we see a pattern, doesn't mean it's actually there. As our minds are designed to spot patterns, we have a tendency to always see them. "To the man with a hammer, everything looks like a nail..." As it were.

My final observation, most academics tend to view the world very differently than those who actually work in the real world...not saying either view is the "correct" one, just that there is a big difference between theoretical and practical or applied. One thing that really pisses off the academics is the way that the real world abuses the tools provided by academia, often using it in ways it was never intended to be used...thus, things don't go as they were supposed to.


----------



## GreatLaker (Mar 23, 2014)

Hi James, good, thought provoking posts.

I have seen Jim Otar's book and website. I like his concept of retirement green, gray and red zones for assessing risk, and when to consider annuitizing.

I agree with the criticism of Monet Carlo simulation, it just applies randomness to returns, and black swan or long secular trends are not well represented by it. Consider the stagflation of the 1970s, where the Dow did not return consistently to its late 1960s levels until the early 1980s, and that sideways market was compounded by inflation destroying purchasing power and high interest rates crushing bond returns. Ouch. Look at a chart of the S&P Composite from 1950 to today. Massive bull starting around 1950, then flat from late 1960s to early 1980s, another massive bull market up to 2000, and relatively flat with lots of volatility since.

Unless I am misinterpreting something, the Trinity Study and folllow-on work by Wade Pfau and Michael Kitces are not Monte Carlo based. Those works are similar to Otar's in that they base a SWR on the withdrawal rate that would have been safe (>95% chance of portfolio survival) over 30 year retirement, based on actual historical returns with a low-cost balanced portfolio. As long as the future is not worse than the worst of the historical study period the SWR should be safe. Could it be worse? I dunno, but I will monitor my portfolio carefully to see if it trends that way and adjust spending if necessary.

The problem with a SWR based on a % of the original portfolio plus inflation is that under most conditions, the portfolio will continue to grow, and be massively larger at the retiree's end of life.

Have you looked at variable percentage withdrawal strategies? It can increase the withdrawal rate when returns are good, and quickly lower it when returns are bad, so it is impossible to totally deplete the portfolio. The author is Canadian. See these threads over at the Bogleheads wiki and forum:
https://www.bogleheads.org/wiki/Variable_percentage_withdrawal
https://www.bogleheads.org/forum/viewtopic.php?f=10&t=120430

At this point I plan on having a base of government pensions and fixed income large enough to cover all or most projected non-discretionary expenses, augmented by equities for discretionary expenses, and managed with variable percentage withdrawals to guide overall spending. Obviously we can't perfectly predict the future, but reasonably projecting it based on the past is better than giving up and saying we just don't know what is ahead.


----------



## OptsyEagle (Nov 29, 2009)

james4beach said:


> I'm a bit disappointed that Otar has not published any papers in peer-reviewed academic journals about his method and findings. This would give me more confidence that the method is sound.
> 
> Thoughts?


He gives his book away for free download. Help yourself.

http://www.flip4u.org/docs/Unveiling the Retirement Myth.pdf


----------



## Spudd (Oct 11, 2011)

You can also check out Firecalc and cFireSim which use this methodology (US market only, but still nice tools).


----------



## My Own Advisor (Sep 24, 2012)

Jim Otar's book is quite technical and certainly not written in layman's terms.

I find there is huge bias to SWR/rules. 

I think most investors would be best served not worrying about SWR and simply save as much as they reasonable can, starting with 10% of net income if they can and keep that up for a few decades, using primarily lower-cost funds for their investments - focusing on registered accounts first over taxable accounts. Once the income from their investments comes close to matching their expenses, once you factor in government benefits, they can likely call it a day from the workplace.

Maybe that's too simplistic but you can certainly beat your head against the wall trying to figure out what is "safe" in terms of a withdrawal rate.


----------



## MoneyGal (Apr 24, 2009)

james4beach said:


> By the way, I work with experts in statistics and data modelling. One of my colleagues always reminds me: outcomes are only are good as the models. It's important to choose a model that is appropriate and true-to-form for what you're modelling.
> 
> Otar is correct in pointing out that financial markets have secular (multi-year or multi-decade) trends. *If he is correct that Monte Carlo (MC) simulations like Trinity study, Pfau, etc fail to model these secular trends*, that seems like a big deal to me. He has other points of criticism too (overly simplistic distributions; and sequence of outcomes is wrong since secular trends are not modelled) which seem plausible. He demonstrates that these all result in much more optimistic outcomes than is sensible.
> 
> Here is his article describing why MC simulations don't accurately model markets. Why would such an important analysis not get published in academic journals?


Because the data on past returns and on secular trends is much MUCH too sparse to draw robust mathematical conclusions. The "aft-casting" methodology, which draws on only a very limited set of data (as strange as that may sound, but he is modelling returns from less than 100 years of data), is just not comparable to MC or other forms of simulation (PDEs) which are able to incorporate much, much, much more randomness. 

TL,DR: differential equations > "aft-casting"


----------



## heyjude (May 16, 2009)

MoneyGal, your input is always insightful, thank you! 

"All models are wrong, but some are useful". 
George Box

https://en.m.wikiquote.org/wiki/George_E._P._Box


----------



## steve41 (Apr 18, 2009)

Before exploring probablistic models, the deterministic math has to stand up. Almost all models fall down in the area of income tax. If you see an entry calling for tax rate, average tax rate or marginal tax rate, the model is in error. There is only the T1 math, with its tax brackets (indexed to the cpi), clawbacks, surtaxes, age credits..... Income tax impacts the model in a very complex way as various forms of capital and income come in and out of play over time. Concepts such as average tax rate and SWR should be put out of their misery IMHO.


----------



## james4beach (Nov 15, 2012)

Thanks for the great replies from everyone! I need to do more reading and thinking about this.

And thanks to MoneyGal for answering my question about why he could not publish this research in the academic field:



MoneyGal said:


> Because the data on past returns and on secular trends is much MUCH too sparse to draw robust mathematical conclusions. The "aft-casting" methodology, which draws on only a very limited set of data (as strange as that may sound, but he is modelling returns from less than 100 years of data), is just not comparable to MC or other forms of simulation (PDEs) which are able to incorporate much, much, much more randomness.


I see what you mean. He's back testing, but that data is very sparse and not useful for mathematical conclusions.


----------



## OnlyMyOpinion (Sep 1, 2013)

Being on retirement's doorstep, I enjoy reading about the various methods of modeling your retirement stream(s).
(MOA, I hope that doesn't make me a headbanger :unconscious.

So I enjoyed reading Jim's articles. As noted, his 'aftcasting' uses historical market and inflation performance to model the adequacy of your savings to fund your retirement. Some of his articles can also be found at: http://www.advisor.ca/columnists/jim-otar

Very different from monte carlo simulation as MoneyGal notes. There is a bit more discussion of that here as well: http://www.hullfinancialplanning.com/the-shortcomings-of-aftcasting

I agree with Steve, the impact of income tax when trying to model your various retirement income streams is not an easy task. 
Some of the articles related to his RRIFmetic software cover this: http://www.fimetrics.com/articles.shtml

How's that for a litany of links :cower:


----------



## Eclectic12 (Oct 20, 2010)

james4beach said:


> ... Why would such an important analysis not get published in academic journals?


As I understand it ... unless one submits one's work to an academic journal, it won't be considered.


Cheers


----------



## james4beach (Nov 15, 2012)

Eclectic12 said:


> As I understand it ... unless one submits one's work to an academic journal, it won't be considered.


Sure, I just meant, I would expect _someone_ to publish these important criticisms of the previous field of work on MC-like simulations and their inadequacy.

Maybe Jim doesn't want to bother publishing. Hopefully someone else has published this counter-argument to the traditional MC techniques. And if not, then perhaps some economics or finance student reading CMF can pick up the idea and submit the analysis to a few journals.


----------



## andrewf (Mar 1, 2010)

steve41 said:


> Before exploring probablistic models, the deterministic math has to stand up. Almost all models fall down in the area of income tax. If you see an entry calling for tax rate, average tax rate or marginal tax rate, the model is in error. There is only the T1 math, with its tax brackets (indexed to the cpi), clawbacks, surtaxes, age credits..... Income tax impacts the model in a very complex way as various forms of capital and income come in and out of play over time. Concepts such as average tax rate and SWR should be put out of their misery IMHO.


Sure, but you have to consider that it is virtually guaranteed that tax rates, brackets and clawbacks will be different by the time one actually retires in 30 years. Pretending they are invariant simplifies the model, but gives a false sense of certainty.


----------



## heyjude (May 16, 2009)

james4beach said:


> Sure, I just meant, I would expect _someone_ to publish these important criticisms of the previous field of work on MC-like simulations and their inadequacy.
> 
> Maybe Jim doesn't want to bother publishing. Hopefully someone else has published this counter-argument to the traditional MC techniques. And if not, then perhaps some economics or finance student reading CMF can pick up the idea and submit the analysis to a few journals.


As a retired academic, I know that getting your work published in peer reviewed journals requires original research that stands up to withering review by anonymous "peers". Journal editors reject most articles submitted to them, sending the remainder back for multiple nitpicking corrections, without guaranteeing acceptance of the second submission. Editors would be even less likely to accept an article written by someone like Jim Otar who does not have an university appointment. If you do eventually get your article published, you have to sign away the copyright. I think Jim Otar quite rightly decided to bypass that process and self publish online. As to why finance researchers have not studied and published on his work, I suspect they see more value to their own career advancement in other areas. For example, Moshe Milevsky does a lot of research on annuities, some of it financed by the insurance companies.


----------



## james4beach (Nov 15, 2012)

It's true that those without university positions are less likely to get published, but even we (at a private corporation) submit academic papers and a few have been published. So it is possible, but yes I imagine he just chose the route of self publishing.

I reached out to my Toronto engineering circles and found a family friend who knows Jim! Maybe I'll see if I can get an introduction or something.


----------



## GreatLaker (Mar 23, 2014)

MoneyGal said:


> Because the data on past returns and on secular trends is much MUCH too sparse to draw robust mathematical conclusions. The "aft-casting" methodology, which draws on only a very limited set of data (as strange as that may sound, but he is modelling returns from less than 100 years of data), is just not comparable to MC or other forms of simulation (PDEs) which are able to incorporate much, much, much more randomness.


Thanks MoneyGal.

I'm not that familiar with Monte Carlo simulations. Does anyone know how likely MC would be to predict situations like a crash where equity prices fell 90% (like 1929-32), the stagflation of the 1970s, the tech crash and bear market 2000-2002, or the financial crisis of 2008? Aftcasting includes those events, so it seems to me that is why Jim Otar prefers it to MC.


----------



## steve41 (Apr 18, 2009)

andrewf said:


> Sure, but you have to consider that it is virtually guaranteed that tax rates, brackets and clawbacks will be different by the time one actually retires in 30 years. Pretending they are invariant simplifies the model, but gives a false sense of certainty.


Even so, I have found that since Martin started indexing the tax brackets, the tax formulation has stayed pretty consistent year over year..... taking the bracket changes (indexing) into account that is.


----------



## MoneyGal (Apr 24, 2009)

GreatLaker said:


> Thanks MoneyGal.
> 
> I'm not that familiar with Monte Carlo simulations. Does anyone know how likely MC would be to predict situations like a crash where equity prices fell 90% (like 1929-32), the stagflation of the 1970s, the tech crash and bear market 2000-2002, or the financial crisis of 2008? Aftcasting includes those events, so it seems to me that is why Jim Otar prefers it to MC.


...MC (simulation) is not "predicting" situations, it is simulating many, many different scenarios (potentially hundreds of thousands). The 'basic' recommendation for MC simulation (with many, many caveats) is 20,000 simulations. And more importantly, in my view, is the question of whether MC can better model scenarios *that have not occurred* keeping in mind that "aft-casting" is limited to scenarios *that have occurred.*

And as a further note MC is one model for testing scenarios. An alternative to simulation is to solve analytically. 

People who develop simulation and analytic models for financial modelling use either MC or analytic solutions because they provide access to a much, much greater range of possible outcomes than "aft-casting" ever can. I think all of this has already been said in this thread, though.


----------



## Beaver101 (Nov 14, 2011)

heyjude said:


> As a retired academic, I know that getting your work published in peer reviewed journals requires original research that stands up to withering review by anonymous "peers". Journal editors reject most articles submitted to them, sending the remainder back for multiple nitpicking corrections, without guaranteeing acceptance of the second submission. Editors would be even less likely to accept an article written by someone like Jim Otar who does not have an university appointment. If you do eventually get your article published, you have to sign away the copyright. I think Jim Otar quite rightly decided to bypass that process and self publish online.* As to why finance researchers have not studied and published on his work, I suspect they see more value to their own career advancement in other areas. For example, Moshe Milevsky does a lot of research on annuities, some of it financed by the insurance companies*.


 ... good reminder.


----------



## steve41 (Apr 18, 2009)

MoneyGal said:


> ...MC (simulation) is not "predicting" situations, it is simulating many, many different scenarios (potentially hundreds of thousands). The 'basic' recommendation for MC simulation (with many, many caveats) is 20,000 simulations. And more importantly, in my view, is the question of whether MC can better model scenarios *that have not occurred* keeping in mind that "aft-casting" is limited to scenarios *that have occurred.*
> 
> And as a further note MC is one model for testing scenarios. An alternative to simulation is to solve analytically.
> 
> People who develop simulation and analytic models for financial modelling use either MC or analytic solutions because they provide access to a much, much greater range of possible outcomes than "aft-casting" ever can. I think all of this has already been said in this thread, though.


RRIFmetic does both, that's why it takes several minutes to do an MC run. Each iteration consumes 2-3 seconds of computer time. For this reason most of my users don't bother with MC.


----------



## james4beach (Nov 15, 2012)

I do agree, as Moneygal says, that MC is capable of generating a suitably large number of scenarios for the purpose of analysis.

But I still think Jim Otar is correct about the flaws he's identified in the MC-based modeling methods. I think they're serious flaws which lead to over-optimistic outcomes in portfolio modeling. I'm not saying that Otar's aft-casting is superior, but I'm just saying that I think he's right: traditional portfolio modeling is done incorrectly and is overoptimistic.


----------



## lonewolf (Jun 12, 2012)

The problem with 99.99% (estimate) of portfolios is they hold mainly the currency of the country the owner of the portfolio is a citizen of. This is a problem in my portfolio also.


----------



## P_I (Dec 2, 2011)

@lonewolf -- Vanguard Canada has a very helpful paper Global equities: Balancing home bias and diversification – A Canadian investor’s perspective that contains the statistic


> However, according to the most recent survey from the International Monetary Fund, Canadian investors only allocate about 40% of their total equity investments outside Canada


Continuing a bit with the topic drift, the paper builds upon another from Vanguard, The role of home bias in global asset allocation decisions


> This paper asks the question, “In a world in which a portfolio’s diversification benefits from broad allocations to global securities, how much home bias is reasonable?” We explore home bias in four developed markets: the United States, the United Kingdom, Australia, and Canada. To address our governing question, we outline a decision framework that considers both quantitative and qualitative criteria. Based on these criteria, we conclude that, in general, U.S. investors may have some justification for marginal home bias, but investors in Australia and Canada might consider increasing their allocations to foreign securities.


I'd recommend a read of both these papers as they might be helpful when considering Jim Otar's method of retirement modeling.


----------

