Retirement Calculator Evaluation
My primary goal is to provide a useful service to individuals in search of DIY tools for financial and retirement planning and, in effect, to separate the sheep from the goats in the multitude of tools offered over the Internet. A secondary and more personal goal is to discover the good and the not-so-good characteristics of the offerings in the retirement calculator market and apply that new knowledge to the continual improvement of the Pralana Retirement Calculator. Stuart C. Matthews, Pralana Consulting LLC
To meet these goals, I’ve identified the following objectives:
I am in the business of designing and selling a family of retirement calculators, so I am somewhat biased. I have tried to maintain my objectivity in doing these evaluations, but it is inevitable that my biases have crept in at some level. The place where this has probably surfaced in the greatest degree is in the selection of attributes by which the comparisons are being done.
I identified three retirement planning test cases, which I think are representative, and ran (or tried to run) all of them against the candidate calculators. Two of the three (with only minor modifications) came from Darrow Kirkpatrick’s (CanIRetireYet.com) articles on the best retirement calculators and are the cases he used. Here are the cases:
I started with the Pralana Retirement Calculator to make sure I had well-defined scenarios and to establish a truth model for the comparisons against all three test cases. Then, I proceeded to load these into a couple of the other high-end tools that I was already generally familiar with and worked with that small set of tools until I was confident that I was using them correctly and getting representative results. At that point, I made a preliminary list of calculator attributes that I thought potential users of retirement calculators would consider important.
Then, I went through the complete set of calculators in a methodical manner. For each calculator, I brought the tool up and followed the guidance offered to load it with first one test case, then another. I made detailed notes as I went along, including my general and specific impressions, and made my assessment of the tool relative to each of the attributes I’d previously established as important to users. As my experience with the body of tools grew, I modified the list of attributes somewhat and then revisited and updated my assessments of the tools previously evaluated.
Ultimately, I tried to do a quantitative evaluation of each of the tools, which always included an assessment of the extent to which I felt I (and other users) could verify the results being presented.
If you’ve reviewed my methodology, analysis and commentary and think you’ve found a flaw, I invite you to tell me about it. I’ll respond to every well-considered and coherently-presented challenge and will strive to correct any errors in a timely manner. Additionally, I’ll revisit my findings from time to time of my own accord, as well as add additional calculators to the list and maintain current information on the website.
This is a major update of the article I first published here in mid-2014. In the original article, I evaluated about a dozen calculators that ranked high in Google search results and ended up separating those tools into two groups: high fidelity (HF) tools and low fidelity (LF) tools. This time, I’ve evaluated a larger group of tools. These tools come in all shapes and sizes and with a variety of design objectives, so it is very difficult to do direct comparisons, either qualitatively or quantitatively. Consequently, I decided to separate the tools into two groups using these criteria:
The tools within the “typical” category may very well fill the need of many users and should not be disregarded just because they’re simple or affiliated with an organization that may have ulterior motives. If you can summarize your financial situation with relatively few numbers and assumptions and want to get a quick estimate of whether you’re on track or not, these tools can probably meet your requirements at no cost and with a minimum of effort on your part. I’m going to list quite a few calculators that fit into this category, but I’m going to refrain from any further elaboration on the characteristics or mathematical veracity of the individual tools. These tools are typically easy to use, so it’s easy for you to plug your numbers into several of them and do your own comparison of results. Here’s the list of tools I spent some time with and placed in this category:
The tools I put in the “unique” category are indeed unique in many ways, with some being simple and straightforward, some doing their modeling with higher degrees of fidelity than others and with varying analysis methods, and others being generalized yet very refined in design with robust feature sets. One immediate and clear observation was that the term “retirement calculator” does not adequately describe many of these tools, because their designs have evolved well beyond the level I typically associated with that term. So, in seeking a way to compare and contrast them, I defined four subcategories:
So, here’s the list of tools that I evaluated and elected to place into the unique category:
I exercised each of these and attempted to perform both qualitative and quantitative evaluations of each of them, and my findings are presented in the table below.
Explanation of my evaluation criteria
Inputs easy to navigate and insert
The ease of identifying where to insert specific items related to demographics, income, expenses, assumptions and portfolio definition, the amount of pointing and clicking required, the clarity with which the various fields are defined and the quality of the user error prevention. This is clearly subjective, but I exercised each tool and rated them on a 1 to 10 scale, with 10 being the easiest.
Outputs easy to navigate and understand
The ease of locating specific items with the tool’s outputs, the amount of pointing, clicking and scrolling required to study the outputs, the clarity with which the various parts of the output are defined and the ease with which the outputs can be interpreted. This is clearly subjective, but I exercised each tool and rated them on a 1 to 10 scale, with 10 being the easiest.
These tools fits into one of four groups: 1) those that perform detailed tax calculations, 2) those that do tax calculations based on multiple user-specified tax rates, 3) those that do tax calculations based on a single user-specified tax rate and 4) those that ignore taxes. For further discussion on this topic, I’ll refer you to the Retirement Calculators 303 (Sensitivity of Long Term Projections to the Tax handling Approach) section of my Home Page article on retirement calculator design.
There are three methods in common use by which retirement planning tools attempt to develop projections into the future: fixed rate projections, Monte Carlo simulations and aft-casting or historical simulations. For further discussion on this topic, I’ll refer you to the Retirement Calculators 306 (Analysis Methods) section of my Home Page article on retirement calculator design.
Quality of documentation
I defined four options for this category, as follows:
The tool models taxable, tax-deferred and tax-free savings categories with their unique tax characteristics and implements a well-defined or user-controlled policy for modeling the related contributions and withdrawals, including the handling of situations where one or more of these categories become depleted. For further discussion on this topic, I’ll refer you to the Retirement Calculators 304 (Sensitivity of Long Term Projections to Pooled vs. Separate Savings Accounts) section of my Home Page article on retirement calculator design.
Life cycle model
The tool models all life stages (accumulation phase, distribution phase, and survivor phase) with equal high fidelity while providing computation support for sustaining the maximum standard of living across all stages. For further discussion on this topic, I’ll refer you to the Retirement Calculators 401 (Life Cycle Models) and Retirement Calculators 402 (Consumption Smoothing) sections of my Home Page article on retirement calculator design.
Support for characterization of income and expenses goes well beyond listing amounts with start and stop dates and growth rates
Examples include modeling of complex income streams, modeling of mortgages, modeling the financing of college educations (529 plans and student loans), calculating Social Security benefits as a function of start age and benefit amount at full retirement age, modeling of rental properties, modeling of changing healthcare expenses from pre-retirement through early retirement to Medicare stages, and detailed checklists of expenses across a large variety of categories.
Derives ROR's from underlying asset classes and asset allocations
The tool allows the user to characterize the portfolio associated with each category of savings as opposed to the simple entry of an expected rate of return.
Provides verifiable outputs
The tool provides tabular outputs containing sufficient information to enable the average user to determine that his or her inputs have been correctly interpreted and modeled, and to follow the translation of those inputs into intermediate results and the ultimate projections being produced.
Models case x and matches PRC/Gold within 10%
I’m confident of PRC/Gold’s math and therefore used it as the standard. I tried to set up each test case in the other tools and then compared the final savings balances or the total spending produced by the tool under test to the values produced by PRC/Gold. Neither of the aft-casting tools (cFIREsim and OTAR) model taxes and they generally don’t operate in a manner that facilitates this type of head-to-head comparison, nor do they produce user-verifiable results. Consequently, I have no basis for confirming their mathematical veracity. Any other tool that doesn’t match PRC/Gold to within 10% appears to contain some mathematical error(s) or simplification that causes significant deviations. The tools with check marks in this row actually matched PRC/Gold more closely than within just 10%; I set the range at 10% to illustrate that the others were off the mark by more than just some insignificant amount in the tests I performed.
DIY tools for retirement planning, in particular, and personal financial planning, in general, come in lots of different forms and vary from simple to elaborate. There is definitely not a one-size-fits-all solution out there. For someone who just wants a quick and easy estimate, there are many adequate tools to choose from. For someone who wants to invest some time delving into the details, saving the inputs, doing what-if’s, quickly comparing alternatives, and revisiting and updating those inputs and the corresponding projections on a regular basis, there are fewer choices, but some good tools available. Some of the free tools are perfectly capable of doing detailed, high fidelity projections, but you may have to pay a few dollars to get a high-end tool with additional features. In the final analysis, I think you have some good options to choose from.
Pralana Consulting LLC, Plano, TX
© 2017 Pralana Consulting. All Rights Reserved.
PRALANA RETIREMENT CALCULATOR | The Mercedes-Benz of Retirement Calculators
ADVANCED PERSONAL FINANCIAL MODELING | MONTE CARLO & HISTORICAL ANALYSIS | INTERACTIVE WHAT-IF ANALYSIS