When running Monte Carlo analysis, I'm noting significant variability in the results run-to-run (no changes to the model between runs). I see about 20% variability with multiple MC runs at all level of percentile values. Is this level of variability expected? Is there a way to increase the number of projections within a given Monte Carlo run to improve statistical stability?
When running what-if scenarios (different pension assumptions, Roth conversions, etc), this variability limits understanding of which scenario might be better outside of just using Fixed Rate analysis. The variability skewed the results enough that it changes the decision outcome (eg Roth vs No Roth or Pension A assumption vs Pension B assumption), when re-running the MC. Outside of running the MC analysis multiple times and estimating the most likely outcome (which will be very tedious), is there a better way to use MC for plan decision making or is Fixed Rate analysis the only way to go to compare scenarios?
@sawright Yes, depending on the length of the modeling period and the standard deviations of your asset classes, this degree of variation is not unusual. No, there's no way to change the number of projections. For the very reason you stated, in general, we think that deterministic projection comparisons are the best way to evaluate alternative options such as pension assumptions and Roth conversions.
Stuart
@sawright How does your MC simulations compare to your historical simulations??
FWIW, I don't get nearly that range with my monte carlos! I only get 1-4% difference in different runs. 40% of my portfolio is stocks, which has a SD of 18% that I set. What assets of yours have the highest SD and what is their AA in your portfolio? If you have a lot of assets with high variance that might be the explanation. Out of curiosity do you have the "use correlated ROR" checkbox checked?
@jkandell Thanks for your thoughts. I'm using the default automatic SD for the asset classes which looks like 1.5x ROR for the class. So SD of 7.5% for Stocks and 3% for Bonds with a 60/40 AA for TD and 70/30 for Roth. Correlated asset classes checkbox is unchecked. I'll look at the Stock SD and see how much it changes things, but those SDs look pretty small.
The worst high-side offending MC results tend to come with the 50% value significantly different than the FR deterministic value. But the delta is significant enough that using MC to compare scenarios isn't reliable for my settings. I'm going to run some experiments to see if I can find an 'average' result from multiple MC runs.
@pizzaman Good Question. Since the historical simulation is deterministic, it of course doesn't vary with multiple runs. It's noisier than MC, but it is more consistent with the FR analysis. So Scenario 1 and Scenario 2 with only subtle difference in FR results shows essentially identical results with historical sims. I'll consider the historical sim as a way to help understand the results more.
