If you sold all your shares early this year, fed up with the range-bound movement of the markets for months on end, you have our sympathies.

You have just missed the opportunity of a lifetime to make big bucks from a bull run. You are the victim of bad market timing.

Also Read Dummies guide to the stock market

The good news: You are in august company. Worldwide, even the world's most gifted investors cannot time their market entries and exits correctly all the time. No one can call the peaks and troughs exactly.

Over the last 10 years, individual stocks like Infosys have done spectacularly well, but not the market as a whole. Take a look at the Sensex.

In 1994, the Sensex peaked close to 4500. Nine years later, it is still below that level despite the pyrotechnics of the last few months. Just four months back, the Sensex was at 3000 levels.

So if you had invested in 1994, you would have actually ended with losses on your plate after nearly a decade of staying invested. That is assuming you didn't cash out during the tech boom in 1999-2000.

Broadly, this means more people would have lost money on the markets than otherwise. So how do you make sure that you are not among the losers?

Easy. Assuming you've bought fundamentally sound stocks, or invested substantially in index stocks, don't get out - ever - without making a profit. The vast majority lose money on the markets because they lose patience.

It is in the nature of markets to remain range-bound or falling for long periods of time; but when they make up their minds, they hit peaks in no time at all.

The market spent months and years hovering around the 3000-mark; but it took just a few weeks this year to head past 4000.

If you have been holding stocks for 10 years, and you got out in March 2003, you are a loser twice over: you lost money by not getting out in 1999-00, and you have missed the boat once again in the current bull run.

The converse is also true: You may spend a year or two racking up slow, but steady, index gains. But one 9/11 can destroy the whole gains in just a few days.

The lesson for traditional losers is simple: Since you cannot time the market, the only way to make money is by staying in it for as long as it takes.

Investors who got in at the peak in 1994, but didn't get out in 2000, will only now be breaking even on their investments. Their gains will come only in the coming months and years.

The Smart Investor did a study covering the period from 1979 to now comparing your returns if you had stayed invested in the market at all times - bull, bear and neutral periods - and if you had woven in and out of markets in good periods. This is what we found:

• If you are an excellent market timer - that is you stayed for the bull periods and opted out of the worst times - you could have made returns of more than 50 per cent a year.
• If you are a lousy timer, you could have lost your shirt - losing around 17 per cent annually. Your money would have been safer under your pillow.
• If you didn't time the market at all, and stayed invested all through the years - through good times and bad – you would have comfortably made 16 per cent annually, well above returns on comparable assets.

The study essentially tries to bring out the risks and rewards associated with market timing - or the lack of it.

We define market timing as an investment strategy that transfers assets from equities to cash equivalents, or from cash equivalents to equities, based on a prediction of the direction and extent of the next price movement in equity markets.

Successful market timing would require not only foretelling the future correctly most of the time but also moving back and forth between equities and cash (or cash equivalents) without incurring prohibitive transaction costs.

For the study, we looked at the country's most widely tracked index, the Sensex, for two time periods: 1979-2003 and 1993-2003.

For both time periods we have considered monthly returns for the Sensex, while for the second time period we have considered daily returns as well to see how a few days of staying in or out of the market could have made a difference to your returns.

The study examines what would have happened to your total returns (as indicated by the index) if the market's best and worst months and days - essentially the stock market extremes - are eliminated from the calculations.

As the term suggests, the best months or days are the ones with the highest positive returns; worst refers to the ones with highest negative returns.

We have done two sets of calculations to ascertain the effect of market timing.

Firstly, we looked at three scenarios - what returns you would have managed if you avoided the worst months or days; what if you missed all the best months or days; and what if you avoided both the best and the worst months or days.

Secondly, we looked at two hypothetical market timers - the perfect timers and inept ones (which means the vast majority of uninformed investors).

The perfect timer owns stocks - we assume he is investing in fundamentally good scrips with a strong correlation to the index, or an index fund - in all the periods with positive returns and avoids those with negative returns by moving his money to savings deposits.

(For the sake of simplicity, we have assumed a standard rate of 5 per cent for savings deposits throughout the period. Transaction costs have also been ignored).

As for the inept timer, he owns stock only when returns were negative. That is, he held his money in savings deposits when stock market returns were positive.

This exercise quantifies the limits to which market timing might have rewarded or punished its practitioners.

Our Findings

(September 1979-August 2003): The study shows that by being 'out of the market' for just a few months during this period, the risks and rewards change dramatically when the months missed were those with the highest or lowest returns.

During this 24-year period, the Sensex gave a compounded annual return of 15.90 per cent as it surged from 116  to 4004.

So Rs 100 invested in the Sensex in September 1979 would have cumulated to Rs 3,449 at the end August 2003, after 288 months, or 24 years.

• When we eliminate the best months, the returns fall dramatically.

For instance, if you missed the best 10 months over 24 years, your return would have fallen to 5.54 per cent.

Your Rs 100 would have grown to Rs 365 at the end of August 2003. Similarly, eliminate more 'best months,' say 20, and your annualised return would  even have turned negative.

And if you eliminate the 72 best months (25 per cent of 288 months), you would have simply wiped out your entire portfolio and left with just Rs 1.17!

• If we eliminate the worst months, say the top 10 of them, then your annual return would have been 23.79 per cent over the same 24-year period.

Similarly, if you avoided the worst 72 months, you would have ended up with - hold your breath - Rs 25,44,879 ! A phenomenal 52 per cent annualised return.

• If we eliminate both the best and the worst, your annualised return for the three scenarios (that is eliminate the 10 best and worst, the 20 best and worst, et cetera) would have been 12.73 per cent, 11.65 per cent and 9.39 per cent respectively.

So, Rs 100 invested in September 1979 would have cumulated to Rs 1,773, Rs 1,409 and Rs 862 respectively in the three scenarios.

The last decade

(September 1993- August 2003): In this time period, when the market saw the arrival of big institutional players and better stock research, the effect of extremes was even more dramatic.

During this period, the Sensex moved up from 2,634 to 4004, and gave a compounded annual return of 4.28 per cent. This means you would have been almost as well off letting your money rot in a savings bank account, and clearly better off with money in a fixed deposit.

• Instead, if you had not been in the market during its best 10 months, you would have ended up with negative returns straight away (-9.08 per cent).

An investment of Rs 100 would have shrunk to Rs 38.59 at the end of 10 years. And it gets worse as we eliminate more of the best months.

• If you had managed to avoid the worst 10 months, Rs 100 would have grown to Rs 549.98, an annual growth rate of 18.59 per cent. If you missed the 20 worst months, you would have landed up with a 28.73 per cent return.

Similarly, if you missed the 30 worst months (25 per cent of the total period) you would have made annualised returns of 38.93 per cent and the value of Rs 100 would have appreciated to Rs 2,678.85.

• Again, if we eliminate both the best and the worst, your annualised return for the three scenarios would have been 3.39 per cent, 4.16 per cent and 5.14 per cent, respectively.

So, Rs 100 invested in September 1993 would have cumulated to Rs 140, Rs 150 and Rs 165 respectively in the three scenarios. In other words, during this period, unless you avoided the worst months and stayed invested in the best months, your chances of making better returns by timing the market would have been pretty bleak.

Calculations based on daily returns also confirm this trend. Your returns would have been a mind-boggling 242 per cent if you had avoided the 605 days of worst losses during the 1993-03 period, and the value of Rs 100 would have Rs 2,19,72,816.

But you would have ended up with a princely Rs 0.001 if you missed an equal number of the best days.

Perfect timer versus inept timer

A perfect timer - who avoided all the negative months while being in the market in all the positive months - would have grown his investment of Rs 100 into Rs 96,40,556 during the period 1979-2003 - an annualised gain of 61.31 per cent. On the contrary, an inept timer would have reduced his portfolio to Rs 0.72, by compounding annual losses at the rate of 24.48 per cent.

Similarly, a perfect timer would have grown his investment of Rs 100 into Rs 6,562 during the period 1993-2003 - a annualised gain of 51.95 per cent. An inept timer would have reduced his portfolio to Rs 3.82, an annualised loss of 27.86 per cent during the same period.

The point to ponder is: While the rewards of market timing seem quite enticing, what is the probability of you getting your timing right?

Statistically, if you have a hit rate of 50 per cent, the probability of getting your timing perfect is 0.5 raised to the power 288 (the number of months covered in the study) - which is nearly zero.

Conclusion

It is well known that the market does not rise or fall steadily. Instead, there are days or months when the market soars or plunges. What's surprising is that a tiny portion of those brief swings account for practically all of the market's gains or losses over decades.

For instance, 65 per cent of the market gains since 1979 came on account of Sensex gains in just 10 months - or three per cent of the 288 months during the period.

So market timing is perhaps even more difficult and risky than investors have been led to believe. Every time an investors talks about deferring  equity purchases "until the market gets better," that investor is trying to time the market.

Yet waiting can mean missing the crucial days or months when the market surges - a narrow window of opportunity most investors probably cannot anticipate.

So when pros recommend investment with a long-term perspective in equities, all they are trying to say is that it is futile to get in and out of the market frequently since stock movements are unpredictable.

A better way is to stay invested at all times so that you don't miss the big moves when they happen. And hopefully, the sharp downfalls will be more than compensated by the upmoves, resulting in net gains at the end of it.

Market timing is, thus, only for those who are confident of predicting the future. Others would be better off staying invested at all times and routing investments in stocks through systematic purchases, or what they call "rupee-cost averaging".

This means buying stocks worth a fixed amount every month or quarter. So when stocks are expensive you buy lesser quantities of them and when stocks are cheaper you automatically end up buying more.

A contrary opinion

A study done by Franklin Templeton for the period from 1990 to 2003 shows that a price-earnings (P/E) ratio-based market timing approach works better than a fully invested portfolio.

Over the last 10 years ended August 1, 2003, a fully invested portfolio would have yielded an annualised gain of 5 per cent while a timed P/E ratio-based portfolio would have given 9.3 per cent.

Over the last five years, the returns would have been 4.2 per cent and 12.6 per cent respectively.

And over the last three years, while a fully invested portfolio would have given a negative return of 3.6 per cent, the timed portfolio would have delivered a 6.2 per cent annualised return.

However, in the last one year, the PE ratio-based strategy has underperformed compared with a fully invested portfolio.

The PE ratio-based timing approach (on which the Franklin Templeton PE Ratio Fund is based) involves an in-built buy-sell mechanism which is triggered by the PE levels of a target index.

For the study, the Sensex has been chosen as the target index though the Franklin PE Ratio Fund is based on the Nifty.

The timing involves changing the asset allocation based on the PE ratio band which is based on the past five years' range of 11 to 28.

So, if the index PE is below 12, the strategy is to remain fully in equities. If the PE goes beyond 28, one must exit equities and stay in cash completely.

As per the study, such a strategy of varying the allocation of equity and debt/money market instruments based on the PE ratio has delivered superior risk-adjusted returns over the long-term.

As is clear from the above graph, while a fully invested portfolio of Sensex stocks has done well in rising markets, a self-balancing portfolio preserves value better in falling markets and has the potential to deliver superior risk-adjusted returns across market cycles.

The problem with the PE ratio-based strategy is that it does not allow for changes in PEs on account for structural changes in the marketplace like a fundamental change in the growth rate in the economy, interest rates, et cetera.

However, Franklin Templeton PE Ratio Fund allows for changes in PEs in case of exceptional situations. Here again, the risk really emanates from misinterpretation of market moves.

For instance, during the infotech boom, the market believed that infotech stocks could continue to deliver bombastic returns and hence the astronomical PEs were justified.