Just now, I read a Mormon Times article by Orson Scott Card, called Guessing leads to knowing. I actually liked most of the article, which surprised me since I don't generally think much of OSC. He's correct that almost all human progress comes from someone who had a hunch or a guess or a crazy idea that they decided to try out. Guessing is indeed a good thing.
However, his reasoning is flawed because his proposed "tests" (praying for confirmation, following church commandments) are not really experiments in a scientific sense. People can believe they feel the Holy Spirit confirming wildly different ideas. How do you know which is correct? Science measures hypotheses against objective reality. OSC proposes testing hypotheses against subjective experience. At best, the experiments may prove which ideas resonate with you personally. If that's all you're after, that's a fine result. But you can't then extrapolate your findings as objective truth.
My father-in-law told me to "doubt my doubts" when he first found out about my disaffection with the LDS church. It didn't make much sense to me at the time. But now I think I understand the premise behind the phrase: that your "doubts" are really just new beliefs. As such, they should be subject to scrutiny just as your original beliefs were. If I accepted the premise that doubts are just new beliefs, I would certainly accept the conclusion that you should "doubt your doubts".
However, there is a fundamental difference between belief and doubt. A belief is a positive assertion that something is true. A doubt is a neutral assertion that I don't know whether something is true. For example, I might say that I doubt the Book of Mormon is a historical record. That statement by itself does not imply that I believe it is not a historical record. It simply means I don't know. I make no assertion in either direction.
Now, one can certainly examine the evidence and come to a tentative conclusion with a reasonable degree of certainty. Would I say the Book of Mormon is more likely to be historical than not? No, based on the evidence I have encountered I would say I believe it's more likely to be biblical fan fiction. However, this statement is not a statement of doubt. It is a statement of belief based on evidence. Do I doubt this belief? Of course! I am willing to change my assessment based on new evidence. And you can believe that's true because I have done so already.
If this is what is meant by "doubting your doubts", then I suppose I already do. But do I doubt the mechanism of doubt itself? Should I boomerang back to my original beliefs before disappearing in a puff of logic? That would be silly. As Orson Scott Card said himself, doubt is the vehicle of progress. To doubt the act of doubting would be like using the Internet to spread the message that all technology is evil. It would be inconsistent, and achieve nothing but a smug self-satisfaction in a castle built on semantics.
So doubt your beliefs. Doubt all your beliefs. If they are worth believing, they are worth doubting. But don't test your beliefs against your feelings. Test them against evidence. And is it worth the self-inconsistency of trying to "doubt your doubts"? I doubt it.