A confidence level of 50% would mean the difference is truly random, with only a 50-50 chance that you'd see the same results in a repeat of the test. Even at 75% the odds are not good—there's a one in four chance that your results are meaningless. For the purposes of direct response, 95% should be the minimum confidence level for a difference to be considered statistically significant.
Of course, this test of statistical significance is only a formula. In order to use it correctly, you need to have a hypothesis of why you believe the response rates should be different. If the calculator shows a significant difference, the test supports your hypothesis. But if not, it's time to go back to the drawing board.
Two final notes. First, you'll notice that the formula allows you to use test panels of different sizes. Contrary to popular opinion, test groups do not need to be equal to ensure a statistically valid response—they only need to be split randomly. And, second, this calculator only addresses the question of response rate. There is a separate test for average gift, but you need to have the full list of actual gifts received in order to calculate it, most easily using statistical analysis software.