The narrowing of the numbers between Mayor Giuliani and Sen. McCain in the most recent CBS/NYT poll was the cause of much debate here yesterday. It seems that the poll caught the attention of Mark Blumenthal of Pollster as well, who offered the following analysis:

Both the February and March surveys involved a hypothetical two-way match-up between Giuliani and McCain (Q49), and the decline in Giuliani’s support appears to be just barely statistically significant. For those wondering (as I did), surveys interviewed 314 registered Republicans voters who say they are likely to vote in Republican primary in February and 603 in March (the most recent survey involved an oversample of Republicans; unweighted counts are at the end of the CBS releases here and here).

Given the sample sizes, the seven point drop for Giuliani (from 50% to 43%) is just barely statistically significant at a 95% confidence level, although the five point increase for McCain (29% to 34%) is not.

However, there are some very good reasons for caution here. First, when I say “barely” I mean the Giuliani decline would not be significant if we needed to be 96% confident. And the precision of my calculation requires a statistical assumption (simple random sampling) that is iffy when pollsters weight their data (as this and virtually all other public polls do).

Next, remember that four other surveys conducted over the last 10 days - including the just released CNN/ORC survey - show no comparable Giuliani decline. In fact, while the changes on any individual poll may have been within sampling error, all four show a slightly larger Giuliani lead (see also the trend in evidence on our Presidential tracking chart).

Keep in mind, of course, that the CBS/NYT survey involved a hypothetical two-way Giuliani-McCain match-up, while the others asked respondents to choose among all of the likely Republican candidates. Notice that the CBS/NYT results are better for both Giuliani and McCain - the perfectly logical result of offering just two choices rather than roughly ten. Perhaps the Giuliani decline occurred mostly among those who only opted for Giuliani (in February) because their first choice was not an option.

Consider finally the issue of the “oversample” of Republicans on the most recent survey interviewed by redialing households “at which a self-identified Republican had been interviewed in [CBS/NYT] polls over the last seven months.” Of course, the pollsters can compare the oversample to Republicans in the base sample to see if the former was more hostile to Giuliani. In response to my email query, CBS Survey Director Kathleen Frankovic confirms that she saw “no indication that the oversample was hostile to Giuliani.”

When it comes to methodology, the CBS/New York Times surveys are top notch. But yes, early or late, even the best polls can produce “fickle” results. So given all of the above, readers should be cautious about making too much of the observed Giuliani decline until confirmed elsewhere.

The last two polls that measured the Rudy/McCain head-to-head matchup were the March 4th Newsweek poll that showed Rudy with a 25 pt. lead on McCain, and the February 17th CBS poll that showed Rudy with a 21 pt. lead on the Senator. So as Blumenthal advises, we should use caution when make a conclusion based on this data alone.