I have been looking at the party-ID numbers in the
Gallup data. I find evidence that party ID is not fixed over time. Here is the argument.
The Gallup poll internal numbers contain the fraction of voters who call themselves Republican, Democratic, or independent. The average GOP fraction is 39%, but fluctuates. The fluctuation has been the source of much discussion.
As it turns out, the amount of fluctuation can be predicted if the fraction of Republicans is fixed over time. From binomial statistics, the expected standard deviation is sqrt(r*(1-r)/ N), where r is the fraction 0.39 and N is the number of people per poll, about 1000.
These numbers predict a standard deviation of 1.5%. From Gallup's data, the actual standard deviation is 2.9%, almost twice this. This suggests that the underlying true party-ID numbers shift over time. This supports criticisms of weighting by party ID.
However, using unweighted data has its own problems. In 2000, Rasmussen did not weight and predicted a margin that was 9 points more favorable to Bush than the final outcome.
Rasmussen now weights, and now his presidential tracking poll fluctuates very little. Because party ID and preferred candidate (Bush/Kerry) are strongly correlated (see my analysis), this means that his weighting procedure will always work to reduce the margin of the leading candidate. This may explain why his poll is so stable - statistically, too stable to be right. In recent 3-day tracking data (analyzing every third day) he has a standard deviation of 0.7% when random fluctuation alone would predict an SD of 1.6%.
There is a second, obvious problem. The horserace result depends on assumptions on party ID, which means that it is very hard to get information from weighted data on who is ahead, a basic fact we want from polls.
I therefore think that both weighting by party ID (Rasmussen now) and not weighting at all (Gallup now, Rasmussen in 2000) have serious problems. A better way to weight would be to use a political question with a fixed answer, such as "Who did you vote for in the last election, Bush, Gore or Nader?" I understand that Zogby does this. Time does it, but does not weight. If anyone knows of other organizations that do this, please let me know.