Measuring and Making Public Opinion

THE OPEN MIND
Host: Richard D. Heffner
Guest: Burns Roper
Title: “Measuring and Making Public Opinion”
VTR: 8/22/84

I’m Richard Heffner, your host on THE OPEN MIND. Public opinion pioneer George Gallup’s death recently reminded me of THE OPEN MIND’s long connection with his profession, of our many programs with Elmo Roper and Paul Lazersfeld and Earl Newson and Daniel Jankalovich and other luminaries and leaders in the measurement of what we Americans think and what we do, and what the disparity is between them, too. Of course, surveying public opinion is very much in the limelight now because of the presidential race. And so I’ve invited Burns Roper, Chairman of the Roper Organization, to talk with us today about the high points and the pitfalls of a profession that looms ever larger in our lives.

Thanks for joining me today, Mr. Roper. You and I have spoken at this table before, perhaps never with so much going on around us that involves your profession. And I wondered if you feel that it’s fair for me to talk about the high points and the pitfalls in this presidential year of public opinion polling in reference to politics?

ROPER: Well, I’m not sure there have been any high points yet, but there may be. There certainly have been some pitfalls, and there are more, I think, coming.

HEFFNER: What are they?

ROPER: Oh, they’re a variety of things. Public opinion, I think because…Well, let me get to why I think so later. I think public opinion is becoming harder to measure when it comes to political measurement.

HEFFNER: Harder to measure?

ROPER: Harder to measure.

HEFFNER: Even as your techniques improve?

ROPER: Right.

HEFFNER: How do you explain that?

ROPER: Well, the only way I can explain it is that people are more liberated now. They are less, sort of, “I vote Democratic because my father voted Democratic, and his father voted Democratic. We’re lifelong Democrats”. Similarly on the Republican side. I think people have more cross-pressures on them now. They are Democrats, yes. And they’re Catholics, yes. And they’re this, yes, and that, yes. And one part of it says, “I should vote for this guy”, and part of it says, “I should vote against him”. And I think people aren’t that sure what they’re going to do.

HEFFNER: Well, is that a function of extrapolation and how difficult that is from the fact that the population is one-fourth Catholic and one-third this and one-seventh that?

ROPER: No, it’s a problem of getting people to tell you what they’re going to do, knowing, I mean, accurately.

HEFFNER: Because they don’t know?

ROPER: Because they don’t know.

HEFFNER: So what do we do with your profession? What do we do with public opinion polling?

ROPER: Well, I don’t know. I’m concerned about it. I’m not sure I know what the answer is. An awful lot of people in the field disagree with me, and they think the reason the recent polls, ’82 for example, and also ’80, were not all that accurate was that they didn’t poll late enough, and they didn’t catch the last-minute change. Well, I don’t think that’s it. I don’t think there was a last-minute change to Reagan in 1980. I know that there is evidence cited that there was, but I think I’ve got equally good evidence that there wasn’t.

HEFFNER: What was there then?

ROPER: In that particular case, I think there was a lot of closet voting. Lifelong Democrats who couldn’t take another four years of Carter, but weren’t about to tell their wives and friends and so on and so forth that they were going to vote for Reagan. And they sort of said, “Well, I suppose I’ll end up voting for Carter even though I don’t particularly like him”. And they knew damned well they were going to vote for Reagan.

HEFFNER: Do you think though, that that’s a matter of idiosyncrasies that related essentially to ’80; though you said ’82, the same thing?

ROPER: Well, ’82 was different. ’82 was a different situation. In ’82, practically every frontrunner in the polls failed to do anything like as well as the polls said he would. Now, in most cases those frontrunners were incumbents. So I don’t know whether it’s frontrunners or incumbents, or what it was. But the polls, for example, said Mario Cuomo by about ten points in New York. He won by three. The polls all said Millicent Fenwick would edge out a close race in New Jersey. She lost. The polls said that Senator Weichert, by about 12 points. Number five. And so on. The polls said that Mark White would lose in Texas. He won. The polls said Bradley over Dick Magian. He lost; narrowly, but he lost. In almost every case – I can’t think of a case where the frontrunner did better than the polls said he would do. In every case it was worse. And I don’t know why.

HEFFNER: Well how many of the frontrunners – to go back to a suggestion you made – were incumbents?

ROPER: Most of them.

HEFFNER: Most of them.

ROPER: And in some cases…Well, now, for example, in New York State there was no incumbent.

HEFFNER: Right.

ROPER: But in a sense. Cuomo was the incumbent. He had been lieutenant governor, so he was…Sure, he was going for governor. But Lehrman wasn’t an officeholder. And Cuomo was the lieutenant governor. So, in a sense, he was the incumbent.

HEFFNER: But you know, as you say that, it occurs to me to ask you whether, just before the elections themselves, the real elections –not the polling that the pollsters did – before the real elections were the ones who lost or who did not win as big as the polls had predicted?

ROPER: And that’s mostly what it was. Not winning as big.

HEFFNER: Okay. Were they touted as going to be the big winners? And could it be a reaction against that touting?

ROPER: Could be. It could be. The thesis does not fit all that well with the thesis that we have an apathetic electorate. Only 53 percent turn out for a presidential election. I’m not sure that’s apathy. I’m not sure it isn’t more frustration than apathy. But I think, the only theory I can come up with that fits the data is that people weren’t that happy about either candidate. And when they heard X is going to beat Y by ten points or 12 points or something, they didn’t want him to win that big. So they either didn’t vote or they voted the other way. And maybe in the case of New Jersey they outfoxed themselves. They wanted Fenwick, but not that much, and they elected Lautenberg. I don’t know.

HEFFNER: There doesn’t seem to be, so much in America anymore, a kind of overwhelming sentiment to the right or to the left or in any direction. Does that account for some of this?

ROPER: It may. I think people are feeling conservative in general. But on specific issues, they’re being quite liberal. And I think there is that ambivalence. Against government regulation, but they ought to do something about prices, they ought to do something about excesses of labor unions, they ought to so something about better, safer working conditions, and I want less government regulation.

HEFFNER: Do you think that’s the point at which this kind of strange involvement focuses on, the almost doubletalk?

ROPER: The philosophical versus the practical. Yeah, I think that’s part of it.

HEFFNER: What about foreign affairs then? Can you measure, do you find the same kind of hip-hopping around in the area of foreign affairs? I mean, it’s harder to measure in terms of candidates.

ROPER: People don’t have as strongly formed views in foreign affairs. They’re not that knowledgeable about it, and that’s sort of where they defer to the president. They are very concerned about potential involvement in another Vietnam, whether it’s in Latin America or Lebanon or whatever. But they don’t really understand that much about what the issues are or how serious it is. There’s sort of a generalized fear about it, but not any strong feeling that the Democrats would do better than Reagan would do.

HEFFNER: All right. You’re very frank, and you’re very up front with this question as to how well the polling goes now. Are there any remedies, or would you say it’s not a situation one cares to remedy?

ROPER: I hope there are some remedies. As I look back on the ’80 election, we did not have a prediction, but, because we didn’t have a last-minute poll. But I saw in the data then the fact that Carter’s weakness was he was indecisive. He flip-flops. He doesn’t have a handle on things. But his strength was that he had compassion. He was for the little guy, for minorities, and so on and so forth. And conversely with Reagan. Reagan was decisive and strong and for the rich. And I thought these two things would probably cancel each other out. Well, in retrospect, it’s very clear that the compassion thing was much less important to people than the strong, decisive leadership thing.

HEFFNER: Or being for the rich?

ROPER: No, no.

HEFFNER: No? Not that?

ROPER: No, not that. Not that. And I don’t know what’s going to be. I mean, again this year, Mondale comes across as the man of compassion, for the little guy, for minorities and so on. And Reagan again comes across as for the rich, the special interests. But, Reagan again comes across as decisive, not wishy-washy, not changing his mind, and so forth. So I don’t know which is going to cut this year. Last year, when they had the compassion and wanted the decisiveness, it went to Reagan. This year, when they’ve got the decisiveness, I don’t know what it’s going to do, whether it’s going to go the other way or not.

HEFFNER: Let’s go back for a moment to the question of how this impacts upon the profession, the business of polling itself. If you were to make a guess as to what that impact will be, what would that guess be?

ROPER: The impact of what?

HEFFNER: The impact upon the profession itself of this inability over the past few years to keep up with the changes.

ROPER: I think it will, if we get beyond the point that we just didn’t poll late enough – and my hope is this year, since so many pollsters think that is the only problem, that there will be some late polling, and if it’s off the mark in any consistent kind of a direction, I think that may lay to rest that that is the cure, and may focus attention on some of these internal conflicts that I think exist.

HEFFNER: Let me ask you about that late polling business. How late is not late enough? I mean, it seems to me that we’ve been treated to polls up to the very last minute.

ROPER: Yeah. That’s why I don’t think that’s the answer. But a lot of polls are completed the week before. But there’s some completed Sunday and tabulated and reported Monday night. And I don’t know how you’d get much later than that.

HEFFNER: Do you think that the polls themselves have an impact upon the phenomena they’re designed to measure?

ROPER: I never have thought so. I just indicated before that possibly people were cutting the frontrunner, that they were having a reverse bandwagon effect, an underdog effect, or a “Let’s bring this frontrunner down to size so he doesn’t get to feeling too big for his boots”. But there has not in the past been a history of polls consistently undermeasuring or overmeasuring. And I have always thought that the polls had more of an effect on the candidates and the campaign workers and the contributors than they do on the people. If polls had a consistent effect, they would always be wrong, and they would always be wrong in the same direction. And that certainly hasn’t been their history over the years, even if there was that pattern in ’82.

HEFFNER: But I have to ask you what impact you think polls should have upon politicians.

ROPER: I think they should do several things. From a self-interest point of view, I think they should inform politicians of what are the sensitive areas that they want to deal with in a gingerly fashion. I think it ought to inform them where they are in a strong agreement so that they can emphasize those things in their own self-interest. I think, on a more philosophical level, I think they ought to inform politicians of what the areas of public ignorance or folly are so that they can disabuse them. An example I’ve used is, if I knew that 93 percent of the American public wanted to drop the H-bomb on China, I’d damned well want to know it. Not so that I could drop the H-bomb on China, but so that I could try and dissuade the public from their madness. Now, if I knew only three percent wanted to drip the H-bomb on China, I’d ignore it. Three percent always want to do anything. And that’s not enough to worry about. I could spend my time on more important problems. So I think it serves that kind of a…

HEFFNER: You’re very quick to say you wouldn’t drop the bomb because you found such a large percentage wanted to.

ROPER: That’s right.

HEFFNER: And I remember years ago when Elmer Roper, when your father, I think it was in a guest editorial in The Saturday Review, and I think – at this age, my thinking isn’t that good – but I think, or my recollections are, I think it had to do with the point at which Nelson Rockefeller dropped out of competition for the presidential nomination. And your father was beside himself. He said, “This is a misuse of the polls”, because the governor, Nelson Rockefeller, was saying, “Well, they don‘t really want the things I want. They’re not going to pick me, and I’m not going to win, so I’m going to drop out”. Now, I gather you feel somewhat the same way…

ROPER: I do.

HEFFNER: …that isn’t the function of the polls.

ROPER: And I think that certainly is a function of the very early polls, you know, before the primaries take place, let’s say. And during the early primaries. People don’t know who they’re picking in the polls at that point. “Oh, Mondale. Yes, I’ve heard of him. So I’m for him. I don’t know who Hollings is, or these other cats”. And it drives some good people out too early I think.

HEFFNER: Well, you’re talking about the people. And really what I was thinking about were the issues. That you take the public pulse, and then…

ROPER: Well, but you can make a case that the polls showed Hollings wasn’t going to do well, and so Hollings dropped out. Now, whether that’s realism or chickening out I don’t know.

HEFFNER: But let’s go back to the question of the issues. If the polls show a large percentage want to drop the bomb, you say that what you would be very certain to do is wage an educational campaign, “For God’s sakes, don’t”.

ROPER: Yeah.

HEFFNER: What do you have to say about the politicians though, who say, “Well, I’m not in keeping with what the pulse rate is. That beat isn’t my own beat. I’d better drop out.”?
ROPER: I guess that doesn’t bother me quite as much as it bothered my father. I think there’s a point where realism comes in. If a person honestly doesn’t have a chance and is spinning his wheels, I’m not sure it’s wise to do it just to charge windmills.

HEFFNER: Then we’re really depending, in terms of our political leadership, upon the accuracy, not in gross situations where you talk about 80 percent and three percent, but we would be talking about depending more and more upon the public opinion polls as monitors, as guides, as measurements of what the people want and don’t want. And I have to ask the question as to whether the polls are that accurate as to be such guides for candidates?

ROPER: Well, I find that hard to answer. The reason I find it hard to answer, polls are showing tremendous bounces now. Mondale was ten points behind Reagan prior to the Democratic convention, according to the Gallup Poll. Roughly ten points. Fourteen points is my memory. Right after the convention, Mondale was ahead by two points. Dramatic shift, except it’s meaningless. The Democrats had had their day in the sun. They didn’t carve each other to ribbons. They nominated a woman vice president, had some tremendous speeches by Cuomo and Jackson and so forth. So it was riding high. And the public has not made up its mind, so it says, “Yeah, I like those two”. Now, if you read that wrong, if you read that as a sharp, solid trend towards Mondale, you go wrong with it. If you know what it’s saying, you don’t go wrong with it. And that two-point lead has disappeared now.

HEFFNER: Okay. Which leads me to the next question of course, which is: Who read it right, and who read it wrong? And what are the responsibilities of those who take and make the polls in terms of the interpretation of those polls?

ROPER: Well, I think the people who take the polls have a great responsibility in this area. I, in several speeches this year, I have talked against the between-conventions poll, just because it is misleading. And we are not doing between-the-conventions poll. In 1976, one of the major polls (I forget which one it was now) showed Carter 36 percentage points ahead of Ford. By the time the Republican convention was over, the 36 was down to 14. It was a meaningless kind of thing. All it did was show that people were sort of applauding what that convention did. Now, it means one thing. It means two things. It means public opinion is highly unstable. It isn’t formed. It can’t go from 14 points minus to two points plus in a week, and then back to ten points minus, or whatever it is. It also means that, while there’s no question that Reagan is ahead and has been ahead all through this, that if the Democrats put on a good campaign, they can win. Public opinion is not yet so set against them that they can’t crash through with a victory.

HEFFNER: Okay, but let’s take the…

ROPER: And that pollster has the responsibility, I think, to say that.

HEFFNER: Okay. Now, between the pollster and the public, obviously there stand those who bring the results of the polls to the people. And I’m not looking for us now to beat the media over the head, or heads, because they have many. But I am concerned – and I’m sure you are – about this matter of who does interpret the polls. If you are commissioned to make a survey, you make the survey, and to the best of your ability, I presume, you present the results of what you have. And I assume that you interpret those results to a certain extent. What are the ethics involved? What ethics are involved in presenting the interpretation you offer, presenting the caveats you offer, when the media presents the results to the public? Who’s responsible there?

ROPER: It’s a little hard, frequently, for a pollster to get the opportunity to correct what has been said erroneously, if the pollster thinks it’s erroneously. But I think he has a responsibility to issue a release or to appear on a program if he can, and try and set the record straight, to put a proper interpretation on that.

HEFFNER: Now, Bud, in the history of mankind – and I know I’m asking too broad a question, so you’ll come right back – but how often does that happen?

ROPER: Well, I don’t know. I think I’ve pretty clearly gotten my ideas across on the limitations of polls and the Quixotic nature of early polls and the meaningless nature of early polls.

HEFFNER: I don’t mean to ask a generic question. But specifically – and I don’t mean an issue now on which you would quote – but does it happen with sufficient frequency to make us breathe a little more easily? There is the poll taker and maker who stand ready to correct misinterpretations of the material he presents to the media or to those who commission his polls?

ROPER: I think most people are pretty good about that. Not all pollsters see things the same way. I’m sure that there are going to be a couple of pollsters out there listening to this program that say, “He’s out of his mind when he says that!” Well, there are differences of opinion.

HEFFNER: What would you do, by the way – I’ve referred a number of times to your profession. And I know that, I guess it was the last time we did a program together, you were, I think, past president at that point of your professional organization.

ROPER: American Association for Public Opinion Research. Yeah.

HEFFNER: Okay. Now, in a few minutes, when you leave this table, we’re going to have a couple of colleagues sitting here who, present Chairman and past Chairman of the American Association of Advertising Agencies. They too are concerned with ethics and politics.

ROPER: Right.

HEFFNER: Their profession and the great sport or the great game of politics…Do public opinion people, do they manifest their concerns to the extent that the advertising people do about what the one profession does to the other, what politics does to advertising, and what politics does to public opinion?

ROPER: Yes, yes. I think they do. But they don’t come up…Now, in the American Association for Public Opinion Research; for example, there is a group that wants to set minimum standards for what constitutes a good poll. It’s got to be this kind of a sample. It’s got to be this size, and so forth. I’m against that.

HEFFNER: Why?

ROPER: Because I don’t think you can set minimum standards. How high is up? I think all you can do is have discloser. I think to say, “Our latest poll shows that Ronald Reagan will win by ten million votes”, and that’s all I tell you, shouldn’t happen. I ought to tell you, we interviewed 2,000 people, or 400 people, or whatever it is, in heavily Republican districts, if that’s what we did, or a representative sample if that’s what we did, and tell you what we did and what we found; not just the result.

HEFFNER: Okay, Bud, suppose one does that. That’s fair enough. But what you have suggested at the very beginning of our discussion is that something is going on in this country that would make even such, not disclaimer, such identification of what we did, perhaps incapable of correctly enough interpreting the results. Something’s going on. So perhaps we ought to just say, “Hey, not more of this polling stuff”.

ROPER: Well, I think that would be unfortunate. I think polling does too many good things. And there are very few areas where it’s got to be within ten percentage points of right. I mean, you said earlier, “Don’t take a gross situation where it’s 93 percent versus three percent”. But I’m not sure that isn’t the kind of, I don’t mean that far off, but do you have a minority who believes something or a majority? Is it most people who are ignorant of the facts, or just a few people who are ignorant of the facts? I think those uses of polls are valuable. And polls are accurate within those limits. I think, providing people know what was done. Again, if you just flash the result up on the screen without knowing what people asked, you can be misled.

HEFFNER: What about the self-fulfilling prophecy aspect of the polls? And the question that often comes up: You read these polls; you are led to function in a similar way. You had suggested at the beginning that maybe you’re turned off when you see a guy is so far ahead, and you turn against him.

ROPER: Well, I think maybe that happened in ’82. Historically, that has not been the record of the polls. If the polls had an effect, they would always be wrong and always be wrong in the same direction. If they caused a bandwagon effect, and the polls announced that X was going to beat Y 60/40, then he’d beat Y 70/30 or 80/20, not 60/40. And that has not been the record, with the one exception of ’82, where they overmeasured the frontrunners. So I don’t think, until then there was no evidence of self-fulfilling prophecy.

HEFFNER: And the exit polls, that controversial issue that has come up so much this year?

ROPER: Well, I think exit polls are very valuable. First of all, I think the networks have the right to do them under the First Amendment. Secondly, I don’t think they affect the outcome of an election. I’ve seen no evidence that they do. And logic…If you’re for Reagan and I’m for Carter, and we both hear, “It’s all over. Reagan’s in”, there’s as much disincentive for you to vote as for me to vote. Reagan doesn’t need your vote; my vote won’t do Carter any good. So we both stay away. Now that hurts the Senators, but presumably if you’re for Reagan you’re for Candidate X for Senate, and if I’m for Carter, I’m for Candidate Y for Senate. So it’s lost them each a vote. It reduces the turnout, but I don’t think it affects the outcome. And, having said that, they have the right to do it and I don’t think there’s any influence, I think they’re damned fools to do it.

HEFFNER: Why are they damned fools to do it?

ROPER: Because I don’t see what they’re gaining. Commercially, I mean, NBC was first in 1980. That did not vault them to first place in news viewing or overall viewing. I don’t see what they gain.

HEFFNER: But they aren’t damned fools, so they do think they’re gaining something.

ROPER: Well, they think so, right. But they’re incurring a lot of ill will. And I think they ought to conserve their reservoir of goodwill for things like Watergate, where they really need it, and not chew it up on silly things like this.

HEFFNER: But, that’s the point at which I have to thank you. And I want to thank you for joining me today. Come back. We’ll see what happens after the election.

ROPER: Good.

HEFFNER: And thanks, too, to you in the audience. I hope that you will join us here again next time on THE OPEN MIND. Meanwhile, as an old friend used to say, “Good night, and good luck”.