Methodology


Base Numbers:

Our current numbers for each MOCs ‘actual score’ are based on an amalgamation of scores from the following three sources: Progressive Punch (PP) to capture progressivism, DW-Nominate (DW-N) for ideology, and each member’s party-line votes to measure partisanship.

We compiled these three scores using different weights to produce one simple 0-100% score (with zero being absolute conservatism and 100 being absolute progressivism) that is most effective specifically for primary challenges.

Firstly, these numbers utilize Progressive Punch’s four scores to make one score for each member. Progressive Punch does a fantastic job of measuring each MOCs progressivism by comparing them to the most progressive members’ votes in the House and Senate. They are an incredible resource, and we hope our work at PC draws more attention to their efforts.

PP’s four scores are: (1) Crucial Lifetime Progressive Score, (2) Overall Lifetime Progressive Score, (3) Crucial Current Congress Progressive Score, and (4) Overall Current Congress Progressive Score.

Progressive Punch calculates these exactly as they should, and there’s no reason to reinvent the wheel on any of them. Their ‘crucial’ scores include “all roll call votes where the margin between yes votes and no votes was narrow and could have been changed by a small group of Democrats voting differently.” We think these votes should be weighted more heavily than others with regards to primaries. It matters far more when a Democrat votes with progressives when a vote is close, than when a bill is certain to pass or certain to fail.

Another key thing Progressive Punch gets right is differentiating between “Lifetime Scores” and “Current Congress” scores. With regards to primaries, we believe weighting the current Congress’s scores more heavily than the lifetime scores is important because it matters far more how each member is voting now than how they’ve voted in the past. Our score takes both lifetime and current congress scores into account — but with a much heavier weight on the current session.

Finally, with regards to Progressive Punch, our algorithm’s weight for current Congress vs. lifetime changes with each day. In order to be  fair to MOCs, the current Congress scores are weighted much less than normal in the very beginning of a new Congress so that a veteran MOC’s score doesn’t tank in the first week of a new session due to one non-progressive vote.

The second score we utilize is the DW-Nominate score. The DW-N method looks at each member’s entire universe of votes and how divergent different representatives’ voting records are from each other. We had to use DW-Nominate to tweak our scores as it’s commonly looked at as the best aggregator, and they include every vote from every member. However, we wanted our score to be a simple 0-100 index and DW-N rates members from -1 (most liberal) to +1 (most conservative).

Since these are different variables than we wanted to use for our score, we changed the range to better fit our algorithm, then cut it in half at the median in each party to gauge moderation. From there, we added as much as 2 points to the member who broke away from the DW-N pack as most liberal in each party, left the Democrats and Republicans near the median of their own party largely untouched, and subtracted as much as 2 points from the most conservative MOCs by party.

The maximum and minimum amounts that the DW-N scores add to or subtract from each member’s existing score is dependent on how close the most conservative member is to zero and the most liberal member is to 100. This is designed to utilize DW-N’s impressive scaling and to help further differentiate the ideological progressives from the ideological conservatives in both chambers.

The final score incorporated is based on party-line votes. Largely, progressive votes follow Democratic legislation and conservative votes follow Republican legislation — but obviously this is not always the case (think CISPA or Patriot Act).

So while the first two scores are based on ideology, we include each member’s 0-100% party-line voting as a way to incorporate partisanship into our final score. We felt partisanship was an important metric for weighing primary challenges because it would be far worse to vote against progressive legislation and against the Democratic leadership than to vote against progressive legislation in step with the Democratic leadership. This concept is obviously fairly subjective and debatable, but we feel that party discipline is an important virtue, and that our score should reflect its importance to legislative politics.


Each Democratic member of Congress will be assigned four scores:

    1. Actual Score
    2. Expected Score
    3. Progressive Value
    4. Primary Score


Actual Score:

This is the 0-100% rating for how this particular member is voting, where zero is conservative and 100 is progressive.

As explained above, this takes into account multiple factors besides just votes for progressive legislation. If a member’s actual score is 80%, it doesn’t necessarily mean that if there are 10 pieces of progressive legislation, this person will vote for eight of them. It’s pretty close, but since we weight crucial votes more heavily than non-crucial votes, current Congress votes more than lifetime votes, and reward party-line voting, it’s a little bit more complicated. We’ll use this score to say things like “Congressman X votes with progressives N% of the time” and while anybody who’s read this far will know this isn’t precisely accurate, it’s probably the best way to convey the meaning in plain English on a general interest political advocacy blog.

Expected Score:

The expected score is a very important piece of our algorithm. We think members of Congress should be evaluated in relation to other MOCs in districts with similar partisan leans – not in relation to our wooly ideals. Essentially, if all the House Democrats who represent D+6 districts are voting around an average of 83.4% progressive based on our actual scores above, then that is how we expect a House Democrat in a D+6 district to vote.

Our expected scores are also split up into 12 different categories and smoothed to a complete line of best fit to accurately predict how each member should be voting.

These 12 categories are:

– Senate Democrats in D+ districts
– Senate Democrats in R+ districts
– Senate Democrats in Even districts
– Senate Republicans in D+ districts
– Senate Republicans in R+ districts
– Senate Republicans in Even districts
– House Democrats in D+ districts
– House Democrats in R+ districts
– House Democrats in Even districts
– House Republicans in D+ districts
– House Republicans in R+ districts
– House Republicans in Even districts

The reason we partitioned our expected scores like this is because we noticed a pretty extreme difference between members of Congress who represent D+1 districts and members who represent R+1 districts and we wanted our scores to reflect that.

It is also important to note that for the very safe Democratic districts (~D+8 and higher) that our expected scores should be viewed as a floor. Since there is no actual threat of these seats flipping to the Republicans, there is no reason why these members (especially those representing D+15 and up) shouldn’t all have actual scores in the upper 90s.

Granted, obviously there’s a margin of error to these scores, and some districts with similar partisan leans are distinct in different ways. That is why our scores are designed to incorporate some extra slack and don’t recommend automatic primaries to these MOCs. Look at a member’s score as a starting point for a conversation about a potential primary challenge, not the final word.

Progressive Value:

The progressive value of a MOC is a very simple: It is the member’s expected score subtracted from their actual score.

If we expect an 80% score from a particular member, and their actual score is a 90%, then they have a +10 progressive value. Inversely, if we expect a 90% from a particular member, and their actual score is 80%, then they have a progressive value of -10. This score is intended to show clearly how positively or negatively each member of congress is voting from a progressive standpoint in one shot, and it is the value used to formulate our primary score.

An important point to note about the progressive value is that it is based on party along with the MOC’s actual and expected scores. By that, we mean that if a Republican has a progressive value of +15 and a Democrat has a progressive value of +5, that does not mean that the Republican is more valuable to progressives than the Democrat is. It simply means that the Republican is voting 15 points more progressive than how we would expect a Republican to vote in that particular seat.

Primary Score:

Primary scores are only assigned to Democratic MOCs,  and are simply based on scores from 0-10, where 0 (or a ✩) means no primary is warranted, and 10 means the MOC must be primaried.

This score is derived by comparing a MOC’s progressive value to their state or district’s partisan lean. At its most basic, if a Democrat has the highest negative progressive value, that will turn into a ’10/10′ on our primary scale, meaning that this member is the most important primary target out of all the representatives and Senators.

There is one exception to this rule: the partisan voting index of that state or district. Our scores have already taken PVI into account while measuring each member’s expected score — so members in more conservative districts will have much lower expected scores.

But the Primary Score takes the state/district’s partisan lean into account once more, so even if a Democrat in a conservative area has the highest negative progressive value, the primary score will likely not give this person a high score, because it takes into account the likelihood that the seat will flip to the Republicans if the incumbent is unseated in the primary.

So in addition to the expected score slack for swing or conservative states and districts, positive primary scores are not normally assigned to Democrats if there is a high risk of the seat changing parties. It is better to have a Democrat in a conservative district voting with progressives 50% of the time rather than a Republican who will vote with progressives 5% of the time.

The amount of additional slack given to Democrats in swing or conservative areas is based on the number of Republicans who hold a seat with their state or district’s particular partisan lean.

So, for example, if there’s one House Republican (out of 435 House members) who holds a D+6 seat, 1/435 = 0.23%, so each House Democrat would receive 0.23% slack on their negative progressive value before they are assigned a positive primary score.

Similarly, if there are 66 House Republicans in R+2’s or more liberal districts, 66/435 = 15.17%, so each House Democrat in an R+2 is given an additional 15.17% of slack before they’re assigned a positive primary score.

There are, however, a few examples of Democrats in swing states or districts who have a primary score attached to them. This is the algorithm’s way of stating that this Democrat votes against progressives so often that it would be worth taking the risk of losing the seat because of how much more good a progressive could do in that seat.

The primary scores also take into account the fact that primary challenges regularly require national progressive support to oust a Democrat who is too conservative for his/her district and is likely well-financed. The algorithm takes this into account by pushing the Democrat with the worst progressive value to the top of our Primary Score (10/10) and it leaves other primariable Democrats back depending on how bad the Democrat with a Primary Score of 10 is.

For example, if the Democrat in the top slot for a primary in the House has a progressive value of -25 and the Democrat with the second worst progressive value is a -11, that’s a fairly large disparity, meaning that the top offending Democrat is much more in need of a primary than the second ranked Democrat. So in order for progressives to prioritize and succeed, the primary scoring system will leave the top offender with a Primary Score of 10/10, while the second could potentially be as low as a 5/10, really highlighting where progressives should focus their energy.


One final note: While we are very proud of our algorithm and feel very confident that it represents members well and successfully identifies those MOCs in need of heightened scrutiny from activists, we know it’s not perfect. We are fully open to tweaking and fine-tuning the algorithm in the future, and welcome any constructive input and suggestions from readers. Please email us if you have any questions or ideas that could improve our method.