Evaluating Democracy Reform Proposals

(Originally posted in https://nmerange.github.io//politics/evaluation/cause%20prioritisation/evaluating-democracy-reforms-working/ ), last updated May 1, 2018

Author: Nick Merange

There are several promising democracy reforms being developed at the moment, some based on technology while some are not. This is not a comprehensive list, but rather an overview of the most significant efforts I'm aware of after being exposed to several as part of Designing Open Democracy. The descriptions, evaluations and summaries here are rough and open to further revision, readers are encouraged to provide suggestions as to what things should be included and excluded in the following summaries or how I might consider adjusting my evaluations and why.

The primary goal here is to provide a rough evaluation of these interventions, their relative strengths and weaknesses in comparison to each other and our existing democracy according to certain criteria of what we're after in a good democratic system. The following comprises a list of interventions I'll be considering here.

Democracy Reforms Being Considered

Tech-based reforms:

Non-Tech Based reforms:

Citizens' Juries

Other reforms not being considered:

Voting Machines
Discussion platforms, such as kialo, argumen, polity, pol.is, etc.
GILT/Prediki's Proposal for a citizen parliament plus some sort of prediction-making ranking system to see who is providing accurate predictions of what's going on to sort information. This isn't being included due to the lack of familiarity with the system
NewVote's proposal, as I am not familiar enough with what it entails, suffice to say it has a mixture of things involved - upvoting/downvoting comments to sort information, use of pol.is's system of idea grouping, and some other stuff
Groups or reforms that I essentially see as more incremental, such as reforms increasing transparency, openness in decision-making, or increasing ease of access to legislators. These reforms may be worthwhile, but likely would not make a significant impact, or would be made redundant if a new system was adopted. These reforms will be more useful the less user-friendly the government decision-making process is.

In principle, any intervention could be assessed using the following criteria. However, it is important to consider which criteria are appropriate and why; we should not necessarily aim for an intervention that does everything, but perhaps a series of interventions that can specialise in certain areas. The most important part of any system though is who gets decision-making power, and how does this manifest?

Summary of Interventions Being Considered

MiVote

Mivote
Issue-Based information aggregation and informed voting

How does it work?

Via app, person opts in/opts out of question being discussed
Presents information on a topic, say, housing, along with different frameworks for understanding the topic
After noting that the person has read some info on the topic (the 'information pack' provided), it gives at least 4 different 'policy approaches' to an issue that could plausibly be taken, person voting picks one

Flux

Flux
Issue-Based Liquid Democracy with tweaks

How does it work?

Each bill in parliament is given a chance to be voted on (In future, it is likely to be possible for people to create their own bills, with votes available for each of those)
Can choose to give your vote to someone else who votes on your behalf, and you can take back your delegation at any time
Can choose to not vote, and instead gain political capital that can be spent later on future votes (How much political capital can be saved, whether it becomes less valuable, and if so over what period is something unresolved right now as far as I can tell)
Plan to implement a system for bidding for political capital, where supply/demand of political capital on a particular vote determines the value of each political capital - the more people are bidding for tokens, the lower the value of each token used, and the higher value given for saving tokens

Online Direct Democracy

Online Direct Democracy
Direct Democracy

How does it work?

Each bill is voted on, for/against
People able to propose their own bills, which are in turn voted on
Online Direct Democracy Reps vote with majority view from votes
Initial bills can be agreed on in principle, then worked out further later on
Pros and cons of each bill can be given

Citizen' Juries

Citizens' Juries

How does it work?

According to NewDemocracy, "a Citizens’ Jury is a group of randomly selected members of a community convened to consider a given topic and provide a response or recommendation to the governing body".

Normally, there is a power to call expert witnesses for testimony, engage in group deliberation and discussion on the basis of information presented, and arrives at a majority viewpoint for how to address a particular issue. In NewDemocracy case it's 80%.

Crucial to this is defining the topic given.

Evaluating Democracy Reform Proposals

Any evaluation of the merits of democracy reform has to be based on what we see as valuable in democracy itself. Without getting into deep philosophical discussion of this here, I will put forth the major principles I think most people would see as valuable and important for democratic decision-making.

Principles underlying Evaluation Criteria - What should good government seek to do?

There are several aspects of Democratic decision-making that are considered valuable: * Effectiveness * Will we get better decisions? (Whether better public policy on the existing information available gets produced, however that's defined. e.g. Does it help us solve problems in the best way, deal with the biggest challenges facing us, and generally make excellent decisions? * How well will it respond to crises? * Freedom * To decide who to vote for or the policy involved * To decide what issues matter * Fairness * Each person gets one vote * Each person's vote is counted and given equal weight * Transparency * Reasoning transparency - people can see why policy X was adopted * Procedural transparency - people can in principle have access to the system process and see what happened at each stage (e.g. when did my representative vote on issue X and how did they vote) * Participation and Inclusion * I want to feel like my values, needs, interests or preferences can be appropriately considered - Each person has a chance to contribute to decision-making - Minorities aren't crushed, nor are they given inappropriate levels of power * Security and privacy * Vote isn't interfered with * Anonymity is assured * Not vulnerable to corruption

Evaluation

Based off these principles, and preceding discussions about creating criteria to determine the value of differing democracy interventions, a list of criteria were created to evaluate democracy reforms. Each intervention was rated out of 10 in each criterion, with a score of 5/10 given for existing Australian Democracy. Thus a score of 5 means that the reform scores about the same on that criterion as existing democracy, with 10 being 'a lot more' than Australian democracy, and 0 being a lot less.

This is to help evaluate the comparative advantage (and disadvantage) of the proposals relative to some reference point that made sense to people. No reform has to be perfect, it just has to be better than what we have. The ratings are rough and plausible, should make comparative sense.

Evaluation of Democracy Proposals

Criteria	Existing Democracy (Australia)	Mivote	Flux	Online Direct Democracy	Citizens' Juries
Will we get better decisions?	5	6.5 (Probably, 65% confident)	4 (Less likely, 40% confident)	5.5 (Slightly better than existing)	8(Effective already, highly (80%) confident)
Responsive to crises?	5	4	6	7	2
Protection from vote fraud and tampering	5	9	7	3	6
Vulnerability to corruption	5	6	6	3	6
Overall Security	5	6.5	5	5	5
Will it crush minorities?	5	6	3	7.5	6
Involvement of population in setting issues to be decided	5	3	8	8	1
Everyone's perspective is included	5	6	5	7	1
Transparency	5	4	4	6	5
Works at a large scale	5	5	7	9	2
Resources required to run (Expensive?)	5	4	3	2	4

Brief notes: * Citizens' Juries have the biggest difference between the effectiveness of decisions made and its other elements, particularly scalability and representativeness * High level of uncertainty on whether Flux and Mivote will get better decisions - Mivote at least unlikely to be worse than existing Democratic decision-making (I don't think existing decision-making is that bad, but policy frames mean that better approaches can be adopted. Only system that explicitly builds in policy and research feed-in)

Major difference I see between Flux and Mivote based off this analysis is that Mivote is a "safer bet" for better and more integrated policy than Flux, whereas Flux is better for getting people what they feel is important to them and their needs represented, at the cost of policy coherence and effectiveness.

Explanation of the numbers given

Will we get better decisions?

I have given Citizens' juries 8 out of 10 because they tend to have a good track record coming up with sensible proposals to deal with complicated problems, see here for some case studies, and I just think it's more likely to do this than any other democracy reform. A weakness of citizens' juries is its ability to allocate resources, which might mean that the recommendations a jury gives might be well the best option, but simply too expensive given other priorities. This can be avoided if there is a requirement that the recommendations cost the same or less than the existing policy. For example, if a citizens' jury is looking at improving the arts industry, they can't just say "Spend more", but must look at the structure, administration and direction of funding. Citizens' juries look particularly promising for addressing complex policy dilemmas, like how to manage the Murray-Darling River Basin in Australia or tax reform.
Mivote I gave 6.5, compared to, say, 5 for existing democracy, because I think it is more likely than the existing democracy to create better decisions on the whole, but not quite as much as citizens' juries. This is because according to this proposal, plausible policy frames are researched, presented and voted on, with the detail of this being resolved presumably by subject-matter experts. That said, although I assume the information given out will also have some hints of what the policy approach taken (say, a humanitarian approach to refugees) might look like in practice should that approach be selected over, say, a "border security" approach, so there will be constraints on the policy design.
Flux I initially gave 5, the same as existing democracy, because it just seemed highly uncertain at the moment whether it would produce better decisions or not. I changed my mind after going through the following reasoning. Generally I think direct democracy produces worse decisions because it is vulnerable to mob mentality leading to policies created in the heat of a moment, that, on reflection, should be dealt with in a more appropriate way. Balancing this direct democracy approach in Flux is that it enables delegation of votes and gives political capital to people who elect to abstain from voting, meaning that specialisation of interest can occur. This said, maybe that will make the system even more vulnerable to mob mentality decisions, as it would discourage undecided or less motivated voters from participating in a decision, which may make decision-making set by people who are most passionate, not necessarily those who are most informed or think it is a difficult issue to resolve and would prefer a more nuanced way to resolve a problem rather than simply being, say, for or against mandatory sentencing laws. So based on this reasoning, I actually think on the whole it is less likely to produce as good decisions as existing democracy, and thus lowered my assessment to 3, or 30% confident you'd get better decisions. I changed my mind again on reading the Feld and Kirchgassner paper, which finds that direct democracy elements in Switzerland seems to allow more informed decisions than representative democracy, or at least decisions more representative of the median voter's views. Based on this, I realised I might be underrating direct democracy's effectiveness and increased the score of Flux on this metric. I'm going to settle with 4 for the moment, because I'm worried about whether the optional voting system combined with the ability to gain political capital will merely incentivise emotional responses being fanned by media for people to spend this capital on their preferred response, rather than encouraging reasoned responses to complex questions. Unsurprisingly, Flux's Co-Founder Max Kaye disagrees, and argues that Flux's system will be better able to respond to mistakes and incentivise good policies being adopted, but I'm not so sure.
I initially gave Online Direct Democracy (ODD) 2 due to scepticism about direct democracy, but having done a very basic google scholar search for 'evidence of direct democracy and decision-making', I found the paper mentioned earlier by Feld and Kirchgassner and some other results that seemed to be positive for direct democracy's effectiveness in Switzerland. My intuition is still that, on a larger scale, direct democracy might not be as effective as existing democracy given that representative democracy has embedded policy-making expertise while direct democracy does not, and also whether the political culture of the polity using the direct democracy is important. This theorising should likely be outweighed by the Swiss experience of direct democracy, so I'm giving Online Direct Democracy a 5.5, expecting it to be slightly better than existing democratic decision-making.

Important to consider as well is the magnitude or 'how much better' the decisions will be, not just the likelihood that on the whole decisions would be better. So 8 out of 10 does not imply that Citizens’ Juries decisions themselves will necessarily be a lot better than existing decisions, just that there is an 80% likelihood that its decisions would be better overall in my view. (If anyone has any research on the magnitude of effectiveness between systems, I'd love to be pointed towards it!)

Responsive to Crises?

Here I have given points based on how quickly each system could respond given an emergency situation. Direct democracy approaches such as ODD and Flux score highly on this, because they can simply ask the question as soon as it happens, vote and decide. A citizens Jury would be very slow at deciding anything, though maybe some of the meetings could be expedited. Still, you need to select, inform, set the question, and give time for deliberation, and then decide. There could be an emergency citizens jury on standby to help speed it up, but overall its slower even than existing democracy, as the representatives are able to make decisions already would know the situation and have the relationships with the bodies (e.g. emergency services) that need to respond. I have given MiVote a 4 for this, and that's possibly generous, because MiVote is meant to undertake an extensive research process before asking the question of what to do, and give at least 4 approaches.

Protection from vote fraud and tampering

Blockchain makes avoiding vote fraud and tampering highly robust, ensuring votes and preferences are accurately recorded without tampering. There is a risk for Flux's approach that stores political capital, as accessing stored cryptocurrency has been able to be exploited by hackers. MiVote does not store capital, so is much more secure. Online Direct Democracy scores low on this for the moment because they do not use blockchain for voting as far as I'm aware, but this could be readily fixed, which would enable it to become as secure as MiVote. Citizens juries are a bit more protected from tampering than existing systems, but more vulnerable than Blockchain methods to tampering, especially if jury panel members are identified publicly.

It is worth considering whether this is in fact a large problem - in certain countries it can be, but frequency of vote tampering such as multiple voting in Australia is very low despite the ease with which it can be done, and it is unlikely to affect elections.

Vulnerability to corruption

There is more to preventing corruption than simply reducing vote fraud - bribery, nepotism, and threats are also tools that can corrupt democratic processes. Existing democracy has been fairly bad at avoiding small-scale corruption, but has managed to avoid large-scale corruption in Australia for the most part. As corruption has occurred, representative democracy has developed requirements for declaring political donations and has introduced or improved oversight mechanisms. This is one area where it might have an advantage over proposed systems, as they haven't gone through this process. MiVote as it stands I believe is a bit more vulnerable to corruption than existing democracy and with greater consequence to its integrity, as it is part of its system that it decides the policy approaches to present through small groups of people, including its own internal committee. This leaves it vulnerable to corruption if there is not adequate transparency and accountability for people on the committee, such as whether they have any conflicts of interest. This I believe could be addressed with transparency and registering od political gifts. Flux doesn't really centralise any aspect of its decision-making, but it's still vulnerable to corruption in my view because it makes votes a tradable commodity. Also, voters can accumulate political capital to be spent (or delegated) later, which becomes potentially a recipe for vote-buying or bribing apathetic and greedy voters. Online Direct Democracy is less vulnerable to either of these risks because it isn't centralised, has no way to give your vote to anyone else, nor can you stockpile political capital for later use. Citizens Juries are easier to corrupt than ODD if it is known which people will be on the jury, because you only need to target a small number of people.

Arguably corruption is not something that can be addressed so much through a democratic decision-making system, but through practices of transparency that are maintained through institutions such as independent media, a strong judicial system, and external bodies keeping track of illicit financial flows.

Overall Security

This is the net positive average of "Protection from vote fraud and tampering" and "Vulnerability to Corruption" scores. Remember, higher protection is good, but higher vulnerability is bad, so it's Protection minus Vulnerability divided by two, plus 5. Online Direct Democracy actually promises the most overall security if it can adopt blockchain voting to address vote fraud and tampering fears, and I believe that could be done with relative ease.

Will it Crush Minorities?

This is not necessarily a democratic concern, but rather a justice or liberal concern. Maybe it isn't a big concern for the reader, or the reader thinks that the minority interest can be protected in other ways, though rule of law and a principle of legal equality. In any case, it is worth looking at whether there is a difference here, and there certainly is. Without a doubt in my mind Flux performs best in this area, because not only does it allow issue-based voting, but also enables voters to gain political capital by not voting and spending it on issues that really matter to them. Online Direct Democracy I gave a low score, based on a 'mob rule' or 'average median voter' models of direct democracy - either way I don't think minority views will hold much sway at all, for better or worse. MiVote faces a similar problem because I think it also goes with the majority answer in terms of the policy approach taken, but minority interests might come into play when the policy is written, depending on who writes it. Citizens Juries are slightly more likely to crush minorities than existing representative democracy as well, because they undertake deliberative decision making as a group, which is selected from a random sample of the population. Probabilities dictate that most people in the jury won't be from minority groups or have beliefs or values that are in the minority It also relies heavily on expertise, which is likely to be more mainstream than minority.

Involvement of population in setting issues to be decided

Here the direct democracy approaches of Online Direct Democracy and Flux do very well, as they both allow anybody to propose legislation in their proposals. This is not the case for Mivote, who to my knowledge do not have a formalised method of selecting a topic on which to research policy approaches. I think MiVote would be happy to investigate whatever issue of the day that was considered most important due to polling. Citizens Juries perform worst in this category because they limit participation to just a small group of people. There's no point at which people get to decide what issues should be discussed, even if they have a citizens' jury to answer that question, it's only a small group who will be involved.

Everyone's perspective is included

Again, Online Direct Democracy performs well against this criterion, as it gives every person a vote on every issue. Flux can work out the same way, but with the possibility of abstaining, it's really not inclusive of everyone's perspective, as Flux thinks it's better that voters ambivalent toward an issue are allowed to abstain and spend their political capital elsewhere. MiVote's proposal also improves on existing representative democracy because each person decides which policy to adopt. Citizens Juries again perform very badly here, because it only involves a small group of people.

Transparency

Here both Flux and MiVote are given lower scores than existing democracy, but for different reasons. In MiVote's case, I gave it a lower score because to me it's not clear how it will select topics or suggest possible policy approaches, and needs a way for this to be clearly communicated to people. Flux is given a lower score because it might not be clear with the saving and delegation of political capital exactly how a certain policy became accepted, and is vulnerable to 'backroom deals' more than any other approach. Online Direct Democracy is the most transparent, as you can see what the bill is you're voting for, and then have the result determined via for or against. Citizens juries can be transparent if they identify why the topic was chosen and how the jury was selected (e.g. if it was stratified to ensure 'population representativeness, how was it stratified). They should always be accompanied by reasoning transparency and a form of Hansard to keep track of proceedings to ensure transparency.

Works at a large scale

I gave both Flux and Online Direct Democracy a high score for this, because both their systems have little management required to maintain at a large scale, a lot less than representative democracy at any rate. I gave MiVote a comparatively lower score than the previous two mentioned, because it requires a committee to select the questions, policy approaches and to research the information to present to voters, which is time consuming and resource intensive. Citizens Juries work on a large scale only insofar as you then risk getting less and less representative samples of a diverse population - imagine a jury of 12 setting legislation for the entire United States, a population of 300,000,000! So it usually is better on a smaller scale. The more jurors added the more difficult it is to have good quality deliberations.

Resources Required to Run/Expensive?

All the suggested interventions are cheaper than existing democracy, to various degrees. Cheapest is Direct Democracy, because there's little bureaucracy required as mentioned in earlier sections. Slightly more bureaucracy is required for Flux's approach and more again for Mivote's research. Citizens Juries are maybe slightly cheaper - they take a decent length of time to run, but citizens don't need to be paid as much, nor do they get the perks that representatives get under existing democracy.

Conclusions

If we're confident that Direct Democracy can create better decisions, then Online Direct Democracy seems to perform best overall. My view is that the political culture that accompanies the direct decision-making will largely determine this effectiveness. The reason I'm pessimistic about this is that scaremongering is a powerful tool, such that good policies may be defeated or bad reforms may be adopted.

What to do with this information?

This was an interesting exercise, but what are we to do with this analysis? The first thought that comes to mind is to determine in what sort of cases each proposal would be best employed, given their relative strengths and weaknesses. In addition to this, what would we do if we were to be interested in investing philanthropic money in this space?

If someone were interested, this would be the key questions they'd need to answer: * How should I weight the criteria? What aspects are more important in a democratic system? * What key uncertainties about the reform could we resolve through experimentation? (e.g. How could we get a higher confidence that the reform will perform to a certain standard for a certain criterion?) * Which reform looks most promising overall? e.g. What could have the most positive impact if implemented well? * What projects are ready to begin right now? * How much would they cost? * What existing information do we have about what these projects have done already? * What are the social costs of failure of each? Which one will allow us to either experiment with a low cost of failure, or with an easy possibility to revert to a previous system?

Such a project could be funded based on a need for systemic change, or could be simulated via a video or board game. Democracy 3 is a video game that already attempts to show the influence of decisions on social impacts, so it could be based on that idea, but with a different voting system.

Overall, it is difficult to assess democracy reform. I hope these criteria might provoke a discussion on working out where to start with research and structured experimentation across democracy reforms to help resolve some of these uncertainties, most importantly to limit the possibility that the system would contribute to catastrophic risks.

Responses

Tom From Flux replied to this with:

Interesting comparison, but can you really evaluate democracy reform without discussing Popper’s Criterion? http://xk.io/2017/05/27/ibdd-and-poppers-criterion/

http://xk.io/2017/05/27/ibdd-and-poppers-criterion/