How to measure parliamentary openness? (methodology)

How to measure parliamentary openness? (methodology)

How well do various parliament and other representative bodies produce and publish information about their activities? How are voting results, minutes from parliamentary sessions or information about members of parliaments available? And how could the current situation be improved? These are the questions that experts from help to answer.

A necessary condition of functioning democracy is availability of information about activities of various representative bodies, most notably national parliaments. Voters need an abundance of high quality information to make responsible decisions and to control whether elected representatives perform their duties. Parliaments, as well as local representative bodies and other similar institutions vary considerably in a degree to which the produce and publish various data. In this respect, there are several key criteria of good practice:

  • Whether parliamentary data is copyrighted and whether citizens can freely re-used such data,

  • Whether the parliament publishes data proactively and free of charge,

  • Whether parliamentary data is published online (e.g. on an official parliamentary website),

  • Whether parliamentary data is available in machine readable formats

  • Whether the parliament enables advanced use of data, e.g. whether it provides API (application programming interface),

  • Whether parliamentary data is published timely, i.e. well in advanced before relevant events (e.g. before a session) or soon after data is produced (e.g. after a session)

  • Whether parliamentary data is published permanently or whether only information on the current parliamentary term is available

  • Whether the official parliamentary website is easily searchable and whether advanced parametric search queries are possible

  • Whether parliamentary sessions are open to public

  • Whether the way parliamentary data is published discriminate against certain groups of citizens, e.g. language minorities.

It is obvious that the issue of availability and quality of parliamentary data is closely connected to issues of human rights, democratic principles of governance, general accessibility of information, transparency of public finances etc.

Declaration on Parliamentary Openness

International standards in parliamentary data openness is defined by the Declaration on Parliamentary Openness. It is an international drafted mainly by the National Democratic Institute, the Sunlight Foundation and the Latin American Network for Legislative Transparency.

Declaration on Parliamentary Openness. Source:

The Declaration contains some 40 specific articles defining how national parliaments and other representative bodies on the sub-state and super-state levels of government should inform citizens about their activities. The Declaration has been recognized by more than 140 parliamentary monitoring organizations from over 75 countries and by several parliamentary assemblies, including e.g. the Parliamentary Assembly of the Organization for Security and Co-operation in Europe or the Mexican Congress. is one of the parliamentary monitoring organizations that endorse the Declaration. We have translated it into Czech and Slovak and we regularly contribute to a blog about the Declaration and parliamentary openness. We are also a member of an international working group on legislative openness that was established in the Open Government Partnership initiative.

Measuring data openness of representative bodies

When advocating for increased openness, one of the most effective tools is a comparative study of several cases. Therefore, it is useful to have a comparative study of parliamentary data openness across several representative bodies when advocating for a better standard of data openness in one parliament. Moreover, comparative studies generate knowledge about best practices and proven solutions to specific problems.

For these reasons, it is necessary to start a global monitoring of parliamentary data openness across the world Such global comparative research would be similar to monitoring of democracy and civil rights conducted annualy by the Freedom House, to monitoring of corruption byTransparency International or to monitoring of budgetary transparency by International Budgetary Partnership.

There have been several attempts to capture parliamentary data openness in various regions. Most notable examples include a comparative study covering several Latin American congresses by the Latin American Network for Legislative Transparency, a study by a Serbian parliamentary monitoring organization Center for Research, Transparency and Accountability covering Turkey and several Balkans countries or a study covering Eastern European parliaments coordinated by an American organization National Democratic Institute. There has been, however, no truly global research on this issue.

In order to improve this situation, we decided to create a methodology for measuring data openness of representative bodies. It was primarily designed to evaluate data openness of the Regional Assemblies in the Czech Republic but it is easily modifiable for any national parliament or other types of representative bodies. The methodology measures whether and how a representative body publishes six types of data on its official parliamentary website:

  • Minutes from sessions. The methodology distinguishes between brief summaries and word-by-word transcripts,

  • Audio and video recordings of sessions. The methodology captures whether it is only possible to watch live audio or video streams or whether it is also possbile to watch or download older recordings,

  • Voting results. The methodology captures whether voting is recorded by names of individual members of whether only total numbers of members voting for, against or abstaining is published,

  • Invitations to sessions and session agenda. This includes e.g. exact place, date and time of sessions, detailed agenda and other related documents,

  • Texts of legislation, both proposed and adopted,

  • Information on members including e.g. party affiliation, membership in committees and other institutions, information on their education, experience and other biographical data.

When applied to a representative body, the methodology captures data openness of two of its components: plenary sessions and committees sessions. The body can for example record votes by name of individual members only during plenary sessions and not in committees. Individual committees’ data openness can either be measured separately or only one value of openness for all committees that captures usual practice can be produced. Exceptions can then be described separately.

Application of the methodology consists of several steps. First, a researcher gathers information about the six above mentioned types of data. This information is recorded using an electronic survey questionnaire.

The survey questionnaire. Source: The questionnaire can be openned in a new window.

Second, data openness is quantified using a numeric indicator. Data openness is broken down into several mutually independent concepts. These concepts are, among others, quality, usability and completeness of data. Quality of data captures how rich or exhaustive the data is, or how much information is loss from the moment a piece of data is produced to the moment it is published.

For example, voting results aggregated to the level of political parties is not as rich as voting results recorded by names of individual members. In case of the former, information that captures intra-party variation of voting behaviour is lost and cannot be retrieved. Another example is minutes from sessions: a brief summary of the debate is not as rich as a detailed, word-by-word transcript of every speech.

Usability of data captures how easy it is to re-use a given type of data. This boils down to the question of formats used to publish data on the official parliamentary website. The research questionnaire makes a distinction between several categories of formats:

  • Machine unreadable formats such as pictures and scanned PDFs,

  • Machine readable or “native” PDFs that can be searched and copied from,

  • Document files (XLS, DOC or similar formats); the questionnaire makes no distinction between proprietary and non-proprietary formats,

  • Texts on the offical websites (effectively HTML files),

  • Plain text files (CSV, TXT or similar),

  • Enriched formats (XML, JSON or similar).

These formats are ordered from the least open (machine unreadable formats) to the most open (enriched formats). Apart from formats, the questionnaire also captures whether the official website provides bulk downloads and APIs.

The last concept is completeness of data. This concept captures how complete the data is, e.g. whether a representative body publishes results of every vote or only some (or only a few) votes but also how many years is covered by data available on the official website.

Third, each of these three concepts is assigned a certain number of points according to a table below. Then, points of the concepts are multiplied and the product is the value of data openness indicator for the given type of data. For example, a representative body that publishes voting results during plenary sessions and publishes it in the XLS format receives 30 points for this type of data (10 points for quality of data multiplied by 3 points for usability of data). A body that publishes brief summaries of committee sessions in native PDFs is awarded only 2 points (1 point for quality of data multiplied by 2 points for usability of data).

Points for completeness of data are calculated by dividing the number of years for which there is data available by the total number of years the representative body exists. This makes various representative bodies comparable. Points for usability of data are calculated by determining the format of data and then adding 1 point if bulk download is possible and 1 point if there is an API.

Fourth, the number of points obtained for each type of data is divided by the maximum number of points which is 80 for each type of data (10 for quality multiplied by 8 for usability (data is in enriched format and both bulk download and API is provided) multiplied by 1 for completeness (data is perfectly complete). The product ranges from 0 % to 100 % and captures data openness of the given type of data. It can then be aggregated by adding up points for the six types of data described above and dividing the sum by the maximum number of points (80 x 6 = 480 points). As I mentioned above, the methodology is applied separately for plenary sessions and for committee sessions. Therefore, two values are obtained at the end of the process. They can either be reported separately or their average an be taken to express overall data openness of the representative body.

Example: Data openness of the Czech Parliament

Lets use the methodology to capture data openness of the Parliament of the Czech Republic. It is a bicameral national parliament consisting of two chambers: The Chamber of Deputies and the Senate. I gathered information about their data openness using the research questionnaire above. Quality, usability and completeness of the six types of data was captured for plenary sessions and for committee sessions. The following table contains modified answers about the Chamber of Deputies:

All types of data are available since 1993 when the Parliament was established. Data from plenary sessions are much more open than data from committee sessions, especially in terms of their quality. For example, there are word-by-word transcripts and audio recordings from plenary sessions while there are only brief summaries of committee sessions. Also, votes taken during plenary sessions are recorded by names of individual deputies while only total numbers of deputies voting for, against or abstaining are recorded during committee sessions. The following table contains comparable information for the Senate:

Data openness of the Senate is slightly worse. The Senate publishes video recordings of plenary sessions but usability of its data is generally lower. This is mostly due to the fact that while some the Chamber data can be downloaded in bulk in CSV files this option does not yet exists for the Senate data. Committee sessions are again less open than plenary sessions in the Senate. On the positive side, completeness of data in the Senate is also excellent since all types of data are available since 1996 when the Senate was established.

The chart below shows values of data openness indices for various types of data in both chambers. The aggregate indicator of data openness is higher in the Chamber (34 %) than in the Senate (30 %). In both cases, the value of data openness for committee sessions are lower (25 % in the Chamber and 23 % in the Senate) than the value for plenary sessions (44 % in the Chamber and 38 % in the Senate).

Kamil Gregor, 2014, CC-BY 4.0.

No comments