Complexity and Social Networks Blog

« Call for papers: ENGAGING DATA First International Forum on the Application and Management of Personal Electronic Information | Main | ISPRAT 1st international government CIO knowledge exchange »

14 May 2009

One rank to rule them all - Notes on benchmarking eGovernment

Almost a decade ago, the EU Commission started to measure the progress of eGovernment in its Member States (now 27) and some other countries. Whenever the new version is published, results usually receives a lot of media attention. Headlines may state "Country X is a leader in eGovernment, it ranked 2nd behind country Y".

Whenever I attend EU conferences that are in some way connected to eGoverment, representatives of Member States like to point out their country's position in the EU eGovernment benchmarking study to underline how far they have come. In fact, whenever politicans or high-level administrators from EU Member States talk about eGovernment, they refer to one particular result the EU eGovernment benchmark--online sophistication Therefore, the benchmark has positively influenced eGovernment policies in EU Member States (MS) and beyond.

Yet, what can the benchmark tell us?

The EU eGovernment benchmark measure 20 public services and the national portal, using four indicators : online sophistication (5-stages), online availability, user centricity and national portals. So in its essence the E-Government benchmark only tells us what is happening on the supply-side of eGovernment in 20 areas. eGovernment, of course, is much more complex than that.

Other eGovernment benchmarks like the one conducted by the United Nations face similar difficulties. How do you measure a complex issue with a limited amount of budget? How do include new trends such as Government 2.0 (Paper / Blog) in a benchmark?

Furthermore, how can you allign benchmarks? They tend differ in scope(EU=20 public services; UN= mix of info society indexes (e.g. from ITU) and eParticipation), underlying cause-effect framework, measures, analysis or transparency of the methodology . Results differ widely and politicians tend to focus on one result. For example, Iceland ranks EU=19, Brown/Brookings=68, UN=21. Why not agree on one cross-financed benchmark and indicators?

The EU and the United Nations are currently revising the eGovernment benchmark methodology. This happens mostly in closed circles of government representatives and experts from academia. While I don't want to criticize this process in general, revising an eGovernment benchmark could be improved by consulting the public (anyone...academics, citizens..) on e.g. how to come up with a framework of measures to capture citizen-centricity, what should be measures or how that should be done. Furthermore, why not make the complete set of data-set available for researchers after 2 years?

Let me close this entry with two recommendations for those involved in redesigning eGovernment benchmarks:

Selecting Measures:
(You should consider this for each measure and any combination of measures)
- Understandibility
- Impact
- Timeliness
- Validity/Accuracy
- Uniqueness
- Comprehensiveness
- Weight
- Collection costs
- Controllability

Scope:
(You can focus on)
- Input
- Process
- Output
- Outcome
- Efficiency (outputs relative to inputs)
- Effectiveness (outcome relative to output and goals)
- Demand
- Usage / Adoption

Posted by Alexander Schellong at May 14, 2009 6:00 PM

.
• Business (3)
.
• Citizen Relationship Management (20)
.
• Commentary (12)
.
• Complexity (2)
.
• Current Trends (59)
.
• eGovernment (24)
.
• Events/Announcements (95)
.
• Innovation (15)
.
• Internet (24)
.
• Knowledge Sharing (22)
.
• Methodology (36)
.
• Networked Governance (22)
.
• Networks in political science (7)
.
• PNG Working Paper Series (6)
.
• Powerlaws (4)
.
• Reviews (2)
.
• Simulations (1)
.
• Small group networks (3)
.
• Small Worlds (3)
.
• SN in the news (76)
.
• Social Capital (6)
.
• Social Computing (15)
.
• Social Finance (2)
.
• Social Networking Platforms (17)
.
• Social Psychology (4)
.
• Social Software (27)
.
• Societal Networks (40)
.
• Technology (45)