Framework for Impact Assessment (IA) and Baseline Study

Impact on Citizens

1.

Key Dimension
of Impact
Economic:
Direct &
Indirect

Indicators
Direct cost to user: travel costs,
travel time, elapsed time for
service delivery, cost of repeated
visits

Extent of reduction of cost of

paying bribes (Overlap between
corruption in economic impact and
governance impact needs to be
addressed)
Rate of errors and time for
recovery

Payment of user fee/processing

charges
Extent of reduction in
data/documents to be submitted

2. Governance:
Corruption,
Accountability,
Transparency,
Participation

Levels of corruption -Specify what

the users perception of corruption
is.
Need for engaging intermediaries
(Include Reason for engaging
intermediary, Distinction between
intermediaries who are an integral
part of the process Vs. touts)
Adherence to a citizens charter:
compliance to committed service
time frame (include awareness of
citizen charter, possibility of agent
influence)
Accountability of Government
functionaries (Mechanism to assess
accessibility)

Questions
Number of trips made for the service
Average travel cost of making each
trip
Average travel time for each trip
Average waiting time in each trip
Estimate of wage loss if any
Payment of bribe to Government
functionaries/agents: Yes/No
Total amount paid in bribes to
Government functionaries/agents
Amount of payments made to agents
to facilitate the service
Any errors in the documents which
required correction: Yes/No
Number of trips required for
correction to be done
Travel cost for the trips
Waiting time in offices for getting
correction done
Estimate of wage loss, if any, in getting
correction done
Total processing fee paid for the task
Total license fee, stamp duty, taxes
paid
Number of documents to be submitted
Cost of preparation of documents in
terms of hours/days
Effort in preparing documents:
Measure on a scale
Measured on a scale
Yes/No

Perception measured on a scale

Degree to which of Government

functionaries can be held accountable:
Measure on a scale

Key Dimension
of Impact

Indicators
Quality and quantity of information
shared by agencies

Questions
Are the rules and procedures clearly
stated without ambiguity and
mistakes: Measure on a scale
Transparency of data: Measure on a
scale
Transparency of decisions: Measure
on a scale
Awareness level and source of
information

3.

Quality of
Service:
Decency,
Fairness,
Convenience,
etc.

Quality and quantity of information

exchange between client and
agencies.
Ability to influence policy,
rules/procedures through
feedback (Include grievance
Redressal)
Quality of problem resolution and
exception handling

Does the agency take responsibility

for the accuracy of information
shared: Yes/No
Has any suggestion or feedback been
provided: Yes/No
Measured on a scale

Was any problem taken for resolution:

Yes/No
Satisfaction with the resolution
process: Measure on a scale

User independence of time and/or

place, 24 x 7 availability

Convenience of location of access

point for service: Measure on a scale
Is the service available 7 days a week:
Yes/No (include reasons of
inaccessibility)
Satisfaction with service timings:
Measure on a scale

Quality of facilities at access points

Measured on a scale

Simplicity of user actions required

for obtaining the service

Measured as a direct indicator

Single window access to several

services (dependant on whether
single window access required by
design)

How many different services are

availed

Overall convenience in obtaining

service
Friendliness in interaction with
Government staff (Including
parameter of dignity/decency with
which client is treated)
Location of service access points
Reliability
Utility

Convenience through a single window

measured on a scale
Measured on a scale
Measured on a scale

Key Dimension
of Impact

Indicators
Extent of protection of privacy

Questions
Any instance of privacy being violated:
Yes/No
Perception of protection of privacy
and confidentiality of data: measure
on a scale

Open ended question:

Fairness, speed of processing, accuracy of transactions, legibility and durability of
certificates and printouts, convenience of obtaining service
Willingness of users to adopt practices in the future
Impact on Agency

1.

Key Dimensions
Economic: Direct
& Indirect

Indicator
Increased revenue collection:
enhanced payee base and
improved compliance

Collection of user fee/

processing charges
Total yearly costs for providing
service: manpower, operating
costs.
Prevention of fraud and
improved cost control

Productivity gains

Question
Total collections over at least 4 years
(2 years prior to application and 2
years after the application)
Payee base over at least 4 years (2
years prior to application and 2 years
after the application)
Perception on compliance: Measure
on a scale (To be supplemented with
qualitative inputs)
Total collection over at least 4 years (2
years prior to application and 2 years
after the application)
Total cost over 3 years after
implementation
Total number of employees over 3
years after implementation
Number of frauds reported in
previous 2 years
Fraud prevention measured on a scale
Ability to control cost measured on a
scale
Number of transactions processed per
month
Average number of transactions per
employee
% reduction in cycle time of key
processes

Accuracy of output/error rate

2.

Governance:
Corruption,
Accountability,
Transparency,
Participation

Transparency of rules,
procedures, data and decision
making

Transparency of data measured on a

scale from perception of a sample of
employees
Transparency of decisions made
measured on a scale from perception
of managers
Measured on a scale from a sample of

Key Dimensions

Indicator

Question
employees

Perception in change of work

and working conditions of civil
servants
Extent of unnecessary
discretion and exercise of gate
keeping role.
Reduction in corruption and
presence of intermediaries

3.

Performance on
Key Noneconomic
Objectives

Strengthening feedback
mechanisms
Degree of employee
involvement in project design,
development & implementation
With reference to the key
organization goals in section I
Ability to define and comply
with a citizens charter
Enhanced basket of services
Improved performance
monitoring and decision
support (account for no. of
services)

Improvements in complaint
handling and problem
resolution

Improved policy formulation

and implementation
Reduced effort in management
and statutory reports

Measured on a scale from perception

of managers
Are intermediaries present: Yes/No
Reduction in corruption measured on
a scale from perception of managers
Measured on a scale from perception
of managers
Measured on a scale from a sample of
employees
Extent to which computerization has
helped in furthering the goals:
measured on a scale
Coverage as % of target population
over at least 4 years (2 years prior to
application and 2 years after the
application)
Number of new services added in the
previous 2 years
Measured on a scale from perception
of managers
Number of complaints over 6 years (3
years prior to application and 3 years
after the application)
Proportion of complaints addressed in
stipulated time (Designed compliance
Vis a vis performance compared to
exception handling)
Is user satisfaction on complaint
handling measured: Yes/No
Satisfaction with complaint handling
measured on a scale from perception
of managers
Measured on a scale from perception
of managers
Measured on a scale from perception
of managers
% of reports that are generated
automatically
Measured on a scale from perception
of managers (To be supplemented
with qualitative inputs)
Transparency of rules and procedures
measured on a scale from perception
of a sample of employees

4.

5.

Key Dimensions
Process
Improvements

Quality of
implementation

Indicator
Reduction in cycle times of key
processes

Integration of services across

agencies and single window
delivery
Process simplification and
reduction in data handling (To
capture redesign of form and
procedures, auto population of
data separately)
Improved record maintenance:
accuracy, consistency, security
and disaster recovery (including
Inclusion of data retrieval,
security policy present)
Data transaction traceability
and audit trails
Process Change
Role Clarity and Degree of
employee-buy-in (Change
management)

Question
Degree of improvement measured on
a scale from perception of managers
(To be supplemented with qualitative
inputs)
Reduction in throughput time
Measured on a scale from a sample of
employees
Measured on a scale from a sample of
employees
Measured on a scale from perception
of managers
Lapses in security recorded in last 2
years
Measured on a scale from perception
of managers
Understanding of ones work
measured on a scale from a sample of
employees
Comfort level with the new system
measured on a scale from a sample of
employees

Stakeholder Consultation
Employee Satisfaction
Adherence to SLAs

Framework for Detailed Assessment (DA)

Impact on Users
All measurements are to be done on the basis of a sample of clients for each major
service availed by the client. Measurements would be done for electronic delivery of
services as well as for the earlier mode of delivery of the same service. In cases where
alternate (non electronic modes) are currently being used by the same set of users in
similar contexts elsewhere, measurement would be recorded for such usage.

Service (Citizen
Perspective)

eGovernance

Key
dimension

Indicators
Awareness

Access
mechanism

Efforts made
Mechanisms
Actual
Complete
Online
Third party

-Different level

-Mobile
-Internet
-Facilitation Centre

Key
dimension

Indicators
-Authorized agencies
Time

Interaction

Government offices
Ease of access
Total time
Waiting time
Turnaround time
Adherence to SLA
Number of visits
Number of channel partners
interacted with
Number of types of channel
partners interacted with
Quality of interaction

Grievance handling
Infrastructure

Information

Complete

-Ease of interaction
-Politeness
-Satisfaction
-Dignity
-How was queue being
managed
-Pro-activeness
-Help desks & call centers
-Availability
-Satisfaction
-Process
-Seating
-Parking
-Accessibility
-Drinking water etc
-Instruction
-Information
-Across channels

Relevant
Consistency across channels
Simple
Language support
Accuracy/updation of
information
Status update/verification
Flexibility of modification
Costs (to the
user)

Delivery of
service

User charges
Agent
Bribe
Opportunity cost?
Cost of access

Integrated
Complete
End to end

-Travel
-Technology
-Preparing documents
-Search cost

Key
dimension

Indicators

Organization and stakeholder (Organization Perspective)

Governance (Citizen Perspective)

Overall perception
Corruption
Trust
Transparency
Privacy
Accountability
Responsiveness
Security
SLA
Use of
connections/
relations
Citizen charters

Technology

Infrastructure
Resources

Benefits
Project
governance

Process before
and after

Existence
Adherence

Awareness
Visibility
Language
Availability
Adherence to standards
Scalability
Versioning
Change management
Inter-operability
Security

Financial
Human

Revenue
Indirect
Leadership
Continuity
Monitoring
Service model
Capacity utilization
Service lead time
Human resource utilization
Cycle time
Record management
Audit trail

-Governance
-Disaster recovery
-Incident handling
-Capacity building
Training
Knowledge
management
-Change management
Pro-activeness
-Employee morale
-Computer awareness

-PPP
-In-house
-Any other