General guidance for national e-accessibility policymakers based on the ROAMEF cycle

Identify the reasons for developing the policy

Drawing up a new Government policy is a delicate process that requires policymakers to have a keen sense of the reasons for developing or amending a policy, including the level of need. This sense should be based on an accurate understanding of the relevant data, requirements and pressures used to justify amending or drawing up the policy.

Policymakers should consult stakeholders and experts and examine statistical analyses and academic studies. They should also seek technical opinions from specialist civil society organizations and private-sector companies to whom the policy or amendments would apply. The continuous improvement of public services is one of the main drivers that govern the development of national policies.

The reasons used by policymakers to justify their decisions are often tied to national, regional and global dynamics, known as PESTLE factors:

• Political
• Economic
• Sociological
• Technological
• Legal
• Environmental (or sometimes “ethical”).

Policymakers should use a PESTLE^b analysis to structure the target policy. It is also an essential tool for identifying challenges and response mechanisms, as covered in the section on risk analysis.

• Why is the State intervening to introduce this national policy?
• What impact would delaying the policy have?
• What issue/problem/opportunity is the State addressing through this policy?
• What evidence-backed and statistical data are available on this issue/problem/opportunity?
• What are the urgent priorities that must be addressed in this policy?
• What governmental, private-sector and civil society organizations are concerned with the policy? How far do their responsibilities extend?
• What obstacles are there to implementing the policy?
• What types of surveys were carried out during the preparations for drawing up the policy?
• Have efforts been made during such surveys to gather the views of civil society organizations regarding their e-accessibility needs?
• Have efforts been made during such surveys to gather the views of persons with disabilities and their carers regarding their e-accessibility needs?
• Was the private sector surveyed regarding its needs and its views on possible solutions for inclusion in the policy?
• Were experts and specialist researchers involved in the preparations for drawing up the policy?
• Was an analysis carried out of PESTLE factors that could affect or be affected by the solutions proposed in the policy?

Define the objectives

Policymakers often use SMART criteria to identify, evaluate and prioritize their objectives. SMART objectives are:

• Specific
• Measurable
• Achievable
• Relevant
• Time-bound

Objectives are defined based on their short, medium or long term impact.

• What are the desired results of the policy?
• What is the expected impact of Government intervention through the policy?
• What are the performance indicators for the policy at the national level?
• Which segments of society will benefit from the policy objectives, and how?
• Do the objectives of the policy serve the needs and aspirations of its beneficiaries?
• What tools, methods and units of measurement will be used to measure the success of the policy?

Define options and apply weighting

The various possible versions of the policy should be assessed over several steps, most importantly by defining the cost of implementation and the benefits offered by each option.

Policymakers may decide to apply weighting, which involves giving extra weight to certain aspects deemed relevant at the national level, such as the distributive effects of the policy or the extent to which it supports equality. Some of the most important aspects to which weighting may be applied are the cost of implementation and the direct human and financial return in the medium and long term.

The policy should be drawn up only after studying all the implementation risks and obstacles that could have an effect on the desired result.

• What are the possible options and solutions set forth in the policy?
• What numerical data are there to indicate the likely success and return of each possible form of the policy?
• What returns does each proposal offer compared with its outcomes? What weighting is applied to each option?
• Have the entities that will be responsible for implementing the policy been defined?
• Have the responsibilities of each entity involved in implementing the policy been defined?
• Do the approved options for the policy benefit all beneficiaries fairly, equally and without discrimination?
• Has each entity responsible for implementing the policy produced a schedule and financial plan for achieving the policy objectives assigned to it?
• Has each entity responsible for implementing the policy produced a plan to manage the risks that could hinder the achievement of the policy objectives assigned to it?

Monitor

Developing a system to monitor the implementation and practical impact of the policy is an important part of governance activities to guide the pace of implementation.

So why not include an action plan within the policy and skip straight to the monitoring stage? Developing a new policy is a regulatory process; in national regulatory policies, the national regulator recognizes the role of the executive authorities and requests them to draw up action plans in accordance with the monitoring (governance) controls required by the regulatory policy.

A baseline must be set for each action plan undertaken as part of the policy. A baseline also helps to identify and define the starting and target figures for each performance indicator.

The following questions should be asked regarding the process for collecting implementation data:

• Has a start date for implementing the policy been set in line with the priorities in the action plan?
• Has a baseline been set at the policy implementation start date, so that indicators can be measured from that point onwards?
• Have the implementing agencies been trained on the types of consolidated periodic reports (monthly, quarterly, biannually and annually) that they must submit to the regulatory authority?
• Have employees of the regulatory authority been trained in how to use the tools to assess the reports submitted by implementing agencies?
• Are the intended objectives achieved during implementation?
• Are the key performance indicators for the policy objectives improving over time?

Conduct interim evaluation and overall evaluation

Conduct design evaluation and implementation evaluation

When developing new policies, it is best to perform (distinct) formative evaluations of the policy, its objectives and its expected results.^c This helps to identify potential weaknesses in specific areas of the policy during the design process or early in its implementation. Amendments can then be made solely to the area of the policy in question, rather than the entire policy.

Once the policy has been completed, a final comprehensive evaluation^d can be carried out that combines separate formative evaluations with data-based evaluation using the performance indicators tracked during the implementation of the policy by the various implementing agencies. Through this approach, policymakers can evaluate the final outputs of the policy.

After research and analysis, the following questions can be answered:

• Has the action plan been divided into phases and measurable components during the implementation period?
• Have standard values been set for the key performance indicators for each component and objective of the action plan for each implementing agency and for the project as a whole?
• Did Government interventions follow the policy as planned? Which of the data-based indicators set out the action plan to support this assessment?
• In the overall evaluation, were standard values applied to measure policy impact, and if so, to whom? Do they present opportunities for improvement?
• Were the impacts included in the anticipated economic feasibility?

Evaluate results, provide feedback and ensure continuous improvement

Government policies and action plans should not be wrapped up following feedback and final evaluations,^e; instead, they should be subject to a process of continuous improvement.

Policymakers should design policies with a degree of flexibility that allows for amendments, developments and additions to be made in response to continuous improvement requirements.^f By designing flexible policies, continuous improvement can be incorporated within the annual policy review system.

• What was learned during policy implementation?
• How can the lessons learned during policy implementation be used to ensure continuous improvement?
• Was the policy designed with the flexibility to allow it or the priorities set out therein to be modified or changed during implementation or after the final comprehensive evaluation?

As an example of flexibility in policy implementation, national e-accessibility policies can be applied to each sector at different stages, thereby minimizing the risks involved and the cost of overcoming any unforeseen obstacles. By engaging stakeholders in the assessment and amendment processes, such obstacles can be more easily overcome, and an improved version of the policy can be issued for each subsequent stage of implementation, with a view to producing a policy that covers as many sectors as possible and meets the needs of various groups of persons with disabilities while achieving the greatest economic impact and return on investment.^f