Statistical Quality Assessment Framework (SQAF)
(based on UN NQAF 2019)

National Statistics Office (NSO)
Ministry of Statistics \& Programme Implementation Government of India

Table of Contents

1.Introduction ….. 3
1.1 Need and benefits of an SQAF ….. 3
1.2 Institutional arrangements for customization of UN-NQAF. ….. 4
1.3 Coverage and Scope ….. 5
2.QualityDimensions ….. 6
2.2 Fundamental Principles of Official Statistics. ….. 8
3. Structure of SQAF ….. 10
3.1 Levels, Principles and Requirements ….. 10
3.1.1 Levels ….. 10
3.1.2 Quality Principles \& Requirements ….. 11
4. Quality Assessment Procedures ….. 22
4.2 Quality Checklist ….. 22
4.3 Scoring procedures for a self-assessment ….. 22
5. Quality Assessment Activities ….. 24
Annexures. ….. 26
Annex A: Important terms ….. 26
Annex B: Reference publications ….. 30

1. Introduction

1.1Need and benefits of an SQAF

1.1.1 Based on the UN National Quality Assurance Framework (NQAF) 2019, the MoSPI envisages to lay down a Statistical Quality Assessment Framework (SQAF) in order to bring improvement in the National Statistical System by laying down quality parameters and corresponding good practices that need to be in place to facilitate and ensure effective management of quality in the National Statistical System and its processes and products.
1.1.2 The SQAF provides a comprehensive framework covering possible aspects starting from the coordination of the statistical system, the legal support and provisions laying down roles and responsibilities of various constituent agencies of the statistical system, and the institutional arrangements and mechanisms to ensure adherence to various Fundamental Principles of Official Statistics covering the environment and the statistical processes involved in the generation of Official Statistics.
1.1.3 The SQAF would enable the data producing Divisions of MoSPI \& other Central Ministries/Departments and State Governments/UT Administrations to carry out internal assessment of their statistical products/processes, in order to identify the gaps and to take remedial measures to bridge these gaps for feasible quality improvement.

Main benefits of having an SQAF (As described in UN-NQAF Manual 2019)

• It provides a generic model for the members of the National Statistical System (NSS) to adopt, develop or revisit their own quality assurance framework;
• It offers a mechanism for the systematic monitoring and ongoing identification of risks and quality issues across the NSS to develop timely corrective measures. It therefore supports quality improvements and their maintenance over time;
• It supports NSS coordination by providing common guidance on quality assurance and reference materials for training;
• It gives greater transparency to the processes by which quality is assured and reinforces the credibility of statistics producers and the coordinating agency (typically the NSO) within the NSS;
• It serves as a common ground to promote dialogue on quality challenges and opportunities at the national, regional and international levels;
• It provides a basis for creating and maintaining a culture of quality within the NSS.

1.2 Institutional arrangements for customization of UN-NQAF

1.2.1 The SQAF follows and aligns with the UN NQAF as described in the UN National Quality Assurance Framework’s Manual for Official Statistics, 2019.
1.2.2 The UN NQAF 2019 Manual calls for the establishment of the necessary institutional arrangements for the development of an NQAF. This includes establishment of a quality task force (or working group) that is responsible for providing strategic direction in the development and implementation of the NQAF and of a quality unit at the NSO that is responsible for documenting the NQAF and for ensuring the resulting quality assessment program is implemented.
1.2.3 MoSPI has a designated Standardization Cell fulfilling the role of a quality unit. The designated Division will be conducting awareness and sensitization programmes for strengthening quality culture in the National Statistical System.
1.2.4 The principles and requirements of SQAF have been constructed by carrying out customization of and modification to the UN NQAF $2019{ }^{1}$ under the guidance of a Task Force and a Steering Committee on Quality Assurance Framework.

1.2.5 Task Force

1.2.5.1 The Task Force was constituted on 21-August, 2020 under the Chairpersonship of the then DG(Statistics), MoSPI, with key stakeholders as members, for developing and recommending for adoption a common quality assurance framework in the Indian context.
1.2.5.2 The Task force after due deliberations recommended for adoption the National Quality Assurance Framework (NQAF) for Statistics. The NQAF, as recommended by the Task Force, was circulated amongst the Divisions of MoSPI as Internal Quality Assessment Framework (IQAF).

¹

1.2.6.2 The Steering Committee on QAF in its several meetings deliberated upon various issues such as deciding the applicability level of the elements of IQAF, viz. NSO level, MoSPI level, Division level and product process level, elaboration of sample elements, etc. Discussions were also made to reduce the subjective biases of the divisions to the extent possible.
1.2.6.3 After series of deliberations, the SC recommended the QAF containing 19 Principles, 85 Requirements ${ }^{2}$ along with 340 elements.
1.2.6.4 Based on the IQAF recommended by the SC on QAF, the SQAF has been developed to be used by statistical agencies across the national statistical system. For ease of understanding and adoption by Ministries/Departments and States/UTs, quality assessment/scoring is to be done at the level of Requirements ( 85 in no.).

1.3 Coverage and Scope

3.1 MoSPI is the nodal Ministry of the country to deal with the statistical matters at the national and international levels. It lays down and maintains norms and standards in the field of statistics, evolves concepts, definitions and methodology of data collection, processing of data and dissemination of results. Other Central Ministries and State Departments which produce and disseminate statistical information also form part of the NSS.
1.3.2 The present SQAF covers the National Statistical System (NSS) at the central level, which comprises the National Statistics Office (NSO), MoSPI and other producers of official statistics such as Central Ministries/Departments and State Governments/UT Administrations.
1.3.3 To begin with, the assessment may be carried out for core statistics which are of national importance and are critical to the development of the economy. The core statistics could be found in the following statistical processes ${ }^{3}$ of the Ministries/Departments:
a) Censuses conducted at national level or in a majority of States
b) Indices compiled at national level or in a majority of States

¹

c) Sample Surveys conducted at national level or in a majority of States d) Administrative statistics (Examples are statistics that could be generated under Companies Act, Societies Act, Factories Act, Mineral Conservation and Development Rules, Registration of Births and Deaths Act, and Land Use Statistics compiled under State regulations etc.)
e) National Accounts Statistics
f) Statistics of Foreign Trade of India and Inter-State trade
g) Statistics in respect of resources including human resources of the country. h) Statistics on the performance of different sectors including social sector infrastructure sectors, financial and external sectors of the economy at all India level
i) Compilation of development indicators
j) Statistics required to be generated due to international commitments from time to time

2. Quality Dimensions

2.1 As per the UN-NQAF Manual 2019, for statistics, a very general definition of quality is fitness for use. This can be operationalized by specifying a set of factors or dimensions that characterize the quality of statistical outputs. These dimensions as given below and illustrated in Figure 1:

• Relevance

The extent to which the statistics satisfy the needs of the users.

• Accuracy

The closeness of estimates to the exact or true values that the statistics were intended to measure.

• Reliability

The closeness of the initially estimated value(s) to the subsequent estimated value(s) if preliminary figures are disseminated.

• Timeliness

The length of time between the end of a reference period (or date) and the dissemination of the statistics.

¹

• Punctuality

The time lag between the release date and the target date by which the data or statistics should have been delivered.

• Accessibility

The ease and conditions with which statistical information can be obtained.

• Clarity

Availability of appropriate documentation relating to the statistics and the additional assistance that producers make available to users.

• Coherence

The ability to reliably combine statistics and data sets in different ways and for various uses. Consistency is often used as a synonym for coherence.

• Comparability

The extent to which differences in statistics from different geographical areas, non-geographical domains, or over time, can be attributed to differences between the true values of the statistics.

Figure 1: Quality Dimensions

The definitions of other key quality related terms are given in Annexure A.

2.2 Fundamental Principles of Official Statistics (FPOS)

2.2.1 It is recognized that official statistics are a public good and that they should be compiled in accordance with certain basic principles, such as professional independence, impartiality, accountability and transparency about methods of collection, compilation and dissemination of statistics. These principles are enshrined in the following United Nations Fundamental Principles of Official Statistics (UN FPOS):

Principle 1

• Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honor citizens’ entitlement to public information.

Principle 2

• To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.

Principle 3

• To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics.

Principle 4

• The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.

Principle 5

• Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on Respondents.

Principle 6

• Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.

Principle 7

• The laws, regulations and measures under which the statistical systems operate are to be made public.

Principle 8

• Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.

Principle 9

• The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all official levels.

Principle 10

• Bilateral and multilateral cooperation in statistics contributes to the improvement of systems of official statistics in all countries.
  2.2.2 The Fundamental Principles were formally adopted by the Government of India through a decision of the Union Cabinet on 4-May 2016 and notified in the Official Gazette on 15th June 2016.
  2.2.3 UN-NQAF quality principles and requirements consider, and are designed to promote, the UN FPOS. Conversely, to ensure adherence to the UN FPOS, implementing the principles of the NQAF is imperative.

3. Structure of SQAF

3.1Levels, Principles and Requirements

3.1.1 Levels

3.1.1.1 The SQAF groups quality principles into four levels, similar to those in the UN
NQAF 2019. These are illustrated in Figure 3 below:

Figure 1: Levels for Quality Principles

3.1.1.2 The logic behind this grouping is as follows. The term quality is interpreted in a broad sense, encompassing all aspects of how well statistical processes and outputs fulfill user and stakeholder expectations. Good quality outputs are statistics that are fit for purpose from the user perspective, more specifically meaning that they are relevant, accurate, reliable, accessible, clear, coherent, comparable, timely and punctual (Level D principles). They are produced by good quality processes, meaning processes that use sound methodology, appropriate procedures, are cost-effective and minimize respondent burden (Level C principles) within institutional environments that incorporate independence, objectivity, impartiality, transparency, confidentiality, data security, adequate resources and quality management (Level B principles), and that are coordinated, with good relationships between producers, users and stakeholders, and use common standards (Level A principles).

3.1.2 Quality Principles \& Requirements

3.1.2.1 Each level contains a set of quality principles specific to that level. There are 19 principles in total. As further detailed in the following sections, associated with each principle

are a number of quality requirements that reflect what is needed to guarantee quality with respect to that principle. There are 85 requirements in total.
3.1.2.2 The principles along with the numbers of associated requirements are given in the chart below.

Number of Requirements

1. Coordinating the national statistical system
2. Relationships with stakeholders
3. Statistical standards
4. Professional independence
5. Impartiality and Objectivity
6. Transparency
7. Statistical confidentiality and data security 8. Quality commitment
8. Adequacy of resources
9. Methodological soundness
10. Cost-effectiveness
11. Appropriate statistical procedures
12. Respondent burden
13. Relevance
14. Accuracy and reliability
15. Timeliness and punctuality
16. Accessibility and clarity
17. Coherence and comparability
18. Metadata
    

3.1.2.3 Quality Requirements

The requirements associated with the various Levels and Principles are defined below.

Level A

The national statistical system comprises the statistical agencies or units within a country that develop, produce and disseminate official statistics on behalf of the national Government, normally with the national statistics office as the leading agency. Ensuring the use of common statistical standards throughout the system is an important part of this management.

Principle 1: Coordinating the national statistical system

Coordination of the work of the members of the NSS is essential for improving and maintaining the quality of official statistics. Principle 1 is supported by principle 8 of the Fundamental Principles of Official Statistics.

Requirement 1.1: A statistical law establishes the responsibilities of the members of the national statistical system including its coordination. Its members are identified in a legal or formal provision.

Requirement 1.2: There are a body and mechanisms for the coordination of the national statistical system for activities at the local, national, regional and international level.

Requirement 1.3: There is a mechanism for considering statistics produced outside the national statistical system, and if appropriate, for these statistics to become official.

Requirement 1.4: There is a national plan or program for the development and production of official statistics.

Principle 2: Managing relationships with stakeholders

The statistical agencies should build and sustain good relationships with all their key stakeholders, including users, data providers, funding agencies, senior government officials, relevant community organizations, academia and the media. The statistical agencies should have access to all data necessary to satisfy the information needs of society in an effective and efficient way. Principle 2 is supported mainly by principles 1 and 5 of the Fundamental Principles of Official Statistics.

Requirement 2.1: Stakeholders are identified and consulted regarding their interests, needs and obligations.

Requirement 2.2: The statistical agencies have a strategy and institutional arrangements to engage with their users.

Requirement 2.3: The statistical agencies continuously maintain and develop cooperation with funding agencies, academic institutions and international statistical organizations, as appropriate.

Requirement 2.4: The national statistics office and, if appropriate, other statistical agencies have the legal authority or some other formal provision to collect data for the development, production and dissemination of official statistics.

Requirement 2.5: The national statistics office and, if appropriate, other statistical agencies have the legal authority or some other formal provision to obtain administrative data and adequate access to these data from other government agencies for statistical purposes.

Requirement 2.6: The national statistics office and, if appropriate, other statistical agencies have the legal authority or some other formal provision and related agreements to access and use data (including “big data”) maintained by private corporations or other nongovernmental organizations for statistical purposes on a regular basis, including for testing and experimentation.

Requirement 2.7: The national statistics office cooperates with and provides support and guidance to data providers.

Principle 3: Managing statistical standards

Standards refer to a comprehensive set of statistical concepts, definitions, classifications and models, methods and procedures used to achieve the uniform treatment of statistical issues within or across processes and across time and space. The use of standards promotes the consistency and efficiency of statistical systems at all levels. Principle 3 is supported by principle 9 of the Fundamental Principles of Official Statistics.

Requirement 3.1: The statistical agencies cooperate in the development and implementation of international, regional and national statistical standards.

Requirement 3.2: The national statistics office provides support and guidance to all data providers and producers of official statistics in the implementation of statistical standards.

Requirement 3.3: Divergences from the international, regional or national statistical standards are kept to a minimum, documented and explained to all stakeholders.

Level B

The quality of the institutional environment is one of the prerequisites to ensuring the quality of statistics.

Principle 4: Assuring professional independence

Statistical agencies should develop, produce and disseminate statistics without any political or other interference or pressure from other government agencies or policy, regulatory or administrative departments and bodies, the private sector or any other persons or entities. Such professional independence and freedom from inappropriate influence ensures the credibility of official statistics. This should apply to the national statistical office as well as to

other producers of official statistics. Principle 4 is supported mainly by principle 2 of the Fundamental Principles of Official Statistics.

Requirement 4.1: A law or other formal provision explicitly declares that statistical agencies are obligated to develop, produce and disseminate statistics without interference from other government agencies or policy, regulatory or administrative departments and bodies, including from within the statistical agencies, private sector or any other persons or entities.

Requirement 4.2: The appointment of the heads of the national statistics office, and other statistical agencies where appropriate, is based on professional criteria and follow transparent procedures. Reasons for dismissal cannot include reasons affecting professional independence. The heads of the statistical agencies are of the highest professional caliber.

Requirement 4.3: The head of the national statistics office and other statistical agencies where appropriate has sole responsibility over the decisions on statistical methods, standards and procedures, and on the content and timing of statistical releases.

Principle 5: Assuring impartiality and objectivity

Statistical agencies should develop, produce and disseminate statistics respecting scientific independence and in a way that is professional, impartial and unbiased, and in which all users are treated equitably. Principle 5 is supported mainly by principle 1 of the Fundamental Principles of Official Statistics.

Requirement 5.1: There is a law or formal provision in force, which is publicly available, and which specifies that statistical agencies should develop, produce and disseminate statistics following professional standards and treat all users in the same way.

Requirement 5.2: The statistical agencies implement a declaration or code of conduct or ethics which governs statistical practices, and compliance with it is followed up.

Requirement 5.3: Data sources and methodologies are chosen on an objective basis.
Requirement 5.4: Statistical releases are clearly distinguished from political/policy statement.
Requirement 5.5: Statistical release dates and times are pre-announced.
Requirement 5.6: In the case that errors are detected, they are corrected as soon as possible, and users are informed about how they affected the released statistics.

Requirement 5.7: The statistical agencies comment publicly on statistical issues, misinterpretation and misuse of official statistics, as appropriate.

Principle 6: Assuring transparency

Statistical agencies’ policies and management practices, and the terms and conditions under which their statistics are developed, produced and disseminated and, if applicable, subsequently revised (including the legal basis and purposes for which the data are required), are documented and available to users, respondents, owners of source data and the public. Principle 6 is supported mainly by principle 3 of the Fundamental Principles of Official Statistics.

Requirement 6.1: The terms and conditions for producing and disseminating official statistics are available to the public.

Requirement 6.2: The terms and conditions for the governance and management of statistical agencies are available to the public.

Principle 7: Assuring statistical confidentiality and data security

Statistical agencies should guarantee that the privacy of data providers (persons, households, enterprises and other data providers) will be protected, and that the information they provide will be kept confidential, will not be able to be accessed by unauthorized internal or external users and will be used for statistical purposes only. Principle 7 is supported by principle 6 of the Fundamental Principles of Official Statistics

Requirement 7.1: Statistical confidentiality is guaranteed by law.
Requirement 7.2: Appropriate standards, guidelines, practices and procedures are in place to ensure statistical confidentiality.

Requirement 7.3: Strict protocols to safeguard data confidentiality apply to users with access to micro data for research or statistical purposes.

Requirement 7.4: Penalties are prescribed for any willful breaches of statistical confidentiality.
Requirement 7.5: Security and integrity of data and their transmission is guaranteed by appropriate policies and practices.

Requirement 7.6: The identification risk of individual respondents is assessed and managed.

Principle 8: Assuring the quality commitment

Statistical agencies should be dedicated to assuring quality in their work, and systematically and regularly identify strengths and weaknesses to continuously improve process and

product quality. Principle 8 is supported by principle 2 of the Fundamental Principles of Official Statistics.

Requirement 8.1: There is a quality policy or a statement of the statistical agency’s commitment to quality, which is publicly available.

Requirement 8.2: The statistical agencies promote a culture of continuous improvement.
Requirement 8.3: There is a specific body responsible for the quality management or the coordination of quality management within the statistical agency, and it receives necessary support to fulfil this role.

Requirement 8.4: The national statistical system staff receives training on quality management.

Requirement 8.5: Guidelines for implementing quality management are defined and made available to the public.

Requirement 8.6: Indicators on statistical output quality are regularly measured, monitored, published and followed up to improve statistical products and processes.

Requirement 8.7: Statistical products and processes undergo periodic reviews.

Principle 9: Assuring adequacy of resources

The financial, human, and technological resources available to statistical agencies should be adequate both in magnitude and quality, and sufficient to meet their needs regarding the development, production and dissemination of statistics.

Requirement 9.1: Financial, human and technological resources are sufficient to implement the statistical work and development program.

Requirement 9.2: Planning and management principles are aimed at the optimal use of available resources.

Requirement 9.3: The statistical agencies’ use of resources is reviewed.

Level C

International standards, guidelines and good practices are fully observed in the statistical processes used by the statistical agencies to develop, produce and disseminate official

statistics, while constantly striving for innovation. The credibility of the statistics is enhanced by a reputation for good management and efficiency.

Principle 10: Assuring methodological soundness
In developing and producing statistics, the statistical agencies should use sound statistical methodologies based on internationally agreed standards, guidelines or best practices. Principle 10 is supported mainly by principle 2 of the Fundamental Principles of Official Statistics.

Requirement 10.1: The methodologies applied by the statistical agencies are consistent with international standards, guidelines and good practices and are regularly reviewed and revised as needed.

Requirement 10.2: The statistical agencies recruit qualified staff and have regular programs to enhance their methodological skills.

Requirement 10.3: Statistical agencies are to choose the data source with regard to accuracy and reliability, timeliness, costs, the burden on respondents and other necessary considerations.

Requirement 10.4: The registers and the frames for surveys are frequently evaluated and adjusted.

Requirement 10.5: The statistical agencies cooperate with the scientific community to improve methods and promote innovation in development, production and dissemination of statistics.

Principle 11: Assuring cost-effectiveness

Statistical agencies should assure that resources are effectively and efficiently used. They should be able to explain to what extent set objectives were attained, that the results were achieved at a reasonable cost and are consistent with the principal purposes of the statistics. Principle 11 is supported mainly by principle 5 of the Fundamental Principles of Official Statistics.

Requirement 11.1: Procedures exist to assess and justify demands for new statistics against their cost.

Requirement 11.2: Procedures exist to assess the continuing need for all statistics, to see if any can be discontinued to free up resources.

Requirement 11.3: Modern information and communication technologies are applied to improve the performance of statistical processes.

Requirement 11.4: Proactive efforts are made to improve the statistical potential of administrative data and other data sources.

Requirement 11.5: The statistical agencies define, promote and implement integrated and standardized production systems.

Principle 12: Assuring appropriate statistical procedures

Effective and efficient statistical procedures underpin quality and should be implemented throughout the statistical production chain. Principle 12 is supported mainly by principle 2 of the Fundamental Principles of Official Statistics.

Requirement 12.1: Statistical processes are tested before implementation.
Requirement 12.2: Statistical processes are well established and regularly monitored and revised as required.

Requirement 12.3: Procedures are in place to effectively use administrative and other data sources for statistical purposes.

Requirement 12.4: Revisions of statistics follow standard and transparent procedures.
Requirement 12.5: Metadata and documentation of methods and different statistical processes are managed throughout the processes and shared, as appropriate.

Principle 13: Managing the respondent burden

Individuals, households or businesses that provide the data upon which statistical products are based are fundamental contributors to the quality of data and information. The requirement to collect data should be balanced against production costs and the burden placed on respondents. Mechanisms to maintain good relationships with providers of data and to proactively manage the respondent burden are essential to improving quality. Principle 13 is supported by principle 5 of the Fundamental Principles of Official Statistics.

Requirement 13.1: The range and detail of requested information is limited to what is necessary.

Requirement 13.2: Mechanisms are in place to promote the value and use of statistics to respondents.

Requirement 13.3: Sound methods including IT solutions are used in surveys to reduce or distribute respondent burden.

Requirement 13.4: Data sharing, data linkage and use of administrative and other data sources are promoted to minimize respondent burden.

Level D

Output quality is measured by the extent to which the statistics are relevant, accurate and reliable, timely and punctual, readily accessible by and clear to users, and coherent and comparable across geographical regions and over time.

Principle 14: Assuring relevance

Statistical information should meet the current and/or emerging needs or requirements of its users. Without relevance, there is no quality. However, relevance is subjective and depends upon the varying needs of users. The statistical agency’s challenge is to weigh and balance the conflicting needs of current and potential users to produce statistics that satisfy the most important and highest priority needs within the given resource constraints. Principle 14 is supported mainly by principle 1 of the Fundamental Principles of Official Statistics

Requirement 14.1: Procedures are in place to identify users and their needs and to consult them about the content of the statistical work program.

Requirement 14.2: Users’ needs and requirements are balanced, prioritized and reflected in the work program.

Requirement 14.3: Statistics based on new and existing data sources are being developed in response to society’s emerging information needs.

Requirement 14.4: User satisfaction is regularly measured and systematically followed up.

Principle 15: Assuring accuracy and reliability

Statistical agencies should develop, produce and disseminate statistics that accurately and reliably portray reality. The accuracy of statistical information reflects the degree to which the information correctly describes the phenomena it was designed to measure, namely, the degree of closeness of estimates to true values. Principle 15 is supported mainly by principle 1 of the Fundamental Principles of Official Statistics.

Requirement 15.1: Source data, integrated data, intermediate results and statistical outputs are regularly assessed and validated.

Requirement 15.2: Sampling errors are measured, evaluated and documented. Nonsampling errors are described and, when possible, estimated.

Requirement 15.3: Studies and analyses of revisions are carried out and used to improve data sources, statistical processes and outputs.

Principle 16: Assuring timeliness and punctuality

Statistical agencies should minimize the delays in making statistics available. Timeliness refers to how quickly – after the reference date or the end of the reference period – the data and statistics are made available to users. Punctuality refers to whether data and statistics are delivered on the promised, advertised or announced dates. Principle 16 is supported mainly by principle 1 of the Fundamental Principles of Official Statistics.

Requirement 16.1: Timeliness of the statistical agency’s statistics comply with international standards or other relevant timeliness targets.

Requirement 16.2: The relationship with data providers is managed with respect to timeliness and punctuality needs.

Requirement 16.3: Preliminary results can be released when their accuracy and reliability is acceptable.

Requirement 16.4: Punctuality is measured and monitored according to planned release dates, such as those set in a release calendar.

Principle 17: Assuring accessibility and clarity

Statistical agencies should ensure that the statistics they develop, produce and disseminate can be found and obtained without difficulty, are presented clearly and in such a way that they can be understood, and are available and accessible to all users on an impartial and equal basis in various convenient formats in line with open data standards. Provision should be made for allowing access to microdata for research purposes, in accordance with an established policy that ensures statistical confidentiality. Principle 17 is supported mainly by principle 1 of the Fundamental Principles of Official Statistics.

Requirement 17.1: Statistics are presented in a form that facilitates proper interpretation and meaningful comparisons.

Requirement 17.2: A data dissemination strategy and policy exist and is made public.
Requirement 17.3: Modern information and communication technology is used for facilitating easy access to statistics.

Requirement 17.4: Access to micro data is allowed for research purposes, subject to specific rules and protocols on statistical confidentiality that are posted on the statistical agency’s website.

Requirement 17.5: Mechanisms are in place to promote statistical literacy.
Requirement 17.6: The statistical agencies have a dedicated focal point that provides support and responds to inquiries from users in a timely manner.

Requirement 17.7: Users are kept informed about the quality of statistical outputs.

Principle 18: Assuring coherence and comparability

Statistical agencies should develop, produce and disseminate statistics that are consistent, meaning it should be possible to combine and make joint use of related data, including data from different sources. Furthermore, statistics should be comparable over time and between areas. Principle 18 is supported mainly by principle 1 of the Fundamental Principles of Official Statistics.

Requirement 18.1: International, regional and national standards are used with regard to definitions, units, variables and classifications.
Requirement 18.2: Procedures or guidelines are in place to ensure and monitor internal, intra sectoral and cross-sectoral coherence and consistency.

Requirement 18.3: Statistics are kept comparable over a reasonable period of time and between geographical areas.

Principle 19: Managing metadata

Statistical agencies should provide information covering the underlying concepts and definitions of the data collected and statistics produced, the variables and classifications used, the methodology of data collection and processing, and indications of the quality of the statistical information – in general, sufficient information to enable the user to understand all of the attributes of the statistics, including their limitations. Principle 19 is supported mainly by principle 3 of the Fundamental Principles of Official Statistics

Requirement 19.1: The metadata management system of the statistical agency is well defined and documented.

Requirement 19.2: Metadata are documented, archived and disseminated according to internationally accepted standards.

Requirement 19.3: Staff training and development programs are in place on metadata management and related information and documentation systems.

4. Quality Assessment Procedures

4.1 The major purpose of quality assessment is to identify quality gaps and potential areas of interventions for quality improvements in the statistical processes and their products and in the organizations within which those processes are conducted. The Statistical Quality Assessment of products/processes of MoSPI and other Central Ministries/Departments and State Governments may be undertaken as a self-assessment.

4.2 Quality Checklist

4.2.1 A Quality Assessment Checklist has been developed as the basis for self-assessment. It is based on the quality principles and requirements. Each requirement in the checklist is a quality aspect and the response during assessment is the degree of conformance to that requirement of the process or organization (institutional environment) being assessed.

4.3 Scoring procedures for a self-assessment

4.3.1 Self-assessment can be applied to a particular statistical production process conducted by the divisions to bring out their statistical products.
4.3.2 Assessment is undertaken based on evaluation of the requirements, the scores for which are used for scoring the principle and ultimately producing an overall score.
4.3.3 Scoring procedures are summarized in Figure 2 below

Figure 2: Scoring Process

Scores at requirement level

For each requirement, full compliance, partial compliance or no compliance to be indicated in the checklist by marking “Yes”, “Partial” or “No”, respectively. A numerical score of 1 would be given in case of full compliance or “Yes”, 0.5 in case of partial compliance while a score of 0 would be given in case of no compliance or “No”. The requirement which does not apply to particular assessment may be marked as “Not Applicable”.

In case, the assessing personnel feels that the requirement is partially fulfilled then they must record it so, however, simultaneously also record the details at what stage the process is so that in the subsequent review it may be scaled up to the next level towards compliance of that requirement.

Note: Since Principle 1 relates to the entire national statistical system, the Ministries/ Departments and States/UTs may not assess their statistical product(s) based on the requirements under Principle 1. In other words, the assessment needs to be done based on the requirements under Principles from 2 to 19.

$\leftarrow$ Scores at Principle level

The simple average of the scores of the underlying requirements multiplied by 100 is calculated. The requirements that are not applicable to a particular assessment are excluded in calculating the average score for a principle.

$\leftarrow$ Overall Score

The simple average of the individual scores of principles is calculated to arrive at overall score for the assessment as a whole. This will be expressed as number between 0 and 100.
4.3.4 The primary aim of a quality assessment is to identify quality problems and quality improvements for dealing with them. Thus, the results of a self-assessment will set out statistical quality improvement plans which will then be reviewed and implemented in accordance with the organization’s priorities and resources available. They will also be used as a starting point for future assessment of the same process or organization.
4.4.5 The results of assessments may also be used collectively to review and refine the assessment process itself and the lists of principles and requirements on which it is based.

5. Quality Assessment Activities

The Quality Assessment activities outlined below:

STEP 1: Baseline/ self-assessment of the Quality of the Selected Statistical Products.

Activities

Assessment of Baseline Quality level may be done for requirements of the IQAF in terms of compliance (Yes $=1$, Partial $=0.5$ and No $=$ 0 ). Requirements which are not applicable may be removed from assessment.

Placing the product specific Quality Factsheets (as per scoring process) before Divisional Head

STEP 2: Setting up of Target Quality Levels to be achieved for the selected products

Activities Setting up of quarterly targets for improving compliances of these requirements

STEP 3: Achieving of Targets of Quality Level for the Selected Statistical Products

Activities Setting up of processes for achieving targeted quality.

Using the identified processes to achieve targeted quality levels.

Placing the product specific Quality Factsheets before Divisional Head on quarterly basis or Secretary/DG level on half yearly basis.

Annexures

Annex A: Important terms

The list below provides definitions of important relevant terms related to SQAF with main reference for the definitions being the UN-NQAF- 2019 Manual.

1. Data and statistics: statistics are numerical information relating to an aggregate of data on units or observations. In general, this Manual uses the term statistics when referring to an output of a statistics production process and the term data when referring to input or possibly throughput in the statistics production process (the term data includes micro data which, depending on the context, can be also an output).
2. Data ecosystem: a system in which a number of actors interact with each other to exchange, produce and utilize data. A system can be understood as a set of connected parts forming a complex whole. There are multiple other definitions of data ecosystem. The United Nations Development Programme model consists of data producers, data objects, infomediaries (i.e., media and other commercial information services) and data users, while other models put the national statistics office-led national statistical system at the center of a system that consists of government agencies, academia and research institutions, the private sector, civil society and international and regional organizations.
3. Data providers and statistics producers: data providers, who provide an input to the statistics production process (such as respondents and holders or owners of statistical, administrative) and statistics producers, who produce a statistical output.
4. Data sources: three data sources according to their purpose and by the entity responsible for their compilation: statistical data sources such as surveys; administrative data sources; and other data sources. In general, other data sources include data sources associated with the term “big data” unless already included, in some instances, in statistical or administrative data sources. New data sources can often be associated with other data sources; however, they may be considered part of statistical or administrative data sources as well, depending on national circumstances.
5. Generic Activity Model for Statistical Organizations (GAMSO): a model that extends and complements the Generic Statistical Business Process Model by modelling additional activities that support statistical production.
6. Generic Statistical Business Process Model (GSBPM): a model that describes the processes used for the production of statistics, including the specification of needs, design,

building, data collection, processing, analysis, dissemination of the products and evaluation of the process.
7. Generic Statistical Information Model (GSIM): an internationally agreed set of definitions, attributes and relationships that describe the pieces of information used in the production of official statistics.
8. Metadata: data that define and describe other data. Structural metadata and reference metadata can be distinguished from each other. Structural metadata define and accompany the data and consist of identifiers and descriptors that are essential for discovering, organizing, retrieving and processing a statistical data set (e.g., titles, subtitles, short descriptions, dimension names, variable names, etc.) Reference metadata are of a more general nature and describe statistical concepts and methodologies used for the collection and generation of data and provide information on data quality, thereby assisting users with the interpretation of the data. Contrary to structural metadata, reference metadata can be decoupled from the data (i.e., they can be generated, collected or disseminated separately from the statistics to which they refer).
9. National quality assurance framework (NQAF): a coherent and holistic system for statistical quality management that assures trust in and the quality of official statistics.
10. National statistics office (NSO): the leading statistical agency within a national statistical system. In general, the NSO has a coordination role within the national statistical system, and is responsible for the development, production and dissemination of official statistics across multiple statistical domains.
11. National statistical system (NSS): the ensemble of statistical organizations and units (statistical agencies) within a country that develop, produce and disseminate official statistics on behalf of the national Government (and other levels of government).
12. Official statistics: statistics that describe, on a representative basis, economic, demographic, social and environmental phenomena of public interest. Official statistics are developed, produced and disseminated as a public good by the members of the NSS in compliance with the Fundamental Principles of Official Statistics and accepted quality frameworks such as the UN-NQAF, as well as other internationally agreed statistical standards and recommendations.
13. Open data: digital data that is made available with the technical and legal characteristics necessary for it to be freely used, reused and redistributed by anyone, at anytime, anywhere.
14. Other statistics producers: entities that do not produce official statistics and are normally not members of the NSS. Other statistics producers have to be distinguished from other producers of official statistics, who are members of the NSS.
15. Principle, requirement, element to be assured: a principle is a general proposition, or procedure, to which statistical agencies and organizations are committed and that will

guide them in meeting their quality-related objectives. A requirement is something needed in order to ensure the implementation of the NQAF – INDIA. An element to be assured is a specific aspect of the UN-NQAF that identifies possible activities, methods and tools to meet the requirement.
16. Quality: the degree to which a set of inherent characteristics of an object fulfils requirements. A simple definition is “fit for use” or “fit for purpose”. It is the users’ needs that define the quality. Different users may have different needs that must be balanced against each other.
17. Quality assessment: the part of quality assurance that focuses on an assessment of how well quality requirements (the stated needs or expectations) are fulfilled.
18. Quality assurance: a planned and systematic pattern of all the actions necessary to provide adequate confidence that a product will conform to established requirements. 19. Quality dimensions: for statistics, the general definition of quality is operationalized by specifying a set of factors or dimensions that characterize the quality of the product.
20. Quality management: the set of systems and frameworks in place within an organization to manage the quality of statistical products and processes. In the case of an NSO and other producers of official statistics, quality management also includes managing the statistical system and the institutional environment, as applicable. Quality management includes quality assurance, but the terms are often used interchangeably; quality management is a more overarching concept, while quality assurance implies a greater focus on concrete actions.
21. Respondent burden: the effort, in terms of time and cost, required for respondents to provide satisfactory answers to a survey.
22. Revision: a change in a value of statistics released to the public. Changes can be the result of errors, but normally the term “revision” is reserved for planned changes in published numbers. Statistics can be revised when more and better source data become available, or due to a change in methodology.
23. Source data: data collected (from respondents, administrative entities and other data providers) by members of the national statistical system to be used in the compilation and production of official statistics.
24. Statistical agencies: members of the NSS, encompassing the NSO and other producers of official statistics. Statistical agencies other than the NSO normally have other main purposes and tasks than the production of official statistics and only a section or a small group of people within the institution produces statistics. The quality requirements for processes and output are the same for all official statistics. However, for a ministry or administrative body where only a part of that body produces statistics, the requirements linked to the institutional environment apply only to the entity producing official statistics. For example, while the ministry or administrative body is typically not independent, the unit

within the ministries/administrative bodies that is responsible for producing statistics should decide on how to produce and when to disseminate its statistics independently.
25. Statistical Data and Metadata exchange (SDMX): an international initiative that aims at standardizing and modernizing (“industrializing”) the mechanisms and processes for the exchange of statistical data and metadata among international organizations and their member countries.
26. Statistical purpose: tasks aimed at developing, producing and disseminating official statistics, including experimenting and testing.
27. Statistical standards: a comprehensive set of statistical concepts, definitions, classifications and models, methods and procedures used to achieve the uniform treatment of statistical issues within or across processes and across time and space.

Annex B: Reference publications

(i) United Nations National Quality Assurance Frameworks Manual for Official Statistics, 2019.
(ii) Quality Assurance Framework of the European Statistical System, 2019
(iii) European Statistics Code of Practice, 2017.
(iv) IMF’s Data Quality Assessment Framework (DQAF), 2010.