Chapter 3. A multi-phase roadmap for upgrading CompraNet from compliance to integration

The Roadmap developed by the OECD is the product of the recommendations of the Subgroups under the guidance of the Ministry of Public Administration (Secretaría de la Función Pública, SFP), co-ordinated and streamlined by the OECD based on international best practices. Additional guidance is provided on how best to implement the Roadmap, given the many considerations surrounding the execution of a programme of this complexity. This chapter outlines the course of action proposed by the Roadmap and evaluates each action in the light of the other dimensions affected, including technological, legal, procedural and governance considerations. This reinforces the need to apply a holistic approach to implementation that takes into account the implications of each change in the e-procurement system.

  

A comprehensive programme for the CompraNet Roadmap

A multi-phase rollout will set up building blocks for subsequent phases

A phased approach to implementation has generally been advocated by the OECD in other e-procurement implementation projects. This is common in large, transformative projects, as it allows for closer monitoring and control. It also reduces the impact of change on users of the system and other stakeholders, and allows for modifications in plans and approach in response to developments in the political environment or technology (OECD, 2017[1]).

Guidance provided by many development banks on implementing national e-procurement systems suggests that a gradual approach to e-procurement reform is advisable, particularly in countries that have developed their foundations through steady advances in the definition of e-government procurement policies, successive legal reforms and, in general, the use of cumulative processes (Procurement Harmonization Project of the Asian Development Bank, 2004[2]). This study proposed a gradual approach that would divide implementation into four phases, namely:

  1. Preparation: This is a definitive phase for the success of e-government procurement. Three basic prerequisites for start-up are in play: what (vision), who (institutional framework and leadership), and how (action plan).

  2. Tendering: The objective of this phase is to conduct tendering processes within shorter time frames, at a lower cost for both suppliers and the government, with transparency, and with a high impact on development.

  3. Contract management: This phase entails developing the capacity to perform on line, and therefore at lower cost, with high transparency, contract management activities such as monitoring of partial and final delivery of goods, services and works, intermediate and final payment management, and contract variations or adjustments.

  4. Purchasing: The objective of this phase is to develop a directly transactional system. Suppliers may offer their goods and services, allowing public entities to select the best option among them on line, manage supply and receipt, update inventories accordingly, and request and process payment.

The Roadmap for CompraNet was developed with this in mind, taking into account the specific context of Mexico, the current state of the CompraNet system and the lessons learned in other OECD reviews of Mexico’s institutions. The improvements are divided into areas corresponding to the remit of the Subgroups. However, some recommendations have been amended or combined with others to develop a cohesive Roadmap for the system.

  • Phase 1 – Compliance-Driven System: the first phase will run to November 2018, which does not leave sufficient time to introduce significant operational changes. Instead, the focus of the first phase is on increasing human compliance with the system (i.e. making sure that officials upload accurate, complete and timely information and running a permanent audit programme on the information uploaded). It will also focus on marketing the system to suppliers, and developing operational changes that resolve some of the system’s most pressing challenges, that is, reducing the opportunities for corruption (i.e. providing data to focus audits on procurement, and setting up mechanisms to make it possible for whistle-blowers to come forward if necessary).

  • Phase 2 – Open Data Friendly System: the second phase, which will cover a three-year period from the end of Phase 1 (2018-2020) will involve a transition to an open data system, changing the way in which data is entered into the system. The goal is to safeguard the integrity of the information in the system, and allow stakeholders to use the data more easily.

  • Phase 3 – Fully Integrated Transactional System: the third phase, starting at the end of Phase 2 (2020-2027), will make the transition to a fully transactional system. It will link with contracting authority platforms and allow oversight of the full procurement cycle.

Multiple levers can be used to implement changes to CompraNet and the wider procurement system

Changing how procurement professionals operate, particularly when using an e-procurement system, does not necessarily require legislative reform. Many different levers can be employed, each with different levels of flexibility. Many operational changes within the system may be instituted without the need for new legislation.

Of the levers available to Public Procurement Authorities or Central Purchasing Bodies (CPBs) to drive procurement reform, legislation is the least flexible option, although it is typically used to implement it. A report by the United Kingdom’s National Audit Office says regulation “can also create costs for businesses, the third and the public sectors. It can, if overused, poorly designed or implemented, stifle competitiveness and growth” (National Audit Office, 2014[3]). Where regulations are used, tools such as regulatory impact analysis (RIA) and consultation can be used to measure the likely benefits, costs and effects of new regulation and examine changes to existing ones (OECD, 2005[4]).

Figure ‎3.1. Procurement levers by level of flexibility
picture

Source: Courtesy of G. Burr; Chile Compra.

Furthermore, in addition to the impact analysis conducted in advance of legislation, the evaluation of existing laws and regulations through ex post impact analysis is necessary to ensure that they are effective and efficient. The overall burden of complying with regulations tends to increase over time, complicating the daily life of citizens and impeding the efficient operation of business. Ex post evaluation can be the final stage of the regulatory policy cycle, evaluating the extent to which regulations meet the goals they were designed to achieve. It can also be the initial point for understanding the impact, shortcomings and advantages of a policy or regulation, and provide feedback for the design of new regulations. This process could be a beneficial exercise in advance of the forthcoming review of Mexican procurement legislation (OECD, 2014[5]).

Weighing the technology purchasing option that offers the best long-term solution

It is important to note that the Roadmap does not seek to determine at what point (or whether) CompraNet should be migrated to a new platform, and whether the platform should be developed in house, developed externally or bought as an “off-the-shelf” system. This decision will require an analysis of the cost of each option and the capacity for upgrading the current platform in line with the recommendations in the Roadmap.

According to Gartner’s Magic Quadrant eSourcing report of 2015, which evaluates supplier trends in the e-procurement market, 81% of organisations providing input to the report deploy multitenant Software as a Service (SaaS) (a single instance of software running on a server and serving multiple parties). Fourteen percent use a system stored within a private cloud and the remaining 5% have their own, stand-alone installation on the premises (Gartner, 2015[6]). This demonstrates the extent to which SaaS is the most prevalent form of application for organisations selecting e-procurement applications.

Each form has different advantages and disadvantages, depending on the context and requirements of the organisation. A thorough cost/benefit analysis should consider all possible deployment options.

A diverse programme of work will require individuals with specialised skills

To date, the CompraNet project has demonstrated a commitment to establishing effective governance involving multiple stakeholders. This approach should be continued, to ensure that the project has suitable direction and oversight. However, given the multitude of changes (in terms of behaviour, culture, processes, roles, etc.) that a new or heavily amended system would introduce, the team set up to deliver the CompraNet Roadmap should consider structuring delivery within a “programme” as opposed to a “project” structure.

The Roadmap calls for a number of phases, some of which involve activities outside CompraNet (for example establishing training programmes for suppliers and developing template tender documentation). According to the United Kingdom’s government-designed project management discipline, PRINCE2, a project represents a single effort, whereas a programme is a collection of projects that form a cohesive package of work.

To be successful, the programme will require the involvement of a number of different roles in various areas of specialisation. For example, a major system change, such as implementing a universally recognised standard for the classification of goods and services, is likely to have implications for legal aspects, human resources and processes. This must be managed carefully by individuals with the requisite expertise, including experts in managing change within organisations.

A number of activities will require ongoing delivery, both during and after the programme. For example, initiatives to develop capacity will need to be continuously delivered and modified, depending on the state of the system. With each change to the system or processes, system handbooks, guidance and training modules will need to be updated. Given the importance of the capability of users in working with the system, an ongoing system training programme is a worthwhile investment.

Figure ‎3.2. CompraNet Roadmap
picture

Roadmap actions

The actions outlined in the above Roadmap take into account the assessment of the current situation, the future activities of SFP in amending the CompraNet system, and the framework for the implementation of e-procurement systems. The table of Roadmap Actions presented below outlines the time frame in which the Roadmap suggests implementation, a description of each initiative and a short title for easy reference. Note that actions are aligned with Subgroup titles. However, some of the recommendations suggested by multiple Subgroups may have been consolidated into one action. Some actions may have been amended or removed, depending on the recommendations of this report based on international best practice. Recommendations are listed chronologically according to each Subgroup’s alignment and the proposed term of implementation.

Each action is assessed against four “dimensions”, which consider the types of changes necessary to implement the action. The four dimensions are: “Tech” – Technology, “Process” – Processes, “Legal” – Legislation and Regulation, and “Gov” – Governance, Structure and People.

Table ‎3.1. Roadmap actions

ID

Title

Recommendation

Subgroup alignment

Dimensions

Timeframe

Tech

Process

Legal

Gov

1

Consult stakeholders on their information needs

Consult stakeholder groups on their different information needs; these must be understood in order to design effective search and analysis tools, and to ensure databases are appropriately structured.

Information disclosure

Short

2

Data management strategy

Develop a data management strategy for the e-procurement system, ensuring information is comprehensive, readily available, and can be used effectively by stakeholders.

Information disclosure

Short

3

Open data standards

Implement open data practices to normalise public procurement data and enhance accountability mechanisms.

Information disclosure

Medium

4

Naming conventions and referencing standards

Enforce the use of naming conventions and agreed referencing standards to benefit the quality and searchability of data in the system.

Information disclosure

Medium

5

Reduce open data fields

Reduce number of open data fields, to minimise the possibility for data entry to compromise searchability and standardisation.

Information disclosure

Long

6

User friendly interface

Design a user-friendly interface for data access and analysis, and provide proper training for its use. Aim to ensure that ease of use of the system is a key feature.

Information disclosure

Long

7

Article 1 exceptions within the system

Enable mechanisms to allow contracting authorities to use the e-procurement system for so-called Article 1 exceptions.

Information disclosure

Long

8

Identify barriers to supplier participation

Set a work plan to identify, with suppliers, barriers to supplier participation within the CompraNet tool, with special focus on small and medium-sized enterprises (SMEs).

Interaction with suppliers

Short

9

Expand training to improve suppliers’ knowledge of the system’s processes and functionality

Undertake efforts to improve and expand system training for suppliers to improve knowledge of the system’s processes and functionality

Interaction with suppliers

Short

10

Expansion of supplier registry to include prequalification information

Expand the current registry of suppliers to include additional information, such as bank guarantees, insurance certificates, professional qualifications and other documentation requested of suppliers during each tender process.

Interaction with suppliers

Medium

11

Standardised tender documents for common goods and services

Communicate standardised documents within the system for the procurement of commonly used goods and services

Interaction with suppliers

Medium

12

Standardisation of electronic documentation format and requirements

Enable the standardisation and streamlining of documents for procurement practitioners across government, including contract terms, general specifications and reporting requirements.

Interaction with suppliers

Long

13

Transition to best practice standard for the classification of goods, services and public works

Implement classification changes so that goods and services are classified under a best practice standard; ensure regime is effectively applied and used by procurement officials.

Interaction with suppliers

Long

14

Procurement practitioner capabilities

Develop training procedures to increase e-procurement capabilities of the Mexican federal procurement workforce.

Competition and capacity building

Short

15

Identify required legal reforms

Identify legal reforms required to address recommendations for the improvement and expansion of CompraNet.

Competition and capacity building

Short

16

Support services for system users

Provide users of the platform support to maximise the benefits of the system through training and guidance (supported by help desk services), including promoting use of help desk services to those not currently engaging with the service.

Competition and capacity building

Short

17

Audit approach developed by Internal Control Bodies on minimising exceptions

Internal control bodies to ensure CompraNet is always used, except in the case of valid exceptions.

Competition and capacity building

Short

18

Promote an inclusive reform programme

Promote an inclusive, multifaceted programme that aligns with other aspects of procurement reform (such as legal and policy settings, or the private-sector environment).

Competition and capacity building

Medium

19

Contract management module

Build contract management capability by establishing centralised tools, such as a contract-management module.

Competition and capacity building

Medium

20

Open tender exception document disclosure

Establish mechanisms to disclose all public information to support a contracting authority’s request for an exception to an open tender process.

Competition and capacity building

Medium

21

Development of monitoring and control approach

Development of monitoring and control approach to verify that users are using the system correctly.

Efficiency and effectiveness

Short

22

Ensure overall adherence to legislation

An initiative to encourage the consistent, systematised and transparent adherence to procurement legislation, such as in the selection of procurement procedures and use of tender exemptions. SFP should include the updating of the current CompraNet guidelines to address changes made as a result of this review.

Efficiency and effectiveness

Short

23

Targeted audit investigations

Design data standards to allow audit authorities to conduct targeted, as opposed to random, audit investigations

Efficiency and effectiveness

Short

24

Savings measurement methodologies

Implement savings measurement methodologies in relation to the amount of time saved through increased efficiency and by the amount of public and private funds saved.

Efficiency and effectiveness

Medium

25

E-Signature enablement

Apply electronic signature functionality so contracts can be entered into the system in open-data format, removing the need for the scanned PDF format.

Efficiency and effectiveness

Medium

26

Integration with e-government systems

Promote integration of the system with other e-government technologies, such as public finance management, budgeting and service delivery systems, leading to better utilisation of public resources through better information transmission, automation, and increased accountability.

Efficiency and effectiveness

Medium

27

End-to-end system coverage of procurement cycle

Benefit from opportunities to increase efficiency through the expansion of the e-procurement system to cover the whole public procurement cycle, including ordering and invoicing.

Efficiency and effectiveness

Long

28

Promotion of whistle-blowing

Set up mechanisms to facilitate acts of whistle-blowing.

Prosecution of complaints

Short

29

Complaint mechanisms

Develop adequate feedback channels and complaint mechanisms, including specific instructions for filling complaints related to public procurement on the SFP website. Provide specific options or drop-downs in relation to public procurement and eliminate requirements for users to indicate which federal law they wish to complain under

Prosecution of complaints

Medium

30

Ensure adherence to document upload processes

Ensure adherence to document upload processes by increasing user capability and incentivising compliance

Integrity and trust in the tool

Short

31

Complete and reliable data

Implement system changes to promote reliability of data within the system. Such changes can include adding checkpoints and requiring the submission of complete and unchangeable information.

Integrity and trust in the tool

Short

32

Classification of tender information

Develop an agreement on classifying tender information as publicly available or only available to control entities.

Integrity and trust in the tool

Medium

33

Enable block-chain tracking of decisions and documentation

Design mechanisms to allow full traceability of decisions and documentation through public procurement processes.

Integrity and trust in the tool

Medium

34

Tender evaluation module

Develop an evaluation module that can conduct automated evaluation against set criteria and weightings for appropriate tender processes (while also facilitating more subjective evaluation processes).

Integrity and trust in the tool

Medium

Source: Produced by the OECD Secretariat.

References

Animal Politico (2017), La Estafa Maestra: Graduados en desaparecer dinero público, http://www.animalpolitico.com/estafa-maestra/ (accessed on 08 September 2017).

Asian Development Bank (2013), e-Government Procurement Handbook, https://www.adb.org/sites/default/files/institutional-document/34064/files/e-government-procurement-handbook.pdf (accessed on 20 November 2017).

Bolongaita, E. (2017), Mandate without means: Strengthening the OECD Anti-Bribery Convention, Carnegie Mellon University, Australia, https://www.oecd.org/cleangovbiz/Integrity-Forum-2017-Bolongaita-oecd-anti-bribery-convention.pdf (accessed on 18 September 2017).

Cejudo, G. (2012), Evidence for Change: The Case of Subsidios al Campo in Mexico, https://www.internationalbudget.org/wp-content/uploads/LP-case-study-Fundar.pdf (accessed on 20 November 2017).

Coordinación de Estrategia Digital Nacional (National Digital Strategy Coordination) (2017), Datos abiertos en la agenda anticorrupción, https://datos.gob.mx/blog/datos-abiertos-en-la-agenda-anticorrupcion?category=proyectos&tag=%20finanzas-y-contrataciones (accessed on 22 November 2017).

Deloitte (2017), “Growth: the cost and digital imperative The Deloitte Global Chief Procurement Officer Survey 2017”, http://www.deloitte.co.uk/cposurvey2017 (accessed on 08 September 2017).

EBRD; UNCITRAL (2015), Are you ready for eProcurement?, European Bank for Reconstruction and Development; United Nations Commission on International Trade Law, https://www.ppi-ebrd-uncitral.com/index.php/en/component/content/article/427-ebrd-is-launching-a-guide-to-eprocurement-reform-are-you-ready-for-eprocurement (accessed on 14 September 2017).

El Financiero (2017), Gobierno prepara programa para encontrar 'fantasmas' en licitaciones, http://www.elfinanciero.com.mx/economia/gobierno-prepara-programa-para-encontrar-fantasmas-en-licitaciones.html (accessed on 08 September 2017).

Handfield, R., J. Yacura and B. Soundararajan (2017), “How Governance of Data and Technology Drive the Intelligence Spectrum in Supply Chain and Procurement”, https://cdn2.hubspot.net/hubfs/514030/1_Amer_BravoSolution/AMER_2017/2017_Whitepapers/15.%20DataGovernanceReport/2017_WP_DataGovernanceProcurement_EN_US.pdf (accessed on 08 September 2017).

Heggstad, K. and M. Frøystad (2011), The basics of integrity in procurement, http://www.u4.no/publications/the-basics-of-integrity-in-procurement (accessed on 13 September 2017).

Howson, C. (2010), Ease of Use and Interface Appeal in Business Intelligence Tools, March 22, http://www.beyeresearch.com/study/13006 (accessed on 07 September 2017).

OECD (2010), “Collusion and Corruption in Public Procurement”, https://www.oecd.org/competition/cartels/46235884.pdf (accessed on 18 September 2017).

OECD (2011), E-procurement, Brief 17, SIGMA-OECD.

OECD (2011), “Competition and Procurement”, http://www.oecd.org/daf/competition/sectors/48315205.pdf (accessed on 13 September 2017).

OECD (2014), “Recommendation of the Council on Digital Government Strategies”, http://www.oecd.org/gov/digital-government/Recommendation-digital-government-strategies.pdf (accessed on 14 September 2017).

OECD (2015), “Effective Delivery of Large Infrastructure Projects: The Case of the New International Airport for Mexico City”.

OECD (2015), “OECD Recommendation of the Council on Public Procurement”, https://www.oecd.org/gov/ethics/OECD-Recommendation-on-Public-Procurement.pdf (accessed on 14 September 2017).

OECD (2015), “Compendium of Good Practices for Integrity in Public Procurement: Meeting of the Leading Practitioners in Procurement”.

OECD (2016), 2016 OECD Survey on Public Procurement.

OECD (2016), “The Korean Public Procurement Service Innovating for Effectiveness”, http://www.oecd-ilibrary.org/docserver/download/4215251e.pdf?expires=1505401890&id=id&accname=ocid84004878&checksum=335B2E0D525A850CD687D7FC819A97D7 (accessed on 14 September 2017).

OECD (2016), “Towards Efficient Public Procurement in Colombia: Making the Difference”, https://doi.org/10.1787/9789264252103-en.

OECD (2016), “Putting an End to Corruption”, Vol. 3/33, https://www.oecd.org/corruption/putting-an-end-to-corruption.pdf (accessed on 26 September 2017), pp. 23-45.

OECD (2017), “Development and implementation of a national e-procurement strategy for the Slovak Republic”, http://www.oecd.org/governance/procurement/toolbox/search/slovakia-e-procurement-strategy.pdf (accessed on 18 September 2017).

Open Contracting Partnership (2017), Open Contracting Partnership Showcase Projects - Ukraine, https://www.open-contracting.org/why-open-contracting/showcase-projects/ukraine/ (accessed on 07 September 2017).

Open Contracting Partnership (2017), The Open Contracting Journey: Step-by-Step, https://www.open-contracting.org/wp-content/uploads/2017/01/7-steps-guidance.pdf (accessed on 07 September 2017).

PwC; Ecorys (2013), “Identifying and Reducing Corruption in Public Procurement in the EU”, https://ec.europa.eu/anti-fraud/sites/antifraud/files/docs/body/identifying_reducing_corruption_in_public_procurement_en.pdf (accessed on 13 September 2017).

Shkabatur, J. (2012), “Transparency with(out) accountability: Open government in the United States”, Yale Law & Policy Review, Vol. 31/1, http://www.jstor.org/stable/23735771, pp. 79-140.

Subsidios al Campo (2017), About this project, http://subsidiosalcampo.org.mx/acerca-de/sobre-este-proyecto/.

United Nations (2006), UN Procurement Practitioner's Handbook, https://www.ungm.org/Areas/Public/pph/ch04s02.html (accessed on 14 September 2017).

Annex A. System ‘Vision’

The development of a shared vision statement for Mexico’s e-Procurement system (CompraNet) was proposed by members of the Subgroups. The Subgroups led the initiative to develop a document that could serve as a guide for the future state of CompraNet, by declaring a list of strategic objectives for Mexico’s federal e-Procurement system to achieve in the coming years. From its origin, the document was treated as a collaborative product that every Subgroup member could agree upon; this practice was in line with the spirit of collaboration that was a foundational principle of the Working Group. All Subgroups agreed on this initiative, and a first draft of the vision statement was prepared soon after. In total, five draft versions were developed from July to October 2017, and a final statement was presented for the Working Group’s consideration during the sixth Plenary Meeting. The final vision statement received a unanimous vote; the document is included in Chapter 1 of this Review (see Box ‎1.6.).

The first draft document included a list of 50 desired strategic objectives for Mexico’s e-Procurement system. The main objectives of this first draft were kept throughout all draft versions and the final vision statement, which related to an e-Procurement system that:

  • can be used by all levels of government (federal, state and municipal)

  • includes all procurement processes, regardless of the contract awarding method (direct award, restricted invitation, public tender or other extraordinary award methods, such as Article 1 exemptions).1

On 26 July, the OECD as Technical Secretariat of the Working Group took on the task of co-ordinating the development of the vision statement and managing the draft versions, to include comments from the Subgroups and Mexico’s Ministry of Public Administration (SFP). On 23 August, a second version of the draft was shared with SFP, and comments were received from the Unit for Public Procurement Policy (UPCP) and the Unit of Open Government and International Co-operation Policies (UPAGCI). Suggestions from SFP’s units were included in a third draft version and shared with Subgroups for a further review on 6 September. From 7-11 September, the Technical Secretariat received comments on the draft vision statement from Mexico’s Institute for Competitiveness (Instituto Mexicano para la Competitividad, IMCO), the local chapter of Transparency International (Transparencia Mexicana), México Evalúa, and the Business Co-ordinator Council (Consejo Coordinador Empresarial). This combined effort between Subgroups and SFP resulted in the fourth draft of the vision statement, which was shared with SFP on 14 September. The fourth version included a condensed list of 12 strategic objectives for Mexico’s e-Procurement system (reduced from the 50 originally proposed).

After SFP’s review of the fourth draft version on 18 September, the Technical Secretariat shared a final draft version with the Subgroups and SFP on 2 October. Remarks from México Evalúa and SFP’s UPAGCI were incorporated, and a fifth draft version was distributed to attendees before the sixth Plenary Meeting on 9 October. During the sixth Plenary Meeting, the Technical Secretariat presented the final draft version of the vision statement for the Working Group’s consideration. After only one minor suggestion by the Business Co-ordinator Council, the document obtained unanimous support, which resulted in a request to the Technical Secretariat to include the final version in the OECD Review of the Mexican e-Procurement system.

Table A ‎A.1. Timetable for developing a vision statement for Mexico’s e-Procurement System

Proposal for the development of a shared Vision Statement

11-Jul

• The Technical Secretariat (TS) invites Subgroup Co-ordinators (SGCs) to develop a vision statement (VS), following a proposal by IMCO and Transparencia Mexicana.

First draft of the vision statement

18-Jul

Transparencia Mexicana shares the first draft version of the VS.

19-Jul

México Evalúa shares its comments on the draft version with the SGCs.

• SFP (Complaints) shares its comments on the draft version with the SGCs.

• Business Co-ordinator Council shares its comments on the draft version with the SGCs.

23-Jul

• COFECE shares its comments on the draft version with the SGCs.

25-Jul

• IMCO shares its comments on the draft version to with SGCs.

TS’ involvement with the co-ordination of the vision statement

26-Jul

• TS receives a request from SFP to co-ordinate the VS document, as well as the drafting and reviewing process.

Second draft of the vision statement

23-Aug

• TS shares the draft version with SFP for comments.

1-Sep

• SFP (UPAGCI) shares its comments on the draft version with the TS.

4-Sep

• SFP (UPCP) shares its comments on the draft version with the TS

6-Sep

• TS shares the draft version with SFP for comments

Third draft of the vision statement

6-Sep

• TS shares the draft version with SGC for comments.

7-Sep

• IMCO shares its comments on the draft version with the TS.

11-Sep

• Business Co-ordinator Council shares its comments on the draft version with the TS.

México Evalúa shares its comments on the draft version with the TS.

Transparencia Mexicana shares its comments on the draft version with the TS.

Fourth draft of the shared vision statement

14-Sep

• TS shares the draft version with SFP for comments.

18-Sep

• SFP shares its comments on the draft version with the TS.

Fifth draft of the Vision Statement

02-Oct

• TS shares the draft version with SGC and SFP for comments.

04-Oct

México Evalúa shares its comments on the draft version with the TS.

05-Oct

• SFP (UPAGCI) shares its comments on the draft version with the TS.

Final vision statement included in the OECD Review

06-Oct

• TS shares the draft version with SGC for comments.

09-Oct

• TS presents the draft version document to be voted and approved by the Working Group during the sixth Plenary Meeting. The draft version document is approved unanimously, with comments presented by the Business Co-ordinator Council.

10-Oct

• Vision statement is included in the OECD Review of the Mexican e-Procurement System.

Annex B. Subgroup recommendations

As part of the collaborative work undertaken by the Working Group, from February to July 2017 each Subgroup developed a series of recommendations for the expansion and improvement of CompraNet. Recommendations were developed in four steps: information gathering, information analysis/diagnosis, recommendation development (using a standardised format agreed upon by all Subgroups), and recommendation aggregation by the Technical Secretariat. In total, 21 recommendations were considered for this Review.

Steps 1 and 2: Information gathering and diagnosis of the current state of CompraNet

During the first Plenary Meeting, held on 28 February 2017, the Working Group defined a working timetable that involved each Subgroup developing a diagnosis based on the evidence they collected on the current state of CompraNet. These diagnoses, which were then shared by every Subgroup during the third Plenary Meeting on 8 May 2017, were developed using different information sources, such as ad hoc surveys (applied to suppliers, procurement officials and CSOs), document reviews, personal interviews, institutional questionnaires and data analyses. Each Subgroup was free to choose the information-gathering methods that best suited its needs.

The main methods used by Subgroups for gathering information – as inputs for their diagnoses – were as follows:

  • Document review of previous works on public procurement by Mexican and international organisations such as Transparencia Mexicana, México Evalúa, Mexico’s National Institute on Statistics and Geography (INEGI), the World Bank, the European Commission, and the OECD.

  • Ad hoc electronic surveys prepared for businesses (suppliers), public officials, CSOs and journalists. Each e-survey had its own response rate and number of valid answers considered for analysis: suppliers (50 responses), public officials (148 responses), and CSOs and journalists (5 responses each).

  • A paper-based questionnaire addressed to different institutions (COFECE, the Superior Audit Body, and the Co-ordination Unit of Internal Control Bodies of the SFP) was included as part of the information-gathering efforts.

Electronic surveys were one of the most relevant sources of information, as they were designed to reach a broad audience of stakeholders, mainly suppliers and public officials. Some of the main insights obtained through these surveys are:

  • Suppliers: i) SMEs are less likely than large businesses to be awarded a contract.2 ii) SMEs find CompraNet system functions more difficult to use than large businesses.3 iii) SMEs are offered considerably fewer opportunities for training and capacity building in the use of CompraNet than large businesses.4 iv) Bid opportunities, award decisions and clarification meetings are considered by suppliers to be the categories where information in CompraNet is most useful, accessible and adequate. Complaints and audits are ranked last against these measures (see Table B.1). v) One-third of businesses argue they have been victims of corruption but have not filed formal complaints because of a lack of trust in investigating authorities or because they consider the complaints process to be inefficient for whistle-blowers.5

  • Public officials: i) The smaller the procurement unit, the more likely it is for procurement officials to consider electronic storage of procurement documents to be of the same legal standing as maintaining physical archives.6 ii) Procurement officials use CompraNet for some steps of the procurement process more than others, i.e. request for bids are more commonly uploaded to the e-Procurement tool than bid evaluations;7 iii) Large and medium procurement units have received considerably more training for the use of CompraNet than small procurement units.8 iv) Smaller procurement units consider CompraNet to be less effective in reducing integrity risks on public procurement.9

Table A ‎B.1. Information quality ranking
Quality of information in CompraNet regarding information’s sufficiency, accessibility, timely information, and usefulness

Sufficiency

Accessibility

Timely information

Usefulness

Average

Call for bids (n=157)

3.75

3.79

3.62

3.85

3.75

Award (n=156)

3.68

3.64

3.56

3.79

3.66

Clarification meetings (n=156)

3.63

3.67

3.33

3.74

3.59

Proposals and market rates (n=142)

3.39

3.17

3.31

3.59

3.36

Direct award report (n=141)

3.24

3.19

3.11

3.83

3.34

Information of the programme or project for which the procurement process is carried out (n=144)

3.57

3.22

3.11

3.11

3.25

Normativity (n=126)

3.03

2.91

2.84

3.56

3.08

Standards, rules, procedures and authorisations required for recruitment (n=134)

3.09

3.03

2.74

3.14

3.00

Tutorials and training (n=117)

2.86

3

2.76

3.3

2.98

Contracts (n=127)

2.94

2.79

2.9

3.16

2.94

Dispute resolution (n=101)

2.67

2.81

2.77

3.2

2.86

Public Procurement Programme (information on awards and contracts to be awarded) (n=123)

3

2.54

2.72

3.15

2.85

Statistics on public procurement (n=107)

2.8

2.58

2.5

2.94

2.70

Modifying agreements to contracts (n=113)

2.73

2.52

2.52

3

2.69

Information on market research: prices, suppliers, inputs (n=126)

2.48

2.34

2.42

3.46

2.67

Contract management and execution (n=113)

2.61

2.69

2.41

2.87

2.64

Global report on contracting (award and execution of contracts) (n=110)

2.67

2.6

2.41

2.84

2.63

Procurement officers: integrity, experience, performance (n=134)

2.5

2.55

2.47

2.91

2.60

Observations and recommendations of the Superior Audit Body audits (n=95)

2.52

2.52

2.52

2.82

2.59

Observations and recommendations of the Internal Control bodies audits (n=99)

2.36

2.39

2.52

2.89

2.54

Complaints (n=92)

2.5

2.38

2.17

2.96

2.50

Note: Using an ascending scale (with values ranging from 1 to 5, where 1 is the lower value of the scale and 5 the maximum).

Source: OECD analysis on the database for the electronic survey to suppliers, Plural Working Group on Public Procurement.

Based on the results of these diagnoses, each Subgroup was asked to develop a list of recommendations for the expansion and improvement of CompraNet. Subgroup recommendations were expected to acknowledge the current state of CompraNet and address current areas of opportunity to advance Mexico’s e-Procurement system.

Table A ‎B.2. Information-gathering methods, by Subgroup

Subgroup

Subgroup co-ordinator (participants)

Information-gathering method and sources

Information disclosure

México Evalúa (Business Co-ordinator Council, CANACO, CMIC, COFECE, IMCO, INAI, Transparencia Mexicana)

Document review (Transparency metric in public works – MeTrOP; national survey on access to public information – ENAID/INEGI 2016; open data standards for procurement, Transparencia Mexicana; Alliance for Open Government “Public Procurement”); e-Survey (suppliers; procurement officials; CSOs; journalists); paper-based questionnaire (Superior Audit Body; COFECE; Internal Control Bodies-SFP)

Interaction with suppliers

Business Co-ordinator Council, Confederation of Industrial Chambers (CMIC, COFECE, COPARMEX, the Mexican Employers’ Association, IMCO, INADEM, INAI)

e-Survey (suppliers); Analysis (Problem Tree Analysis methodology)

Competition and capacity building

COFECE (Business Co-ordinator Council)

Interview (to the General Director’s Office for Business Intelligence at COFECE); paper-based questionnaire (COFECE)

Efficiency and effectiveness

IMCO (Cámara Nacional de la Industria de Transformación, Business Co-ordinator Council, CMIC, COFECE, INADEM, INAI)

Document review (“Best practice indicators for public procurement”, Government of Scotland; “Compendium of good practices for integrity in public procurement”, OECD; “End-to-end e-Procurement to modernise public administration”, European Commission; “E-Procurement Golden Book of Good Practices”, PwC; “E-Procurement: Opportunities and challenges”, World Bank); e-Survey (suppliers)

Prosecution of complaints

SFP (Business Co-ordinator Council: CANACO, COPARMEX)

e-Survey (suppliers); Analysis (Review to the Comprehensive system of citizen complaints – SIDEC)

Integrity and trust in the tool

Transparencia Mexicana (Business Co-ordinator Council: CMIC, COFECE, CONCAMIN, IMCO, INADEM, INAI)

Interview (stakeholders); analysis (review on CompraNet information; review of current legislation regarding public procurement)

Step 3: Recommendation development

For the formal proposal of recommendations for the expansion and improvement of CompraNet, the Subgroups agreed on the use of a standard format. This format, the Single Format for Observations and Recommendations (SFOR) was first suggested by the Subgroup Interaction with Suppliers co-ordinated by CCE and CONCAMIN (Confederation of Industrial Chambers – Confederación de Cámaras Industriales). The idea to develop a SFOR document was presented to all Working Group members during the third Plenary Meeting on 8 May 2017. On 15 May, the Technical Secretariat shared a first draft for the SFOR template document with the SFP for comments and suggestions, and the Technical Secretariat then shared the revised template draft with Subgroups for comments on 26 May. The SFOR template was agreed upon by all Subgroups, and a clarification meeting between all group co-ordinators regarding the filling of the template document was held on 1 June. The main characteristics and components of the SFOR template document are as follows:

  • includes a cost-benefit analysis as well as an analysis on legal changes that could result from the implementation of the proposed recommendations;

  • provides a section for the description of the justification considered by each Subgroup for the development of recommendations.

Subgroups shared their final SFOR documents with the Technical Secretariat on 12 June. In total, 30 SFOR documents and 51 recommendations were received (Subgroups were free to develop a single SFOR document for each recommendation, or to group several in a single SFOR document).

The Technical Secretariat received the following recommendations from the Subgroups:

Table A ‎B.3. Single Formats for Observations and Recommendations, by sub group

Sub groups

SFORs received

Recommendations received

Main issues addressed in the Sub group’s SFORs

Information disclosure

13

13

Information disclosure; Documents supporting the complete procurement cycle

Interaction with suppliers

14

14

Information disclosure; Information availability for query

Competition and capacity building

1

8

Information disclosure; Information availability fo query

Efficiency and effectiveness

0

0

Analysis of process compliance in CompraNet

Prosecution of complaints

1

4

CompraNet’s complaint module

Integrity and trust in the tool

1

12

Information registry; Information audits; Information disclosure

Step 4: Recommendation aggregation by the Technical Secretariat

Once the SFORs were shared with the Technical Secretariat, the OECD undertook an aggregation analysis on the 51 recommendations received. These recommendations varied, covering a wide range of issues that were later grouped into 21 general themes. The results of this frequency analysis identified two main topics that were prevalent throughout most of the SFORs:

  • broad document and information disclosure for the complete procurement cycle (end-to-end information)

  • development of, and strict general compliance with, open-data standards for the classification of goods and services (similar to the existing Single Classifier for Public Procurement, CUCOP).

Recommendations were aggregated by the Technical Secretariat and shared with the Working Group on 18 July 2017, during the fifth Plenary Meeting. From the Technical Secretariat’s analysis, the original 51 recommendations were grouped into 21 categories. All recommendations expressed by Subgroups are considered within the Technical Secretariat’s shortened version. The analysis also indicates the frequency with which each recommendation was mentioned within all SFORs received, as well as the total number of Subgroups that address each issue. Results of the aggregation analysis are illustrated below.

Table A ‎B.4. Aggregated recommendations

General Recommendation

Frequency of sub groups that mention the recommendation in their SFORs

Frequency with which the recommendation is mentioned in SFORs

1. Information disclosure

4

17

2. Availability of information for query

3

9

3. Information registry/ data capture

3

8

4. Business Intelligence / Market Research

4

6

5. Tender notification

4

6

6. Complete transactional process

4

5

7. End-to-end documentation

3

5

8. Complaints module

2

5

9. Open data standards for the classification of goods and services (CUCOP)

3

4

10. Change log

3

4

11. Information download

3

4

12. Information audits

2

4

13. Supplier registry (RUPC)

2

3

14. Annual planning

2

3

15. Publication of tenders

2

2

16. Creation of new modules

1

2

17. System interconnection

1

2

18. Clarification meetings

1

1

19. Subcontracts and right of collection assignments

1

1

20. Contract execution

1

1

21. Favourable opinion of the Treasury, social services and bank guarantees

1

1

Table A ‎B.5. Timetable for developing Subgroup recommendations on CompraNet

Setting information gathering methods and developing diagnoses on CompraNet

28 February

During the 1st Plenary Meeting sub groups are assigned the task to develop a diagnosis on the current state of CompraNet

March-April

Sub groups define their information gathering methods including e-Surveys, document reviews, interviews and paper-based questionnaires.

8 May

Sub groups share their diagnoses with the Working Group during the 3rd Plenary Meeting

Development and implementation of SFORs

8 May

Interaction with Suppliers Sub groups take a proposal for the development of a SFOR to all Working Group members during the 3rd Plenary Meeting. The proposal was approved by the Working Group and the Technical Secretariat (TS) was assigned the task to co-ordinate and manage the development of a SFOR template document

15 May

TS shares a first draft version of the SFOR template document with SFP for comments

26 May

TS shares a second draft version of the SFOR template document with Sub groups for comments

1 June

TS holds a clarification meeting with all Sub group coordinators to address any doubts regarding the filling of the SFOR template document

12 June

Sub groups share their SFOR documents with the TS for their future aggregation and analysis

Aggregation analysis on recommendations

13 June - 17 July

TS undergoes an aggregation analysis on recommendations shared by the Sub groups in their SFOR documents

18 July

TS shares the final aggregation analysis with the Working Group during the 5th Plenary Meeting

Notes

← 1. Contracts entered into between contracting authorities and other contracting authorities or state-owned entities (for example, universities), under what is known as an Article 1 exemption (Article 1, fifth paragraph, LAASSP & Article 1, fourth paragraph, LOPSRM), are not required (nor allowed) to be entered into CompraNet.

← 2. Contract awards for businesses actively using CompraNet during the last five years, by size of business: micro (43% of micro businesses have not been awarded at least one contract during the last five years), small (44%), medium (25%), big business (6%).

← 3. Ease of use for the different system functions included in CompraNet, by size of business: micro (29% of micro businesses consider CompraNet functionalities easy to use), small (44%), medium (50%), big business (88%).

← 4. Training and capacity-building opportunities for the use of CompraNet, by size of business: SMEs (33% of SMEs have been offered training and capacity-building opportunities for the use of CompraNet), big business (75%).

← 5. Reasons for not filling formal complaints using SIDEC (SFP’s complaint system) against public officials in cases of corruption on public procurement processes: there was no case of corruption to complain about (69%), there was at least one case of corruption to complain about, but businesses do not trust the investigating authorities or consider the complaint process to be inefficient for whistle-blowers (31%).

← 6. Legal validity of electronic archives vs. physical archives for procurement documentation: small procurement units (6.5% of procurement officials at small procurement units consider physical archives to have more legal validity than electronic archives), medium units (13%), large units (17%).

← 7. Use of CompraNet for different steps of the procurement process: request for bids (95% of procurement officials say they use CompraNet for this step), opening of bid proposals (82%), clarification meetings (79%), bid evaluation (61%).

← 8. Sufficient training for the use of CompraNet: small units (58% of procurement officials at small procurement units consider they have not received enough training for the use of CompraNet); medium and large units (46%).

← 9. Based on your experience, has CompraNet had, or not had, effects on integrity risks on public procurement vs. face-to-face methods? By unit size: small units (42% of small procurement units consider CompraNet has had no positive effects on integrity risks compared to previous face-to-face methods), medium and large units (13%).