3. Making room for agility: Recommendations for the Slovak Republic

Complex problems have lots of unknowns and potential for things to happen that cannot be predicted. They require adaptive action1 through co-design: designing things with users (not just for them); respecting, valuing and understanding users’ lived experiences and insights; meaningful discussions (not just formal consultation); and many experiments (OECD, 2020[1]). This follows 6 interconnected co-design principles and approaches: (1) define the outcome; (2) understand users; (3) test assumptions; (4) involve users; (5) observe actual behaviour; (6) deliver, test, learn, adapt. (Figure 3.1)

Agile is the ability to create and respond to change. It is a way of dealing with, and ultimately succeeding in an uncertain and turbulent environment. The Agile Manifesto2 (Manifesto for Agile Software Development) was published in 2001. The authors of the Agile Manifesto chose “Agile” as the label for their approach because this word represented adaptability and response to change (Agile Alliance, 2001[2]).

The Manifesto is the end result of several streams of exploration and experimentation that continued throughout the 1990s. There were various practitioners, either people working inside organisations developing software products or consultants helping organisations build software, who thought that they should come up with new ideas for software development as the majority of software failed to actually be useful to real people or were built over budget with missed deadlines.3

During the 1980s in particular, the growing demands and expectations of the marketplace swamped the ability of the software development industry to build software as it was needed. In addition, the widely-held belief that development teams could predict customer needs far in advance – sometimes many years in advance – turned out to be incorrect. So even when working software was delivered, it often did not meet the needs of customers4. Analysis and experimentation conducted during the 1990’s suggested that the so-called waterfall development process was largely responsible for this climate of failure.

The Agile Manifesto outlined a framework for a different approach to the problem. The (then) new agile approach featured:

  • outreach to potential users of software,

  • decomposition of large software projects into much smaller projects that were much less difficult and risky, and

  • empowerment of development teams to respond to evolving requirements.

Agile software development is an umbrella term for a set of frameworks and practices based on the four core values (Figure 3.2) and 12 principles expressed in the Agile Manifesto (Agile Alliance, 2001[2]).

Some of the authors of the Agile Manifesto formed the Agile Alliance in late 2001, a non-profit organisation that promotes software development according to the Manifesto's values and principles. In 2011, the Agile Alliance created the Guide to Agile Practices5 (renamed the Agile Glossary in 20166), an evolving open-source compendium of the working definitions of agile practices, terms, and elements, along with interpretations and experience guidelines from the worldwide community of agile practitioners.

The question is whether agile principles and approaches can be also applied for (ICT) public procurement by governments, and if yes, how.

A project delivered using an agile methodology stresses

  • collaboration,

  • adaptation and flexibility,

  • iterative and incremental development and

  • reviews.

Agile divides a software or system development project into small cycles, often referred to as “iterations”. During each iteration a team works through a full development cycle including planning, requirements analysis, design, coding, testing and review. Fully tested, working software that is capable of being deployed is delivered at the end of each iteration. Subsequent iterations result in additional software that builds upon or complements the software that has already been delivered. As a result, problems can be identified early and on a relatively smaller scale, and can therefore be resolved quickly. The service only “goes live” when there is enough feedback to show the service works for the users and meets their needs. There is a continuous ability to improve and to build a service that meets user needs. (Figure 3.3)

The link between user-centred, design-led approaches and incremental (iterative) methods is critically important in agile ICT procurement throughout the full public spending lifecycle. (Figure 3.4)

Agile methods encourage teams to build quickly, test what they have built and iterate their work based on regular feedback. Agile methods were first implemented in small teams, projects and companies, but during the last few years the usage of agile methods has also been scaled up for use in large system development and distributed software development. However, public agencies and governmental organisations have been slow in adopting agile practices, with the exception of some specific high-tech research organisations. (Jouko Nuottila, 2016[3])

To understand the specificities of applying an agile approach to public procurement, it is worth looking at closer the “traditional” waterfall method where the process is sequential. It starts by gathering requirements, making plans and conducting the procurement process. Then, based on the contract, a product is designed and built. The final stage involves testing and releasing the software to the public buyer. It is only at this end stage in the process where feedback is received from potential users. There is only one chance to get each part of the project right, because there is no returning to earlier stages. The waterfall approach is typical for certain areas of engineering design. In software development, it tends to be among the less iterative and flexible approaches, as progress flows in largely one direction (“downwards” like a waterfall).7 The original waterfall model comprised of five different phases: requirements, design, implementation, verification and maintenance (Figure 3.5) Over time, some variations of the original model emerged, but the logic behind waterfall remained the same: when a phase is completed, its output becomes the input for the next one, which starts immediately after the former.

The waterfall model offers numerous advantages (Kienitz:, 2017[4]) for software developers:

  • The waterfall model provides a structured approach: the model itself progresses linearly through easily understandable and explainable phases;

  • The staged development cycle enforces discipline: every phase has a defined start and end point, and progress can be conclusively identified (through the use of milestones) by both supplier and client. It ensures that each phase is completed before moving on to the next one.

  • This model places emphasis on documentation (such as requirements documents and design documents). In less thoroughly designed and documented methodologies, knowledge is lost if team members leave before the project is completed, and it may be difficult for a project to recover from the loss. If a fully working design document is present, new team members or entirely new teams should be able to familiarise themselves by reading the documents.

  • Through this model, it is possible to estimate the whole project's cost and effort needed right from the start, in the requirements phase.

Despite the seemingly obvious advantages, the waterfall model has received criticism in recent times. The most prominent criticism revolves around the fact that very often, customers do not really know what they want up-front; rather, what they want emerges out of repeated two-way interactions over the course of the project. In this situation, the waterfall model, with its emphasis on up-front requirement capture and design, is seen as somewhat unrealistic and unsuitable for the vagaries of the real world. Further, given the uncertain nature of customer needs, estimating time and costs with any degree of accuracy (as the model suggests) is often extremely difficult. In general, the model is recommended for use only in projects which are relatively stable and where customer needs can be clearly identified at an early stage.8

Another criticism centres upon the model’s implicit assumption that designs can be feasibly translated into real products; this sometimes runs into roadblocks when developers actually begin implementation. Often, designs that look feasible on paper turn out to be expensive or difficult in practice, requiring a re-design and hence destroying the clear distinctions between phases of the traditional waterfall model. 9 Some criticisms also centre on the fact that the waterfall model implies a clear division of labour between “designers”, “programmers” and “testers”; in reality, such a division of labour in most software firms is neither realistic nor efficient.

Delivery models for ICT development projects in most cases follow a similar sequence of distinct phases, from detailed planning, to design, development, testing and integration, and finally deployment of a fully functioning and finished product. Most standard ICT procurement contracts for software and application development set out what is to be delivered (including functional and non-functional requirements of the system), when it is to be delivered and for how much, as well as risk allocation provisions in the event something goes wrong.

This certainty is intended to minimise risk and reflects the obligation that agencies should use public resources in an efficient, effective, economic and ethical manner. However, when an agency's specific system requirements are not absolutely certain, or are likely to evolve over time, locking these down in a statement of work is more likely to result in the delivery of a solution that does not ultimately meet all of the agency's needs.

It may also result in the agency paying for features that it does not require. In many cases, such projects will be delivered late and over-budget, as contract variation processes are deployed to try to capture the agency's changing requirements over time.

Traditional approaches to public procurement typically focus on the tendering stage of the public procurement cycle (see Figure 3.6).

Due to the necessary formalities at the tender stage of a public procurement procedure (which is generally the most regulated), the greatest opportunities to apply agile methods exist in the pre-tender (preparatory and planning) stage, and post-tender or contract implementation and service delivery stage. (Figure 3.7)

However, despite the formalities of the regulated tender stage, it is still possible to benefit from an iterative “deliver, test, learn, adapt” approach (responding to change over following a plan from the Agile Manifesto) to finalise critical elements of a procurement while it is in-flight in the preparatory stage (e.g. requirements, evaluation questions and assessment criteria, terms and conditions).

This is possible as long as at the pre-procurement stage:

  • user-centred co-design led and iterative approach has been taken in the development of these critical elements

  • an empowered multidisciplinary team (individuals and interactions over processes and tools from the Agile Manifesto) has been established, comprised of specialists representing:

    • primary procurement users (buyers and suppliers who need to work together collaboratively through the eventual contracts that are awarded) involved throughout the delivery process

    • the full procurement lifecycle (i.e. policy, sourcing, category management, contract management)

    • legal aspects

    • user-centred design (i.e. user research, content design)

    • agile delivery management

  • work has been shared as openly as possible (ideally publicly, e.g. via official government blogs)

Doing so will minimise the risk of:

  • the needs of primary procurement users not being met, for example:

    • the wrong products, services or capabilities being available to buyers

    • a lack of diversity and capacity in supply, which limits competition (in the case of framework agreements)

    • a form of contract that limits agility, e.g. due to overly prescribed functional and non-functional requirements, deliverables, timescales, outsourcing risk, etc., rather than target outcomes, problems to solve, users’ needs to meet, risk sharing, and standards that govern quality of incremental delivery10, (working software over comprehensive documentation from the Agile Manifesto)

  • the perspectives, biases, assumptions of one public sector profession dominating the design process, at the expense of the other professions who have an interest, and therefore the needs of these secondary procurement users not being met

  • material changes being needed during questions and clarifications, while the formal procurement is in-flight

  • issues arising at the post-procurement contract implementation and service delivery stage, potentially leading to:

    • adversarial buyer-supplier relationships (customer collaboration over contract negotiation from the Agile Manifesto)

    • suboptimal social value for money being achieved

Box 3.1 shows a good example of this user-centred co-design, iterative, open, collaborative and multidisciplinary approach at the pre-procurement stage from the United Kingdom. The Government Digital Service (GDS) and the Crown Commercial Service (CCS) have been using the Digital Marketplace GOV.UK blog to publish procurement plans and timetables, draft service categories, service questions, and terms and conditions, in advance of and during the build up to the formal procurements to deliver framework agreements.

The sequential waterfall approach is necessary to build things like bridges and buildings, however it might be less effective for building and running services when technology changes quickly, like in the ICT sector. Government services especially need to be able to respond quickly to policy changes and the needs of the public (as the recent and still ongoing COVID-19 situation demonstrates).

Using waterfall models, public buyers might run the risk that their service provider spends 18 or 24 months building a service that no longer meets government policy, cannot work with the latest technology and does not meet users’ needs.

Agile methods allow the service provider to quickly make changes while it is building the service. On the other hand, it does not mean that every ICT procurement needs to follow an agile methodology. It should be evaluated on a case-by-case basis whether agile method leads to the expected optimal outcome. In some cases the waterfall model might be a more suitable choice and in some other cases a contract that includes both agile and waterfall model elements might result in the best solution.

“Non-agile” (traditional) procurement still has its place, in particular for stable markets with defined products and services (such as off the shelf products and commodity solutions). It can be suited to projects where requirements and scope are fixed, the product itself is firm and stable, and the technology is clearly understood. For those simple and complicated problems where there is little of discrepancy between what the customer needs and what the market provides.

There can be challenges in agile adoption for public buyers or agencies. One of these that many governmental organisations might face is linked with the fact that public agencies must develop IT systems for implementing digital services related to legislation, such as tax legislation, and they need to reflect any changes to the existing legislation. Therefore, the date when a change in a law comes to effect sets a deadline for the project, which might conflict with agile methods (Jouko Nuottila, 2016[3]). However, this does not mean that agile does not believe in deadlines. Agile focuses on delivering value to the user within the constraints of the available time.

The Agile methodology embraces uncertainty and operates on the expectation of continuously learning and improving in order to prioritise adding value to users. By starting small with phases designed to build understanding through exploration, teams can research, prototype, test and learn about the needs of their users before committing to building a real service, allowing them to fail quickly and correct course in response to what they find. Successfully delivering in this way relies on ensuring that the culture of approaching digital services reflects leadership and vision, understands whole problems, designs services from end-to-end, involves the public and delivers in a multi-disciplinary and collaborative fashion. (OECD, 2021[5])

OECD countries are experimenting with and implementing agile contracting practices for small but also for big or mega projects, such as in the United States, where the Health and Human Services Agency in the State of California, after several success stories with smaller projects, decided that the replacement for a 20-year-old Child Welfare Services case management system would be the test bed for agile project management methodology on a major capital investment. The State of California had already looked for a significant project to which it could apply agile project management methodology and develop software more iteratively. Until then, several agencies had already been experimenting with agile on smaller projects in the State of California, but the Child Welfare Services case management system was the first major capital investment where this approach was used. (Box 3.2) The Child Welfare Services relied on the guidance and support of the US Technology Transformation Services’ Office of Acquisition that introduced agile contract formats to support the greater uptake of agile approaches in procurement, and issued a guide on agile approaches in public procurement via 18F11 that is an office of US federal employees within the General Services Administration (GSA) which collaborates with other agencies to fix technical problems, build products, and improve how government serves the public through technology.

In Slovenia, where many public services are provided through commercial relationships with external suppliers, it was extremely important to draw on external technical expertise to offsetting capacity constraints and limits on the availability of internal skills, in particular to increase capacity in the short- and medium-term. As early as in 2017, the Ministry of Public Administration (MPA) issued Guidelines on Procuring IT Solutions with advise using the procurement process to prioritise agile solutions and ensure an inclusive approach to testing services (Republic of Slovenia, 2017[6])

An agile approach calls for the implementation of more open, inclusive, iterative and cyclical approaches in the procurement of ICT products and services. This means, for instance:

  • Bringing on board all of the relevant stakeholders to jointly design projects and define ICT project priorities together;

  • Creating marketplaces to facilitate the pooling of suppliers, such as the supplier portals in UK, Australia, New Zealand;

  • Monitoring and reporting on early results in an iterative fashion, and

  • The revision and redesign of the project implementation process when needed.

Clear ICT procurement frameworks and practices are fundamental for the successful implementation of national digitalisation programmes. Strategic planning of ICT procurement facilitates strategic decision making, efficiency, effectiveness and sustainability of public ICT investments, and helps avoid gaps and overlaps. Having strategic planning methods and formal guidelines in place helps governments to overcome “agency thinking approaches” that usually anchor silo-driven decisions, while often failing to prioritise interoperability or common standards for improved integration and sharing across different sectors and levels of government. (OECD, 2020[7]) (OECD, 2020[8])

OECD countries like the United Kingdom are leading a new whole-of-government perspective that places iteration at the core of the ICT procurement cycle. This requires a cross-cutting approach supported by the development of common standards for ICT project development, management and evaluation, and agile monitoring and control.

An agile environment relies on new forms of collaboration between the public sector and non-governmental actors, thus acknowledging the benefits of involving all relevant players early in project planning and development as a means to ensure that ICT projects comply with central standards and take into consideration the needs of the end user. The goal is to ensure that public funds are invested in ICT projects that create benefits for the public sector, businesses and citizens and help build a capable and responsive public sector. New Zealand and the United Kingdom have adopted flexible approaches in terms of supplier selection, establishing marketplaces that help suppliers apply to specific project calls easily, cutting costs across government and creating a more dynamic market environment.

For governments to successfully take advantage of what agile can offer in digitalisation, a change in mind-set for procurement officials is required. Individuals and organisations need to change their ways of working when they start to adopt agile practices. In public organisations, there is an established formal mode of operation, which creates a challenging environment for adopting agile methods. Consequently, a public organisation might even need to revisit its underlying organisational values and culture to be able to adopt agile methods successfully. (Jouko Nuottila, 2016[3])

The procurement officials and decision makers at the contracting authorities need to embrace a new way of thinking about their role. The agile process combines design with development and user acceptance. The final design, the final product emerges through a collaborative effort between developers and users. The traditional procurement approach, heavy on functional specifications written up front, is not consistent with the agile approach The new mind-set of procuring for agile involves many major shifts in thinking (John O’Leary, 2017[9])12. (Figure 3.8)

Capacity building of the staff is identified as one of the key tasks for ensuring the successful adoption of agile methods in public procurement. However, formal training is not enough; people should understand and learn agile values and principles in addition to existing practices to be motivated and committed (K. Conboy, 2011[10]). Incentives, psychological motivators play a significant role, together with abilities to cope with and manage change, in adopting new technologies and methods (Cormican, 2015[11]). Ideally, the capacity building on agile practices in public procurement of digital should be extended beyond procurement professionals to the delivery and corporate functional teams. (OECD, 2021[12])

The 18F guide from the United States defines agile as the follows:

“Agile is something you are, not something you do. Agile is not a checklist, or a methodology, or a series of rituals. Agile is a way of thinking and a way of attacking problems. Embrace mistakes, learn, and keep trying. Mess up and learn again and again and again. Cut your losses. Fail forward fast. It’s okay. You won’t get fired. You’re learning. That is agile.”13  

Agile approaches require greater investments in the pre-tender stage of the public procurement cycle, in terms of preparing and planning the whole tender to ensure the realisation of benefits in the longer term, and better management of the procurement cycle as a whole. In the pre-tender stage, contracting authorities conduct preliminary market engagement activities, assessing the real needs of the public organisation and users, evaluate the different solutions, and choose among different technological solutions, including justifying the need for the tender.

Timescales can significantly influence procurement outcomes. Too often documents are rushed out or suppliers are given inadequate time to respond to complex requirements. A bit of forward-planning can go a long way to ensuring the procurement itself is done in a timely manner. Once a need has become clear (e.g. through "needs assessment"), even if all the details and budget have not yet been decided, there is scope to start analysing the market and identifying suitable procedures. Consulting other public or private organisations who have procured similar needs can also be a valuable use of time in the run up to a formal procedure being launched.

For ICT goods and services, this stage is even more relevant as foundation for decision-making, given the diversity of technological alternatives and modes of answering to the needs of the beneficiaries and end-users. Answers need to be given to important initial procurement questions such as choice between service versus supply (e.g. lease of computers, software), contract versus framework agreement, duration – all important decisions with impact on the attractiveness of the tender, the competition for the contract, the price of purchase and many other factors.

Another key feature for agile procurement approaches is the involvement of different players at the different stages of the ICT project development and procurement cycle, including the contract implementation. The OECD Recommendation of the Council on Public Procurement (OECD, 2015[13]) emphasises the importance of involvement of all interested stakeholders in the process. Proactive and adequate disclosure of information throughout the procurement cycle is critical to support a level playing field for suppliers to compete for government contracts and to support citizens’ involvement in the oversight of government operations.

Early market engagement is a strategic and collaborative approach for public buyers to gather valuable market intelligence in relation to the high level aims of large scale investment programs. Government officials can benefit from suppliers’ knowledge of markets and trends and understand the capability and capacity of suppliers prior to formulation of a procurement plan and strategy. Both the contracting authority and the ICT suppliers can benefit from early engagement with the market. Benefits include:

  • enables government to better plan for and mitigate risks

  • ensures a contracting authority will make a fully informed decision and maximises the buying power

  • ensures the right supplier is chosen to provide the right service

  • provides government the opportunity to realistically gauge expectations of what the market can and cannot contribute to the proposed program – on the other hand, it also allows the contracting authority to manage supplier expectations and confront any preconceptions regarding contracting with the government

  • complex, innovative or high-risk programs can be adapted to better utilise ICT industry capacities

Contracting authorities can benefit from suppliers’ knowledge of markets and trends, and develop an understanding of the ICT industry capacity available to the public sector to meet planned demand. This understanding can then be incorporated into ICT strategic plans, business cases, procurement plans and forward procurement schedules. Information from the engagement process can be used to develop technology-neutral specifications. However, care should be taken to ensure that the development of procurement options, and the accompanying technical specifications used in procurement documents, is not unduly influenced by the suppliers that have been involved in discussions.

Developing a deep understanding of the users’ (people who are expected to use the product or service in question) needs is a crucial element during the preparatory phase of any public procurement procedure. Users are the most important consideration in seeking to achieve the desired investment outcome. Investments need to be easy to use and have a consistent user experience. Every solution has users, even those that are internal. Even hardware and components supporting a broader solution have a user. Agile approaches in public procurement, including the adoption of more iterative methodologies in the development of digital services could help delivering on users’ needs and preferences. Applying agile approaches also means regular and thorough tests of the products and services under development, and users need to be involved in these repeated tests. Putting in place continuous feedback loops therefore is also necessary. (Box 3.3

Beside the concept of “user-centered design”, usability is another concept that contracting authorities should apply to ensure that public ICT/digital investments deliver their users. Usability means that a solution is developed to be easy to use. All solutions should be usable, including those that use off-the-shelf products. A product that meets all the business requirements of users, but requires an intensive investment of time to learn, has likely not prioritised usability. A major component of usability is accessibility. (Digital Transformation Agency, Commonwealth of Australia, 2019[14]) For example, in the European Union, in order to improve the functioning of the internal market, the Directive 2016/2102 on the accessibility of the websites and mobile applications of public sector bodies was adopted to approximate the laws, regulations and administrative provisions of the EU Member States relating to the accessibility requirements of the websites and mobile applications of public sector bodies, thereby enabling those websites and mobile applications to be more accessible to users, in particular to persons with disabilities.

Innovation is essential for delivering better, modern digital products and services, whilst reducing cost. In most OECD countries, the regulatory framework is supportive on using innovative and agile approaches to ICT procurement. However, practice does not always benefit from this flexibility. Overly prescribed solutions are included in tender notices, although over-specification is less likely to meet intended outcomes and does not allow room for innovation. As the traditional waterfall approach inherently leaves less room for flexibility and puts focus on up-front requirement capture and design, is seen as somewhat unsuitable for the capturing the (sometimes uncertain) customers’ needs in the public sector. Agile approach offers the flexibility needed to develop sustainable, innovative and tailor-made products and services. Aspects such as incremental and iterative delivery, team work and close cooperation between public body and supplier and user centred design makes agile approach different from the traditional, waterfall approach. Being open to innovation through having an outcome-based approach allows a greater range of solutions to be offered. Digital investments need to be open to innovative solutions from the very start: being outcome-focused and using descriptive requirements early in an investment allow greater range of innovative solutions to be offered. In Australia, public buyers are encouraged to be innovative by the Government. (Box 3.4).

The success of ICT projects requires among other things a clear business case. The business case informs the investment decision, the procurement strategy and helps ICT projects deliver on expected benefits. The business case as a project management tool is valuable to inform the decision-making process when deciding whether to invest in a particular ICT project or choose between ICT projects. The business case differs in this regard from a simple cost-benefit analysis (CBA), since it also brings strategic goals and non-financial benefits into the decision making process. The business case is also a tool that minimises project risks by breaking down the economy of the project into deliverables and enables users to work with the benefits of the project in a structured approach (Danish Agency for Digitisation, 2018[15]). Successfully benefitting from the business case is not easy, and several key concepts have to be defined and clarified to make sure that the use is comparable and thus enabling decision-makers to have a consistent approach and share knowledge.

The OECD Recommendation on Digital Government Strategies calls on governments to develop business cases that articulate “…the value proposition for all projects above a certain budget threshold to identify the expected economic, social and political benefits to justify public investment and to improve project management” (OECD, 2014[16]).

The primary benefits of clear business cases include the availability of a consistent framework for comparing investment decisions, a better view of costs, benefits and beneficiaries and a contribution to assessing the efficiency and effectiveness of ICT projects. However, business cases are not used regularly or consistently. As the OECD 2019 Digital Government Index shows, only just over half of governments have standardised models/methods to develop and present business cases for ex ante measurement of benefits and costs of digital government projects. Of this proportion, 39% require projects to meet specific criteria (e.g. budget threshold), while models/methods are compulsory for all ICT projects in only 15% of governments. Making the adoption of business case methodology a required policy lever in the early stages of project management would help countries achieve coherence and value proposition of ICT investments, enabling smart and cost-effective investment decisions for public value in line with strategic objectives. (OECD, 2020[7])

However good practices exist. Denmark has been a champion in the use of a common Business Case methodology to efficiently plan, strategically align and monitor the implementation of public ICT investments. All central government institutions in Denmark are required by budget regulations to follow specific guidelines set out by the Ministry of Finance when conducting ICT projects. The business case is in this case a tool in an ICT project management framework which is mandatory to use for all central government projects (Danish Agency for Digitisation, 2018[15]).

When every central government institution conducting ICT projects uses the same methodology for expenses to include, it ensures comparable projects.

In Finland, for example it is recommended that ICT projects between EUR 1-5 million build a business case. If the project costs are above EUR 5 million, it is mandatory to make one.

In Norway, it is mandatory for ICT projects below EUR 75 million to follow a best practice project model of their own choice, which involves making a business case. This enables sector agencies to adapt the recommended best practice model to their specific context and needs, rather than being a "once size fits all approach". For projects above EUR 75 million use of business cases and the Ministry of Finance's Quality Assurance Scheme is mandatory.

The United Kingdom has developed a comprehensive business case model (HM Treasury, 2018[17]) for systematic project evaluation according to five dimensions: 1. strategic, 2. economic, 3. commercial, 4. financial and 5. management.14 The economic dimension considers a cost benefit analysis (CBA) or cost effectiveness analysis (CEA) to quantify the social benefits of the initiative. Badged economists (members of the Government Economic Service) carry out such analyses. Introducing economists and financial expertise in these stages can facilitate the interaction and engagement with financial authorities. Additionally, the process considers a financial evaluation whose objective is to evaluate the availability of resources to fund the initiative, including the support and co-ordination with other units within the public sector when external funding is required. This specific step includes an analysis of how the project affects the balance sheet, income and expenses of the institution. The business dimension of this analysis introduces practical considerations that are especially important when planning and evaluating ICT investments.

To increase the knowledge about how different countries work with business cases the OECD has established the OECD Thematic Group on Business Cases which consisted of nine countries, such as Chile, Denmark, Estonia, Finland, the Netherlands, Norway, Portugal, Sweden and the United Kingdom. Clear business cases enable structuring ICT and digital public investments, while ensuring such decisions are supported by a clear rationale and an evident alignment to strategic priorities. The work of the group has led to the identification of 10 principles on business cases, which are fundamental in order to successfully benefit from the use of business cases in central government (Figure 3.9). All principles should be seen as central guidelines, and the implementation should always take into account national context and starting point (Danish Agency for Digitisation, 2018[15]).

Building on this work and especially on the report from the OECD Thematic Group on Business Cases presented by Denmark at the E-Leaders Meeting in October 2018, the Digital Transformation Agency of Australia and the OECD E-Leaders Working Group developed jointly the Business Case Playbook15. The purpose of the Playbook is to help countries develop business cases which support investment decisions in digital transformation and ICT. The Playbook explores what works, and what does not. The Playbook is based on the experience of Australia, and other OECD members, including Canada, Denmark, Estonia, and the United Kingdom. The Playbook covers the foundational concepts of a business case required to present a compelling argument for a digital or ICT investment. Each Play explores a core component of business case development, supported by helpful links.

Engaging stakeholders in the process of designing business cases is essential in order to promote joint ownership, distribution of benefits and a better understanding of users’ needs. Publishing forward-looking plans openly, can support not only increased transparency but also early market engagement. The UK has published guidance on setting up a commercial, digital and technology spend controls forward-looking pipelines, including templates16.

In the Slovak Republic, there is no specific standardised model of business cases for investment in digital technologies. The Value for Money Unit within the Ministry of Finance developed a Methodology on Developing and analysing business cases.

A low initial cost does not necessarily mean a solution will represent value for money. Costs incurred after the initial purchase can often change the whole-of-life cost. This means a solution with a low initial cost could have a high whole-of-life cost. Considering whole-of-life or life cycle cost (LCC) is a key component of assessing value for money. LCC looks beyond the initial purchase price of a solution to other cost elements such as maintenance costs, transition out costs, licensing costs (where applicable), the cost of additional features added after the initial investment, consumable costs and disposal costs. Technology costs, such as architecture, administration, integration, support and training should be also considered.

With regard to ICT goods and services, several OECD countries developed supporting tools for the calculation of LCC or total cost of ownership (TCO). In Denmark, the Ministry of Environment and the Environmental Protection Agency developed TCO tools for several products and services, such as computers (laptops, desktop computers, tablets, thin clients), displays (computer displays, information displays), multi-function devices (printers, copy machines, scanners, fax), projectors, servers, storage, Large Network Equipment, Small Network Equipment17. In Germany, the Federal Environmental Agency developed product-group specific excel tools18 that provide assistance in the calculation of life cycle costs of computers, multi-functional devices, monitors, computing centres and other products. The Calculation Tools of the Berliner Energieagentur for the products groups of vehicles, household devices and IT can be used without any comprehensive prior knowledge. It enables a fast access to calculating life cycle costs. The life cycle cost tool picker19, which has been available on the website of the Competence Center for Innovative Procurement (Kompetenzzentrums innovative Beschaffung, KOINNO) since September 2016, supports the needs-based selection of a life cycle cost calculation tool.

LCC in ICT procurement is extremely flexible; sensitive to changes in user needs, user behaviour, and a rapidly evolving industry. Contracting authorities are encouraged to think in terms of functional units, defined by quantitative and qualitative aspects (meaning performance characteristics delivered by ICT goods, networks and services), and product systems (ICT networks and services can be seen as logical structures, which are physically made up of ICT goods, including hardware and software). The LCC tool developed by the European Commission, the EU GPP Criteria for Computers and Monitors20 incorporated this logic in advising public procurers to define their needs in terms of functional units and not in relation to ICT goods (not computers, but capacity to process data, exchange information, serve a certain number of users, etc.).

The contract has always been a cornerstone of public procurement: the contract that defines the relationship between the contracting authority and a supplier, reflecting precisely the conditions, obligations that were covered by the tender documentation and the competition during the tender process. A well-written contract, including detailed specifications, is critical to a successful engagement and delivery. A contract, signed after a successful tender procedure, covers every detail: prices, delivery, and system performance. Contracting authorities rely on contractual safeguards to minimise risk to taxpayers, e.g. non-performance clauses. Strict definitions clarify what counts as non-performance in terms of time, cost, and scope. Agile, however, cannot operate well with many constraints and the standard contract terms and conditions used in a “traditional” ICT design and development contract will not support the delivery of services on the basis of an agile methodology.

The essence of the waterfall software development contract is that the buyer/customer tests whether the software meets its requirements, and if it does so by a certain date the software is accepted. All of the contractual rights and remedies of the buyer/customer, together with its payment obligations, revolve around the software meeting the requirements by a certain date. (Atkinson, 2010[18])

In agile procurement, the final product emerges through a joint effort during the process, therefore contract management is really the key to success. Regular engagement of the “business owner” and end users throughout the process is essential. Hands-on involvement is critical to monitor progress and avoid unpleasant eleventh-hour surprises. (John O’Leary, 2017[9])21

Agile requires modular or phased contracting methods that provide room for iteration and inclusive, cyclical approaches. Modular contracting is a procurement strategy that breaks up large, complex projects into multiple, tightly-scoped procurements to implement technology systems in successive, interoperable increments. This strategy helps mitigate risk, reduce vendor lock-in, and encourages the delivery of working software to users more rapidly.22

Modular contracting reduces vendor lock-in by providing more opportunities for vendor engagement and ensuring more than one vendor will know how the system works. In order to leverage modular contracting to reduce vendor lock-in, governments will also need to think about interoperability of system modules up front. By mandating system interoperability, modular contracting enforces good coding practices and increases the consistency of the software. This enables new vendors to come in if the public buyer decides to reopen the competition.

Instead of setting out detailed specifications, pricing and timeframes for delivery, and provisions for when things go wrong, the main objective of a contract for the delivery of a project using agile methodology is to structure the relationship between the customer and the supplier. As the contract is for the delivery of an outcome, rather than a thing, the contract will need to set out how the project is to be governed, delivered and the responsibilities of both parties in working towards that outcome. Additionally, rather than focussing solely on risk allocation provisions and contractual remedies if a project fails, an agile development contract should focus on the rules of engagement to ensure problem and failure resolution in real time.

Well-designed and well-managed contracts for the delivery of projects using an agile methodology can be effective in ensuring value for money outcomes for public buyers in their procurements of software and application design and other ICT development projects. Success will depend on:

  • supplier/service provider experience in delivering projects using agile project methodologies;

  • skilled and committed delivery teams within the contracting authority who continuously monitor supplier/service provider performance; and

  • early engagement with legal department/team and other internal and external stakeholders who have familiarity with contracts for the delivery of agile projects.

One of the key roles the contracting authority can play is to make sure that the contract is being monitored. A “trust and verify” approach can protect taxpayers and lead to working software. But it makes new demands and requires a new skill set for contracting authorities.

Designing a contract well for agile delivery means primarily that the contract is designed in a way that ensures that the parties understand the expectations of the relationship in an agile context. For this reason, a contract for agile delivery should include specific provisions and clear requirements relating to:

  • key roles and key personnel including the roles and responsibilities of key staff members at the contracting authority (such as a product owner), development teams, and agile “coaches”

  • key processes and governance requirements including for the development process, various meetings to support the process, testing requirements and timeframes; and

  • key documentation and tools including the project objectives and target outcomes, development items (such as a product backlog), tracking tools, and management information to support decision making.

While the specific provisions and terminology will vary depending on the specific agile methodology used, establishing the processes and expectations around these three key areas is critical in drafting an appropriate contract to support the effective implementation of a project based on agile delivery methodology. In addition to agile-specific provisions, a contract for delivery of a project using an agile methodology may also require reconsideration of and changes to standard contract clauses, including with respect to:

  • warranties;

  • intellectual property rights;

  • liability;

  • termination;

  • dispute resolution; and

  • change control processes.

Many different risk assessment techniques are used as part of project management in the public sector. The best ones tend to emphasise that risk should be managed by the party best able to do this, rather than a default position such as ‘the supplier takes all risks.’ Honest, accurate and regularly updated appraisals of risk make it less likely that an innovative, agile procurement approach will fail – but only if they are communicated and acted upon. One way to do this is by having a "project steering group", which is able to handle both informal and formal communication, so that risks can be dealt with as they arise as well as through an initial strategy. Many (but not all) risks can be managed by choice of procurement procedure, intellectual property strategy and contract terms.

Several ICT procurement projects involve an investment in making new ideas a reality, both by the contracting authority and the supplier(s) or service provider(s) involved. Each will want to recoup its investment, and this often takes the form of asserting intellectual property rights (IPR). In order to capture the benefits of innovative details solutions which are most important to it, without paying unnecessarily for rights and options which will not be used, the contracting authority should develop a strategy on IPR which takes into account the likely future applications of the product or service it is purchasing. For example, if a new design for recycling bins is developed as part of a waste management contract, does it make sense for the authority to purchase or licence this, and what about rights to the design of vehicles, which empty the bins? Issues to consider in answering such questions include the future ability of the authority to change service providers, and whether the design could also be licensed to other users of the service. In some cases sharing of information without the actual transfer of intellectual property rights will be sufficient to realise these objectives (European Commission, 2015[19]).

There are a number of different pricing models for the delivery of agile projects. Commercial terms need to be structured in a way that appropriately rewards the supplier for its efforts, while ensuring protections are in place for the contracting authority. Contracting authorities and supplier approaches to pricing an agile project are likely to be very different. While contracting authorities will naturally want the certainty that comes with a fixed price, this can erode the benefits of an agile delivery model by parties seeking to mitigate their risk by setting out rigid specifications, payment and change processes. Suppliers on the other hand, are likely to want time and materials pricing to reflect the inherently uncertain scope of a project delivered using an agile methodology. However, a pure time and materials engagement creates disincentives for the supplier to develop realistic estimates and then to adhere to them.

The digitalisation of the public sector means that a wide range of ICT systems are now as critical to society and the economy as, for example, electricity or transport infrastructure. Society and economy can only function if the ICT systems that support the work of the healthcare services, the education system, the tax authorities, the police or the public transport companies just to name a few, are actually fit for purpose.

Effective ICT is crucial when it comes to creating a better and more coherent public sector. Efficient and effective ICT is a prerequisite for the work carried out by public sector employees and for the quality of service provided by the public sector each day to citizens and the business alike. It is a part of the foundation on which the modern welfare state rests. This means that central government authorities bear a great deal of the responsibility for the national ICT portfolio functioning effectively and with a high level of information security.

ICT systems must be user-friendly, coherent, and secure and ICT projects must stay on track. As OECD country examples show, building a co-ordinated governance structure for managing and implementing ICT procurements and moving towards innovative and agile purchasing approaches contribute to achieving efficiency in government ICT expenditure and support the successful implementation of the national digitalisation agenda.

Building on the strengths identified in this Report, there are opportunities for the Slovak Republic to improve its current frameworks and practices for ICT procurement and to ensure that new technologies can be deployed quickly to improve public service delivery and implement national digitalisation. Recommended actions for the Slovak government for consideration are:

One of the main findings of the Report is that contracting authorities are not supported by a clear, whole-of-government ICT procurement strategy and thus they have limited guidance on how to align their ICT spending to meet the Government’s digital transformation agenda. The lack of a whole-of-government strategy also results in the lack of clarity in industry about the types of solutions they should provide to the government.

A national ICT procurement strategy that defines the strategic direction of the government’s main ICT procurement can be a powerful way of co-ordinating ICT investments across contracting authorities and government. Having a national ICT procurement strategy in place also ensures that ICT projects are co-ordinated across the different levels of public administration. A strategic, uniform and standardised approach to ICT procurement helps governments to overcome “agency thinking” approaches that usually fail to prioritise interoperability or common standards and sharing across different sectors and levels of government.

The national ICT procurement strategy should:

  1. 1. Promote coherent and aligned approaches and processes to ICT procurement.

  2. 2. Promote the strategic use of public procurement, including the promotion of quality-based selection of the tenders.

  3. 3. Demonstrate political leadership for more innovative, agile and iterative approaches and cultivate a more open culture towards new ways of purchasing ICT goods and services.

  4. 4. Promote competition in ICT procurement by increasing the chances of small specialised firms (including start-ups) to have access to ICT contracts in their area of expertise.

  5. 5. Encourage transparent and effective stakeholder participation throughout the whole public procurement cycle, with special focus on involving users of final goods and services, different levels of governments affected by the project and private sector or non-for profit service providers to ensure buy-in and distribution of realised benefits.

  6. 6. Stimulate understanding and collaboration among technical experts, policy specialists and procurement officials, in order to move beyond the current practice where procurements are conducted without a strong focus on outcomes.

  7. 7. Emphasise the importance of the preparatory phase of ICT procurement and promote a more strategic approach to users’ involvement, needs assessment, early market engagement and development of business cases.

On the other hand, strategic planning of ICT procurement could be addressed in a government-wide public procurement strategy or national digital government strategy. Furthermore, the national strategy for ICT procurement should be aligned with or be an integral element of the national digital government transformation strategy, preferably modelled on the 'Six Dimensions of a Digital Government' from the OECD Digital Government Policy Framework (OECD, 2020[8]). The OECD Digital Government Policy Framework consists of six dimensions that comprise a digital government:

  1. 1. Digital by design

  2. 2. Data-driven public sector

  3. 3. Government as a platform

  4. 4. Open by default

  5. 5. User-driven

  6. 6. Proactiveness.

In line with the national strategy, individual contracting authorities should develop their own institutional ICT strategy and long-term plans that will define the strategic direction of the contracting authorities’ main ICT investments and the actions required to ensure that all ICT systems fit within this strategy and within the national ICT (procurement) strategy. ICT procurement decisions taken within the context of an institutional strategy are likely to result in purchases that meet the needs of the contracting authority as a whole, rather than only those of individual departments. This is particularly relevant when migrating to new systems or solutions where the move is most cost-effective if undertaken on a large scale. The development of ICT procurement strategies can also lead to further rationalisation of ICT infrastructure in departments, limiting duplication and promoting sharing and reuse of services while allowing flexibility.

There are several good examples from OECD countries for comprehensive ICT purchasing strategies adopted in recent years.

In Denmark, the Government adopted a strategy in 2017, with the title of “A solid ICT foundation – Strategy for ICT management in central government23, with the aim of creating a solid foundation for ICT systems and defining common objectives for central government organisations on how to manage their ICT portfolios.

In Ireland, the Public Service ICT Strategy24 includes a special chapter on ICT Procurement. The ICT Strategy includes several key areas, such as Delivery of services via a government private cloud, Common applications delivery, Networks and telecommunications, ICT Support and ICT Procurement. The Office of Government Procurement (OGP) has been tasked to deliver the commercial implementation of the Public Service ICT Strategy through the development and delivery of sourcing strategies aimed to reduce the fixed ICT cost base. These strategies intend to leverage the considerable buying power of the Public Service and include, where possible, aggregation of spend, standardisation of specifications and on-going analysis/renegotiation of current ICT contracts.

In 2017, the United Kingdom published its government transformation strategy25 and digital strategy26. Both consistently referenced the need for taking user-centred, design-led, data-driven and open approaches to public procurement, building on the Digital Marketplace to embed these approaches more widely across the whole marketplace for public sector procurement. Local Government Association (LGA) in the UK issued the 'National technological and digital procurement category strategy27 in July 2017, which reinforces the importance of standards and assurance approaches, Digital Marketplace.

The Digital Transformation Strategy 2018-202528 in Australia rationalises ICT spending and dictates a number of principles to be observed while not being too prescriptive to leave room for agencies’ needs. The Roadmap accompanying the Strategy describes a rolling two-year window of the implementation work.

The review of the current ICT public procurement practices showed that the co-ordination between different government institutions that have some role and mandate in both developing and implementing the national digital agenda and conducting ICT procurement is not sufficient and efficient. Co-operation between different levels of government is also missing. As a result, individual contracting authorities’ purchasing decisions focus on agency-specific solutions rather than whole-of-government solutions, increasing the risk of duplication. Due to a lack of proper co-ordination mechanisms, there are no real examples for sharing and re-using of already existing ICT solutions in the public sector. As the spend review highlighted, there are only a very limited number of joint ICT procurements. There is also limited co-ordination and exchanges with external stakeholders, such as ICT business associations, other relevant interest groups, although some good examples do exist. A well-established governance structure for ICT procurement could successfully support the implementation of the Government’s digital goals.

To this end, the Government of the Slovak Republic should:

  • Consider establishing a unit in charge of ICT procurement policy at the central government level to ensure coherent and efficient ICT procurement across the public sector in line with the strategic priorities.

  • Establish formal or informal co-ordination mechanisms for ICT procurement with the subnational government level to avoid duplication and improve the value for money of ICT investments.

There are various examples for governance structure from OECD countries. Box 3.5 presents different types of governance and organisational frameworks.

In Denmark, the Government established a National Council for ICT Projects in 2011 after a 2010 comprehensive report identified several shortcomings in governmental IT projects. The task of the Council is to provide guidance to governmental IT projects as well as review all Danish governmental bodies with annual ICT cost above 30 million Danish krones. The 2010 report highlighted that for many years central government in Denmark ran its ICT projects with a high degree of outsourcing. Although this approach has served central government properly thanks to the good working relationship with private providers, central government has slowly handed over ever-greater responsibility for ICT to external consultants and private suppliers with regard to preparing requirements specifications, formulating calls for tenders, the selection of its suppliers and the subsequent implementation of the ICT systems in question. As a result, several central government organisations have lost control and critical knowledge of their ICT projects and systems and are unable to build the important bridge between ICT and their core remits. They have also lost their capability to enter into positive and value-creating collaborations with the private ICT market. The report therefore highlights the need for professionalising IT efforts. Based on the findings, the Government introduced new governance approaches for government ICT projects. Currently all central government authorities must follow the national ICT systems management model, and all central government organisations must undergo regular reviews of their ICT systems management by the National ICT Council. The Danish Agency for Digitalisation is co-ordinating the implementation of the strategy and providing support to the central government agencies in managing their ICT portfolios. (Box 3.6)

In the United Kingdom, the Government Digital Service (GDS), which is part of the Cabinet Office, is focusing on improving government services by simplifying access, improving (opening) government data, and making government more effective and efficient with the introduction of new technologies. GDS supports government digital transformation with digital and technology experts, leads the government’s use of data to support data-driven innovation across the public sector, provides best practice guidance and advice for consistent, coherent, high-quality services, sets and enforces standards for digital services, builds and supports common platforms, services, components and tools, helps government choose the right technology, favouring shorter, more flexible relationships with a wider variety of suppliers as well as supports increased use of emerging technologies by the public sector.29 Digital, data and technology standards and policy, and assurance at the pre-procurement planning and investment appraisal stage, and post-procurement service delivery and implementation stage, rests with GDS. This is mandated for by central government, and used on a voluntary basis by over 200 local government organisations that have signed up to the Local Digital Declaration30. Furthermore, GDS published guidance on governance principles for agile service delivery31, and the recently updated 'Agile digital and IT projects: clarification of business case guidance' from the UK's National Treasury and Government Finance Function32.

As the Report shows, in The Slovak Republic, there is a need for the public sector to take a strategic and systematic approach to the ICT market, rather than just engaging with it on a short-term and program-by-program basis. A key part of this is to engage with the ICT industry at an early stage of the planning of the ICT investment projects. Therefore, the government should promote better engagement between the ICT sector and the public sector. One way to do so is through developing a forum to capture supplier feedback on procurement issues in a planned, strategic and collaborative way with the aim of improving procurement processes for both suppliers and buyers. (Box 3.7)

In New Zealand, the Digital Marketplace serves the purpose of better engagement with the market. (Box 3.8)

Another important tool in this regard is to promote the wider use of early market engagement in the preparatory phase of the public procurement procedures. Market engagement is a key success factor for ICT procurement, especially for non-standard or irregular purchases or for purchases that result in realisation of unique ICT results and solutions. However, contracting authorities need some further practical advice and guidance on how to conduct market analysis and how to engage with the market in a way that respects the principles of transparency, non-discrimination and ensures competition.

Early market engagement with the ICT industry depends on a proactive and constructive approach by contracting authorities and suppliers for its success. Contracting authorities also need to be prepared to receive constructive criticism from potential suppliers and take useful learning from it.

Early market engagement or preliminary market consultation is allowed under the European Union policy and legal framework and as well as under the Slovakian public procurement legislation. According to the Directive 2014/24/EU a contracting authority may engage directly with economic operators as part of a market analysis. The process needs to be planned and managed very carefully so as to avoid the risks of lack of transparency, unequal treatment, or distortion of a subsequent competition. The EU Directive includes provisions in Article 40 related to such direct engagement, using the term “preliminary market consultations”: “Before commencing a procurement procedure, contracting authorities may conduct market consultations with a view to preparing the procurement and informing economic operators of their procurement plans and procurements.” The European Commission published several guiding documents to promote the use of preliminary market consultation and to explain how it can be conducted in a successful way while respecting the principles and rules of the European Union. For example the Guidance for public authorities on Public Procurement of Innovation gives detailed guidance on the objectives and steps of preliminary market consultation (European Commission, 2015[19]) (Figure 3.10).

The Slovak legislation also provides the possibility of contracting authorities to engage with the market, but this approach is not frequently used as was confirmed during the fact-finding missions. Contracting authorities need methodological support and guidance on how to engage with the market without infringing the principles and rules of public procurement. The Public Procurement Office has recently published, as part of the Public Procurement Methodology an infographic for the preparatory market consultation33 to support both the business sector and the contracting authorities on how to conduct market consultation properly, aligned with the requirements of the legal framework. In its Methodological Document about ICT procurement, the Working Group on Public Procurement and ICT Contracting34 also provides methodological advice to contracting authorities on preliminary market analysis.

Market engagement, however, can (and should) not be limited to the early phases of the procurement process, indeed it can continue during the tendering phase as well as in the post award phase as Table 3.1 shows.

In the United Kingdom, the Government Digital Service (GDS, part of the Cabinet Office) has engaged with the market on numerous occasions since the Global Digital Marketplace Programme was first publicly announced in September 201735. In Ireland, the Department of Enterprise, Trade & Employment (DETE) developed a guidance on how to effectively and transparently engage with market participants. The “Buying Innovation: the 10 step guide to smart procurement and SME access to public contracts”36 is based on the European Commission’s “Guide on Dealing with Innovative Solutions in Public Procurement – 10 Elements of Good Practice” and “Buying Green – A handbook on environmental public procurement”. The publication provides general guidance with clearly identified steps on how to apply the procurement process in a way that enables the procurement of innovation. (Box 3.9)

In Belgium, the Centrale de Marchés (CMS, Central Market for Federal Services), the Central Procurement Agency for Federal Services, responsible for awarding and monitoring the framework contracts for federal public services, regularly conducts market consultations in the procurement of ICT products to help develop relevant environmental requirements. Strong federal consultation on purchasing, which identifies the common needs of the various federal entities, co-ordinates and takes decisions in this area. To this end, a consultation body was created: the CSAF (strategic consultation network for federal purchases), which includes participants from the main federal institutions.37

As the Report shows, one main obstacle for the greater uptake of innovative, strategic approaches in ICT public procurement in the Slovak Republic is the lack of confidence and capability on the side of the contracting authorities to take new approaches to ICT procurement. In general, for the time being in The Slovak Republic, public procurement is not seen as a tool to achieve strategic priorities, but is rather perceived as an operational tool for purchasing goods and services at the lowest possible price. Risk averse behaviour can be also experienced on the side of the contracting authorities: innovative, agile approaches are usually considered riskier than well-known, traditional approaches. The organisational culture is not supportive of accepting a certain level of risk associated (or perceived to be associated) with agile methods. This might be also related to the lack of capacity (and in some cases expertise) of using proper risk management strategies to address any potential risks associated with innovative, agile approaches. The almost exclusively legal compliance-oriented strict controls and the fear of legal challenges do not encourage, or in some cases even prevent, the use of quality based criteria in the tender process and the experimentation with new public procurement approaches. This is, however, not a specific ICT procurement related challenge, rather a systemic issue in the Slovak public procurement system.

On the other hand, as meetings with various stakeholders during the fact finding missions confirmed, there are highly motivated staff members who wish to apply agile approaches, however, they do not know how to do so as there are no published good examples and there is only a limited availability and awareness of practical methodologies for public procurers to apply flexible and agile methods in ICT procurement. Therefore, building the capacity of staff members can be identified as one of the key tasks for ensuring the successful adoption of agile methods. However, formal training is not enough; people should learn agile values and principles in addition to practices to be motivated and committed (K. Conboy, 2011[21]).

Contracting authorities need support in improving their professional knowledge in terms of the strategic and innovative use of public procurement, including using quality (MEAT) criteria in evaluation, engaging strategically with the business sector, applying agile methods in public procurement, contract management for agile implementation. Procurement officials need help to engage in pilots on agile approaches without fear. The expectations of what is required from procurement staff have increased over time. Governments must now provide staff with additional training and support in order to embrace and implement strategic procurement. As shown in Figure 3.11, there has been an evolution of practices to build the skills of procurement staff in relation to strategic procurement.

The Slovak Government should develop and implement capacity building strategies to build skills and competencies of government staff who are involved in procurement of ICT solutions, including civil servants involved in the control of the public procurement procedures. Capacity building can include several actions, such as

  1. a) Developing operational tools on applying agile methods (such as guidelines, templates for contracts)

  2. b) Creating a national competency centre or a dedicated knowledge sharing platform to share capability

  3. c) Creating safe spaces for experimentation to introduce flexible and agile approaches in ICT procurement process through implementing pilot ICT projects using agile approaches and then communicating their results widely as well as developing communities of practice in order to facilitate connections and the exchange of knowledge

Contracting authorities can benefit a lot from guidelines, model contracts or templates that are publicly available and regularly updated. There are several good examples from OECD countries on how to support contracting authorities with operational tools in ICT procurement. For example in Italy, the Agency for Digital Italy (Agenzia per l’Italia Digitale) issued guidelines38 on the acquisition and reuse of software for public administrations. In Finland, where agile methodologies have been used in public procurement since 2010, by the requirement of the State IT director, a model agile agreement is available which was generated by the Ministry of Finance in 201539. In the United States, templates for agile blanket purchase agreements (BPAs) were developed by 18F, the innovative digital transformation governmental team. 18F is an office of US federal employees within the General Services Administration (GSA) that collaborates with other agencies to fix technical problems, build products, and improve how government serves the public through technology. It is part of the Technology Transformation Services, which is within the Federal Acquisition Service. In Ireland, the Office of Government Procurement issued several guidance notes to support public buyers with delivering public value on digital procurement, and most recently (February 2021) a Procurement Guidance Note on Cloud Services (Box 3.10)

Good examples in The Slovak Republic also exist. The Working Group on Public Procurement and ICT Contracting40 led by the Ministry of Investments, Regional Development and Informatisation developed a Methodological Document about ICT procurement, covering the most important issues in ICT procurement in the Slovak Republic, and providing methodological advice to contracting authorities on several challenges, such as preventing vendor lock-in, terminating unbalanced contracts from the past, dividing contracts into lots, preliminary market analysis, common availability of goods and services on the market, selection criteria (a tool helping procuring entities to determine whether the procured goods/ services are commonly available) and design contest for procuring software as well as IP rights41. The Methodological Document could serve as a good basis for developing further operational tools and guidelines.

Competence centres can operate as centralised advisory services to support the implementation of strategic ICT public procurement. In a decentralised environment, where new initiatives and approaches must be implemented by huge number of individual contracting authorities, the centralisation of expertise and resources on specific topics can be highly beneficial.

There are several examples in OECD countries for competence centres on public procurement, and specifically for innovation procurement (even if they do not solely focus on ICT procurement). For example, Germany set up a dedicated competence centre called the German Competence Centre for Innovation Procurement (Kompetenzzentrum innovative Beschaffung, KOINNO) in 2013 that supports innovation in public procurement. KOINNO’s objective is to increase public procurement of innovative goods and services in Germany, and, by doing so, trigger innovation and increased competiveness in the German economy. In order to measure progress towards this objective, KOINNO has targeted a considerable increase in the percentage of procurement procedures for new technologies, products and services. KOINNO provides contracting authorities with training, workshops, networking opportunities, on-call consulting and a website containing best practices, templates and guidance. KOINNO also supports contracting authorities in obtaining funding from the EU’s Horizon 2020 fund for research and innovation. Given that KOINNO operates on periodic mandates from the German government in the form of a memorandum of understanding, the centre must continue to demonstrate value in order to have its commission renewed periodically. KOINNO’s work also targets businesses in order to encourage the adoption of innovative practices and to ensure SMEs understand and participate in unique tender procedures like pre-commercial procurement. (OECD, 2019[22])

However, Germany is not the only country in the European Union that established a competency centre, there are several other countries, such as Austria, the Netherlands, Spain and Sweden, just to name a few. There is even a network for these competency centres, the European Network of National Competence Centres for Innovation Procurement. The Network is operating within the framework of an EU funded project, the Procure2Innovate project42 which aims to improve institutional support for public procurers of ICT and other sectors that implement innovation procurement. The project supports competency centres for innovation procurement in 10 European Union countries: five are already established (in Austria, Germany, the Netherlands, Spain and Sweden); while five new ones will be established in the near future (in Estonia, Greece, Ireland, Italy and Portugal). According to the definition used by the Network, a competency centre on innovation procurement is an organisation/organisational structure that has been assigned the task by its government and has a mandate according to national law to encourage wider use of pre-commercial procurement (PCP) and public procurement of innovation (PPI) that includes providing practical and/or financial assistance to public procurers in the preparation and/or implementation of PCP and PPI across all sectors of public interest.

There are three organisational models used in practice for national competency centres of innovation procurement in the European Union43. The first model is located within the central purchasing body. The second model is an institution that is either under the direct authority of another government institution or has been integrated into an existing agency. The final model is a competency centre that has been contracted out to a non-profit organisation. In practice, most competency centres mix two or even three approaches together. The most extreme being Finland where it is a “virtual competence centre” combining initiatives and expertise from eight institutions. The Netherlands are also a unique case where the competency centre on innovation procurement is part of a larger competency centre for public procurement, which in turn is aligned with the Ministry of Economic Affairs. In contrast, Germany, Ireland, Italy and Austria apply the typology in its purest form.

Beyond competency centres dedicated to public procurement, Digital Academies can be also relevant sources of the digital skills development. The benefits of agile are throughout the public procurement cycle and the insight for procurement professionals will be richer from learning alongside colleagues from all professions (the multi-disciplinary model) and not just in the context of agile in procurement. To support the development of digital skills in public sector, the OECD published the OECD Framework for Digital Talent and Skills in the Public Sector discusses this approach (OECD, 2021[12]). This is a three-pillar framework for equipping the public sector (whether national or local) with the skills to achieve digital government maturity:

  • Pillar 1 covers the importance of the context for those working on digital government and discusses the environment required to encourage digital transformation.

  • Pillar 2 addresses the skills to support digital government maturity, covering all public servants, particular professionals and those in leadership roles.

  • Pillar 3 considers the practical steps and enabling activities required to establish and maintain a workforce that encompasses the skills to support digital government maturity.

Good examples for digital skill development, however, exist throughout OECD countries, such as in Slovenia where the Ministry of Public Administration runs "Innovation Training in Public Administration". (Box 3.11)

Creating safe spaces for experimentation to use flexible and agile approaches in ICT procurement process is really important to build not only the capacity but also the confidence of public organisations in going beyond the traditional purchasing approaches. A good tool can be conducting pilot ICT projects using agile approaches and then communicating their results widely.

Similarly developing communities of practice in order to facilitate connections and the exchange of knowledge amongst stakeholders from different parts of government could be a powerful tool as it creates a valuable opportunity to share knowledge and experience, to learn from each other. Experience shows that these kinds of communities have cross-cutting benefits and can clearly help address challenges associated with fragmentation. Around the world, countries are increasingly setting up effective yet often relatively simple networks and communities of practice to help civil servants overcome bureaucratic silos and fragmented government structures. Such communities or networks help advance implementation in a consistent, unified manner. (OECD, 2020[23])

These communities of practice or networks can take a number of different forms; for instance:

  • They can be formally structured with governance structures and set processes, or more informal, such as meetup groups

  • They can be government-only or open to external parties from civil society and the private sector

  • They can be fully virtual, in-person or a combination of the two.

The UK government has built a series of communities for civil servants hosted on Google Groups and through Slack Channels on a wide variety of topics, some of which touch on procurement. Furthermore, the UK Crown Commercial Service 'buying digital community'44, sits alongside all other cross-government digital, data and technology communities of practice. These are central to capability and capacity building efforts.

In the Netherlands, the Dutch Professional and Innovative Tendering Network for Government Contracting Authorities (PIANOo)45 was created in 2005 as a network for public procurers with a goal to disseminate knowledge. Since then the institution’s role has expanded. PIANOo now serves as an expertise centre for public procurement, building on a network of 3,500 contracting authorities. These practitioners provide the input for PIANOo’s work. PIANOo’s approach combines different activities:

  • Publications: based on members’ questions and concerns, PIANOo publishes guidance documents that can support procurers in their daily work.

  • Meetings: PIANOo organises regular forums in which members come together to discuss current challenges and exchange good practices. These meetings are regional, for specific industries or procurement markets, and one overarching annual PIANOo conference.

  • Online portal: on the organisation’s website, tools, publications and guidance are collected, serving as an “encyclopaedia” for public procurement in the Netherlands, including an innovation procurement toolbox.

  • Training: PIANOo provides training on the public procurement legal framework.

Even in The Slovak Republic, good examples exist, although not related specifically to ICT procurement but rather to green public procurement (GPP). The Slovak Environmental Agency operates a GPP HelpDesk46 to provide information on GPP for public buyers.

In Canada, GCpedia and GCconnex provide connection points for individuals working in government, with different digital discussion groups focusing on a variety of subjects. Portugal’s Common Knowledge Network provides more open collaboration opportunities by inviting non-governmental participants to join the community. (OECD, 2020[23]) (Box 3.12.)

In the Slovak Republic, public procurement is relatively centralised in that sense that the majority of public spending happens on the central government level. Specific contracts are handled by contracting authorities at central, regional, and local levels, whilst some contracting authorities are required to purchase commonly available goods, services or works from the Ministry of the Interior (MoI), which acts as the central purchasing body (European Commission, 2014[24]). This remains the case for IT procurement, where 58% of IT spend occurs within the central government. As noted in Table 3.2. Overall spend in IT services the central government purchased over EUR 420 million worth of IT services between 2016 and 2019 (Public Procurement Office, 2019[25]).

This high level of centralisation in IT services is correlated to the 2030 Strategy for Digital Transformation of The Slovak Republic, the government strategy that defines the policy and particular priorities of The Slovak Republic in the context of currently on-going digital transformation of economy and society under the influence of innovative technologies and global megatrends of the digital era (Office of the Deputy Prime Minister of the Slovak Republic, 2019[26]). One of the opportunity areas identified in this 2030 strategy is the need for centralised reform in order to increase competences and accelerate digital innovation in areas such as public procurement. Indeed, with a government focus on promoting centralisation to achieve digital transformation, it is foreseeable that there will be increased levels of centralisation in IT services.

While there are identified high levels of ICT purchasing occurring at a central level in The Slovak Republic, there does not currently exist an ICT specific Centralised Purchasing Body (CPB). A central purchasing body (CPB) is a contracting authority that: i) acquires goods or services intended for one or more contracting authorities; ii) awards public contracts for works, goods or services intended for one or more contracting authorities; or, iii) concludes framework agreements for works, goods or services intended for one or more contracting authorities. Centralisation of procurement operations through the creation of such an ICT-focused CPB can lead to significant benefits, including better prices through economies of scale, lower transaction costs and improved capacity and expertise, but if not properly managed, centralisation can also entail risks (OECD, 2015[13]). The creation of an ICT-based CPB is not unknown in the OECD, with countries such as Germany beginning to centralise information technology (IT) procurement at the federal level. To support the centralisation efforts Germany created the Central Office for IT Procurement within the Federal Procurement Office of the Federal Ministry of the Interior (Zentralstelle für IT-Beschaffung) in 2017 (see Box 3.13). The ZIB is tasked with defining specific procurement strategies. These strategies range from the aggregation of IT-related procurement needs, to ad-hoc support, to contracting authorities for individual contracts.

Ireland also introduced centralisation for ICT procurement to deliver the commercial implementation of the Public Service ICT Strategy. The Office of Government Procurement (OGP) developed sourcing strategies aimed to reduce the fixed ICT cost base to leverage the considerable buying power of the Public Service. The sourcing strategies include, where possible, aggregation of spend, standardisation of specifications and on-going analysis/renegotiation of current ICT contracts. (Box 3.14)

In 2018, the Government in Hungary also introduced further centralisation in the field of government ICT procurement. A new agency, the Digital Government Agency (Digitális Kormányzati Ügynökség Zrt., DKÜ) was set up with the aim of unifying and centralising the government’s ICT procurement as well as making public ICT spending more transparent and improving the efficiency of ICT procurement. DKÜ set up a repository of the ICT assets of the government. The relevant public bodies and companies are required to upload their annual IT development and procurement plans to the Centralised IT Public Procurement System (KIBER) by 31 March each year.

The 2015 OECD Recommendation on Public Procurement states: “Adherents should develop and use tools to improve procurement procedures, reduce duplication and achieve greater value for money, including centralised purchasing, framework agreements, e-catalogues, dynamic purchasing, e-auctions, joint procurements and contracts with options” (Principle on efficiency, paragraph VII) (OECD, 2015[13]). Centralisation of purchasing activities has been a major driver of the efficient performance of public procurement systems.

Centralisation of procurement activities and aggregation of needs are observed across an overwhelming majority of OECD countries. CPBs are increasingly established to reap the benefits of aggregated demands and outputs of procurement activities. The benefits of centralised purchasing activities – such as better prices through economies of scale, lower transition costs, and improved capacity and expertise – are widely acknowledged. Another key aspect of centralisation is the use of framework agreements. A framework agreement is an agreement with one or more economic operators for the supply of goods, services and, in some cases, works. Its purpose is to establish the contract conditions to be awarded by one or more contracting authorities during a certain given period, in particular, with regard to maximum price, minimum technical specifications and, where appropriate, the quantities envisaged. Usually the terms of a framework agreement shall not exceed four years (OECD, 2014[27]). The aggregation of demand caused by a framework agreement is a strong tool to enhance efficiency, reduce administrative burden and lower the cost.

Framework agreements designed to meet users’ needs (primary users being public sector buyers and suppliers) can also support increased competition and SME participation as several examples, such as the one from the United Kingdom shows: the Crown Commercial Service (CCS) ‘Digital Outcomes and Specialists’ (DOS) framework agreement (available through the Digital Marketplace platform), was launched at the end of April 2016 and by January 2021 had re-opened for new supplier applications 4 times. DOS5 (the current iteration, which went live on 20 January 2021) has 3,340 suppliers (94% SMEs) available to the UK public sector. Since its launch (and as at 28 January 2021) 3,999 contracting opportunities have been published, of which:

  • 84% (3,343) received 6 or more bids;

  • 45% (1,787) received between 11 and 25 bids;

  • 2% (76) received single bid responses.

Participation of SMEs range between 75% and 85% of contracting opportunities. Furthermore 1,078 DOS contracts were awarded that were valued above £122,97647, which have an average value of £1,684,279, and received an average number of 16 bids per contracting opportunity. This illustrates efficiency and effectiveness gains from time saving, standardisation (consistent terms and conditions, application and enforcement of digital, data and technology standards, centralisation of data capture and reporting, etc.). (Figure 3.12.)

Figure 3.13 shows, SMEs have so far been awarded between 70-82% of the 1,910 DOS contracts, which have had award data updated by buyers.

The use of framework agreements are not uncommon for ICT purchasing in The Slovak Republic. For IT services, 12% of purchasing is done via a framework agreement, during the period of 2016 to 2019 (Figure 3.14). Comparing this with the total procurement spend, 22% of spending is done via framework agreement. It is often difficult to establish framework agreements in the field of ICT as the compatibility and usability of a specific IT services are unique to the contracting authority that initially procures the service. There is certainly scope for further consolidation of spending into framework agreements for IT services, especially for contracting authorities to collaborate earlier in the procurement process to ensure that the IT services they procure can be made consistent across government.

When IT services are divided into a year upon year analysis, it does become clear that the use of framework agreements is increasing. As identified in Figure 3.15, in 2018, 25% of IT-related contracts were administered through framework agreements. This figure does however decrease in 2019 to 17%.

The 2015 OECD Recommendation on Public Procurement calls upon Adherents to use information and communication technologies to “drive cost savings and integrate public procurement and public finance information” and to “employ recent digital technology developments that allow integrated e-procurement solutions covering the public procurement cycle” (Principle on e-procurement, paragraph VIII, i). E-procurement systems collecting consistent, up-to-date and reliable data on procurement processes can feed into other government information technology (IT) systems through automated data exchanges, reducing risks of mistakes, errors and duplication. Meanwhile, integration with other digital government systems such as digital invoicing is essential to make e-procurement systems fully functional during all phases of the procurement cycle (OECD, 2015[13]).

To this end, as framework agreements are effective at generating efficiencies and savings across government, the PPO should consider the creation of an online portal or platform that would not only provide an online location for ICT framework agreements, but would also be used to aggregate public demand and streamline procurement processes. A number of countries such as the UK and New Zealand have created online platforms that gives government agencies access to innovative products and services, particularly cloud services.

National, regional and local public administrations can reduce costs, increase their efficiency and foster interoperability by jointly developing, reusing or sharing IT solutions that meet common requirements. Central governments can support this process by creating a climate of innovation in their administrations, encouraging staff to take an active role in the process and promoting the use of information and communication technologies.

Public services can be implemented faster and more efficiently by sharing and re-using already available solutions and by learning from the experiences of other public authorities, agencies or even of other countries.

Sharing of solutions refers to making solutions available to others, or developing common solutions, such as:

  • Releasing an application under an open source license on a repository

  • Providing common IT frameworks and architectures, common list of standards and metadata, guidelines for project management

  • The shared development of solutions, based on common requirements, with or without pooling of procurement

  • Making shared services available for several public administrations, for example as cloud, or web services.

Re-using already available solutions means that public administrations confronted with a specific problem seek to benefit from the work of others by looking at what is available, assessing its usefulness or relevance to the problem at hand, and deciding to use solutions that have proven their value elsewhere. In some cases, the solutions are reused once they have been adapted to specific requirements or linguistic environments. The use of open source software and collaborative coding has enabled public administrations to leverage the developer community in the continuous improvement of its solutions, thus proving to be a powerful tool to increase procurement’s efficiency. Nevertheless, most importantly it provides a space for collaboration by creating the opportunity to reuse solutions, to collectively improve by learning from each other, and to share solutions, knowledge and wisdom.

Public sector for example, develops software. Beyond the immediate need of the public authority, this software, solution represents an asset that could be reused by other public sector agencies. Software reuse means “Distribution under a licence”, because software is protected by copyright and without the authorisation of the copyright owner; any use (including modification, adaptation, and re-distribution) is copyright infringement. Allowing the reuse of software by third parties is not a unilateral “gift” in the sense of a “deprivation”: on the contrary, increasing use and sharing of software has the effect of augmenting its value: more users means more developers, more experts, more potential for improvement, more need and interest for training, more service providers interested to become competent, technical alignment of other initiatives on the published solution (that become a reference), and reduction of cost to make it interoperable. Not all public sector software is aimed for sharing and redistribution: some software is too specific in terms of business needs, or there could be security requirements not implemented. Therefore the decision for sharing / allowing others to reuse and localise the source code is not an obligation and needs to be taken on a case by case basis by the relevant authority (Schmitz, 2013[28]).

Sharing and reusing technology, data, and services (e.g. common platforms, components, design system elements, etc.) is central to the concept of ‘Government as a platform' and a foundational element of the OECD Digital Government Policy Framework: Six dimensions of a Digital Government (OECD, 2020[8]). We are no longer in the binary 'build' versus 'buy' way of thinking; reuse is a critical decision making factor to achieve value for money and reduce whole life costs.

In almost all countries, there are several technology resources and common government platforms that are available to all government organisations. This help public agencies to reuse government services, information, data and software components instead of developing their own solutions.

In Italy, the Codice dell’Amministrazione Digitale obliges public buyers to use collaborative coding, release the software developed or purchased with an open license (one of the licenses approved by the Open Source Initiative) and to publish it in a public repository. As already mentioned in the Report earlier, the Agency for Digital Italy (Agenzia per l’Italia Digitale) issued guidelines48 on the acquisition and reuse of software for public administrations to support public buyers. The administration must always obtain full ownership of the software. The guidelines include technical attachments that can be directly included in contracts and specifications related to software development, software modification and maintenance, in order to fulfil the release obligation. The guidelines also include detailed instructions on how to publish software as an open source. (Box 3.15)

As Chapter 1 already presented, the European Commission is also strongly encouraging EU Member States to share and re-use already available ICT solutions across borders and sectors in an efficient and effective way. The EU-wide sharing and reuse of interoperable solutions for public administrations could reduce costs and risks, foster innovation and businesses’ use of digital technologies, and ensure digital sovereignty. A collaborative platform, Joinup49, was set up to facilitate the sharing and reuse of IT solutions developed for public administrations in EU member states. Joinup is a single-access point to almost 2,800 interoperability solutions for public administrations, included in the collections of more than 40 standardisation bodies, public administrations and open source software repositories. The interoperability solutions are described using the Asset Description Metadata Schema. Joinup can serve also as an example for setting up a national collaborative platform and catalogue of reusable IT solutions. It provides freely reusable software under an open source licence and some support to help countries set up their own collaborative platform with services similar to those of the Joinup platform.

In its Sharing and Reuse Framework for IT solutions50 (2016), the European Commission put forward a variety of good practices aiming at promoting the re-use of procured solutions. For example, a collection of good examples for contractual clauses for service procurement was developed to propose common clauses for contracts, which public administrations could use during procuring services. Clauses are developed both for contracts related to the: 1. the development of new IT tools that may be re-used and/or shared later, 2. re-use of already available IT tools possibly through customization. (Box 3.16)

In The Slovak Republic, the National Agency for Network and Electronic Services (“NASES”) provides Central Government Portal available at https://slovensko.sk. Central Government Portal provides central and unified access to information resources and electronic public services. Information (advice, guides, descriptions) users are searching for is usually a part of particular government department website. Central Government Portal focuses on the integration of such information along with electronic public services and provides them to users through a single entry point in an accessible and comprehensive way.

In the United Kingdom, the GOV.UK Service Toolkit51 covers this issue. The portal provides all the information needed to design, build and run services that meet government standards. Besides the list of digital and technology standards, guidance on specific topics, the portal provides also the technologies that can be used when developing governmental solutions, such as:

  • GOV.UK Notify – technology to keep the users updated with emails, text messages and letters, cheaply and easily

  • GOV.UK Pay – technology to take and process payments - a simple experience for users and easy integration

  • GOV.UK Platform as a service – hosting the service on a government cloud platform without having to build and manage your own infrastructure

  • GOV.UK Sign in (beta) – technology to sign in to service quickly, easily and securely

ICT standards play an essential role in achieving interoperability of new technologies and can bring significant benefits to both industry and consumers. They help ICT markets remain open and allow consumers the widest choice of products. They can prevent reliance on single vendors for products and system components that implement desired technologies by identifying the key element of the technology required and ensuring that its use is not limited to a specific product or service.

In the European Union, the European Commission identified ICT standards a key element in creating a level playing field for all technology providers and therefore encourages public authorities to make better use of the full range of relevant standards when procuring ICT products and services (European Commission, 2013[29]). Procuring ICT solutions based on standards that are available for any user increases the potential for interoperability with other applications that use the same standards and thus achieve ‘vendor independence’. Standards determine the key element of a technology and create a level playing field for all ICT suppliers. More suppliers will be able to submit offers to invitations to tender for standards-based systems, leading to more competition and choice.

Open standards are one of the most powerful tools to open up government. They make it possible for the smallest supplier to compete with the largest ones. They make data open for any citizen to audit. They unlock the transformative power of open source software. To ensure that purchases are not limited to the original supplier and that they can be further used to deliver trans-governmental services, it is recommended to support solutions that use standards and no proprietary elements.

Public procurements should include only standards that are supported by the market and that are recognised by a formal standardisation organisation, or a technical specification that has been identified by the European Commission or by a national organisation. So long as they are not recognised, they remain "technical specifications" that can also be used in public procurement, but their legal validity may be questioned, and an additional explanation may be necessary. Where openness requirements are justifiable due to interoperability needs of the procuring public authority, openness properties for open standards should be included as well. Furthermore, given that standards and technical specifications can be implemented in different ways, it is important that they provide reference to implementation or conformity tests.

Referencing standards in technical specifications aims at increasing common understanding of procurement documents between buyers and suppliers. It may help to define works, supplies and services, contribute to reducing total costs, ensure equality, increase transparency and makes it easier to develop procurement documents. As public procurement officers are unfamiliar with standards and standardisation, and need guidance on how they should reference standards in procurement documents, under the leadership of the Swedish Standards Institute (SIS), and with the financial support of the European Commission a Guide for referencing standards in public procurement in Europe52 was developed and published in 2018. The Guide aims at providing a better understanding of what standards are and how they can be referenced in public procurement. It also aims at providing ideas on how to reference standards in general, based on the EU procurement legislative framework.

Common, consistent standards that flow throughout the full public spending lifecycle, starting at the pre-procurement planning, investment appraisal stage. In the United Kingdom, the Technology Code of Practice53 is used for this purpose for this, combined with the authority delegated to GDS from the Treasury for assuring spending plans against those standards. This flows into procurement. Furthermore, at the post-tender implementation / service delivery stage, the Service Standard54 is used combined with the authority to assure incremental delivery on a phased basis.


[2] Agile Alliance (2001), “Manifesto for Agile Software Development”, http://agilemanifesto.org/.

[18] Atkinson, S. (2010), Why the traditional contract for software development is flawed?, Thomson Reuters (Legal) Limited and Contributors, pp. 179-182.

[11] Cormican, T. (2015), Towards holistic goal centered performance management in software development: lessons from a best practice analysis, pp. 23-26.

[15] Danish Agency for Digitisation (2018), Report from the OECD Thematic Group on Business Cases.

[20] Department of Internal Affairs (n.d.), “Cloud Marketplace”, https://marketplace.govt.nz/about-the-marketplace/.

[14] Digital Transformation Agency, Commonwealth of Australia (2019), “Digital Sourcing Consider First Policy guidance”, https://www.buyict.gov.au/sp?id=resources_and_policies&kb=KB0010625.

[19] European Commission (2015), “Guidance for public authroities on Public Procurement of Innovation”, https://s3platform.jrc.ec.europa.eu/en/web/guest/w/guidance-for-public-authorities-on-public-procurement-of-innovation.

[24] European Commission (2014), “Slovakia Country Profile”, Public procurement – Study on administrative capacity in the EU.

[29] European Commission (2013), “Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions - Against lock-in: building open ICT systems by making better use of standards in public procurement”, https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013DC0455&from=EN.

[17] HM Treasury (2018), Guide to developing the project business case - Better business cases: better outcomes, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/749086/Project_Business_Case_2018.pdf.

[9] John O’Leary, W. (2017), “Going Agile: The new mind-set for procurement officials – How does Agile change the role of the acquisition officer?”, Agile in Government, A playbook from the Deloitte Center for Government Insights - Deloitte Insights, https://www2.deloitte.com/content/dam/insights/us/articles/3897_Agile-in-government/DUP_Agile-in-Government-series.pdf.

[3] Jouko Nuottila, K. (2016), Challenges of adopting agile methods in a public organization, pp. 65-85, https://doi.org/10.12821/ijispm040304.

[10] K. Conboy, S. (2011), People over process: Key people challenges in agile development, pp. 48-57.

[21] K. Conboy, S. (2011), People over process: Key people challenges in agile development, pp. 48-57.

[4] Kienitz:, P. (2017), The pros and cons of Waterfall Software Development, https://www.dcslsoftware.com/pros-cons-waterfall-software-development/.

[5] OECD (2021), Digital Government Review of Slovenia: Leading the Digital Transformation of the Public Sector, OECD Publishing, https://doi.org/10.1787/954b0e74-en.

[12] OECD (2021), “The OECD Framework for digital talent and skills in the public sector”, OECD Working Papers on Public Governance, No. 45, OECD Publishing, Paris, https://doi.org/10.1787/4e7c3f58-en.

[1] OECD (2020), Digital Government in Chile – Improving Public Service Design and Delivery, OECD Digital Government Studies, OECD Publishing, Paris, https://doi.org/10.1787/b94582e8-en.

[7] OECD (2020), Digital Government Index - 2019 results.

[23] OECD (2020), System Change in Slovenia: Making Public Procurement More Effective, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/b050ef2f-en.

[8] OECD (2020), The OECD Digital Government Policy Framework: Six dimensions of a Digital Government, OECD Public Governance Policy Papers, No. 02, https://doi.org/10.1787/f64fed2a-en.

[22] OECD (2019), Public Procurement in Germany: Strategic Dimensions for Well-being and Growth, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/1db30826-en.

[13] OECD (2015), OECD Recommendation of the Council on Public Procurement, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0411, https://www.oecd.org/gov/public-procurement/recommendation/.

[27] OECD (2014), “Manual for Framework Agreements”, https://www.oecd.org/gov/ethics/manual-framework-agreements.pdf.

[16] OECD (2014), “OECD Recommendation of the Council on Digital Government Strategies”, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0406.

[26] Office of the Deputy Prime Minister of the Slovak Republic (2019), “Strategy of the Digital Transformation of Slovakia 2030”, https://www.vicepremier.gov.sk/wp-content/uploads/2019/11/Brochure-SMALL.pdf.

[25] Public Procurement Office (2019), Exported data, National Public Procurement System.

[6] Republic of Slovenia (2017), Guidelines on Procuring IT Solutions, https://nio.gov.si/nio/asset/smernice+za+javno+narocanje+informacijskih+resitev?lang=en.

[28] Schmitz, P. (2013), STANDARD “SHARING AND RE-USING” CLAUSES FOR CONTRACTS - Contractual Clauses for Service Procurement, https://joinup.ec.europa.eu/sites/default/files/document/2014-03/Standard%20sharing%20and%20re-using%20clauses%20for%20contracts.pdf.


← 1. Agile Procurement for the Public Sector, by Emilio Franco, 2017-08-19, Public Spend Forum, https://www.publicspendforum.net/blogs/emilio-franco/2017/08/19/agile-procurement-public-sector/

← 2. http://agilemanifesto.org/

← 3. A Short History of Agile, https://www.agilealliance.org/agile101/

← 4. Guide on Agile Principles and 18F Practices, https://agile.18f.gov/

← 5. https://www.agilealliance.org/wp-content/uploads/2017/09/AgilePracticeGuide.pdf

← 6. https://www.agilealliance.org/agile101/agile-glossary/

← 7. The waterfall development model originated in the manufacturing and construction industries; where the highly structured physical environments meant that design changes became prohibitively expensive much sooner in the development process. When first adopted for software development, there were no recognised alternatives for knowledge-based creative work.

← 8. Understanding the pros and cons of the Waterfall Model of software development, 2006, TechRepublic, September 22, 2006; https://www.techrepublic.com/article/understanding-the-pros-and-cons-of-the-waterfall-model-of-software-development/

← 9. https://catalystcycle12.wordpress.com/2015/04/10/waterfalls/

← 10. See the Service Standard from the United Kingdom as an example, https://www.gov.uk/service-manual/service-standard

← 11. https://agile.18f.gov/

← 12. John O'Leary, William D. Eggers: Going Agile: The new mind-set for procurement officials – How does Agile change the role of the acquisition officer? in: Agile in Government, A playbook from the Deloitte Center for Government Insights, Deloitte Insights, 2017; https://www2.deloitte.com/content/dam/insights/us/articles/3897_Agile-in-government/DUP_Agile-in-Government-series.pdf

← 13. https://agile.18f.gov/

← 14. https://www.gov.uk/government/publications/the-green-book-appraisal-and-evaluation-in-central-governent/agile-digital-and-it-projects-clarification-of-business-case-guidance

← 15. https://businesscaseplaybook.service.gov.au/index.html

← 16. https://www.gov.uk/guidance/set-up-a-commercial-or-digital-and-technology-spend-controls-pipeline

← 17. https://csr-indkob.dk/tco-vaerktoejer/

← 18. https://www.umweltbundesamt.de/dokument/berechnungswerkzeug-fuer-lebenszykluskosten (in German)

← 19. https://www.koinno-bmwi.de/informationen/toolbox/detail/lebenszyklus-tool-picker-1/

← 20. Commission Staff Working Document EU GPP Criteria for Computers and Monitors, SWD(2016) 346 final, Brussels, 21.10.2016, https://ec.europa.eu/environment/gpp/pdf/toolkit/computers%20and% 20monitors/EN.pdf, the revised version from 2021: Commission Staff Working Document EU green public procurement criteria for computers, monitors, tablets and smartphones, SWD(2021) 57 final, https://ec.europa.eu/environment/gpp/pdf/210309_EU%20GPP%20criteria%20computers.pdf

← 21. John O'Leary, William D. Eggers: Going Agile: The new mind-set for procurement officials – How does Agile change the role of the acquisition officer? in: Agile in Government, A playbook from the Deloitte Center for Government Insights, Deloitte Insights, 2017; https://www2.deloitte.com/content/dam/ insights/us/articles/3897_Agile-in-government/DUP_Agile-in-Government-series.pdf

← 22. Mark Headd , Ed Mullen: Modular contracting and working in the open, October 25, 2018, https://18f.gsa.gov/2018/10/25/modular-contracting-and-working-in-the-open/

← 23. https://en.digst.dk/media/15367/a-solid-ict-foundation-strategy-for-ict-management-in-central-government.pdf

← 24. https://ictstrategy.per.gov.ie/index.html

← 25. https://www.gov.uk/government/publications/government-transformation-strategy-2017-to-2020/government-transformation-strategy-tools-processes-and-governance#priorities-until-2020

← 26. https://www.gov.uk/government/publications/uk-digital-strategy/3-the-digital-sectors-making-the-uk-the-best-place-to-start-and-grow-a-digital-business#widening-procurement

← 27. https://www.local.gov.uk/National-technological-and-digital-procurement-category

← 28. https://www.dta.gov.au/digital-transformation-strategy/digital-transformation-strategy-2018-2025#:~:text=We%20will%20deliver%20world%2Dleading,of%20work%20we%20have%20planned

← 29. https://www.gov.uk/government/organisations/government-digital-service/about

← 30. https://localdigital.gov.uk/declaration

← 31. https://www.gov.uk/service-manual/agile-delivery/governance-principles-for-agile-service-deliver

← 32. https://www.gov.uk/government/publications/the-green-book-appraisal-and-evaluation-in-central-governent/agile-digital-and-it-projects-clarification-of-business-case-guidance

← 33. https://www.uvo.gov.sk/metodika-zadavania-zakaziek-5c1.html

← 34. http://www.informatizacia.sk/expertne-skupiny-gad/22464s

← 35. https://gds.blog.gov.uk/2019/08/12/engaging-uk-suppliers-in-the-global-digital-marketplace-programme-alpha-phase/; https://gds.blog.gov.uk/2018/07/24/engaging-uk-suppliers-in-the-global-digital-marketplace/; https://gds.blog.gov.uk/2017/10/04/make-procurement-open-it-makes-government-better/

← 36. https://ogp.gov.ie/buying-innovation-the-10-step-guide-to-smart-procurement-and-sme-access-to-public-contracts/

← 37. https://www.publicprocurement.be/fr/services-federaux/la-centrale-de-marches-pour-services-federaux-cms

← 38. https://docs.italia.it/italia/developers-italia/gl-acquisition-and-reuse-software-for-pa-docs/en/stabile/index.html. https://www.gazzettaufficiale.it/eli/id/2019/05/23/19A03233/sg

← 39. https://www.suomidigi.fi/ohjeet-ja-tuki/jhs-suositukset/jhs-166-julkisen-hallinnon-it-hankintojen-yleiset-sopimusehdot-jit-2015

← 40. http://www.informatizacia.sk/expertne-skupiny-gad/22464s

← 41. An earlier methodological instruction for standard details in describing the subject matter of contract, standard terms and conditions of participation in public procurement and optimum contractual terms and conditions in relation to IT projects available online at: http://www.informatizacia.sk/ext_dok-metodicky_pokyn_std_obstaravanie_1-0/15176c

← 42. Procure2Innovate is funded by the European Union Horizon 2020 programme. https://procure2innovate.eu/project/

← 43. https://procure2innovate.eu/fileadmin/user_upload/Documents/ Procure2Innovate_HowtosetupacompetencecentreonInnovationProcurement.pdf

← 44. https://www.gov.uk/service-manual/communities/digital-buying-community; https://www.gov.uk/service-manual/communities

← 45. http://www.pianoo.nl/en/public-procurement-netherlands

← 46. https://www.sazp.sk/zivotne-prostredie/environmentalne-manazerstvo/zelene-verejne-obstaravanie-gpp/gpp-helpdesk.html

← 47. The GPA threshold for UK central government contracting authorities above which goods and services contracts would have been competed via Tenders Electronic Daily, if a framework agreement like DOS was not available.

← 48. https://docs.italia.it/italia/developers-italia/gl-acquisition-and-reuse-software-for-pa-docs/en/stabile/index.html ; https://www.gazzettaufficiale.it/eli/id/2019/05/23/19A03233/sg

← 49. https://ec.europa.eu/isa2/solutions/joinup_en

← 50. https://joinup.ec.europa.eu/sites/default/files/document/2015-03/guideline_on_procuring_it_solutions_-_v1_00.pdf

← 51. https://www.gov.uk/service-toolkit

← 52. https://ec.europa.eu/docsroom/documents/33421

← 53. https://www.gov.uk/government/publications/technology-code-of-practice/technology-code-of-practice

← 54. https://www.gov.uk/service-manual/service-standard

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2022

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at https://www.oecd.org/termsandconditions.