4. Challenges in the development and procurement of ICT/digital projects

While Greek authorities have been gathering considerable experience in implementing ICT/digital projects over the past years, challenges often persist, leading to ineffective project implementation that hinders benefits realisation. In particular, large digital projects are at greater risk of failure, as highlighted already by Greek authorities in the National Digital Strategy 2016-2021 (Ministry of Digital Policy Telecommunications and Information, 2016[1]) as well as during the interviews and data gathering process for this review.

Namely, digital projects generally take a long time from the moment of conception and design until they are effectively implemented, are often not flexible enough to respond to changing needs, face challenges to interoperate and adopt cross-governmental standards, do not incorporate the latest technologies, and lead to complex and closed systems with the risk of vendor lock-in and legacy issues that increase maintenance costs, among other challenges. Digital transformation’s expected benefits may not be achieved if ICT/digital projects do not adhere to system-wide standards, are not tailored and driven to users’ needs, take too long to be implemented, or are complex to manage during and after the implementation and procurement processes (Ministry of Digital Policy Telecommunications and Information, 2016[1]).

This section outlines key areas where the Greek government can achieve greater competency and maturity in order to secure ICT/digital projects’ benefits realisation. It comprises reflections and suggestions throughout the project life cycle as well as other key components of the digital government ecosystem in Greece.

As highlighted in the previous chapter, the creation of the Ministry of Digital Governance represents a step forward in terms of the governance of digital government in Greece. Created in 2019, the Ministry is empowered to control all relevant functions for digital government, including policy design, strategic planning, funding management and public procurement for most of the critical and relevant projects. Within this governance model, Information Society S.A. is also under the mandate of MDG and acts as its implementation arm. Information Society plays a critical role to support public sector organisations with limited institutional capacities to implement and deliver, acting as an executive arm supporting the implementation of ICT projects in the public sector.

The reforms undertaken by the Greek administration are in line with the OECD Recommendations for Digital Government Strategies, which calls governments to secure leadership, political commitment and institutional capacities to co-ordinate digital transformation projects and achieve the digital government imperative (OECD, 2014[2]). When looking at the organisational structure and operational model of MDG, evidence collected from various departments and units within and outside the Ministry shows that it operates under a fragmented approach as a consequence of the successive institutional reforms undergone in the past decade. This siloed functioning becomes particularly clear in the ways departments and units involved in the life cycle of ICT/digital projects do not collaborate and interact throughout this process. This also includes the limited awareness of roles and mandates of each department/unit involved in the implementation and procurement of ICT/digital projects within and outside MDG, which constrains a smooth and co-ordinated approach in project assessment, prioritisation, funding and procurement. Further capacity building activities to bring together different areas within MDG are needed to create opportunities for policy dialogue, knowledge transfer and co-ordination beyond the formal procedures for project approval, funding and implementation. Institutional culture and appropriate incentives play an important role in this context, too. Namely, a culture that promotes co-operation between teams and institutions with concrete incentives (e.g. career rewards for co-operative behaviours) can reduce fragmentation and silo-based thinking.

In addition, the Digital Transformation Bible (DTB) includes specific provisions on the governance framework supporting the implementation of the strategy. This comprises setting an inclusive governance model to involve relevant public sector stakeholders in this process, including the Digital Transformation Steering Committee (relevant ministers and state secretaries) and the Digital Transformation Executive Network (institutional CIOs and heads of public sector bodies and organisations that have a key role in the implementation of the DTB).

Accordingly, both co-ordination bodies reflect well-intended efforts to secure effective governance mechanisms for the implementation of the strategic roadmap at top political and execution level. However, overcoming institutional siloes requires clarifying the responsibilities of each body, including the advisory and decision-making roles of both the Committee and the Network to effectively support the implementation of the Bible. Similarly, clear communication of their roles and mandate are needed within and outside the public sector to set expectations, ownership and awareness of the inclusive governance model for digital government in the country.

An effective digital transformation requires adopting user-driven approaches in the formulation and development of ICT/digital projects. Establishing concrete mechanisms for understanding and meeting user needs increases the pertinence, expected impact and successful implementation of ICT/digital projects throughout the development process, capturing feedback and acting accordingly to shape them around and driven by user needs. In this sense, adopting agile methodology approaches to frame the design and implementation of digital transformation process can help encompass increased engagement with users to collect feedback with the importance of addressing the uncertainties and risks of large and complex projects by using proof of concepts and minimum viable products (MVPs) instead of detailed upfront requirements (Agile Alliance, 2001[3]; Meyer, 2014[4]).

The Greek public sector requires a cultural shift in the development of ICT/digital projects to better understand and place user needs at the core of the development processes. Beyond anecdotal cases, MDG and public sector organisations in Greece do not have institutionalised methods to conduct user research processes that help better understand and define the requirements when planning an ICT/digital project nor as well as concrete mechanisms to channel user feedback at early development stages.

Additionally, evidence collected during the interviews suggests that user research and service design skills are not common abilities among Greek public workforce. Within the OECD, several countries are progressing to setting standards and a culture of user research, such as the case of the US and the development of the Digital Services Playbook (see Box 4.1). The Playbook guides on the driving principles of digital services, including a specific checklist and recommendations on the role of users in the design and delivery of services.

Promoting user research and a culture around user needs implies establishing formal engagement mechanisms with users, where they can channel their needs into the design phase so that solutions provided fully meet their demands and expectations. In addition, the government needs to adapt the process of formulating digital initiatives to ensure that users’ input effectively impacts the design of solutions. Successful engagement with users should be continuous and iterative throughout the project development cycle, promoting ongoing exchanges and constant user feedback.

Similarly, the limited adoption of agile project management principles and standards is impeding a culture of experimentation, testing and iteration. Agile development is critical when addressing the risks of implementing large and complex digital transformation projects. Similarly, there is a limited culture and room for experimentation practices and the use of proof-of-concept in the design and delivery of digital solutions, in line with existing gaps on institutional capacities for delivery (see next section). Embracing a user-driven approach calls for flexibility in product development, promoting scalability, and encouraging continuous learning and improvement. The use of agile methodologies can help Greece foster a culture driven around user needs, such as it has been observed in New Zealand and the UK with dedicated guidelines and practices to support adoption of agile methodologies in designing and delivering services (see Box 4.2).

MDG is currently responsible for the approval, funding allocation and management of most ICT/digital initiatives, along with the support of Information Society S.A. in certain projects. Currently, the development process is fragmented and limitedly co-ordinated within the Ministry. The relevance attestation (project approval mechanism) acts as a method to join-up departments/units within MDG but it does not act as a policy lever to effectively support the implementation of the DTB. Instead, interviewees indicated that the process often creates delays in each phase as well as limited internal and external monitoringregarding the progress and status of each project as well as accountability of relevant stakeholders across the project life-cycle.

This process results in the limited information managed by MDG concerning the approval, funding, procurement, management and monitoring/evaluation of projects above EUR 60 000 set as budget threshold to be approved by MDG. Despite collecting general information about goals, technical specifications and expected timelines, MDG does not have a comprehensive knowledge management approach to leverage this information to effectively steer the implementation of the DTB, prioritise efforts and create synergies between relevant authorities to avoid project duplication and fragmentation. Interviewees identified that existing budget thresholds and approval mechanisms are inducing project fragmentation to avoid lengthy procurement processes with MDG and/or Information Society. Such an approach impedes synergies and alignment, producing a silo-based digitalisation of the public sector.

Additionally, the Ministry does not have clear visibility and overview of relevant projects below the budget threshold. The limited information held by MDG regarding these projects impedes the Ministry to take actions to secure efficiency, coherence and co-ordination among relevant line ministries (for example to centralise the procurement of ICT/digital commodities or to co-ordinate public sector institutions with similar needs). This approach may be also fostering project fragmentation and shadow IT in the Greek public sector, with the eventual impact on legacy issues in the medium and long-term.

All these issues have an ultimate impact on the role of MDG within the digital government ecosystem in Greece and its capacity to focus on the key priorities set in the DTB. Considering the comprehensive set of initiatives defined in the strategy, MDG could prioritise and give larger institutional attention to the projects with a systemic impact for the whole-of-government digital transformation in Greece. This includes critical digital transformation enablers, such as interoperability systems, digital identity, or payment platforms.

Instead, the Ministry currently devotes significant time and effort in dealing with the long-tail of small projects which could eventually be implemented by line ministries. While this requires further capacity building at institutional level to de-centralise the implementation of the strategy, MDG would benefit by concentrating efforts and resources in ensuring the right implementation of the objectives set for the public sector in the DTB. For this, the Ministry could make a strategic use of management tools to streamline the ICT/digital project development life cycle in order to support project prioritisation and the level of involvement (strategic, implementation or monitoring) of MDG and line ministries in this process.

Several countries have addressed similar issues by implementing a comprehensive ICT portfolio management approach that supports the implementation of digital government strategies. An ICT portfolio management approach can serve as a policy lever to address the development cycle of ICT/digital projects (from approval to monitoring/evaluation) recording relevant information to oversee the extent to which public sector organisations are meeting the objectives and goals set for each project. Additionally, such an approach can provide more granular information at project level in order to set concrete key performance indicators (KPIs) to assess and monitor progress (see Box 4.3 above).

MDG has different funding sources to support the implementation of ICT/digital initiatives, including European and national funds. According to the evidence collected, there are different procedures and internal responsibilities within MDG depending on the funding source, following different approval workflows and legal formalities. Currently, the limited alignment in the way projects are approved and funded is challenging MDG in terms of administrative and policy coherence.

For example, in the case of ESIF funds, ICT/digital projects are handled directly by MDG, which is not the case for the Public Investment Programme or line ministries budgets. These different procedures risk causing inefficient resource allocation by affecting investment decisions based on different criteria and parallel paperwork needed. Stakeholders within MDG acknowledge the need to develop coherent management frameworks that replicate the experience of the ESIF in other funding sources, in particular given the experience with the implementation of the Public Investment Programme or upcoming Resilience and Recovery Funds (RRF).

This coherence can be supported by revisiting and aligning the relevance attestation and funding mechanisms, unifying the criteria and procedures to have a coherent approach to approving and prioritising ICT/digital projects in the Greek government. In line with good practices on coherent capital budgeting frameworks, investment assessments and decisions should be independent of specific funding mechanisms (OECD, 2015[9]).

Additionally, and in line with the OECD Recommendation of the Council on Digital Government Strategies, the approval mechanism should include a multi-dimensional value proposition (analysis of the costs and benefits of ICT/digital projects) in the short, medium and long-term. This approach supports the identification of financial, social, environmental or administrative benefits and costs to support planning and prioritisation of initiatives. As outlined previously, the revision of the approval mechanism can be supported by the development of ICT portfolio management in order to have a comprehensive overview of the different stages.

MDG holds the critical role of approving and funding most of ICT/digital projects in the public sector. However, to date the Ministry is not leveraging this process to secure that digital investments adhere to the priorities and standards for digital government in Greece prior to allocating funding.

Currently, the DTB sets several priority areas to digitally transform the Greek government, economy and society. Concerning the public sector, the implementation of interoperability frameworks, the national central service delivery portal gov.gr and the development of digital skills require solid policy levers for MDG to secure a coherent and aligned approach across public sector organisations. This includes making effective use of budget thresholds and approval processes to enforce the implementation of digital government objectives as well as the needed standards and tools for its implementation. MDG could strategically use this process for public sector organisations to adopt existing and future digital tools and standards that secure a coherent implementation across the public sector. The evidence collected shows that line ministries acquire ICT goods such as software licenses or basic hardware using this waiver. These kinds of items are often highly standardisable. Therefore the government could benefit from co-ordinated procurement practices to secure the best value for money.

This is the case of Chile, where the Digital Government Division is co-ordinated with the Budget Office to assess all ICT/digital projects in the public sector in order to assess their adherence to strategic goals and adoption of digital government standards and tools (e.g. cloud first, agile management, digital identity, interoperability). This information is leveraged by the Budget Office to assess all digital initiatives included in the annual budget (See Box 4.4). This process also involves ChileCompra, Chile’s public procurement authority, which identifies common ICT/digital needs that can be centralised through collaborative or co-ordinated purchase mechanisms, as currently done with IT hardware, mobile phone plans, software licences and data infrastructure, among others.

A critical success factor for ICT/digital projects is the speed of implementation. In Greece, however, several elements contribute to extended implementation timeframes, including the project approval (relevance attestation), communication between relevant ministries, and management of the procurement process. Evidence indicates that some ICT/digital projects may require several years between the project proposal and contract award activities. Such delays pose challenges to attract suppliers, particularly SMEs that may not have the organisational and financial capacities to engage in overly long processes. At the same time, delayed implementation of procurement processes, particularly if not implemented in a flexible and agile way, risks delivering outdated solutions i.e. the technology may be already outdated, or the needs of end-users may have evolved. This means that solutions may no longer be fit-for-purpose or relevant (e.g. institutional mergers or reorganisations, regulatory changes, technology upgrades to different systems or platforms etc.).

As a starting point, there is a need to speed up the early preparatory stages of a digital project, which entail early project preparation, the validation on the basis of the DTB, as well as the selection of the appropriate financing schemes. While these steps are broadly understood by stakeholders within MDG and line Ministries, their implementation is often not as streamlined as it could be.

For instance, the approval of digital projects based on the DTB is in essence a simple and fast process, but institutional communication channels between the ministries involved often create delays. Greek authorities could think how to set-up direct communication and validation channels to ensure that the process is not slowed down unnecessarily.

Furthermore, the project preparation may entail a lengthy decision-making and negotiation process to define the responsibilities of the procurement process itself (i.e. whether a line ministry or Information Society S.A. will take over the procurement process) as well as the scope of the procurement project, in case it is executed by Information Society S.A. Limited capacity by the line ministries is linked to the fact that the project preparation is still at early stage when the support of Information Society S.A. is requested, requiring important preparation work as well as lengthy negotiations to establish a Memorandum of Understanding that frames the relationship for the development of ICT/digital projects, including its procurement.

Finally, as discussed above, the selection of the financing scheme is a process that also requires significant time, in particular when ESIF funding is involved, as additional requirements may be connected with EU-funded projects.

In addition to the slow implementation of the preparatory phase of digital projects, the procurement process itself is often burdened by extreme delays throughout each of the procurement phases. The project data analysed by the OECD team Figure 4.2 and Figure 4.3) reveals bottlenecks identified in each stage of the procurement process. In addition to delays, many ESIF financed projects do not reach sub-sequent implementation stages after their approval (Figure 4.2). In fact, while all the ICT projects approved under the ESIF programme period 2014-2020 obtained the approval of starting public procurement procedures, only 62% of them reached the stage of tender publication. Out of the projects that publish a tender, only 59% issued a contract award. Ultimately, only 22% of the initially approved projects reached the stage of contract implementation.

Beyond these bottlenecks in implementation, the overall execution of digital projects is severely slowed down. On average, the project procurement process takes 686 days, from the project approval to the start of contract implementation. This data excludes the time needed for actual contract implementation.

The tender preparation alone is takes significant time, taking on average 476 days from the project approval to the approval of starting procurement procedures. Then, it takes 24 days from the approval of starting procurement procedures to publishing tender, and 82 days from tender publication to issuing contract award. For instance, the tender preparation is lengthy due to the lack of an effective co-ordination processes with the project beneficiary. Lack of personnel as well as limited technical understanding of the subject matter of the contract may also contribute to slow implementation. Reportedly, delays are also linked to the evaluation stage, as evaluation committee’s members are not allotted dedicated time for this task, but it is an additional responsibility to their day-to-day work. This limits the incentive for speedy conclusion of the evaluation phase.

A key challenge in the procurement process of digital projects lies in frequent legal challenges. While the Review Body AEPP is relatively speedy in providing a decision, each phase of the procurement may be challenged without deadlines, thus taking up to a year for the process to conclude. If the decision of AEPP is further challenged in court, the process may take several years to conclude. Namely, it took 262 days on average from issuing contract award to starting contract, which could be mainly attributable to the legal challenges on the contract award decision.

Addressing the slow execution times of the remedies system goes well beyond the scope of the Ministry of Digital Governance. Nevertheless, procurement bodies within the Ministry could explore in greater detail whether recurring aspects of the procurement procedure are challenged in court, and identify measures to address these. Greater clarity of the tender documentation and of award criteria could help in limiting legal challenges. Similarly, enhancing the capacity of suppliers in understanding the procurement process may also reduce ambiguity and the related need to make use of the remedies system.

Analysing the duration of each phase of the procurement process gives insight into, it appears that specific bottlenecks and challenges that may emerge during the procurement process. Namely, the data suggest that challenges emerge in defining the detailed concept of ICT items to be procured, as this process takes a long time (476 days) (see Figure 4.3). Beyond the length of this phase, only 62% of the approved projects reach the stage of tender publication, suggesting additional challenges in preparing tender documentation, or other difficulties at this preparatory stage. Specific data or insights into the challenges that approved project face at this stage are not available, e.g. cancellations or other causes.

In contrast, the process to publish a tender once the approval of starting public procurement procedure has been received is relatively speedy (24 days). In parallel, the vast majority projects that published a tender also issued a contract award: 59% of approved projects issued contract award, compared to 62% of approved projects that published a tender. In addition, the tender process took a comparatively shorter amount of time: 82 days from publishing tender to issuing contract award. As such, there seem the tender process from tender issuance to contract award appears comparatively less challenging in terms of execution and speed. The stage from contract award to start of contract execution also appears fraught with difficulties, as a significant number of projects do not advance to this stage and this stage is very delayed. Specifically only 22% of all the approved projects started its contract implementation, and these 22% projects required 262 days from issuing contract award to starting a contract. This data could evidence the difficulties encountered through the legal challenges, as one of the potential causes for this low progress rate and the long time spent in this stage.

Overall, challenges around lengthy and burdensome procurement processes affect many OECD countries, and some have taken concrete steps to streamline and speed up these processes, such as the UK, as discussed in Box 4.5 below.

Beyond the speed of procurement processes, the effective implementation of ICT/digital projects requires advanced procurement practices. In the context of the Ministry of Digital Governance, a mixed picture has been observed with the level of advancement of procurement practices suited for digital projects. For instance, Information Society S.A., as a longstanding actor in the field, is more advanced in engaging with the market on a regular basis. In contrast, the Department of Procurement and Logistics covers the full spectrum of products and services, and relies on in-house expertise for the procurement of digital projects.

It is important to give enough focus and allocate enough time and resources to the preparatory phase for the success of the tender process. The more time spent in the preparatory phase of the procurement process, the less time will be needed to fix misunderstandings, discrepancies and mistakes in the procurement process and when managing contracts. The preparation entails gathering a sound understanding of what the market can deliver by conducting early market engagement. However, as discussed above, the practices for market engagement vary within MDG, but overall do not exploit the full possibilities permitted by the law, and the related benefits for the procurement of digital transformation projects.

In some instances, contracting authorities lack full understanding of the benefits of market engagement. Often, however, the barrier to conducting market engagement lies in contracting authorities’ lack of practical knowledge about how to conduct such market engagement, as well as knowledge of alternative modalities. In particular, contracting authorities fear legal repercussions for interacting with market providers. In contrast, market engagement is foreseen by the EU legal framework, provided the principles of transparency and equal treatment are respected. Thus, it is important for the process to be planned and managed carefully.

Beyond the respect of fundamental principles, the EU Directive leave room of manoeuvre to contracting authorities by providing the following definition of “preliminary market consultations”: “Before commencing a procurement procedure, contracting authorities may conduct market consultations with a view to preparing the procurement and informing economic operators of their procurement plans and procurements” (Article 40, Directive 2014/24/EU).The use of preliminary market consultations has been transposed into the Greek public procurement framework with Article 47 and 48 of L. 4412/2016. Furthermore, HSPPA has produced a dedicated technical instruction to provide contracting authorities with greater clarity on how to apply preliminary market consultations in accordance with the law (Technical Instruction 5/2019: Issuance of a Technical Instruction on: "Preliminary market consultations in innovation public procurement" Έκδοση Τεχνικής Οδηγίας με θέμα: «Προκαταρκτικές διαβουλεύσεις της αγοράς στις δημόσιες συμβάσεις καινοτομίας»).1

Market engagement can take several forms, and can also be used throughout the procurement cycle to continuously inform the contracting authority of market developments. Table 4.1 below summarises various forms of market engagement throughout the procurement cycle.

In the context of digital transformation projects, it is important to regularly exchange with the industry and keep abreast of technology developments. Several OECD countries are using digital platforms to facilitate market engagement between buyers and suppliers, i.e. so-called digital marketplaces. These platforms allow suppliers to present their products and facilitate the interaction with government buyers. Box 4.6 describes the UK’s initiative in this field.

Regular interaction with the market also provides the opportunity of identifying upcoming and innovation trends, as perceived by market players. Through the Emerging Market Initiative, National Association of State Procurement Officials (NASPO) in the U.S. surveys market players on what they consider to be innovative trends and upcoming needs for the public sector (Box 4.7).

Some measures are in place to understand beneficiary needs and engage them throughout the project cycle, but these appear not to be sufficiently developed to deliver outcomes in a consistent and satisfactory way. For instance, Information Society S.A. relies on setting up a multi-disciplinary team for each of its projects. Beneficiaries are represented in this team, and are key in contributing to all phases of the project development, including procurement the preparation of technical specifications. Despite this mechanism, the process appears to be running slowly and there can be challenges in creating a shared understanding between the technical/procurement team and the subject matter experts (i.e. the beneficiary).

In contrast, the Department of Procurement and Logistics does not appear to have a formalised process in place for understanding and engaging beneficiaries; instead, it relies on informal co-operation with the departments. While such a mechanisms may prove effective on an ad-hoc basis and solve short-term issues, they do not substitute for dedicated mechanisms to understand user needs and involve them effectively.

Additionally and as previously outlined, there is room for ICT/digital projects to adopt agile project management approaches that create spaces for increased dialogue and interaction between MDG, Information Society S.A., beneficiaries and end-users. Limiting the definition and development of ICT/digital projects only within the public sector fosters top-down approaches that do not reflect the final needs and expectations of end-users, affecting the pertinence and relevance of final solutions.

Another key aspect of success in the procurement of digital projects is the focus on quality in the award of a contract. In many instances, however, contracting authorities rely on the lowest price instead of awarding contracts to the best price-quality ratio. A common misconception is that market participants are not ready or willing to compete on the basis of quality. Fear of audit is also a frequent cause for relying on the lowest price as evaluation criterion. Not least, the lack of technical skills by procurement practitioners may also limit the use of quality criteria. As such, contracting authorities are not able to award contracts to high-quality providers that offer value-for-money solutions. Instead, choosing the lowest price brings along the risk of poor execution and less advanced technology. Despite a high prevalence of lowest price, the Greek procurement legal framework fully supports contract award based on best-price quality ratio, i.e. taking into account quality dimensions that are related to the subject matter of the contract, as per Article 86 of L. 4412/2016.

In a similar vein, overly prescriptive technical specifications are potential causes of sub-optimal execution of digital projects. In addition, overly prescriptive technical specifications are often the consequences of the almost exclusive use of the lowest price criteria. Importantly, prescriptive specifications do not leave any opportunity for the market to propose innovative solutions. Furthermore, such specifications may turn out overly rigid and inflexible. In the digital environment, however, it is especially important to allow for agility, modifications and iterations. Thus, a product or service procured on the basis of an overly defined specifications may prove not to be fit-for-purpose. Over-specifying details hide further risks and disadvantages. Customised solutions are generally more expensive than standard ‘off-the-shelf’ options. In addition, they are more difficult to be reused. Subsequently, suppliers who develop and manage custom-made systems can retain all the information about the system and make it very difficult to migrate to another supplier or to maintain or upgrade the system in the future. Excessive customisations might also lead to supplier dependence (vendor lock-in as it was already mentioned earlier). Contracting authorities should define the problem to be solved (the expected outcome of the purchase) rather than designing the solution.

Instead, functional specifications offer greater flexibility and room for innovation. When using functional specifications (or functional requirements), the contracting authority only specifies a required performance or outcomes, without providing the technical detail for reaching such a result. While much welcomed by market participants, as reported by private sector stakeholders, the procurement bodies appeared to lack awareness about such types of specifications and do not make use of them in practice. Nonetheless, the use of functional specifications is fully supported by the Greek procurement framework. Namely, Article 54 of L. 4412/2016 specifies that technical specifications shall be formulated, among other methods, “in terms of performance or functional requirements, including environmental characteristics, provided that the parameters are sufficiently precise to allow tenderers to determine the subject-matter of the contract and to allow contracting authorities to award the contract”.

Public procurement has been evolving with emerging advanced tools to drive efficiency throughout the public procurement cycle in satisfying the needs of the government and its citizens. Indeed, the OECD Recommendations of the Council on Public Procurement calls upon countries to develop and use tools to improve procurement procedures, reduce duplication and achieve greater value for money, including centralised purchasing, framework agreements, e-catalogues, dynamic purchasing, e-auctions, joint procurements and contracts with options. It also recommends countries to use public procurement as policy lever to pursue broad policy objectives such as stimulating innovation (OECD, 2015[15]).

Currently, Greece uses advanced procurement practices (such as dynamic purchasing systems and public procurement for innovation, including innovation partnerships) in a very limited way in the area of ICT public procurement, regardless of the fact that the Greek Public Procurement Law foresees these advanced procurement tools and schemes since it transposed the EU Directives on public procurement.

Dynamic purchasing system (DPS) is a procurement tool to support agile ICT procurement through a completely electronic system. Namely, under a dynamic purchasing system, new economic operators may apply for participating at any time throughout the life of the dynamic purchasing system, unlike framework agreements in which the participation of new economic operators is not allowed after the framework is set up (OECD SIGMA, 2017[16]). This dynamic feature allows contracting authorities to change suppliers more easily, and economic operators to continuously incorporate advances in technology. The DPS is thus very suitable for repeated purchases of standardised items, where technology developments occur fast. For instance, the European Commission adopted the use of DPS for procuring its cloud services. In addition to staying abreast of technological evolution, the tool significantly shortened lead times by 80% compared to the previously used framework agreements. In fact, even for complex tenders, the time between tender opening and final evaluation could be reduced to 2-3 weeks (Mercell, n.d.[17]).

From the evidence gathered during the fact-finding missions, stakeholders within MDG are still hesitant about the use of DPS, given the lack of experience with using this tool, and an overall risk-averse attitude towards new approaches in procurement, as procurement procedures are typically highly scrutinised at audit. This is despite the fact that DPS is regulated by Article 33 in L. 4412/2016 and the Greek e-procurement system fully supports the implementation of DPS from a technical perspective. To further support the uptake of DPS, HSPPA has recently translated a guide on DPS (“Dynamic Purchasing Systems: Use Guidelines” – “Δυναμικά συστήματα αγορών Κατευθυντήριες γραμμές χρήσης”) prepared by the European Commission (European Commission, 2021[18]).

Piloting the use of DPS with a limited scope, could be an initial starting point to test the technology, learn from a first iteration and potentially apply the DPS more widely going forward. Procurement officials would need to be supported by sufficient capacity-building to undertake such a pilot with confidence.

When considering setting up a DPS, it is important to take stock of experiences from countries and organisations that have already undertaken this effort. Namely, in some countries, there is a relatively widespread use of this instrument, which allows drawing lessons learnt and gathering insights on practical aspects related to organisational and management arrangements for running a DPS, benefits accrued to the organisations and suppliers, as well as key success factors. The example from the Danish central purchasing body SKI (Statens og Kommunernes Indkøbsservice) provides a detailed example of DPS implementation for standard software (see Box 4.8).

Furthermore, Central Purchasing Bodies across Europe have also implemented DPS in recent years. Their experiences are discussed in detail in Annex A.

The scheme of public procurement for innovation is another emerging public procurement practice, allowing governments to boost innovation at both the national and local levels. The use of public procurement for innovation is essential in the context of improving digital governance that always requires high-level innovative solutions which might not be available in the market. Strategic use of public procurement for innovation (PPI) is defined as “any kind of public procurement practice (pre-commercial or commercial) that is intended to stimulate innovation through research and development and the market uptake of innovative products and services” (OECD, 2017[19]). It includes schemes such as pre-commercial procurement (PCP), procurement of innovative solutions and Innovation Partnership.

Public procurement for innovation has been promoted by the European Commission since 2007 with the Commission Communication entitled ‘Pre-commercial Procurement: Driving innovation to ensure sustainable high quality public services in Europe’ (European Commission, 2007[20]). The 2014 Directives have further anchored the concept of public procurement of innovation in the legal framework by introducing the Innovation Partnership as a specific procedure to facilitate innovation procurement in the EU. Other procedures, such as the competitive dialogue or the negotiated procedure, are also suitable to conduct innovation procurement. In this sense, public procurement of innovation is fully aligned with the European legal framework. This is also reflected by the fact that in 2021, the European Commission published a Guidance on Innovation Procurement in order to offer guidance for policymakers and practitioners throughout the European Union on how innovation procurement can improve the economic recovery of the European Union after the COVID-19 crisis with better public investment (European Commission, 2021[21]) (Box 4.9). The relevant procurement methods apply in Greece, too, as defined in L. 4412/2016. Specifically, including competitive dialogue (Article 30, 267), innovation partnership (Article 31, 268), and negotiated procedure with prior call for competition (Article 266). To support the implementation of innovation procurement HSPPA developed dedicated Technical Instructions on Innovation Contracts (Τεχνική Οδηγία 2/2018, Συμβάσεις καινοτομίας).

Across the OECD, governments are using innovation procurement to address challenges related to digitalising government. This can be done by using a so-called “challenge-driven” approach, whereby the contracting authority formulates its problem as a challenge and invites economic operators to come up with solutions. An example of this is used by Scotland, as described below (Box 4.10).

Greece is still behind in the use of this procurement scheme for innovation in the ICT procurement, and overall investment and adoption of ICT-based innovative solutions. Namely, its share of procurement dedicated to innovative ICT amounts to 2.9% of out of total procurement expenditure, compared to the European average of 3.5%. Furthermore, Greece ranked 13th position (low performer) out of 30 European countries in the adoption of ICT-based public procurement for innovative solutions, according to a 2018 benchmarking study carried out by the European Commission on national innovation procurement policy frameworks (European Commission, 2021[23]).

The role of these advanced procurement practices has not been fully recognised yet by the stakeholders in the field of digital procurement in Greece due to the lack of awareness and specific skills. Greece ranked at the 15th position (modest performer) out of the 30 European countries (27 EU Member States plus the UK, Switzerland and Norway) on national innovation procurement policy frameworks, according to the benchmarking study carried out by the European Commission (European Commission, 2021[23]). The study points out the weakness in the absence of various elements to promote the use of public procurement for innovation such as the action plan, spending target, monitoring system, capacity-building and assistance measures.

However, Greece has been taking initiatives at a country level. The General Directorate for Public Procurements (GDPP) of the Ministry of Development and Investments, as National Central Purchasing Body for goods and general services in Greece, is mandated with setting up within its structural organisation a National Competence Centre on Innovation Procurement in Greece, under the Procure2Innovate project funded by the European Union Horizon 2020 programme. The centre aims at promoting mainstreaming innovation procurement throughout Greek contracting authorities (See Box 4.11) The Ministry of Digital Governance could benefit from getting involved in the process of establishing this competence centre.

In addition, international experiences identified in Poland and Finland show some options to promote the use of advanced practices such as starting by a pilot project, and supporting implementation with a competence centre. Poland has proven experiences in using PPI schemes such as pre-commercial procurement (PCP) and innovation partnership, starting from pilot projects. In particular, Poland has been using PCP to procure innovative solutions to advance the digital agenda. Namely, the National Centre for Research and Development (NCRD) of Poland, the agency in charge of ESIF R&I projects, initiated an e-Pioneer project financed under the ESIF Operational Programme "Digital Poland 2014-2020”. This initiative contributed to encouraging a larger pool of interested contractors to participate in the ESIF R&I projects by using PCP (see Box 4.12). Greece could benefit from considering the pilot initiative of this good practice in order to promote and support the use of advanced procurement practices in digital projects.

Finland set up a target of public procurement for innovation and established a Competence Centre for Sustainable and Innovative Procurement (KEINO), in order to support the use of innovative public procurement procedures (see Box 4.13).

The contract implementation stage should not be overlooked, as it represents the stage in which the solution is actually delivered by a supplier. Examples for poor performance are cost overruns, contract modifications, additional works and vendor lock-in.

The feedback gathered from conversations with stakeholders pointed out that there is room for improvement in the contract delivery. Reportedly, suppliers tend to deliver the minimum on required goods and services. While it is not clear to what extent minimum performance by suppliers undermines the overall contract implementation, and whether it affects all contracts equally, it should be noted that contracting authorities have limited awareness about the links between proper tender preparation and supplier performance. Namely, the pre-tender stages such as early market engagement and user involvement allow to have very solid expectations about what the market can deliver and at what price, and therefore limit the risk of poorly drafted tender documentation (from selection criteria, to award criteria, technical specification and even contract clauses). Conversely, well drafted tender documents and contract (contract clauses) mitigate risks for contract underperformance.

Overall, contracting authorities in Greece, however, place limited emphasis on the planning stages of a procurement. As outlined in this report, this phenomenon also applies in the context of procurement of digital projects.

From stakeholder conversation, it emerged that final beneficiaries often lack the skills and capacity to take full ownership of the solutions that has been procured. As such, they may not fully benefit from the gains of digitalisation. Furthermore, there are no comprehensive monitoring processes and reporting mechanisms that would allow beneficiaries to provide feedback, such as a satisfaction survey. Without this kind of feedback loop, it is difficult for procurement bodies to continuously improve their processes and interactions with beneficiaries.

Increasingly, contracting authorities across the OECD have been turning to agile approaches of project development, including procurement functions, and particularly when dealing with digital technology and ICT. At its core, agility refers to the ability to create and respond to change. The concept derives from the Agile Manifesto2 (Manifesto for Agile Software Development) published in 2001 (Agile Alliance, 2001[3]). The “Agile” approach takes its name because this word represents adaptability and response to change.

In the context of public procurement, applying agile principles refers to using collaboration, working flexibly and allowing for adaptation, proceeding in iterations and reviewing on a continuous basis. As such, the software end product is the result from a collaborative effort between developers and end-users (Deloitte, 2017[27]). In contrast, the common way of procuring ICT goods and services has been the waterfall method, which consists of a structured step-by-step approach through the various phases of conception, initiation, analysis, design, testing, and implementation. This approach implies involving users at an early development stage to have fully-defined and accepted set of requirements and needs that are not supposed to change in the course of implementation; and at the end of the process when end-users are involved and have the opportunity to provide feedback. This approach leaves no room for interaction and engagement with user on an ongoing basis, and puts pressure on ‘getting it right’ at the first step because the approach does not foresee the return to an earlier stage. As such, the waterfall model is considered less flexible and effective in software development (OECD, 2022[13]).

Despite a number of benefits of using the waterfall approach for software development, criticism revolves around the fact that the uncertainty around end-user’s needs, and the accuracy required of cost estimates at early stages of preparation are often not reflective of realities. Instead, the approach can be useful in the context of well-known technologies and stable user preferences. Data from Poland give insights into the success rates of agile projects versus waterfall projects. Namely, the success rate for agile projects lies at 49% compared to 14% when using waterfall. More importantly, the failure rate for waterfall is much higher (29%) compared to agile (9%), underscoring the strong risk mitigating aspects of an agile approach (Polish Economic Institute, 2019[22]).

Currently, Information Society S.A. makes use of the waterfall approach in the procurement of digital projects. As discussed, this model provides a clear framework for operations, but increases the risks of Information Society S.A. working for several months (or years) on services that no longer meet end-user needs once finalised. Furthermore, contracts for digital and ICT projects tend to be very large, thereby posing a further challenge to a more nimble and agile procurement process. As reported by private sector stakeholders, the size of the Greek public sector ICT market is relatively small compared to its private sector equivalent. Partly due to fiscal constraints of the past decades, the market has been characterised by a small number of large contracts, which are highly attractive to the pool of available suppliers. Such large contracts, however, also increase the incentives for litigation, as suppliers are aware that only few large opportunities are present on the market.

Increasing agility in the procurement process also foresees the breakdown of complex and large projects into smaller and iterative projects. The agile approach thus requires the use of modular contracting as a procurement strategy, i.e. procuring a series of tightly-scoped procurements to implement technology systems in successive and interoperable increments. The benefits of such a strategy include risk mitigation, reduction of vendor lock-in, as well as increased speed of delivery (Mark Headd, 2018[28]). Indeed, with shorter contracts, the risk of failure of the entire project is much reduced, and instead is contained to a small piece of the system. Similarly, planning a successful contract for a timeframe between six and twelve months is an easier undertaking compared to planning for a six-year horizon (Laura Gerhardt, 2019[29]).

Furthermore, these smaller and modular contracts can be accessible to a larger pool of suppliers, including innovative SMEs and start-ups. At the same time, procuring multiple contracts increases the need to procure much faster and requires re-thinking the procurement process through this lens. Shortening the procurement time and breaking up large contracts has been the approach taken in the State of California (Box 4.14).

Since agile development relies heavily on the interaction and collaboration between end-user, buyer and supplier, the process of contract management becomes a key success factor for the implementation of digital projects. This requires regular engagement throughout the process and a partnership approach to contract delivery. Trust in the relationship with suppliers is also an essential ingredient to the success of agile delivery, given that contractual non-performance cannot be defined as easily in a context where the contract scope is flexible. Often this represents a cultural barrier for public buyers, where typically civil servants rely on heavily detailed contract specifications. To overcome such barriers, the United States Digital Service developed the TechFAR Hub guidelines, which promote a more lenient interpretation of contract rules (Eggers and O’Leary, 2017[31]).

While moving from a waterfall approach to agile and modular contracting certainly requires a learning curve, it would be beneficial for Greek contracting authorities to gradually move towards an agile implementation methodology, especially for those projects where the technology and user needs are bound to evolve over time. As a starting point, Greek authorities could consider piloting such an approach, with a view of potentially scaling it up over time. In parallel, dedicated guidance and support material should be made available. Overall, a key goal of MDG should be to substantially reduce the duration of the procurement process. Applying modular contracting requires designing the development and procurement process in such a way to ensure its speedy execution. Finally, reducing the contract size in favour of multiple small projects would provide key benefits in terms of responsiveness to user needs, as well as limiting the risk of failure of large and expensive digital transformation projects.

Finally, despite potential hesitation from public buyers, using agile approaches in public procurement is not in contradiction with the regular procurement process, and does not require an adaptation of the regulatory framework. Similarly to other procurement practices discussed throughout this report, an agile procurement process also heavily relies on a solid preparation of the tender through needs assessment and early market engagement. In terms of procurement procedure, the regular process applies. All types of procurement procedures can be applied for agile implementation, though complex projects may be more suited for procedures that allow greater flexibility between the contracting authority and the supplier (e.g. competitive dialogue), in particular if the subject matter of the contract is not fully known from the onset. Specific agile elements may be included in the tender specifications and the contract conditions, such as modularity or multistage delivery. Selection criteria can ensure that suppliers have experience in agile working methods, while award criteria can give more weight to relevant aspects such as quality of UX design, or project management capacity. Importantly, contract conditions can be designed to introduce a greater degree of flexibility compared to more standard approaches.

To succeed in the implementation of transformational digital projects, the Greek government needs to rely on a pool of innovative and capable suppliers that are able to deliver quality solutions to the public administration. To this end, the Greek government needs to be considered an attractive client, in particularly given that the share of the ICT market by the public sector has been relatively small compared to the private sector (although the influx of funds from the Recovery and Resilience Facility (RRF) will likely change this dynamic). It needs to ensure a strong track record to increase competition and attract capable and innovative suppliers, especially in the context of an open and dynamic regional market such as the European Union. Speedy procurement procedures and payment processes are part of what constitutes an attractive client. At the same time, several barriers persist to making the public procurement market accessible to suppliers, particularly small, innovative start-ups and SMEs, notably the long duration of procurement cycle and competition that does not reward the highest quality offer, or innovative solutions. These aspects may deter small and innovative companies from participating in public sector bids, given that a significant amount of liquidity and capital is required to financially sustain a procurement process spanning over several years. Furthermore, as noted by procurement stakeholders, vested interests in the ICT market may also pose a barrier to participation of smaller companies. Namely, large ICT players often have more resources to lobby and engage with Government ICT leaders and politicians. This can contribute to the risk-aversion of smaller businesses who do not have the same financial capacity and access to lobbying resources.

From the buyer’s perspective, access to cutting-edge, innovative, start-up supplier eco-system including international companies remains difficult in their operations. The Greek ICT market overall is quite localised and characterised by small and medium-sized enterprises, as most companies do not surpass the 250 employees. In some instances, these SMEs represent Greek subsidiaries of international corporations.

Important work is ongoing to promote the involvement of new and innovative companies in the tendering arena, and is considered one of the priorities of Information Society S.A. Indeed, it has taken part into several activities that support the creation of a vibrant ecosystem of innovative suppliers. Information Society S.A. is also looking into how to harness international experience and expertise, if it is not yet available on the Greek market. Going forward, Information Society S.A. also plans to build an ecosystem for technology matching, further supporting the capacity and maturity of a local ICT ecosystem.

Similarly, the limited institutional capacities to plan, develop, procure and maintain ICT/digital projects increase the strategic relevance of the private sector as critical partners for the implementation of the DTB (Ministry of Digital Governance, 2021[33]). The strategy underlines the role of the GovTech ecosystem to support its implementation, and requires further actions to foster and strengthen this ecosystem to take this expected role.

While digital projects are often custom-made, procuring ICT and digital goods and services also consists purchasing standardised, highly-demanded and off-the-shelf solutions. For such purchases, centralising demand is an effective strategy to generate economies of scale, ensure efficient procurement operations and increased coherence and interoperability of solutions. Indeed, centralisation of procurement operations has proven to lead to significant benefits, such as better prices, lower transaction costs, as well as increased capacity and expertise since such procurements are typically carried out by specialised procurement professionals. In the context of technological change, specialisation and centralisation are necessary to accelerate delivery. Namely, they facilitate alignment on cultural aspects as well as coordination across departments and line ministries that may otherwise hamper the implementation of digital transformation. Over time, knowledge and capability become more pervasive as they spread across government entities. At the same time, centralisation entails risks, if not managed adequately (OECD, 2022[13]). When dealing with multiple clients, there is a risk of repeatedly encountering similar challenges as well as knowledge loss, which slows delivery and stifles innovation. Hence, it is important to ensure a learning effect within the CPB’s clients.

Recognising the potential for centralisation of ICT, several OECD countries have already introduced specialised ICT Central Purchasing Bodies (CPBs). For instance, Germany introduced the Central Office for IT Procurement within the Federal Procurement Office of the Federal Ministry of the Interior (Zentralstelle für IT-Beschaffung) in 2017. This entity is the central point of contact for ICT procurement at the federal level. In its initial phase, ZIB was tasked with an advisory function, i.e. supporting users throughout the entire procurement process, from the first notification of a need, through planning and commissioning, including contract management throughout the length of the agreement.​ Since its creation, it has moved to an implementation phase, whereby it also carries out tenders for on behalf of contracting authorities (OECD, 2019[34]).

Similarly, Ireland introduced centralised ICT procurement in order to deliver on its Public Service ICT Strategy. This effort is led by the Office of Government Procurement, i.e. Ireland’s CPB. Ireland’s approach focuses on whole-of-government IT applications that are crosscutting and impact multiple departments, as opposed to technologies that are specific to one ministry, and streamlining applications such as payroll or messaging, while leaving devolved decision-making for agency-specific technologies with the relevant agency.

In the context of MDG, no such centralisation or advisory function of digital technology and ICT needs has been identified. It is unclear to what extent common needs and repetitive needs within MDG are bundled together and procured via dedicated framework agreements by the Department of Procurement and Logistics. To create or strengthen such a function, MDG could also consider co-operating with the central-level CPB in Greece, namely the General Directorate of Public Procurements of the General Secretariat of Commerce and Consumer Protection of the Ministry of Economy, Development and Tourism.

This function can be supported by leveraging existing project approval processes or future efforts to adopt a whole-of-government ICT portfolio management to identify common needs of highly standardised and demanded ICT/digital products or goods. This information, along with other data sources that provide further information on purchasing behaviour, can serve as basis to analyse and conduct centralised procurement exercises.

One of the most pressing challenges in the Greek public sector to effectively implement the DTB is the limited institutional capacities to plan, manage, procure and implement ICT/digital projects. The restricted managerial and technical capacities in line ministries to carry out digital transformation projects has created a significant overload in MDG and its dependent units, including Information Society S.A., as they concentrate most of the digital expertise in the public sector and take complete control over the development lifecycle of these projects. As a consequence, line ministries take often a passive role and have limited ownership over the process, preventing a coherent and more decentralised implementation of the DTB.

The DTB includes several provisions on digital skills development to underpin the national digital transformation strategy, including the development of a national digital competence framework. Accordingly, OECD governments as the UK have developed professional capability frameworks to support institutions identifying skills and professional profiles needed to drive transformation (see Box 4.16). However, such an approach may be insufficient to address the existing digital talent and skill challenges. Concrete initiatives are needed to equip line ministries and their workforces with the skills needed to implement the goals set in the DTB. This includes assessing existing competences and developing training programmes to effectively accompany the implementation of the DTB, including the management and procurement of ICT/digital projects.

Interviewees indicated the limited conditions to attract and retain digital talent in the public sector as one of the reasons for constrained institutional capacities to manage ICT/digital projects. As outlined previously, the Greek public sector observes a limited culture for horizontal collaboration and co-operation, which constrains the development of an agile culture to co-ordinate digital transformation projects. Similarly, organisational conditions to foster digital talent and skills are limited to date, including promotion and training schemes as well as attractive employment packages to keep the public workforce abreast of new practices in the ICT/digital domain. Management culture can further exacerbate existing issues of talent attraction and retention by not properly valuing staff, providing opportunities for career advancement as well as promoting a culture of co-operation necessary for bringing about the transformational changes sought by MDG.

Conversely, institutional and management culture may strengthen positive developments of collaboration, continuous improvement and talent attraction by setting the tone at the top. In this sense, Greece could benefit from promoting multidisciplinary teams in ICT/digital project development, bringing together digital professionals with non-digital backgrounds in designing and delivering digital investments. Merging digital and non-digital expertise can be helpful to create a culture for collaboration and a better understanding of the multiple policy, legal, and financial factors that may determine the scope and impact of digital transformation projects Figure 4.4).

Finally, a comprehensive approach towards equipping public sector organisations with the tools to implement digital transformation projects includes setting guidelines and standards that fosters coherence and alignment of efforts. Guidelines and standards are an effective way to enable system-wide transformation while promoting coherence and alignment in the development of ICT/digital projects both within and outside MDG. As outlined previously, this may include service design and delivery, agile management, and ICT/digital procurement standards and guidelines.

When looking at the availability of standards and common practices for ICT/digital projects, MDG has not set a concrete and actionable set of common tools that help empower service teams to have a more leading role in the development of these projects. This fosters incentives for line ministries to rely on MDG and Information Society S.A. to implement these projects on their behalf and take a less active role as beneficiaries rather than project owners. Further developing a comprehensive set of standards and guidelines for digital transformation in the Greek government can help build capacities across the public sector and reinforce ownership over digital initiatives.

The implementation of digital projects requires a specific set of procurement and digital competencies. As reported by stakeholders during the fact-finding missions, there has been limited investment and recruitment of personnel with specific ICT skills over the past several years. As a result, procurement entities are short in specialised personnel that can bridge the divide between expertise in digital technology and procurement competence. Such professional profiles would be needed to effectively translate ICT needs into clear procurement documents (i.e. technical specifications, selection criteria, award criteria). As a result, stakeholders consider understaffing and the lack of appropriately skilled resources as one of their main challenges in the implementation of procurement of digital projects.

Importantly, stakeholders within MDG see the potential for training and upskilling existing staff as a means to overcome some of the skills gaps. In fact, the required capabilities are often available in-house throughout the Greek government, but are not organised to ensure knowledge transfer and capacity-building. In parallel, better planning of the project pipeline, and effective prioritisation of projects are also useful tools to manage a growing workload within existing constraints. Initial efforts in prioritisation have proven effective in delivering more effectively on multiple projects.

Across the OECD, countries are increasingly recognising the need for specific skills related to digital and ICT procurement, covering the full procurement cycle as well as agile methodologies. The US for instance, invested heavily in developing dedicated training (Box 4.17).

Evidence indicates that MDG does not have a comprehensive monitoring and assessment policy for digital transformation initiatives in the public sector. Despite the DTB indicates the future development of a monitoring system for the activities comprised in the strategy, it does not provide further details about what and how the system will measure digital government progress and performance. Similarly, evidence collected indicates that systematic use of monitoring tools in MDG is limited and responds to efforts scattered across the different phases of ICT/digital project development and the respective departments/units involved in the process. Considering the fragmented approach towards ICT/digital project management within MDG, efforts to improve monitoring should be part of a broader strategic approach for benefits realisation within the Ministry, involving all relevant stakeholders.

Evidence indicates that currently there are no clear indicators to track progress and/or performance of ICT/digital projects. Additionally, KPIs can help create incentives within the public sector if managed in a transparent and open way, fostering accountability of all related stakeholders involved in the process. For example, in Australia some regional governments are using online dashboards to track progress and monitor the implementation of their digital transformation strategies, including progress of ICT/digital projects through data visualisation and open data (see Box 4.18). In line with the progress and existing culture for open government data in the country, MDG can leverage this information and make it available to relevant stakeholders to foster transparency, reinforce accountability and align incentives to pursue the strategic goals set in the DTB.

Additionally, performance data can be an effective way to feed development cycles and take concrete actions in the redefinition of strategic priorities. Considering the steering role of MDG in setting goals and priorities for digital government, having a sound approach to assess the performance of ICT/digital projects as well as to proactively use this information to redefine priorities and set new goals is critical to secure the realisation of intended outcomes.

Similarly, a user-driven approach for digital government calls for improved and standardised ways to assess the experience of end-users with ICT/digital projects. This implies establishing coherent feedback mechanisms for end-users (citizens and businesses) to communicate their experience with a certain service. However, in line with the limited initiatives to understand and meet the needs of end-users, MDG does not have a standardised approach to measure user satisfaction in digital government. Given the nature of some ICT/digital projects (service design and omni-channel delivery) and the priority set in the DTB for the implementation of the centralised service platform gov.gr, MDG would benefit by setting a comprehensive user satisfaction measurement policy.

Currently, the DTB includes a dedicated provision to collect and process user feedback (the Integrated Citizen Relationship Management System). In order to improve the assessment of ICT/digital projects, MDG should accelerate its implementation, setting a common methodology for the Greek public sector organisations to assess user satisfaction in line with the limited institutional capacities observed to support the development of ICT/digital projects.


[8] Agency for Digitalisation (n.d.), ICT portfolio management, https://en.digst.dk/ict-portfolio-management/ (accessed on 27 September 2021).

[3] Agile Alliance (2001), Manifesto for Agile Software Development, http://agilemanifesto.org/.

[36] Central Digital and Data Office (2019), Making the Digital, Data and Technology Capability Framework more user friendly, https://digitalpeople.blog.gov.uk/2019/10/24/making-the-digital-data-and-technology-capability-framework-more-user-friendly/ (accessed on 27 September 2021).

[27] Deloitte (2017), Going Agile: The new mind-set for procurement officials, https://www2.deloitte.com/us/en/insights/industry/public-sector/agile-in-government-procurement-mindset.html.

[10] División Gobierno Digital (2021), Instrucciones para la formulación de proyectos TIC - EVALTIC 2022, https://digital.gob.cl/transformacion-digital/estandares-y-guias/instrucciones-para-la-formulacion-de-proyectos-tic-evaltic-2022/ (accessed on 27 September 2021).

[31] Eggers, W. and J. O’Leary (2017), “Agile in Government: A playbook from the Deloitte Center for Government Insights”, https://www2.deloitte.com/content/dam/insights/us/articles/3897_Agile-in-government/DUP_Agile-in-Government-series.pdf (accessed on 27 September 2021).

[21] European Commission (2021), COMMISSION NOTICE - Guidance on Innovation Procurement - C(2021) 4320 final, https://ec.europa.eu/docsroom/documents/45975.

[23] European Commission (2021), The strategic use of public procurement for innovation in the digital economy, https://op.europa.eu/en/publication-detail/-/publication/7f5a67ae-8b8e-11eb-b85c-01aa75ed71a1/language-en (accessed on 5 October 2021).

[18] European Commission (2021), “Δυναμικά συστήματα αγορών Κατευθυντήριες γραμμές χρήσης”, https://www.eaadhsy.gr/images/docs/ET0121340ELN.el.pdf (accessed on 22 February 2022).

[37] European Commission (2020), “ProcurCompEU: Study on professionalisation of public procurement in the EU and selected third countries”, https://doi.org/10.2873/1198.

[26] European Commission (2020), “Study on professionalisation of public procurement in the EU and selected third countries”, https://doi.org/10.2873/1198.

[20] European Commission (2007), “Pre-commercial Procurement: Driving innovation to ensure sustainable high quality public services in Europe”, https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52007DC0799&from=EN (accessed on 7 February 2022).

[11] European Commission (n.d.), “Streamline public procurement procedures”, https://ec.europa.eu/regional_policy/sources/good_practices/GP_fiche_19.pdf (accessed on 27 October 2021).

[29] Laura Gerhardt, M. (2019), Why we love modular contracting, https://18f.gsa.gov/2019/04/09/why-we-love-modular-contracting/.

[28] Mark Headd, E. (2018), Modular contracting and working in the open, https://18f.gsa.gov/2018/10/25/modular-contracting-and-working-in-the-open/.

[17] Mercell (n.d.), 80% decrease in tender lead times with the implementation of a Dynamic Purchasing System supported by Mercell, https://www.negometrix.com/en/cases/european-commission/.

[4] Meyer, B. (2014), Agile! The Good, the Hype and the Ugly, Springer International Publishing: Switzerland.

[33] Ministry of Digital Governance (2021), “Bible of Digital transformation 2021-2025”, https://digitalstrategy.gov.gr/ (accessed on 17 August 2021).

[1] Ministry of Digital Policy Telecommunications and Information (2016), National Digital Strategy, 2016-2021 (ΕΘΝΙΚΗ ΨΗΦΙΑΚΗ ΣΤΡΑΤΗΓΙΚΗ).

[42] Ministry of Interior and Safety (2021), Assessing Impacts of Digital Government in Korea.

[41] MinTIC (2021), Índice de Gobierno Digital, https://colombiatic.mintic.gov.co/679/w3-propertyvalue-36675.html (accessed on 2 November 2021).

[6] New Zealand Government (2019), Assurance guidance for Agile delivery, https://www.digital.govt.nz/dmsdocument/115-assurance-guidance-for-agile-delivery-full/html (accessed on 27 September 2021).

[13] OECD (2022), Towards Agile ICT Procurement in the Slovak Republic: Good Practices and Recommendations, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/b0a5d50f-en.

[35] OECD (2021), “The OECD Framework for digital talent and skills in the public sector”, OECD Working Papers on Public Governance, No. 45, OECD Publishing, Paris, https://doi.org/10.1787/4e7c3f58-en.

[34] OECD (2019), Public Procurement in Germany: Strategic Dimensions for Well-being and Growth, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/1db30826-en.

[19] OECD (2017), Public Procurement for Innovation: Good Practices and Strategies, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/9789264265820-en.

[15] OECD (2015), OECD Recommendation on the Council on Public Procurement, http://www.oecd.org/gov/public-procurement/recommendation/OECD-Recommendation-on-Public-Procurement.pdf (accessed on 28 May 2018).

[9] OECD (2015), “Recommendation of the Council on Budgetary Governance”, https://www.oecd.org/gov/budgeting/Recommendation-of-the-Council-on-Budgetary-Governance.pdf (accessed on 22 September 2021).

[2] OECD (2014), “Recommendation of the Council on Digital Government Strategies”, Public Governance and Territorial Development Directorate, Vol. July.

[12] OECD (n.d.), OECD Public Governance Policy Papers, OECD Publishing, Paris, https://doi.org/10.1787/14e1c5e8-en-fr.

[16] OECD SIGMA (2017), Procurement Tools, https://www.oecd-ilibrary.org/docserver/5js4wzv3mtvj-en.pdf?expires=1629455540&id=id&accname=ocid84004878&checksum=492A5A5C9F0D36277DA73F7F1DA88CC8 (accessed on 20 August 2021).

[14] OECD-OPSI (2021), Cooperative Procurement Emerging Markets Initiative, https://oecd-opsi.org/innovations/cooperative-procurement-emerging-markets-initiative/.

[22] Polish Economic Institute (2019), “GovTech - New technology in the public sector”.

[24] Procure2Innovate (n.d.), National Competence Centre on Innovation Procurement in Greece, http://www.procure2innovate.eu (accessed on 25 August 2021).

[25] Public Procurement Office of Poland (2020), Public Procurement of Innovation, https://www.uzp.gov.pl/__data/assets/pdf_file/0017/45503/Zamowienia_publiczne_na_innowacje_EN_WCAG-2021-01-12.pdf (accessed on 16 April 2021).

[39] Queensland Government (2021), Digital Projects Dashboard, https://www.qld.gov.au/digitalprojectsdashboard (accessed on 2 November 2021).

[30] Rath, D. (2017), Agile Acquisitions: Rethinking Public-Sector Purchasing, Government Technology, https://www.govtech.com/budget-finance/GT-September-2017-Agile-Acquisitions-Rethinking-Public-Sector-Purchasing.html (accessed on 6 December 2022).

[32] The U.S. Digital Service (n.d.), TechFAR Hub, https://techfarhub.cio.gov/handbook/general-considerations/.

[38] U.S. Digital Service (n.d.), Digital IT Acquisition Professional Training (DITAP), https://techfarhub.cio.gov/initiatives/ditap/.

[7] UK Government (2017), Assurance for agile delivery of digital services, https://www.gov.uk/government/publications/assurance-for-agile-delivery-of-digital-services (accessed on 27 September 2021).

[5] US Digital Service (n.d.), Digital Service Playbook, https://playbook.cio.gov/ (accessed on 27 September 2021).

[40] Victorian Government (2021), Victorian Government IT project dashboard, https://www.vic.gov.au/IT-project-dashboard (accessed on 2 November 2021).

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2022

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at https://www.oecd.org/termsandconditions.